[Fixed]-Database "is being accessed by other users" error when using ThreadPoolExecutor with Django

7👍

Make sure you are calling connection.close() in your code when working with threads. This solved the issue for me, where other proposed solutions (decorator for the test methods, changing db settings, DB functions like shown by Nick above, restarting postgres) did not. Useful example

5👍

This looks like more Django’s issue than Postgres’ —
see for example this ticket: https://code.djangoproject.com/ticket/22420

From what you provided, I see that Django didn’t close all connections to test database before making attempt to drop it. In this case, Postges tries to protect other sessions from data losses, so it cannot drop/rename the database before they all disconnect.

If you want, you can drop the test database manually, using pg_terminate_backend(..) function and pg_stat_activity view you already used:

select pg_terminate_backend(pid) 
from pg_stat_activity 
where
  datname = 'DATABASE_NAME'
;

drop database DATABASE_NAME;

If, by some reason, somebody is very quick and manages to connect between theese two commands, drop database will fail again. In such case, you can repeat it, but before that revoke rights to connect to this database from public — this will prevent connections to it:

revoke connect on database DATABASE_NAME from public;

…and then repeat operations described above.

👤Nick

5👍

Connections to the database are thread local. I ended up fixing this problem by adding a callback to each future returned by the executor.

from django.db import connections

def on_done(future):
    # Because each thread has a connection, so here you can call close_all() to close all connections under this thread name.
    connections.close_all()

def main():
    # ...
    with ThreadPoolExecutor() as executor:
        while True:
            future = executor.submit(do, get_a_job())
            future.add_done_callback(on_done)

More found here: https://www.programmersought.com/article/3618244269/

Another thing that could be the problem is that you are subclassing TestCase which holds a global lock on your test. SubclassingTransactionTestCase will fix this problem and allow any threads your test may spawn to communicate with the database

2👍

I got similar problem and solved it by using django.test.testcases.TransactionTestCase as superclass for my test class

1👍

In my PyCharm (Run process) + postgres db case, I’ve fixed it by running:

1.Log in as a user that has correct rights to db, in my case it’s default “postgres“.

$ psql -h localhost -U postgres -W

2. Then drop all connections to existing db, in my case “mydbname” with command:

# select pg_terminate_backend(pid) from pg_stat_activity where datname='mydbname';

3. Then just click “Run” button PyCharm in normal way.

Leave a comment