[Fixed]-Avoiding MySQL deadlock in Django ORM

by

9đź‘Ť

Use select_for_update() method:

samples = self.model.objects.select_for_update().filter(
                          user=self.user,
                          timestamp__gte=start_time,
                          timestamp__lte=end_time)


for sample in samples:
    # do something with a sample
    sample.save()

Note that you shouldn’t delete selected samples and create new ones. Just update the filtered records. Lock for these records will be released then your transaction will be committed.

BTW instead of __gte/__lte lookups you can use __range:

samples = self.model.objects.select_for_update().filter(
                          user=self.user,
                          timestamp__range=(start_time, end_time))
👤catavaran

6đź‘Ť

To avoid deadlocks, what I did was implement a way of retrying a query in case a deadlock happens.

In order to do this, what I did was I monkey patched the method “execute” of django’s CursorWrapper class. This method is called whenever a query is made, so it will work across the entire ORM and you won’t have to worry about deadlocks across your project:

import django.db.backends.utils
from django.db import OperationalError
import time

original = django.db.backends.utils.CursorWrapper.execute

def execute_wrapper(*args, **kwargs):
    attempts = 0
    while attempts < 3:
        try:
            return original(*args, **kwargs)
        except OperationalError as e:
            code = e.args[0]
            if attempts == 2 or code != 1213:
                raise e
            attempts += 1
            time.sleep(0.2)

django.db.backends.utils.CursorWrapper.execute = execute_wrapper

What the code above does is: it will try running the query and if an OperationalError is thrown with the error code 1213 (a deadlock), it will wait for 200 ms and try again. It will do this 3 times and if after 3 times the problem was not solved, the original exception is raised.

This code should be executed when the django project is being loaded into memory and so a good place to put it is in the __ini__.py file of any of your apps (I placed in the __ini__.py file of my project’s main directory – the one that has the same name as your django project).

Hope this helps anyone in the future.

👤Luccas Correa

Leave a comment