ERROR: duplicate key value violates unique constraint "django_dramatiq_task_pkey"

[anapaulagomes]

In a fresh installation of django-dramatiq, I got the following error:

db_1        | 2021-05-30 06:48:15.094 UTC [59] ERROR:  duplicate key value violates unique constraint "django_dramatiq_task_pkey"
db_1        | 2021-05-30 06:48:15.094 UTC [59] DETAIL:  Key (id)=(f1381983-90c3-41ce-bf7f-4c781b8e5149) already exists.
db_1        | 2021-05-30 06:48:15.094 UTC [59] STATEMENT:  INSERT INTO "django_dramatiq_task" ("id", "status", "created_at", "updated_at", "message_data", "actor_name", "queue_name") VALUES ('f1381983-90c3-41ce-bf7f-4c781b8e5149'::uuid, 'running', '2021-05-30T06:48:15.090606+00:00'::timestamptz, '2021-05-30T06:48:15.090622+00:00'::timestamptz, '\x7b2271756575655f6e616d65223a2264656661756c74222c226163746f725f6e616d65223a226765745f636974795f636f756e63696c5f75706461746573222c2261726773223a5b22323032312d30352d3237225d2c226b7761726773223a7b7d2c226f7074696f6e73223a7b22706970655f746172676574223a7b2271756575655f6e616d65223a2264656661756c74222c226163746f725f6e616d65223a22646973747269627574655f636974795f636f756e63696c5f6f626a656374735f746f5f73796e63222c2261726773223a5b5d2c226b7761726773223a7b7d2c226f7074696f6e73223a7b7d2c226d6573736167655f6964223a2237623063333262342d323530312d343466352d626161662d623264643631356166623164222c226d6573736167655f74696d657374616d70223a313632323335373239353037387d7d2c226d6573736167655f6964223a2266313338313938332d393063332d343163652d626637662d346337383162386535313439222c226d6573736167655f74696d657374616d70223a313632323335373239353037387d'::bytea, 'get_city_council_updates', 'default')

Version: 0.10.0
Broker: rabbitmq

[sshivananda]

@Bogdanp any idea about what might cause this error?

[mowidatim]

Noticed this on my Google cloud VM as well. Everything seems to be functioning as we would expect, just have a nice large 3.5m entry log file. I use the same back end (RabbitMQ).

[chachra]

happens for me as well when trying to do a message.get_result()

[yuzhay]

Have the same issue
version: 0.11.5
broker: Redis

I assume that it might be because of 2 django nodes working in parallel.

[dkoenigroer]

Same issue when adding pre_save listener to a model with uuid as primary key or overriding its save-method .

[ikvk]

ERROR: duplicate key value violates unique constraint "django_dramatiq_task_pkey"

The error occurs due to the race-cond in update_or_create of the create_or_update_from_message method.
Reason: after_enqueue is executed by the sending thread of the actor, and before_process_message and after_process_message are executed by the worker.

1. To eliminate errors, you can use a mutex in create_or_update_from_message before update_or_create, for example:

from django_lock import lock
...
lock_name = 'django_dramatiq__create_or_update_from_message'
lock_obj = lock(lock_name, timeout=5, blocking=True, release_on_del=True)  # *in Redis
if not lock_obj.acquire():
    logger.error(f'failed to get the mutex" {lock_name}"when writing the django_dramatiq log') # todo del
    return None
task, _ = self.using(DATABASE_LABEL).update_or_create(
...

2. To increase the chance of consistency of the task Task.status in the database,
   you can put sleep in the AdminMiddleware methods before create_or_update_from_message:
   before_process_message - time.sleep(0.01)
   after_process_message time.sleep(0.02)

I don't think it can be done better here.

* The possibility of race-cond in update_or_create is specified in the django dock:
https://docs.djangoproject.com/en/5.1/ref/models/querysets/#update-or-create