Standardise Task execution order

[RealOrangeOne]

Code of Conduct

• I agree to follow Django's Code of Conduct

Feature Description

I suggest defining exactly what the order of task execution should be.

Problem

The currently implemented first-party Tasks backends don't behave as proper queues for executing tasks. Therefore, there's no canonical definition for the order that tasks should be executed in based on their attributes.

django-tasks-db has defined this as:

[F("priority").desc(), F("run_after").asc(), F("enqueued_at").asc()]

(Defined as ORM fields for convenience).

Request or proposal

proposal

Additional Details

Some backends, such as RQ, do not support this level of customization for ordering. The standard should be best-effort, and adhered to as much as possible.

Implementation Suggestions

I suggest taking the semantics from django-tasks-db and defining them as the standard execution order.

Following on from https://code.djangoproject.com/ticket/37126, this could be additionally defined as a sorting implementation for TaskResult, rather than just in documentation.

[github-actions]

Thank you RealOrangeOne for sharing your idea! We have a lot of them so please be patient. You can see the current queue here.

Community instructions

For commenters, please use the emoji reactions on the issue to express support, and/or concern easily. Please use the comments to ask questions or contribute knowledge about the idea. It is unhelpful to post comments of "I'd love this" or "What's the state of this?"

Reaction Guide

• 👍 This is something I would use
• 👎 This is something that would cause problems for me or Django
• 😕 I’m indifferent to this
• 🎉 This is an easy win

[codingjoe]

Thanks for the ticket, Jake. 👍

Side note: a FIFO queue makes the most sense to me too. However, a strict run_after order will be difficult to guarantee for most backends, since deferred tasks may be held in a separate queue.
We might want to document this with a small asterisk here.

[ZhengKangYang]

Hi everyone,

I am the author of the original PR (#21383) for ticket #37126. I strongly agree with using the sorting standard from django-tasks-db as the canonical execution order.
If possible, I would love to continue the work and submit a new PR based on the consensus reached in this discussion.