Pulp

I'm also seeing these tasks, but in my case there are also a number (right now 8) of these deferred_download tasks in the queue.
Since there is no information about what's going on with these tasks I have no idea if they've been there for just a couple of minutes or several months.
I have seen these tasks for a while, but I don't see any way of finding out if these are the same tasks or if they're being completed and new ones are added to the queue.

$ pulp-admin tasks list 
+----------------------------------------------------------------------+
                                 Tasks
+----------------------------------------------------------------------+

Operations:  deferred_download
Resources:   
State:       Waiting
Start Time:  Unstarted
Finish Time: Incomplete
Task Id:     cbe6d773-54c3-4a47-a4ab-51a1ab7a81a8

Operations:  deferred_download
Resources:   
State:       Waiting
Start Time:  Unstarted
Finish Time: Incomplete
Task Id:     afbff9e7-31be-4409-a881-d81ac29a2e38

Operations:  deferred_download
Resources:   
State:       Waiting
Start Time:  Unstarted
Finish Time: Incomplete
Task Id:     a947e629-83be-4874-bd1c-be2733100400

Operations:  deferred_download
Resources:   
State:       Waiting
Start Time:  Unstarted
Finish Time: Incomplete
Task Id:     d59b7cf1-2442-47ba-abf3-c37999bf423b

Operations:  deferred_download
Resources:   
State:       Waiting
Start Time:  Unstarted
Finish Time: Incomplete
Task Id:     1d8dc6ad-42aa-4eed-8d30-47b37ae07ba8

Operations:  deferred_download
Resources:   
State:       Waiting
Start Time:  Unstarted
Finish Time: Incomplete
Task Id:     8be5a6f8-1968-483f-972f-7a9eead6f2aa

Operations:  deferred_download
Resources:   
State:       Waiting
Start Time:  Unstarted
Finish Time: Incomplete
Task Id:     c0f79f6b-b547-4fdb-a338-ae8387b94521

Operations:  deferred_download
Resources:   
State:       Waiting
Start Time:  Unstarted
Finish Time: Incomplete
Task Id:     6af5d0af-a6cc-4669-86e4-a7eebcf52dcd

$ pulp-admin tasks details --task-id 6af5d0af-a6cc-4669-86e4-a7eebcf52dcd
+----------------------------------------------------------------------+
                              Task Details
+----------------------------------------------------------------------+

Operations:       deferred_download
Resources:        
State:            Waiting
Start Time:       Unstarted
Finish Time:      Incomplete
Result:           Incomplete
Task Id:          6af5d0af-a6cc-4669-86e4-a7eebcf52dcd
Progress Report:  

Aside from seeing extra tasks in the task list, I think there are no other downsides to the current approach. If you want to see fewer tasks, you can go in /etc/pulp/server.conf to the [lazy] section's "download_interval" setting and make it a much bigger number.

Here's the thinking on how it works currently and why:

Pulp needs to periodically look at the list of deferred downloads, and retrieve any files it finds listed.

We have a scheduler that is capable of queuing a task at specific times. It's basically upstream "celery beat". It's sole purpose in life is to consult a schedule, and queue tasks at the right time.

I don't think we would want to introduce any conditional logic to the scheduler. "Queue this task ever hour, but only if some list of conditions are met". We would have to give it per-task-type code to run to check conditions, and that introduces all kinds of new failure opportunities into this otherwise very-simple service.

It's easier to let the task get queued and run. Running a task is unbelievably inexpensive, so that's no problem. If the task finds no work to do, it exits very quickly (none went more than a second from the output you provided). So in terms of workflow and separation of concerns, I think we have the simplest and most robust approach in place.

So far what I'm saying is, "This is the expected behavior."

But there is one option you might like that we are considering for other reasons. There has been interest from some users in having a "cache-only" Pulp. All repos would have the on_demand policy, but Pulp would never permanently store any of the files. They would be cached by squid according to its rules, and that's it. The easiest way to accomplish this is to let the user disable the download_deferred task. This would have the side benefit of letting a user such as yourself disable the task, and not have to look at its entries in the task status list.

Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.