Project

Profile

Help

Actions
Send by e-mail Copy link

Refactor #4173

closed

Change the multilayered design to use Futures to handle nested content

Added by amacdona@redhat.com about 6 years ago. Updated almost 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
ipanova@redhat.com
Sprint/Milestone:
1.0.0
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Sprint:
Sprint 51
Quarter:

Description

With the addition of DeclarativeContent.future, we ought to be able to use that to handle any nested content without loops or the layered pipeline we currently use. With that refactor in place, the pipeline will be flat and each stage will only need to be used once, which will eliminate the need for reentrant ProgressBars described previously.

A previous commit for pulp_deb used this design pattern, and will be helpful for the implementation here:
https://github.com/pulp/pulp_deb/blob/de4539b1b213317ec710c83a5fc8a4eaf0dcf019/pulp_deb/app/tasks/synchronizing.py

It should be noted that pulp_deb no longer uses that pattern to get around the min_size deadlocking problem, so I'm going to set 4296 as as blocker. https://pulp.plan.io/issues/4296

Current implementation does not use the ProgessBar. This is because the ProgressBar returns unexpected results when the same Stage is used multiple times as we do in Docker sync.

Related issues

Blocked by Pulp - Issue #4296: Stages API could deadlock when "discovering" content due to minsizeCLOSED - CURRENTRELEASEmdellwegActions
Blocks Container Support - Refactor #4178: Update sync to use ContentSaver StageCLOSED - CURRENTRELEASEipanova@redhat.com

Actions
Actions
Copy link
 #1

Updated by ipanova@redhat.com about 6 years ago

will it be a change in the progress bar or stages pulpcore code? or in docker plugin?

Actions
Copy link
 #2

Updated by amacdona@redhat.com almost 6 years ago

I didn't dig too deep into this, I was more focused on getting sync to work. I hope that these changes can exist only in the docker code. If it can't be done, we may need to make changes to pulpcore.

Actions
Copy link
 #3

Updated by amacdona@redhat.com almost 6 years ago

  • Sprint Candidate changed from No to Yes
Actions
Copy link
 #4

Updated by ipanova@redhat.com almost 6 years ago

  • Groomed changed from No to Yes
Actions
Copy link
 #5

Updated by jortel@redhat.com almost 6 years ago

  • Groomed changed from Yes to No
Actions
Copy link
 #6

Updated by jortel@redhat.com almost 6 years ago

  • Sprint Candidate changed from Yes to No
Actions
Copy link
 #7

Updated by dkliban@redhat.com almost 6 years ago

The problem in the original description is due to a limitation of the Stages API. If the stages API allowed stages that come after the Artifact download stage to send items into the queue for the Artifact download stage, the Progress Bar inside the Artifact download stage would report accurate results. bmbouter is going to add another story to add support for such a use case. Once that is implemented this story should be updated to state that pulp_docker needs to be refactored to use that feature.

Actions
Copy link
 #8

Updated by amacdona@redhat.com almost 6 years ago

  • Blocked by Issue #4296: Stages API could deadlock when "discovering" content due to minsize added
Actions
Copy link
 #9

Updated by amacdona@redhat.com almost 6 years ago

  • Tracker changed from Story to Refactor
  • Subject changed from As a user my sync task tracks progress with the ProgressBar to Change the multilayered design to use Futures to handle nested content
  • Description updated (diff)
Actions
Copy link
 #10

Updated by amacdona@redhat.com almost 6 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to amacdona@redhat.com
Actions
Copy link
 #11

Updated by amacdona@redhat.com almost 6 years ago

  • Status changed from ASSIGNED to NEW

I will not have a chance to do this for at least 2 weeks, so I am putting this back. If others want to take this, please feel free!

Actions
Copy link
 #12

Updated by amacdona@redhat.com almost 6 years ago

Actions
Copy link
 #13

Updated by ipanova@redhat.com almost 6 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee changed from amacdona@redhat.com to ipanova@redhat.com
  • Groomed changed from No to Yes
  • Sprint Candidate changed from No to Yes
  • Sprint set to Sprint 49
Actions
Copy link
 #14

Updated by rchan over 5 years ago

  • Sprint changed from Sprint 49 to Sprint 50
Actions
Copy link
 #15

Updated by ipanova@redhat.com over 5 years ago

  • Status changed from ASSIGNED to POST
Actions
Copy link
 #16

Updated by rchan over 5 years ago

  • Sprint changed from Sprint 50 to Sprint 51
Actions
Copy link
 #17

Updated by ipanova@redhat.com over 5 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

Applied in changeset commit:pulp_docker|568c4a350964a91cb8c4fedd86e255e29537c19e.

Actions
Copy link
 #18

Updated by bmbouter over 5 years ago

  • Tags deleted (Pulp 3)
Actions
Copy link
 #19

Updated by ipanova@redhat.com about 5 years ago

  • Project changed from Docker Support to Container Support
Actions
Copy link
 #20

Updated by bmbouter almost 5 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions
Copy link
 #21

Updated by ipanova@redhat.com almost 5 years ago

  • Sprint/Milestone set to 1.0.0
Actions
Send by e-mail Copy link

Also available in: Atom PDF