Issue #4246
closedStory #3693: Lazy for Pulp3
Lazy sync does not create additional RemoteArtifact objects
Description
Say you sync a repo with Remote.policy=on_demand and you get unit foo but no Artifact.
Now you sync a different repo w/ a different remote with Remote.policy=on_demand and that repo also contains unit foo. During this second sync the stages API will recognize the unit is already in Pulp and will not create it. It will also not create a RemoteArtifact. It will associate the existing unit foo with the new repo version.
Now delete the first Remote, which will delete all RemoteArtifact objects.
Now a user goes to fetch foo from the second repo. The user will receive a 404 from the content app because when it find the ContentArtifact, and no Artifact to serve it will not find a RemoteArtifact like it should. Thus the content app can only 404.
I expect since there is still a Remote that can remotely fetch that Artifact that if should be fetched.
Overall I think every sync needs to ensure that in the case where a Content unit is already know to Pulp that the Stages pipeline will still create a RemoteArtifact object where today it does not.
Solution¶
1. have the RemoteArtifact codepath be created even if the ContentUnit exists. (currently it doesn't run at all if .pk != None)
2. Have RemoteArtifact switch to bulk_get_or_create() to handle the IntegrityErrors.
Related issues
Updated by bmbouter almost 6 years ago
I think to fix this, we need to have the ContentUnitSaver stage attempt to create a RemoteArtifact for each ContentArtifact for that remote and handle the IntegrityError. The performance of that is going to suck though, but for correctness I'm not sure what else we can do.
To accopmlish ^, maybe we:
1. have the RemoteArtifact codepath be created even if the ContentUnit exists. (currently it doesn't run at all if .pk != None)
2. Have RemoteArtifact switch to bulk_get_or_create() to handle the IntegrityErrors.
Updated by jortel@redhat.com almost 6 years ago
Agreed. A RemoteArtifact should always be created for each DeclarativeArtifact.
Updated by CodeHeeler almost 6 years ago
- Triaged changed from No to Yes
- Tags Pulp 3 RC Blocker added
Updated by bmbouter almost 6 years ago
- Description updated (diff)
- Sprint Candidate changed from No to Yes
Adding solution to the main ticket to get it groomed.
Updated by dkliban@redhat.com almost 6 years ago
- Related to Story #2507: The streamer needs to try all RemoteArtifact entires related to a ContentArtifact added
Updated by dkliban@redhat.com almost 6 years ago
- Groomed changed from No to Yes
- Sprint set to Sprint 46
Updated by jortel@redhat.com almost 6 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to jortel@redhat.com
Updated by jortel@redhat.com almost 6 years ago
- Status changed from ASSIGNED to POST
Updated by jortel@redhat.com almost 6 years ago
- Status changed from POST to ASSIGNED
Updated by jortel@redhat.com almost 6 years ago
- Status changed from ASSIGNED to POST
Updated by jortel@redhat.com almost 6 years ago
- Status changed from POST to MODIFIED
Applied in changeset commit:pulpcore-plugin|cbf69d2dd0ffd139915b05f5f7a1f0a4972ad282.
Updated by bherring over 5 years ago
- Copied to Test #4465: Lazy sync does not create additional RemoteArtifact objects added
Updated by bmbouter almost 5 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE