Issue #3013
closedContent is not downloaded after switching from on_demand to immediate policy
Added by ttereshc about 7 years ago. Updated over 3 years ago.
Description
Steps to reproduce:
- create repo
- create importer with on_demand policy
- sync
- update importer, set policy to immediate
- sync again
Observe that no files were downloaded.
My guess is that importer looks into pulp_file_filecontent
only and builds inventory
based on the data from it without checking an existence of corresponding artifact in case of immediate policy.
Updated by ttereshc about 7 years ago
- Priority changed from Normal to High
- Severity changed from 2. Medium to 3. High
- Triaged changed from No to Yes
Updated by jortel@redhat.com about 7 years ago
The basic problem is that the importer is determining there is no content in the remote repository that needs to be added to the local (pulp) repository. Nothing to do.
We need to take a holistic approach here.
Few options:
- The user is responsible for triggering a repository-download task when changing the policy to immediate.
- The policy change (to immediate) could trigger a repository-download task automatically. But, what if this fails?
- The policy change needs to be noticed by the plugin (perhaps with help from the platform). Upon noticing this, the plugin needs to attempt to download artifacts just as it does when adding content (to the repository) that already exists.
The last option involving the importer is least desirable. It adds unwanted complexity to every importer. Either 1 or 2 or combination of 1 and 2 seems like the best idea.
Thoughts?
Updated by dalley over 3 years ago
- Sprint set to Sprint 98
This might be fixed already but we definitely need to verify that, and if it's not fixed then it will need attention soon.
Updated by ggainey over 3 years ago
I just did the following on my core-master et al dev system:
pulp rpm remote create --name dell --url http://linux.dell.com/repo/hardware/dsu/os_independent/ --policy on_demand
pulp rpm repository create --name dell --remote dell
ppulp rpm repository sync --name dell
pulp rpm remote update --name dell --download-concurrency 5 --policy immediate
pulp rpm repository sync --name dell
The second sync downloaded all the content, as you can see from the task output:
pulp task show --href /pulp/api/v3/tasks/395fec3f-1a2e-4965-baa0-9fad2d09afb6/
{
"pulp_href": "/pulp/api/v3/tasks/395fec3f-1a2e-4965-baa0-9fad2d09afb6/",
"pulp_created": "2021-06-14T16:57:18.061465Z",
"state": "running",
"name": "pulp_rpm.app.tasks.synchronizing.synchronize",
"logging_cid": "b6023513cb594a98b51dcc71870e6ecc",
"started_at": "2021-06-14T16:57:18.180855Z",
"finished_at": null,
"error": null,
"worker": "/pulp/api/v3/workers/49e49578-edd5-4c67-b3f3-dbf9fde8a4a1/",
"parent_task": null,
"child_tasks": [],
"task_group": null,
"progress_reports": [
{
"message": "Associating Content",
"code": "associating.content",
"state": "running",
"total": null,
"done": 0,
"suffix": null
},
{
"message": "Downloading Metadata Files",
"code": "sync.downloading.metadata",
"state": "completed",
"total": null,
"done": 4,
"suffix": null
},
{
"message": "Parsed Packages",
"code": "sync.parsing.packages",
"state": "completed",
"total": 634,
"done": 634,
"suffix": null
},
{
"message": "Downloading Artifacts",
"code": "sync.downloading.artifacts",
"state": "running",
"total": null,
"done": 454,
"suffix": null
}
],
"created_resources": [
null
],
"reserved_resources_record": [
"/pulp/api/v3/remotes/rpm/rpm/f4b27411-a886-4d8b-85f1-06d3925f8282/",
"/pulp/api/v3/repositories/rpm/rpm/c146463f-6dc0-46a2-b50f-896996a9ed6e/"
]
}
I think we're good - but additional testing would be great.
Updated by ipanova@redhat.com over 3 years ago
- Sprint changed from Sprint 101 to Sprint 102
Updated by lmjachky over 3 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to lmjachky
Updated by lmjachky over 3 years ago
Since the issue description is referring to importer
, I am not sure whether this is really a pulp3 thing. I also tried to reproduce the error on pulp3 with the file plugin (similarly to Grant) and I could not reproduce it while having the pulp_file and pulpcore master branches checked out. If it is related to pulp3, then we can close this issue.
Updated by lmjachky over 3 years ago
- Status changed from ASSIGNED to CLOSED - CURRENTRELEASE
Updated by lmjachky over 3 years ago
- Status changed from CLOSED - CURRENTRELEASE to CLOSED - WORKSFORME
Works for the 3.16.0.dev pulpcore version.