Project

Profile

Help

Issue #3013

Content is not downloaded after switching from on_demand to immediate policy

Added by ttereshc about 4 years ago. Updated 20 days ago.

Status:
CLOSED - WORKSFORME
Priority:
High
Assignee:
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 104
Quarter:

Description

Steps to reproduce:

  • create repo
  • create importer with on_demand policy
  • sync
  • update importer, set policy to immediate
  • sync again

Observe that no files were downloaded.

My guess is that importer looks into pulp_file_filecontent only and builds inventory based on the data from it without checking an existence of corresponding artifact in case of immediate policy.

History

#1 Updated by ttereshc about 4 years ago

  • Priority changed from Normal to High
  • Severity changed from 2. Medium to 3. High
  • Triaged changed from No to Yes

#2 Updated by jortel@redhat.com about 4 years ago

The basic problem is that the importer is determining there is no content in the remote repository that needs to be added to the local (pulp) repository. Nothing to do.

We need to take a holistic approach here.

Few options:

  1. The user is responsible for triggering a repository-download task when changing the policy to immediate.
  2. The policy change (to immediate) could trigger a repository-download task automatically. But, what if this fails?
  3. The policy change needs to be noticed by the plugin (perhaps with help from the platform). Upon noticing this, the plugin needs to attempt to download artifacts just as it does when adding content (to the repository) that already exists.

The last option involving the importer is least desirable. It adds unwanted complexity to every importer. Either 1 or 2 or combination of 1 and 2 seems like the best idea.

Thoughts?

#3 Updated by bmbouter over 2 years ago

  • Tags deleted (Pulp 3)

#4 Updated by dalley 4 months ago

  • Sprint set to Sprint 98

This might be fixed already but we definitely need to verify that, and if it's not fixed then it will need attention soon.

#5 Updated by ggainey 4 months ago

I just did the following on my core-master et al dev system:

pulp rpm remote create --name dell --url http://linux.dell.com/repo/hardware/dsu/os_independent/ --policy on_demand
pulp rpm repository create --name dell --remote dell
ppulp rpm repository sync --name dell
pulp rpm remote update --name dell --download-concurrency 5 --policy immediate
pulp rpm repository sync --name dell

The second sync downloaded all the content, as you can see from the task output:

pulp task show --href /pulp/api/v3/tasks/395fec3f-1a2e-4965-baa0-9fad2d09afb6/
{
  "pulp_href": "/pulp/api/v3/tasks/395fec3f-1a2e-4965-baa0-9fad2d09afb6/",
  "pulp_created": "2021-06-14T16:57:18.061465Z",
  "state": "running",
  "name": "pulp_rpm.app.tasks.synchronizing.synchronize",
  "logging_cid": "b6023513cb594a98b51dcc71870e6ecc",
  "started_at": "2021-06-14T16:57:18.180855Z",
  "finished_at": null,
  "error": null,
  "worker": "/pulp/api/v3/workers/49e49578-edd5-4c67-b3f3-dbf9fde8a4a1/",
  "parent_task": null,
  "child_tasks": [],
  "task_group": null,
  "progress_reports": [
    {
      "message": "Associating Content",
      "code": "associating.content",
      "state": "running",
      "total": null,
      "done": 0,
      "suffix": null
    },
    {
      "message": "Downloading Metadata Files",
      "code": "sync.downloading.metadata",
      "state": "completed",
      "total": null,
      "done": 4,
      "suffix": null
    },
    {
      "message": "Parsed Packages",
      "code": "sync.parsing.packages",
      "state": "completed",
      "total": 634,
      "done": 634,
      "suffix": null
    },
    {
      "message": "Downloading Artifacts",
      "code": "sync.downloading.artifacts",
      "state": "running",
      "total": null,
      "done": 454,
      "suffix": null
    }
  ],
  "created_resources": [
    null
  ],
  "reserved_resources_record": [
    "/pulp/api/v3/remotes/rpm/rpm/f4b27411-a886-4d8b-85f1-06d3925f8282/",
    "/pulp/api/v3/repositories/rpm/rpm/c146463f-6dc0-46a2-b50f-896996a9ed6e/"
  ]
}

I think we're good - but additional testing would be great.

#6 Updated by rchan 3 months ago

  • Sprint changed from Sprint 98 to Sprint 99

#7 Updated by rchan 3 months ago

  • Sprint changed from Sprint 99 to Sprint 100

#8 Updated by rchan 2 months ago

  • Sprint changed from Sprint 100 to Sprint 101

#9 Updated by ipanova@redhat.com about 2 months ago

  • Sprint changed from Sprint 101 to Sprint 102

#10 Updated by rchan about 2 months ago

  • Sprint changed from Sprint 102 to Sprint 103

#11 Updated by rchan about 1 month ago

  • Sprint changed from Sprint 103 to Sprint 104

#12 Updated by lmjachky 20 days ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to lmjachky

#13 Updated by lmjachky 20 days ago

Since the issue description is referring to importer, I am not sure whether this is really a pulp3 thing. I also tried to reproduce the error on pulp3 with the file plugin (similarly to Grant) and I could not reproduce it while having the pulp_file and pulpcore master branches checked out. If it is related to pulp3, then we can close this issue.

#14 Updated by lmjachky 20 days ago

  • Status changed from ASSIGNED to CLOSED - CURRENTRELEASE

#15 Updated by lmjachky 20 days ago

  • Status changed from CLOSED - CURRENTRELEASE to CLOSED - WORKSFORME

Works for the 3.16.0.dev pulpcore version.

Please register to edit this issue

Also available in: Atom PDF