Project

Profile

Help

Issue #3013

closed

Content is not downloaded after switching from on_demand to immediate policy

Added by ttereshc over 6 years ago. Updated over 2 years ago.

Status:
CLOSED - WORKSFORME
Priority:
High
Assignee:
Start date:
Due date:
Estimated time:
Severity:
3. High
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 104
Quarter:

Description

Steps to reproduce:

  • create repo
  • create importer with on_demand policy
  • sync
  • update importer, set policy to immediate
  • sync again

Observe that no files were downloaded.

My guess is that importer looks into pulp_file_filecontent only and builds inventory based on the data from it without checking an existence of corresponding artifact in case of immediate policy.

Actions #1

Updated by ttereshc over 6 years ago

  • Priority changed from Normal to High
  • Severity changed from 2. Medium to 3. High
  • Triaged changed from No to Yes
Actions #2

Updated by jortel@redhat.com over 6 years ago

The basic problem is that the importer is determining there is no content in the remote repository that needs to be added to the local (pulp) repository. Nothing to do.

We need to take a holistic approach here.

Few options:

  1. The user is responsible for triggering a repository-download task when changing the policy to immediate.
  2. The policy change (to immediate) could trigger a repository-download task automatically. But, what if this fails?
  3. The policy change needs to be noticed by the plugin (perhaps with help from the platform). Upon noticing this, the plugin needs to attempt to download artifacts just as it does when adding content (to the repository) that already exists.

The last option involving the importer is least desirable. It adds unwanted complexity to every importer. Either 1 or 2 or combination of 1 and 2 seems like the best idea.

Thoughts?

Actions #3

Updated by bmbouter almost 5 years ago

  • Tags deleted (Pulp 3)
Actions #4

Updated by dalley almost 3 years ago

  • Sprint set to Sprint 98

This might be fixed already but we definitely need to verify that, and if it's not fixed then it will need attention soon.

Actions #5

Updated by ggainey almost 3 years ago

I just did the following on my core-master et al dev system:

pulp rpm remote create --name dell --url http://linux.dell.com/repo/hardware/dsu/os_independent/ --policy on_demand
pulp rpm repository create --name dell --remote dell
ppulp rpm repository sync --name dell
pulp rpm remote update --name dell --download-concurrency 5 --policy immediate
pulp rpm repository sync --name dell

The second sync downloaded all the content, as you can see from the task output:

pulp task show --href /pulp/api/v3/tasks/395fec3f-1a2e-4965-baa0-9fad2d09afb6/
{
  "pulp_href": "/pulp/api/v3/tasks/395fec3f-1a2e-4965-baa0-9fad2d09afb6/",
  "pulp_created": "2021-06-14T16:57:18.061465Z",
  "state": "running",
  "name": "pulp_rpm.app.tasks.synchronizing.synchronize",
  "logging_cid": "b6023513cb594a98b51dcc71870e6ecc",
  "started_at": "2021-06-14T16:57:18.180855Z",
  "finished_at": null,
  "error": null,
  "worker": "/pulp/api/v3/workers/49e49578-edd5-4c67-b3f3-dbf9fde8a4a1/",
  "parent_task": null,
  "child_tasks": [],
  "task_group": null,
  "progress_reports": [
    {
      "message": "Associating Content",
      "code": "associating.content",
      "state": "running",
      "total": null,
      "done": 0,
      "suffix": null
    },
    {
      "message": "Downloading Metadata Files",
      "code": "sync.downloading.metadata",
      "state": "completed",
      "total": null,
      "done": 4,
      "suffix": null
    },
    {
      "message": "Parsed Packages",
      "code": "sync.parsing.packages",
      "state": "completed",
      "total": 634,
      "done": 634,
      "suffix": null
    },
    {
      "message": "Downloading Artifacts",
      "code": "sync.downloading.artifacts",
      "state": "running",
      "total": null,
      "done": 454,
      "suffix": null
    }
  ],
  "created_resources": [
    null
  ],
  "reserved_resources_record": [
    "/pulp/api/v3/remotes/rpm/rpm/f4b27411-a886-4d8b-85f1-06d3925f8282/",
    "/pulp/api/v3/repositories/rpm/rpm/c146463f-6dc0-46a2-b50f-896996a9ed6e/"
  ]
}

I think we're good - but additional testing would be great.

Actions #6

Updated by rchan over 2 years ago

  • Sprint changed from Sprint 98 to Sprint 99
Actions #7

Updated by rchan over 2 years ago

  • Sprint changed from Sprint 99 to Sprint 100
Actions #8

Updated by rchan over 2 years ago

  • Sprint changed from Sprint 100 to Sprint 101
Actions #9

Updated by ipanova@redhat.com over 2 years ago

  • Sprint changed from Sprint 101 to Sprint 102
Actions #10

Updated by rchan over 2 years ago

  • Sprint changed from Sprint 102 to Sprint 103
Actions #11

Updated by rchan over 2 years ago

  • Sprint changed from Sprint 103 to Sprint 104
Actions #12

Updated by lmjachky over 2 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to lmjachky
Actions #13

Updated by lmjachky over 2 years ago

Since the issue description is referring to importer, I am not sure whether this is really a pulp3 thing. I also tried to reproduce the error on pulp3 with the file plugin (similarly to Grant) and I could not reproduce it while having the pulp_file and pulpcore master branches checked out. If it is related to pulp3, then we can close this issue.

Actions #14

Updated by lmjachky over 2 years ago

  • Status changed from ASSIGNED to CLOSED - CURRENTRELEASE
Actions #15

Updated by lmjachky over 2 years ago

  • Status changed from CLOSED - CURRENTRELEASE to CLOSED - WORKSFORME

Works for the 3.16.0.dev pulpcore version.

Also available in: Atom PDF