Project

Profile

Help

Issue #3013

Content is not downloaded after switching from on_demand to immediate policy

Added by ttereshc almost 4 years ago. Updated about 21 hours ago.

Status:
NEW
Priority:
High
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 99
Quarter:

Description

Steps to reproduce:

  • create repo
  • create importer with on_demand policy
  • sync
  • update importer, set policy to immediate
  • sync again

Observe that no files were downloaded.

My guess is that importer looks into pulp_file_filecontent only and builds inventory based on the data from it without checking an existence of corresponding artifact in case of immediate policy.

History

#1 Updated by ttereshc almost 4 years ago

  • Priority changed from Normal to High
  • Severity changed from 2. Medium to 3. High
  • Triaged changed from No to Yes

#2 Updated by jortel@redhat.com over 3 years ago

The basic problem is that the importer is determining there is no content in the remote repository that needs to be added to the local (pulp) repository. Nothing to do.

We need to take a holistic approach here.

Few options:

  1. The user is responsible for triggering a repository-download task when changing the policy to immediate.
  2. The policy change (to immediate) could trigger a repository-download task automatically. But, what if this fails?
  3. The policy change needs to be noticed by the plugin (perhaps with help from the platform). Upon noticing this, the plugin needs to attempt to download artifacts just as it does when adding content (to the repository) that already exists.

The last option involving the importer is least desirable. It adds unwanted complexity to every importer. Either 1 or 2 or combination of 1 and 2 seems like the best idea.

Thoughts?

#3 Updated by bmbouter about 2 years ago

  • Tags deleted (Pulp 3)

#4 Updated by dalley 5 days ago

  • Sprint set to Sprint 98

This might be fixed already but we definitely need to verify that, and if it's not fixed then it will need attention soon.

#5 Updated by ggainey 5 days ago

I just did the following on my core-master et al dev system:

pulp rpm remote create --name dell --url http://linux.dell.com/repo/hardware/dsu/os_independent/ --policy on_demand
pulp rpm repository create --name dell --remote dell
ppulp rpm repository sync --name dell
pulp rpm remote update --name dell --download-concurrency 5 --policy immediate
pulp rpm repository sync --name dell

The second sync downloaded all the content, as you can see from the task output:

pulp task show --href /pulp/api/v3/tasks/395fec3f-1a2e-4965-baa0-9fad2d09afb6/
{
  "pulp_href": "/pulp/api/v3/tasks/395fec3f-1a2e-4965-baa0-9fad2d09afb6/",
  "pulp_created": "2021-06-14T16:57:18.061465Z",
  "state": "running",
  "name": "pulp_rpm.app.tasks.synchronizing.synchronize",
  "logging_cid": "b6023513cb594a98b51dcc71870e6ecc",
  "started_at": "2021-06-14T16:57:18.180855Z",
  "finished_at": null,
  "error": null,
  "worker": "/pulp/api/v3/workers/49e49578-edd5-4c67-b3f3-dbf9fde8a4a1/",
  "parent_task": null,
  "child_tasks": [],
  "task_group": null,
  "progress_reports": [
    {
      "message": "Associating Content",
      "code": "associating.content",
      "state": "running",
      "total": null,
      "done": 0,
      "suffix": null
    },
    {
      "message": "Downloading Metadata Files",
      "code": "sync.downloading.metadata",
      "state": "completed",
      "total": null,
      "done": 4,
      "suffix": null
    },
    {
      "message": "Parsed Packages",
      "code": "sync.parsing.packages",
      "state": "completed",
      "total": 634,
      "done": 634,
      "suffix": null
    },
    {
      "message": "Downloading Artifacts",
      "code": "sync.downloading.artifacts",
      "state": "running",
      "total": null,
      "done": 454,
      "suffix": null
    }
  ],
  "created_resources": [
    null
  ],
  "reserved_resources_record": [
    "/pulp/api/v3/remotes/rpm/rpm/f4b27411-a886-4d8b-85f1-06d3925f8282/",
    "/pulp/api/v3/repositories/rpm/rpm/c146463f-6dc0-46a2-b50f-896996a9ed6e/"
  ]
}

I think we're good - but additional testing would be great.

#6 Updated by rchan about 21 hours ago

  • Sprint changed from Sprint 98 to Sprint 99

Please register to edit this issue

Also available in: Atom PDF