Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #7535

closed

Pulp_rpm sync reports "completed" but does not update packages

Added by Aant over 4 years ago. Updated about 4 years ago.

Status:
CLOSED - WORKSFORME
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
4. Urgent
Version:
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

I have set up Pulpcore v3.6.3 with Pulp_rpm v3.6.2. I created the following remote:

{
  "pulp_href": "/pulp/api/v3/remotes/rpm/rpm/d8fb524c-e73b-4afc-ba2e-05eea7f5b6a8/",
  "pulp_created": "2020-08-18T08:47:19.183836Z",
  "name": "centos8_base",
  "url": "http://mirror.centos.org/centos-8/8/BaseOS/x86_64/os/",
  "ca_cert": null,
  "client_cert": null,
  "client_key": null,
  "tls_validation": true,
  "proxy_url": "http://<proxy server>",
  "username": null,
  "password": null,
  "pulp_last_updated": "2020-08-18T08:47:19.183855Z",
  "download_concurrency": 10,
  "policy": "immediate",
  "sles_auth_token": null
}

and this repository:

{
  "pulp_href": "/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/",
  "pulp_created": "2020-08-18T08:47:17.704104Z",
  "versions_href": "/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/versions/",
  "latest_version_href": "/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/versions/1/",
  "name": "centos8_base",
  "description": null,
  "remote": null,
  "metadata_signing_service": null,
  "retain_package_versions": 0
}

If I sync, it reports success:

http --auth admin --auth-type basic :9000/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/sync/ remote=/pulp/api/v3/remotes/rpm/rpm/d8fb524c-e73b-4afc-ba2e-05eea7f5b6a8/
http: password for admin@localhost:9000:
HTTP/1.1 202 Accepted
Allow: POST, OPTIONS
Connection: close
Content-Length: 67
Content-Type: application/json
Date: Mon, 21 Sep 2020 08:10:11 GMT
Server: gunicorn/20.0.4
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN

{
    "task": "/pulp/api/v3/tasks/655d2e7b-2783-4496-a851-3bcd8d5e442e/"
}

Let's take a closer look at the task:

 http --auth admin --auth-type basic :9000/pulp/api/v3/tasks/655d2e7b-2783-4496-a851-3bcd8d5e442e/
http: password for admin@localhost:9000:
HTTP/1.1 200 OK
Allow: GET, PATCH, DELETE, HEAD, OPTIONS
Connection: close
Content-Length: 743
Content-Type: application/json
Date: Mon, 21 Sep 2020 08:10:29 GMT
Server: gunicorn/20.0.4
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN

{
    "child_tasks": [],
    "created_resources": [],
    "error": null,
    "finished_at": "2020-09-21T08:10:11.585792Z",
    "name": "pulp_rpm.app.tasks.synchronizing.synchronize",
    "parent_task": null,
    "progress_reports": [
        {
            "code": "optimizing.sync",
            "done": 1,
            "message": "Optimizing Sync",
            "state": "completed",
            "suffix": null,
            "total": null
        }
    ],
    "pulp_created": "2020-09-21T08:10:11.257796Z",
    "pulp_href": "/pulp/api/v3/tasks/655d2e7b-2783-4496-a851-3bcd8d5e442e/",
    "reserved_resources_record": [
        "/pulp/api/v3/remotes/rpm/rpm/d8fb524c-e73b-4afc-ba2e-05eea7f5b6a8/",
        "/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/"
    ],
    "started_at": "2020-09-21T08:10:11.507912Z",
    "state": "completed",
    "task_group": null,
    "worker": "/pulp/api/v3/workers/80ca382e-fdec-4eef-94b4-6c1c67433511/"
}

Let's use this repo on the same machine:

cat /etc/yum.repos.d/CentOS-Base.repo
[Base]
name=CentOS-$releasever - Base
enabled=1
baseurl=https://<myurl>/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial

Now if I do yum check-update it does not find any package to update. If I change baseurl= to http://mirror.centos.org/centos-8/8/BaseOS/x86_64/os/ (which is the same url I have in the remote) and I do yum check-update, then it finds 28 outdated packages. How is it possible?

Related issues

Copied to RPM Support - Story #7665: Related to #7535 - Document how to synchronise repositoriesCLOSED - DUPLICATE

Actions
Copied to RPM Support - Issue #7786: Pulp_rpm sync reports "completed" but does not update packagesCLOSED - WORKSFORMEActions
Actions
Copy link
 #1

Updated by Aant over 4 years ago

More troubleshooting: I changes the remote url to "https://mirrors.edge.kernel.org/centos/8/BaseOS/x86_64/os/", deleted publication and distribution and recreated them. Ran sync: it seems to have done some work: Task:

...
    "progress_reports": [
        {
            "code": "downloading.metadata",
            "done": 4,
            "message": "Downloading Metadata Files",
            "state": "completed",
            "suffix": null,
            "total": null
        },
        {
            "code": "downloading.artifacts",
            "done": 157,
            "message": "Downloading Artifacts",
            "state": "completed",
            "suffix": null,
            "total": null
        },
        {
            "code": "associating.content",
            "done": 164,
            "message": "Associating Content",
            "state": "completed",
            "suffix": null,
            "total": null
        },
        {
            "code": "parsing.modulemds",
            "done": 66,
            "message": "Parsed Modulemd",
            "state": "completed",
            "suffix": null,
            "total": 66
        },
        {
            "code": "parsing.modulemd_defaults",
            "done": 42,
            "message": "Parsed Modulemd-defaults",
            "state": "completed",
            "suffix": null,
            "total": 42
        },
        {
            "code": "parsing.comps",
            "done": 67,
            "message": "Parsed Comps",
            "state": "completed",
            "suffix": null,
            "total": 67
        },
        {
            "code": "parsing.packages",
            "done": 5367,
            "message": "Parsed Packages",
            "state": "completed",
            "suffix": null,
            "total": 5367
        },
        {
            "code": "downloading.metadata",
            "done": 4,
            "message": "Downloading Metadata Files",
            "state": "completed",
            "suffix": null,
            "total": null
        },
        {
            "code": "downloading.artifacts",
            "done": 74,
            "message": "Downloading Artifacts",
            "state": "completed",
            "suffix": null,
            "total": null
        },
        {
            "code": "associating.content",
            "done": 73,
            "message": "Associating Content",
            "state": "completed",
            "suffix": null,
            "total": null
        },
        {
            "code": "parsing.comps",
            "done": 42,
            "message": "Parsed Comps",
            "state": "completed",
            "suffix": null,
            "total": 42
        },
        {
            "code": "parsing.packages",
            "done": 1675,
            "message": "Parsed Packages",
            "state": "completed",
            "suffix": null,
            "total": 1675
        }
    ],
...
    "state": "completed",
...
}

But still if I use my pulp machine's url in /etc/yum.repos.d yum check-update shows no pending updates.

Actions
Copy link
 #2

Updated by ipanova@redhat.com about 4 years ago

  • Project changed from Pulp to RPM Support
Actions
Copy link
 #3

Updated by ppicka about 4 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ppicka
Actions
Copy link
 #4

Updated by ppicka about 4 years ago

  • Status changed from ASSIGNED to CLOSED - WORKSFORME
  • Assignee deleted (ppicka)

Can't reproduce on version mentioned nor later. Outputs are same but yum client doesn't show any differences. If you are able to reproduce please re-open this one or file a new issue.

Actions
Copy link
 #5

Updated by Aant about 4 years ago

Thanks for your help. I think I know now what's going on. After doing /pulp/api/v3/repositories/rpm/rpm/<ID>/sync/ remote=... I need to delete the publication and the distribution and create them again so that I can find the new packages like curl http://.../pulp/content/repo/Packages/x/.

But is this the expected behaviour?

Actions
Copy link
 #6

Updated by Aant about 4 years ago

  • Copied to Story #7665: Related to #7535 - Document how to synchronise repositories added
Actions
Copy link
 #7

Updated by Aant about 4 years ago

Hello,

the issue persists. Workflow:

  • set up repo, remote, syncronize them, create publication, distribution
  • sync, delete and recreate publication, distribution
  • set baseurl to my Pulp 3 server in /etc/yum.repos.d/whatever.repo
  • dnf clean all, dnf check updates > nothing found
  • set baseurl in /etc/yum.repos.d/whatever.repo to the external URL Pulp synchronizes from
  • dnf clean all, dnf check updates > several updates found

This is a very severe situation, worse than not being able to synchronize as it may make me believe that my systems are up to date.

Actions
Copy link
 #8

Updated by Aant about 4 years ago

  • Copied to Issue #7786: Pulp_rpm sync reports "completed" but does not update packages added
Actions
Send by e-mail Copy link

Also available in: Atom PDF