Project

Profile

Help

Issue #9398

closed

No optimized syncs after a change to repo's remote?

Added by sskracic@redhat.com about 3 years ago. Updated about 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Master
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 107
Quarter:
Q4-2021

Description

My environment:

pulpcore - 3.14.5 pulp-rpm - 3.14.2 RHEL 8

I am fighting a weird performance issue. After a change to a remote that is attached to some repository (for example when a client certificate is changed), all further sync attempts for that repository involves reloading of the repository metadata and parsing it, even when no new content is available and no new repository version is created afterwards. This repeats for any subsequent sync attempt of that repo, and is a significant resource hog (eg. RHEL7 repos carry 30K+ packages). A sample task with progress reports looks like this:

      {
            "pulp_href": "/pulp/api/v3/tasks/a9ec4ba1-a804-483d-b9ae-02874a8f19ad/",
            "pulp_created": "2021-09-15T08:43:25.974820Z",
            "state": "running",
            "name": "pulp_rpm.app.tasks.synchronizing.synchronize",
            "logging_cid": "4bd0de408bce49f2932790a4c6075e5e",
            "started_at": "2021-09-15T09:03:54.732570Z",
            "finished_at": null,
            "error": null,
            "worker": "/pulp/api/v3/workers/0c1e8308-37a8-43e3-8a9b-fb91ebdc18b6/",
            "parent_task": null,
            "child_tasks": [],
            "task_group": null,
            "progress_reports": [
                {
                    "message": "Downloading Metadata Files",
                    "code": "sync.downloading.metadata",
                    "state": "completed",
                    "total": null,
                    "done": 11,
                    "suffix": null
                },
                {
                    "message": "Downloading Artifacts",
                    "code": "sync.downloading.artifacts",
                    "state": "running",
                    "total": null,
                    "done": 0,
                    "suffix": null
                },
                {
                    "message": "Associating Content",
                    "code": "associating.content",
                    "state": "running",
                    "total": null,
                    "done": 0,
                    "suffix": null
                },
                {
                    "message": "Parsed Modulemd",
                    "code": "sync.parsing.modulemds",
                    "state": "completed",
                    "total": 374,
                    "done": 374,
                    "suffix": null
                },
                {
                    "message": "Parsed Modulemd-defaults",
                    "code": "sync.parsing.modulemd_defaults",
                    "state": "completed",
                    "total": 45,
                    "done": 45,
                    "suffix": null
                },
                {
                    "message": "Parsed Packages",
                    "code": "sync.parsing.packages",
                    "state": "completed",
                    "total": 19122,
                    "done": 19122,
                    "suffix": null
                },
                {
                    "message": "Parsed Advisories",
                    "code": "sync.parsing.advisories",
                    "state": "completed",
                    "total": 1566,
                    "done": 1566,
                    "suffix": null
                },
                {
                    "message": "Parsed Comps",
                    "code": "sync.parsing.comps",
                    "state": "completed",
                    "total": 67,
                    "done": 67,
                    "suffix": null
                }
            ],
            "created_resources": [
                null
            ],
            "reserved_resources_record": [
                "/pulp/api/v3/remotes/rpm/rpm/9b11994b-f7ea-4df3-a7ed-094f4b183a6c/",
                "/pulp/api/v3/repositories/rpm/rpm/c610ead3-8517-47c4-9ed8-d9dc9cc731ee/"
            ]
        },


Also available in: Atom PDF