Project

Profile

Help

Issue #8671

closed

Sync with Main Debian Repo fails on a translation file

Added by tp-hlawatsch almost 3 years ago. Updated almost 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Urgent
Assignee:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version - Debian:
Platform Release:
Target Release - Debian:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Hello everybody,

we plan to use pulp as our main tool for repos . We testing at the moment Debian mirroring Our pulp testinstance is runing with the latest offical docker image

This is the configured remote repo

{ "name": "debian", "url": "http://ftp.de.debian.org/debian/", "ca_cert": null, "client_cert": null, "client_key": null, "tls_validation": false, "proxy_url": null, "proxy_username": null, "proxy_password": null, "username": null, "password": null, "pulp_labels": {}, "download_concurrency": "10", "policy": "immediate", "total_timeout": null, "connect_timeout": null, "sock_connect_timeout": null, "sock_read_timeout": null, "rate_limit": null, "distributions": "buster", "components": "main", "architectures": "amd64", "sync_sources": false, "sync_udebs": false, "sync_installer": false, "gpgkey": null, "ignore_missing_package_indices": false }

the sync run in to the following error

{ "pulp_href": "/pulp/api/v3/tasks/a7bd851b-f001-4f5f-89eb-0e0de529a556/", "pulp_created": "2021-04-30T07:53:11.537345Z", "state": "failed", "name": "pulp_deb.app.tasks.synchronizing.synchronize", "logging_cid": "00de2125667f4c7e8b8e01554c401eeb", "started_at": "2021-04-30T07:53:11.648803Z", "finished_at": "2021-04-30T07:53:19.135647Z", "error": { "traceback": " File "/usr/local/lib/python3.6/site-packages/rq/worker.py", line 1008, in perform_job\n rv = job.perform()\n File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 706, in perform\n self._result = self._execute()\n File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 729, in _execute\n result = self.func(*self.args, **self.kwargs)\n File "/usr/local/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py", line 122, in synchronize\n DebDeclarativeVersion(first_stage, repository, mirror=mirror).create()\n File "/usr/local/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 149, in create\n loop.run_until_complete(pipeline)\n File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete\n return future.result()\n File "/usr/local/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline\n await asyncio.gather(*futures)\n File "/usr/local/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in call\n await self.run()\n File "/usr/local/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 266, in run\n RemoteArtifact.objects.bulk_get_or_create(self._needed_remote_artifacts(batch))\n File "/usr/local/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 324, in _needed_remote_artifacts\n msg.format(rp=content_artifact.relative_path, c=d_content.content)\n", "description": "No declared artifact with relative path "dists/buster/main/i18n/Translation-ml.bz2" for content """ },

We do not see a misconfiguration on our side so I raised this issue here

Regards Stephan Hlawatsch


Related issues

Related to Debian Support - Story #8734: Synchronize translation filesCLOSED - DUPLICATE

Actions
Has duplicate Debian Support - Issue #8756: Repeated syncs with mirror=false lead to duplicate metadata files within repo versions (causing the sync to fail)CLOSED - CURRENTRELEASEActions
Copied to Debian Support - Backport #8735: Backport 8671 to 2.12CLOSED - CURRENTRELEASE

Actions
Copied to Debian Support - Backport #8736: Backport 8671 to 2.11CLOSED - CURRENTRELEASE

Actions
Copied to Debian Support - Backport #8737: Backport 8671 to 2.10CLOSED - CURRENTRELEASE

Actions
Copied to Debian Support - Backport #8738: Backport 8671 to 2.9CLOSED - CURRENTRELEASE

Actions
Actions #1

Updated by quba42 almost 3 years ago

Note: At the time of writing, the "latest official docker image" should include the following versions:

pulpcore 3.12.1
pulp_deb 2.11.1
Actions #2

Updated by quba42 almost 3 years ago

  • Priority changed from Normal to Urgent
Actions #3

Updated by quba42 almost 3 years ago

It appears fixing the relative path for translation files has unearthed some other bug with translation files, that is breaking synchronization for lots of repos.

Why I missed this when I originally fixed the relative path problem, I don't know (perhaps it depends on the translation files in the repo being synced) but this needs to be fixed ASAP.

I am bumping the priority to Urgent.

Actions #4

Updated by quba42 almost 3 years ago

  • Triaged changed from No to Yes

Also affects version 2.12.0

Actions #5

Updated by quba42 almost 3 years ago

  • Related to Story #8734: Synchronize translation files added

Added by quba42 almost 3 years ago

Revision 98f25d84 | View on GitHub

Completely disable synchronization of translation files

Closes #8671 https://pulp.plan.io/issues/8671

This feature has never fully worked, the implementation is incomplete and causes various sync failures. Re-adding this feature should be done under the following story: https://pulp.plan.io/issues/8734

Actions #6

Updated by pulpbot almost 3 years ago

  • Status changed from NEW to POST
Actions #7

Updated by quba42 almost 3 years ago

  • Sprint/Milestone set to 2.13.0
Actions #8

Updated by quba42 almost 3 years ago

  • Status changed from POST to MODIFIED
Actions #9

Updated by quba42 almost 3 years ago

  • Has duplicate Issue #8756: Repeated syncs with mirror=false lead to duplicate metadata files within repo versions (causing the sync to fail) added
Actions #10

Updated by quba42 almost 3 years ago

Actions #11

Updated by quba42 almost 3 years ago

Actions #12

Updated by quba42 almost 3 years ago

Actions #13

Updated by quba42 almost 3 years ago

Actions #14

Updated by pulpbot almost 3 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF