Project

Profile

Help

Issue #8671

Sync with Main Debian Repo fails on a translation file

Added by tp-hlawatsch 7 months ago. Updated 6 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Urgent
Assignee:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version - Debian:
Platform Release:
Target Release - Debian:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Hello everybody,

we plan to use pulp as our main tool for repos . We testing at the moment Debian mirroring Our pulp testinstance is runing with the latest offical docker image

This is the configured remote repo

{ "name": "debian", "url": "http://ftp.de.debian.org/debian/", "ca_cert": null, "client_cert": null, "client_key": null, "tls_validation": false, "proxy_url": null, "proxy_username": null, "proxy_password": null, "username": null, "password": null, "pulp_labels": {}, "download_concurrency": "10", "policy": "immediate", "total_timeout": null, "connect_timeout": null, "sock_connect_timeout": null, "sock_read_timeout": null, "rate_limit": null, "distributions": "buster", "components": "main", "architectures": "amd64", "sync_sources": false, "sync_udebs": false, "sync_installer": false, "gpgkey": null, "ignore_missing_package_indices": false }

the sync run in to the following error

{ "pulp_href": "/pulp/api/v3/tasks/a7bd851b-f001-4f5f-89eb-0e0de529a556/", "pulp_created": "2021-04-30T07:53:11.537345Z", "state": "failed", "name": "pulp_deb.app.tasks.synchronizing.synchronize", "logging_cid": "00de2125667f4c7e8b8e01554c401eeb", "started_at": "2021-04-30T07:53:11.648803Z", "finished_at": "2021-04-30T07:53:19.135647Z", "error": { "traceback": " File "/usr/local/lib/python3.6/site-packages/rq/worker.py", line 1008, in perform_job\n rv = job.perform()\n File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 706, in perform\n self._result = self._execute()\n File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 729, in _execute\n result = self.func(*self.args, **self.kwargs)\n File "/usr/local/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py", line 122, in synchronize\n DebDeclarativeVersion(first_stage, repository, mirror=mirror).create()\n File "/usr/local/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 149, in create\n loop.run_until_complete(pipeline)\n File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete\n return future.result()\n File "/usr/local/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline\n await asyncio.gather(*futures)\n File "/usr/local/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in call\n await self.run()\n File "/usr/local/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 266, in run\n RemoteArtifact.objects.bulk_get_or_create(self._needed_remote_artifacts(batch))\n File "/usr/local/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 324, in _needed_remote_artifacts\n msg.format(rp=content_artifact.relative_path, c=d_content.content)\n", "description": "No declared artifact with relative path "dists/buster/main/i18n/Translation-ml.bz2" for content """ },

We do not see a misconfiguration on our side so I raised this issue here

Regards Stephan Hlawatsch


Related issues

Related to Debian Support - Story #8734: Synchronize translation filesNEW

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Has duplicate Debian Support - Issue #8756: Repeated syncs with mirror=false lead to duplicate metadata files within repo versions (causing the sync to fail)MODIFIED<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Copied to Debian Support - Backport #8735: Backport 8671 to 2.12CLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Copied to Debian Support - Backport #8736: Backport 8671 to 2.11CLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Copied to Debian Support - Backport #8737: Backport 8671 to 2.10CLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Copied to Debian Support - Backport #8738: Backport 8671 to 2.9CLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision 98f25d84 View on GitHub
Added by quba42 7 months ago

Completely disable synchronization of translation files

Closes #8671 https://pulp.plan.io/issues/8671

This feature has never fully worked, the implementation is incomplete and causes various sync failures. Re-adding this feature should be done under the following story: https://pulp.plan.io/issues/8734

History

#1 Updated by quba42 7 months ago

Note: At the time of writing, the "latest official docker image" should include the following versions:

pulpcore 3.12.1
pulp_deb 2.11.1

#2 Updated by quba42 7 months ago

  • Priority changed from Normal to Urgent

#3 Updated by quba42 7 months ago

It appears fixing the relative path for translation files has unearthed some other bug with translation files, that is breaking synchronization for lots of repos.

Why I missed this when I originally fixed the relative path problem, I don't know (perhaps it depends on the translation files in the repo being synced) but this needs to be fixed ASAP.

I am bumping the priority to Urgent.

#4 Updated by quba42 7 months ago

  • Triaged changed from No to Yes

Also affects version 2.12.0

#5 Updated by quba42 7 months ago

  • Related to Story #8734: Synchronize translation files added

#6 Updated by pulpbot 7 months ago

  • Status changed from NEW to POST

#7 Updated by quba42 7 months ago

  • Sprint/Milestone set to 2.13.0

#8 Updated by quba42 7 months ago

  • Status changed from POST to MODIFIED

#9 Updated by quba42 6 months ago

  • Has duplicate Issue #8756: Repeated syncs with mirror=false lead to duplicate metadata files within repo versions (causing the sync to fail) added

#10 Updated by quba42 6 months ago

#11 Updated by quba42 6 months ago

#12 Updated by quba42 6 months ago

#13 Updated by quba42 6 months ago

#14 Updated by pulpbot 6 months ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF