Project

Profile

Help

Issue #6787

closed

Pulp 3 - pulp_deb 'structured mode' publication creation throws an error

Added by swisscom almost 4 years ago. Updated about 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version - Debian:
master
Platform Release:
Target Release - Debian:
OS:
CentOS 7
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Dear support team, When creating a publication of a deb/apt repository in structured mode, I get the following error : File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 886, in perform_job\n rv = job.perform()\n File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 664, in perform\n self._result = self._execute()\n File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 670, in _execute\n return self.func(*self.args, **self.kwargs)\n File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_deb/app/tasks/publishing.py", line 137, in publish\n prc.package\n File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_deb/app/tasks/publishing.py", line 174, in add_package\n package_serializer.to822(self.name).dump(self.package_index_files[package.architecture][0])\n

Commands used to create the repository, remote and publication : http -a admin:password post :24817/pulp/api/v3/repositories/deb/apt/ name='deb-test3' http -a admin:password post :24817/pulp/api/v3/remotes/deb/apt/ name='deb-test3' url='http://ftp.ch.debian.org/debian/' policy='on_demand' distributions="buster jessie" architectures="amd64" components='main contrib' http -a admin:password post :24817/pulp/api/v3/repositories/deb/apt/52bdc8ed-fc59-4825-9c75-1240d00e8a30/sync/ remote=/pulp/api/v3/remotes/deb/apt/8f040ed3-8b25-4dcf-ba7f-be10cc55a813/ http -a admin:password post :24817/pulp/api/v3/publications/deb/apt/ repository=/pulp/api/v3/repositories/deb/apt/52bdc8ed-fc59-4825-9c75-1240d00e8a30/ structured=true

Pulp components version : (pulp) -bash-4.2$ pip list | grep pulp pulp-deb 2.3.0b1 pulp-rpm 3.3.1 pulpcore 3.3.1

pulpcore + pulp-rpm have been installed using the Ansible installer. pulp-deb has been installed using this procedure :

su - pulp

$ export PULP_SETTINGS=/etc/pulp/settings.py $ source /usr/local/lib/pulp/bin/activate $ /usr/local/lib/pulp/bin/pip install pulp-deb $ pulpcore-manager migrate --noinput

Full /var/lofg/messages attached to this issue

Thanks in advance for your help


Files

messages (1.38 MB) messages /var/log/messages swisscom, 05/21/2020 12:02 PM

Related issues

Related to Debian Support - Task #6991: architecture=all type packages should be handled consistently using current best practicesCLOSED - CURRENTRELEASE

Actions
Actions #1

Updated by quba42 almost 4 years ago

I went ahead and reformatted your description a bit:

Dear support team, When creating a publication of a deb/apt repository in structured mode, I get the following error :

  File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 886, in perform_job
  rv = job.perform
  File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 664, in perform
  self._result = self._execute()
  File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 670, in _execute
  return self.func(*self.args, **self.kwargs)
  File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_deb/app/tasks/publishing.py", line 137, in publish
  prc.package
  File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_deb/app/tasks/publishing.py", line 174, in add_package
  package_serializer.to822(self.name).dump(self.package_index_files[package.architecture][0])
  KeyError: 'all'

Commands used to create the repository, remote and publication :

http post :24817/pulp/api/v3/repositories/deb/apt/ name='deb-test3'
http post :24817/pulp/api/v3/remotes/deb/apt/ name='deb-test3' url='http://ftp.ch.debian.org/debian/' policy='on_demand' distributions="buster jessie" architectures="amd64" components='main contrib'
http post :24817/pulp/api/v3/repositories/deb/apt/52bdc8ed-fc59-4825-9c75-1240d00e8a30/sync/ remote=/pulp/api/v3/remotes/deb/apt/8f040ed3-8b25-4dcf-ba7f-be10cc55a813/
http post :24817/pulp/api/v3/publications/deb/apt/ repository=/pulp/api/v3/repositories/deb/apt/52bdc8ed-fc59-4825-9c75-1240d00e8a30/ structured=true

Pulp component versions (obtained via 'pip list | grep pulp'):

  • pulp-deb 2.3.0b1
  • pulp-rpm 3.3.1
  • pulpcore 3.3.1

pulpcore + pulp-rpm have been installed using the Ansible installer. pulp-deb has been installed using this procedure:

su - pulp
export PULP_SETTINGS=/etc/pulp/settings.py
source /usr/local/lib/pulp/bin/activate
/usr/local/lib/pulp/bin/pip install pulp-deb
pulpcore-manager migrate --noinput
Actions #2

Updated by quba42 almost 4 years ago

Looks to me like the structured publisher is trying to place "architecture=all" type packages into their own non existent Packages file.

This is clearly a bug that needs to be fixed.

You could try using "simple=true" instead of "structured=true" until this is fixed.

Actions #3

Updated by quba42 almost 4 years ago

The following PR fixes this issue: https://pulp.plan.io/issues/6787

Additionally we should also update the test fixtures to cover this case.

Actions #4

Updated by pulpbot almost 4 years ago

  • Status changed from NEW to POST
Actions #5

Updated by quba42 almost 4 years ago

  • Assignee set to quba42

Added by quba42 almost 4 years ago

Revision 52419a26 | View on GitHub

Fix structured publishing

fixes #6787 https://pulp.plan.io/issues/6787

Ensures packages with architecture 'all' are added to all available package indecies, rather than a non existent 'all' package index.

Actions #6

Updated by quba42 almost 4 years ago

  • Status changed from POST to MODIFIED
Actions #7

Updated by quba42 almost 4 years ago

  • Related to Task #6991: architecture=all type packages should be handled consistently using current best practices added
Actions #8

Updated by quba42 about 3 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF