Project

Profile

Help

Issue #8770

closed

Package validation fails if package paragraphs contain keys without values

Added by ciokan almost 3 years ago. Updated over 2 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version - Debian:
Platform Release:
Target Release - Debian:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

pulp_worker_1            | pulp [480b66bac84f441e969095f0f132c3a1]: rq.worker:ERROR: Traceback (most recent call last):
pulp_worker_1            |   File "/usr/local/lib/python3.9/site-packages/rq/worker.py", line 1013, in perform_job
pulp_worker_1            |     rv = job.perform()
pulp_worker_1            |   File "/usr/local/lib/python3.9/site-packages/rq/job.py", line 709, in perform
pulp_worker_1            |     self._result = self._execute()
pulp_worker_1            |   File "/usr/local/lib/python3.9/site-packages/rq/job.py", line 732, in _execute
pulp_worker_1            |     result = self.func(*self.args, **self.kwargs)
pulp_worker_1            |   File "/usr/local/lib/python3.9/site-packages/pulpcore/app/tasks/base.py", line 34, in general_create
pulp_worker_1            |     serializer.is_valid(raise_exception=True)
pulp_worker_1            |   File "/usr/local/lib/python3.9/site-packages/rest_framework/serializers.py", line 228, in is_valid
pulp_worker_1            |     raise ValidationError(self.errors)
pulp_worker_1            | rest_framework.exceptions.ValidationError: {'section': [ErrorDetail(string='This field may not be blank.', code='blank')], 'priority': [ErrorDetail(string='This field may not be blank.', code='blank')]}
pulp_worker_1            |

I used the DRF form to upload content (/pulp/api/v3/content/deb/packages/). I also tried adding content by linking an artifact instead but the error is the same.


Related issues

Related to Debian Support - Test #7279: Add a test that uploads a "minimal" Debian package.CLOSED - DUPLICATEActions
Has duplicate Debian Support - Issue #9658: PackageIndex with empty Fields fails to syncCLOSED - DUPLICATEActions
Actions #1

Updated by ciokan almost 3 years ago

pulp-deb==2.12.0
pulp-rpm==3.11.0
pulpcore==3.12.2
Actions #2

Updated by quba42 almost 3 years ago

It looks like the package you tried to upload does not have a Section nor a Priority field in its metadata, but pulp_deb considers these to be required fields.

Can you tell me what package it was you were trying to upload?

I will have a more detailed look at this problem some time next week.

Actions #3

Updated by quba42 almost 3 years ago

  • Related to Test #7279: Add a test that uploads a "minimal" Debian package. added
Actions #4

Updated by quba42 almost 3 years ago

  • Sprint/Milestone set to Katello
Actions #5

Updated by quba42 almost 3 years ago

Neither section nor priority is a required field in the pulp_deb package model or serializer. This error message does not make much sense to me...

Actions #6

Updated by quba42 almost 3 years ago

Looks like "blank" here refers to a Null value, so this may be related to empty string vs. Null value.

See: https://www.django-rest-framework.org/api-guide/fields/#string-fields

Actions #7

Updated by rgp over 2 years ago

I've encountered the same issue trying to sync: deb [arch=amd64] https://apt.releases.hashicorp.com bionic main

foreman-2.3.5-1.el7.noarch
katello-3.18.5-1.el7.noarch
python3-pulp-deb-2.9.2-1.el7.noarch
python3-pulp-rpm-3.10.0-1.el7.noarch
python3-pulpcore-3.7.6-1.el7.noarch
pulp_tasks:
- pulp_href: "/pulp/api/v3/tasks/4d674eb1-5bdb-4b94-addb-0f480cc303f0/"
  pulp_created: '2021-10-22T12:07:06.812+00:00'
  state: failed
  name: pulp_deb.app.tasks.synchronizing.synchronize
  started_at: '2021-10-22T12:07:06.928+00:00'
  finished_at: '2021-10-22T12:07:07.750+00:00'
  error:
    traceback: |2
        File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
          rv = job.perform()
        File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
          self._result = self._execute()
        File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
          return self.func(*self.args, **self.kwargs)
        File "/usr/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py", line 106, in synchronize
          DebDeclarativeVersion(first_stage, repository, mirror=mirror).create()
        File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 148, in create
          loop.run_until_complete(pipeline)
        File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
          return future.result()
        File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
          await asyncio.gather(*futures)
        File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
          await self.run()
        File "/usr/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py", line 390, in run
          *[self._handle_distribution(distribution) for distribution in self.distributions]
        File "/usr/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py", line 450, in _handle_distribution
          for component in _filter_split(release_file.components, self.components)
        File "/usr/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py", line 497, in _handle_component
          await asyncio.gather(*pending_tasks)
        File "/usr/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py", line 554, in _handle_package_index
          serializer.is_valid(raise_exception=True)
        File "/usr/lib/python3.6/site-packages/rest_framework/serializers.py", line 242, in is_valid
          raise ValidationError(self.errors)
    description: "{'section': [ErrorDetail(string='This field may not be blank.',
      code='blank')], 'priority': [ErrorDetail(string='This field may not be blank.',
      code='blank')]}"
  worker: "/pulp/api/v3/workers/d28102d7-b296-4fd6-ad48-dad07c0dc94a/"
Actions #8

Updated by quba42 over 2 years ago

I managed to retrieve a package index paragraph from the Hashicorp repo that pulp_deb chokes on:

Package: consul-terraform-sync
Version: 0.4.0
Section:
Priority:
Architecture: amd64
Maintainer: HashiCorp
Installed-Size: 34483
Depends: openssl
Homepage: https://github.com/hashicorp/consul-terraform-sync
Description: Consul Terraform Sync is a service-oriented tool for managing network infrastructure near real-time.
Filename: pool/amd64/main/consul-terraform-sync_0.4.0_amd64.deb
SHA1: 4d5a30b17aab337cd9775461627e9ba7080e9d5d
SHA256: 7aa0613c93c030981b873233c0f284aff33ecef0a19e03ebef9e3b90e2f68dab
Size: 15966854

While Section and Priority are not required fields in pulp_deb, including the key without a value as above causes the error during validation.

Actions #9

Updated by pulpbot over 2 years ago

  • Status changed from NEW to POST
Actions #10

Updated by quba42 over 2 years ago

  • Subject changed from Error when adding content to Package validation fails if package paragraphs contain keys without values

Added by quba42 over 2 years ago

Revision 67cb51a4 | View on GitHub

Drop package paragraph fields without values

closes #8770 https://pulp.plan.io/issues/8770

Actions #11

Updated by quba42 over 2 years ago

  • Sprint/Milestone changed from Katello to 2.17.0
Actions #12

Updated by quba42 over 2 years ago

  • Status changed from POST to MODIFIED
Actions #13

Updated by wilful over 2 years ago

Have same issue when trying to sync deb https://repos.influxdata.com/debian bullseye stable

# pulp status
{
  "versions": [
    {
      "component": "core",
      "version": "3.16.0"
    },
    {
      "component": "rpm",
      "version": "3.16.1"
    },
    {
      "component": "python",
      "version": "3.5.2"
    },
    {
      "component": "deb",
      "version": "2.16.0"
    },
    {
      "component": "container",
      "version": "2.9.0"
    }
  ],
Actions #14

Updated by quba42 over 2 years ago

  • Has duplicate Issue #9658: PackageIndex with empty Fields fails to sync added
Actions #15

Updated by pulpbot over 2 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF