Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-12-14T18:48:55ZPulp
Planio Pulp - Issue #9637 (CLOSED - DUPLICATE): has_remote_param_obj_perms assumes that pulpcore seriali...https://pulp.plan.io/issues/96372021-12-14T18:48:55Zttereshcttereshc@redhat.com
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulpcore/2088":<a href="https://github.com/pulp/pulpcore/issues/2088" class="external">https://github.com/pulp/pulpcore/issues/2088</a></p>
<hr>
<p><a href="https://github.com/pulp/pulpcore/blob/e785f56d72469a7f602db6d2d347a2912e7cbf2e/pulpcore/app/global_access_conditions.py#L146" class="external">RepositorySyncURLSerializer is always used</a> in <code>has_remote_*_perms</code>.</p>
<p>It works only for plugins which do not have this serializer subclassed.</p>
<p>The error one may see is:</p>
<pre><code>E pulpcore.client.pulp_container.exceptions.ApiException: (400)
E Reason: Bad Request
E HTTP response headers: HTTPHeaderDict({'Server': 'nginx/1.14.1', 'Date': 'Mon, 13 Dec 2021 20:29:20 GMT', 'Content-Type': 'application/json', 'Content-Length': '36', 'Connection': 'keep-alive', 'Vary': 'Accept, Cookie', 'Allow': 'POST, OPTIONS', 'X-Frame-Options': 'DENY', 'X-Content-Type-Options': 'nosniff', 'Referrer-Policy': 'same-origin', 'Correlation-ID': 'b1b5beab65cd4d7d8923b5c2304b7338', 'Access-Control-Expose-Headers': 'Correlation-ID'})
E HTTP response body: {"your_plugin_specific_field_for_sync":["Unexpected field"]}
</code></pre>
<p>Everything works fine with an admin user, this error is seen only with non-admin users.</p>
<p>Pulp needs to use a serializer configured for the view[set].</p> RPM Support - Issue #9636 (CLOSED - DUPLICATE): FileNotFoundError: [Errno 2] No such file or dire...https://pulp.plan.io/issues/96362021-12-13T17:05:32Zttereshcttereshc@redhat.com
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2321":<a href="https://github.com/pulp/pulp_rpm/issues/2321" class="external">https://github.com/pulp/pulp_rpm/issues/2321</a></p>
<hr>
<p>Pulp expects custom metadata files to have a specific format for filenames: -.</p>
<p>During pulp-2to3-migration, users get <code>FileNotFoundError: [Errno 2] No such file or directory: ' '</code> because the filename is only a checksum :/, which we remove to determine the filename, so the path we detect becomes an empty string.</p>
<pre><code>Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 478, in migrate_repo_distributor
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: pulp2dist, repo_version)
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repository.py", line 91, in migrate_to_pulp3
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: publish(repo_version.pk, checksum_types=checksum_types, sqlite_metadata=sqlite)
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 344, in publish
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: publication_data.populate()
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 253, in populate
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: self.repomdrecords = self.prepare_metadata_files(main_content)
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 99, in prepare_metadata_files
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: with open(path, "wb") as new_file:
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: FileNotFoundError: [Errno 2] No such file or directory: ''
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: pulp: rq.worker:INFO: 27738@1002a1103081001.xxx.com: c0d58c5a-9ff1-4d40-bbb3-9c24fdf0fdb4
Nov 22 21:31:34 1002a1103081001 pulpcore-resource-manager: pulp: rq.worker:INFO: resource-manager: 43f43c1a-d09e-46bd-99fe-73b6e2ee397c
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-3: pulp: rq.worker:INFO: 27740@1002a1103081001.xxx.com: Job OK (b25f3fa2-401c-425c-92eb-b49a61415617)
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-2: pulp: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=XXX-Red_Hat_Ansible_Engine_2_7_RPMs_for_Red_Hat_Enterprise_Linux_7_Server_x86_64, version=1
</code></pre>
<p>Some repositories like <a href="https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/rhv-mgmt-agent/4/os/repodata/repomd.xml" class="external">https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/rhv-mgmt-agent/4/os/repodata/repomd.xml</a> have filenames in the form of UUID.</p>
<p>Observed with pulp2 to pulp3 migration only so far, need to test sync.</p> RPM Support - Issue #9583 (CLOSED - DUPLICATE): Distribution tree uniqueness constraint is not en...https://pulp.plan.io/issues/95832021-11-23T17:43:48Zttereshcttereshc@redhat.com
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2305":<a href="https://github.com/pulp/pulp_rpm/issues/2305" class="external">https://github.com/pulp/pulp_rpm/issues/2305</a></p>
<hr>
<p>Currently <a href="https://github.com/pulp/pulp_rpm/blob/master/pulp_rpm/app/models/distribution.py#L133" class="external">Pulp requires</a> the combination of the following fields to be unique:</p>
<pre><code> unique_together = (
"header_version",
"release_name",
"release_short",
"release_version",
"arch",
"build_timestamp",
)
</code></pre>
<p>In some cases, it doesn't seem enough.
For some reason multiple repositories might have all those fields exactly the same and they differ in variants definition only. Such are not proper distribution trees, the majority do not have any images associated.</p>
<p>Examples brought by always helpful @gdve from <a href="https://pulp.plan.io/issues/8566#note-33" class="external">https://pulp.plan.io/issues/8566#note-33</a>:</p>
<pre><code>CentOS/8-stream/AppStream/x86_64/os/.treeinfo:build_timestamp = 1625615144
CentOS/8-stream/BaseOS/x86_64/os/.treeinfo:build_timestamp = 1625615155
CentOS/8-stream/HighAvailability/x86_64/os/.treeinfo:build_timestamp = 1625026406
CentOS/8-stream/PowerTools/x86_64/os/.treeinfo:build_timestamp = 1625026406
CentOS/8-stream/RT/x86_64/os/.treeinfo:build_timestamp = 1625026406
</code></pre>
<pre><code>AlmaLinux/8.4/AppStream/x86_64/kickstart/.treeinfo:build_timestamp = 1622014553
AlmaLinux/8.4/AppStream/x86_64/os/.treeinfo:build_timestamp = 1622014553
AlmaLinux/8.4/BaseOS/x86_64/kickstart/.treeinfo:build_timestamp = 1622014553
AlmaLinux/8.4/BaseOS/x86_64/os/.treeinfo:build_timestamp = 1622014553
AlmaLinux/8.4/HighAvailability/x86_64/kickstart/.treeinfo:build_timestamp = 1622014558
AlmaLinux/8.4/HighAvailability/x86_64/os/.treeinfo:build_timestamp = 1622014558
AlmaLinux/8.4/PowerTools/x86_64/kickstart/.treeinfo:build_timestamp = 1622014558
AlmaLinux/8.4/PowerTools/x86_64/os/.treeinfo:build_timestamp = 1622014558
AlmaLinux/8.4/extras/x86_64/kickstart/.treeinfo:build_timestamp = 1622014558
AlmaLinux/8.4/extras/x86_64/os/.treeinfo:build_timestamp = 1622014558
</code></pre>
<p>Pulp 2to3 migration fails with <code>No declared artifact with relative path ".treeinfo" for content "<DistributionTree: pk=64f44866-0207-4005-9c06-0f45e52cbdd1>"</code>.
I would expect sync to behave the similarly, needs testing though.</p> RPM Support - Issue #9551 (CLOSED - DUPLICATE): Error passing upload RPMhttps://pulp.plan.io/issues/95512021-11-02T19:35:27Zwibbit
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2317":<a href="https://github.com/pulp/pulp_rpm/issues/2317" class="external">https://github.com/pulp/pulp_rpm/issues/2317</a></p>
<hr>
<p>After uploading an RPM (rpm -qp --all .rpm works fine on the source RPM), I'm greeted with the following error.</p>
<pre><code>{'child_tasks': [],
'created_resources': [],
'error': {'description': 'RPM file cannot be parsed for metadata.',
'traceback': ' File '
'"/opt/bats/lib/python3.8/site-packages/pulpcore/tasking/pulpcore_worker.py", '
'line 272, in _perform_task\n'
' result = func(*args, **kwargs)\n'
' File '
'"/opt/bats/lib/python3.8/site-packages/pulpcore/app/tasks/base.py", '
'line 34, in general_create\n'
' serializer.is_valid(raise_exception=True)\n'
' File '
'"/opt/bats/lib/python3.8/site-packages/rest_framework/serializers.py", '
'line 220, in is_valid\n'
' self._validated_data = '
'self.run_validation(self.initial_data)\n'
' File '
'"/opt/bats/lib/python3.8/site-packages/rest_framework/serializers.py", '
'line 422, in run_validation\n'
' value = self.validate(value)\n'
' File '
'"/opt/bats/lib/python3.8/site-packages/pulpcore/plugin/serializers/content.py", '
'line 104, in validate\n'
' data = self.deferred_validate(data)\n'
' File '
'"/opt/bats/lib/python3.8/site-packages/pulp_rpm/app/serializers/package.py", '
'line 240, in deferred_validate\n'
' raise NotAcceptable(detail="RPM file cannot be '
'parsed for metadata.")\n'},
'finished_at': datetime.datetime(2021, 11, 2, 19, 12, 12, 382621, tzinfo=tzutc()),
'logging_cid': '3c5f6c83815947bfae0e71cb661ec91f',
'name': 'pulpcore.app.tasks.base.general_create',
'parent_task': None,
'progress_reports': [],
'pulp_created': datetime.datetime(2021, 11, 2, 19, 12, 10, 711408, tzinfo=tzutc()),
'pulp_href': '/pulp/api/v3/tasks/452c9c7e-1715-4751-9462-0f68a649feb3/',
'reserved_resources_record': ['/pulp/api/v3/artifacts/58eb9338-26f8-4f55-a0dc-ddca290fa1fb/'],
'started_at': datetime.datetime(2021, 11, 2, 19, 12, 10, 792329, tzinfo=tzutc()),
'state': 'failed',
'task_group': None,
'worker': None}
</code></pre> RPM Support - Issue #9503 (CLOSED - CURRENTRELEASE): get() returned more than one UpdateRecord --...https://pulp.plan.io/issues/95032021-10-12T12:31:40Zttereshcttereshc@redhat.com
<p>Advisory conflict resolution doesn't properly support the case when multiple advisories with the same errata_id but different in some other way are added to a repository version or somehow existed in previous repo version.</p>
<pre><code>pulp [59cc0e4d-b1d3-4e83-be25-a010e9413044]: pulpcore.tasking.pulpcore_worker:INFO: Task 1584fe74-28d1-4d63-9710-b1b4e36a6e1c failed (get() returned more than one UpdateRecord -- it returned 2!)
pulp [59cc0e4d-b1d3-4e83-be25-a010e9413044]: pulpcore.tasking.pulpcore_worker:INFO: File "/usr/lib/python3.6/site-packages/pulpcore/tasking/pulpcore_worker.py", line 317, in _perform_task
result = func(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/pulpcore/app/tasks/repository.py", line 219, in add_and_remove
new_version.add_content(models.Content.objects.filter(pk__in=add_content_units))
File "/usr/lib/python3.6/site-packages/pulpcore/app/models/repository.py", line 963, in __exit__
repository.finalize_new_version(self)
File "/usr/lib/python3.6/site-packages/pulp_rpm/app/models/repository.py", line 353, in finalize_new_version
resolve_advisories(new_version, previous_version)
File "/usr/lib/python3.6/site-packages/pulp_rpm/app/advisory.py", line 87, in resolve_advisories
previous_advisory = previous_advisories.get(id=advisory_id)
File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 412, in get
(self.model._meta.object_name, num)
pulp [59cc0e4d-b1d3-4e83-be25-a010e9413044]: pulpcore.tasking.pulpcore_worker:INFO: Task e9a64af6-047f-48d6-825d-48108e9e3c26 failed (get() returned more than one UpdateRecord -- it returned 2!)
pulp [59cc0e4d-b1d3-4e83-be25-a010e9413044]: pulpcore.tasking.pulpcore_worker:INFO: File "/usr/lib/python3.6/site-packages/pulpcore/tasking/pulpcore_worker.py", line 317, in _perform_task
result = func(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/pulpcore/app/tasks/repository.py", line 219, in add_and_remove
new_version.add_content(models.Content.objects.filter(pk__in=add_content_units))
File "/usr/lib/python3.6/site-packages/pulpcore/app/models/repository.py", line 963, in __exit__
repository.finalize_new_version(self)
File "/usr/lib/python3.6/site-packages/pulp_rpm/app/models/repository.py", line 353, in finalize_new_version
resolve_advisories(new_version, previous_version)
File "/usr/lib/python3.6/site-packages/pulp_rpm/app/advisory.py", line 88, in resolve_advisories
added_advisory = UpdateRecord.objects.get(id=advisory_id, pk__in=added_advisory_pks)
File "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 412, in get
(self.model._meta.object_name, num)
</code></pre>
<p>Currently, there is <a href="https://github.com/pulp/pulp_rpm/blob/3.15.0/pulp_rpm/app/advisory.py#L75" class="external">a check</a> for adding exactly the same advisory multiple times. It doesn't cover the case when it's different versions of the same advisory.</p>
<p>So it is possible to fail <a href="https://github.com/pulp/pulp_rpm/blob/3.15.0/pulp_rpm/app/advisory.py#L88" class="external">here</a> when two different advisories but with the same errata_id are being added.</p>
<p>To reproduce, use modify/ endpoint and pass two different advisories with the same errata_id.</p> Migration Plugin - Issue #9295 (CLOSED - CURRENTRELEASE): After migration interruption, incomplet...https://pulp.plan.io/issues/92952021-08-25T20:39:56Zttereshcttereshc@redhat.com
<p>To reproduce:</p>
<ul>
<li>interrupt migration at the moment when a publication is created but not completed (good luck)</li>
<li>re-run migration and observe that pulp2repositories endpoint refers to incomplete publications. If you try to access them via content app, you'll get 404.</li>
</ul> Migration Plugin - Issue #9242 (CLOSED - CURRENTRELEASE): Filter out incomplete publications from...https://pulp.plan.io/issues/92422021-08-17T12:08:42Zipanova@redhat.comipanova@redhat.com
<p>If the only publication a repo version has is incomplete, it would be shown in a mapping as migrated.</p>
<p>The problematic code:
<a href="https://github.com/pulp/pulp-2to3-migration/blob/master/pulp_2to3_migration/app/serializers.py#L236" class="external">https://github.com/pulp/pulp-2to3-migration/blob/master/pulp_2to3_migration/app/serializers.py#L236</a></p> Migration Plugin - Issue #8895 (CLOSED - CURRENTRELEASE): django.core.exceptions.FieldError: Unsu...https://pulp.plan.io/issues/88952021-06-14T18:28:21Zttereshcttereshc@redhat.com
<p><del>Trying to run a migration with the future pulpcore 3.14 fails in the following way</del>
pulpcore 3.14 will deprecate it and 3.15 will remove it.</p>
<p>This needs to be fixed for 3.15.</p>
<pre><code>pulp [d42e1bccd6444d1e9992fd63aadf7625]: django.request:ERROR: Internal Server Error: /pulp/api/v3/migration-plans/5172b292-c16d-45d0-80cb-4577328dbd7f/run/
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/rest_framework/viewsets.py", line 125, in view
return self.dispatch(request, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/rest_framework/views.py", line 509, in dispatch
response = self.handle_exception(exc)
File "/usr/local/lib/python3.6/site-packages/rest_framework/views.py", line 469, in handle_exception
self.raise_uncaught_exception(exc)
File "/usr/local/lib/python3.6/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
raise exc
File "/usr/local/lib/python3.6/site-packages/rest_framework/views.py", line 506, in dispatch
response = handler(request, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/pulp_2to3_migration/app/viewsets.py", line 94, in run
if is_migration_plan_running():
File "/usr/local/lib/python3.6/site-packages/pulp_2to3_migration/app/viewsets.py", line 47, in is_migration_plan_running
reserved_resources_record__resource='pulp_2to3_migration')
File "/usr/local/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 892, in filter
return self._filter_or_exclude(False, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 910, in _filter_or_exclude
clone.query.add_q(Q(*args, **kwargs))
File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/query.py", line 1290, in add_q
clause, _ = self._add_q(q_object, self.used_aliases)
File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/query.py", line 1318, in _add_q
split_subq=split_subq, simple_col=simple_col,
File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/query.py", line 1251, in build_filter
condition = self.build_lookup(lookups, col, value)
File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/query.py", line 1110, in build_lookup
lhs = self.try_transform(lhs, lookup_name)
File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/query.py", line 1151, in try_transform
"permitted%s" % (name, output_field.__name__, suggestion)
django.core.exceptions.FieldError: Unsupported lookup 'resource' for ArrayField or join on the field not permitted.
</code></pre> File Support - Issue #4042 (CLOSED - CURRENTRELEASE): Update workflow in docs after changes of fi...https://pulp.plan.io/issues/40422018-09-27T07:54:29Zvdusek
<p>Recently fields of responses were changed (e.g. <a href="https://pulp.plan.io/issues/3978" class="external">https://pulp.plan.io/issues/3978</a>, <a href="https://pulp.plan.io/issues/3979" class="external">https://pulp.plan.io/issues/3979</a>). Workflow in docs should be updated.</p> Pulp - Issue #3978 (CLOSED - CURRENTRELEASE): Operation postponed response contains too much info...https://pulp.plan.io/issues/39782018-09-05T13:51:52Zdkliban@redhat.com
<p>When a user performs a sync, repository_version creation, or a publish she is presented with a 202 response with the following schema:</p>
<pre><code>{
"_href": "/pulp/api/v3/some-task-id/",
"task_id": "some-task-id"
}
</code></pre>
<p>The `task_id` is not necessary because users are supposed to use resource HREFs as identifiers. The `_href` field is not descriptive enough on it's own. A better response would look like</p>
<pre><code>{
"task": "/pulp/api/v3/some-task-id/"
}
</code></pre> Ansible Plugin - Issue #3898 (CLOSED - CURRENTRELEASE): Filters inherit from a Django's FilterSet...https://pulp.plan.io/issues/38982018-08-02T15:13:08Zvdusek
<p>AnsibleRoleFilter and AnsibleRoleVersionFilter inherit from Django's filterset.FilterSet class not from our BaseFilterSet. I suppose this could cause problems in generating documentation.</p> Pulp - Issue #3846 (CLOSED - WONTFIX): Wrong status codehttps://pulp.plan.io/issues/38462018-07-10T09:07:34Zvdusek
<p>When I want to create a repository with a name which is already used for another repository, it returns status code 400 - Bad Request. According to this topic <a href="https://stackoverflow.com/questions/3825990/http-response-code-for-post-when-resource-already-exists" class="external">HTTP response code for POST when resource already exists</a> I would suggest use 409 - Conflict.</p> File Support - Issue #3827 (CLOSED - CURRENTRELEASE): Filters on content list lack help texthttps://pulp.plan.io/issues/38272018-07-06T17:33:51ZdaviddavisPulp - Issue #3826 (CLOSED - CURRENTRELEASE): Range filter help text should specify the format al...https://pulp.plan.io/issues/38262018-07-06T17:31:54Zdaviddavis
<p>We have some range filters like <code>last_updated__range</code> but it's not clear how to format the input to these filters.</p> Pulp - Issue #3825 (CLOSED - CURRENTRELEASE): Date filter help text should specify the format of ...https://pulp.plan.io/issues/38252018-07-06T17:30:31Zdaviddavis
<p>We have a bunch of date filters but it's not clear what formats they allow.</p>