Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2022-01-12T15:43:50ZPulp
Planio Pulp - Backport #9668 (CLOSED - CURRENTRELEASE): Backport #9665: Make the `adjust_roles` function...https://pulp.plan.io/issues/96682022-01-12T15:43:50Zdkliban@redhat.com
<p>While migrating to the Roles backend I find I need to access <code>adjust_roles</code> [1] and CI throws an error when accessing it via <code>from pulpcore.app.apps import adjust_roles</code> [2].</p>
<p>[1] <a href="https://github.com/pulp/pulpcore/blob/main/pulpcore/app/apps.py#L269" class="external">https://github.com/pulp/pulpcore/blob/main/pulpcore/app/apps.py#L269</a>
[2] <a href="https://github.com/ansible/galaxy_ng/runs/4731797454?check_suite_focus=true" class="external">https://github.com/ansible/galaxy_ng/runs/4731797454?check_suite_focus=true</a></p> Pulp - Backport #9664 (CLOSED - CURRENTRELEASE): Backport #9660 "django update broke pulpimport f...https://pulp.plan.io/issues/96642022-01-06T17:54:23Zttereshcttereshc@redhat.com
<p>Django addressed a security issue involving filepaths in a way that broke how pulpimport was using Storage:</p>
<p>In 3.14, the following failure in <code>pulp_rpm.tests.functional.api.test_pulpimport.ParallelImportTestCase testMethod=test_clean_import</code> :</p>
<pre><code class="text syntaxhl" data-language="text">E pulp_smash.pulp3.bindings.PulpTaskError: (PulpTaskError(...), "Pulp task failed (Detected path traversal attempt in '/var/lib/pulp/media/artifact/d4/89a5ea552e5ea595976e39f891fe249e95d8eb40cbd7f50a46c0126a7072ab')")
</code></pre>
<p>Against core/main, the same test hangs.</p>
<p>The problem is that core/import builds a full-path to send to Storage.save(), which used to "work" but is now Not Allowed (for perfectly good security-reasons)</p>
<p>See <a href="https://github.com/pulp/pulpcore/blob/main/pulpcore/app/tasks/importer.py#L396" class="external">https://github.com/pulp/pulpcore/blob/main/pulpcore/app/tasks/importer.py#L396</a></p> Pulp - Backport #9663 (CLOSED - CURRENTRELEASE): Backport #9660 "django update broke pulpimport f...https://pulp.plan.io/issues/96632022-01-06T17:53:12Zttereshcttereshc@redhat.com
<p>Django addressed a security issue involving filepaths in a way that broke how pulpimport was using Storage:</p>
<p>In 3.14, the following failure in <code>pulp_rpm.tests.functional.api.test_pulpimport.ParallelImportTestCase testMethod=test_clean_import</code> :</p>
<pre><code class="text syntaxhl" data-language="text">E pulp_smash.pulp3.bindings.PulpTaskError: (PulpTaskError(...), "Pulp task failed (Detected path traversal attempt in '/var/lib/pulp/media/artifact/d4/89a5ea552e5ea595976e39f891fe249e95d8eb40cbd7f50a46c0126a7072ab')")
</code></pre>
<p>Against core/main, the same test hangs.</p>
<p>The problem is that core/import builds a full-path to send to Storage.save(), which used to "work" but is now Not Allowed (for perfectly good security-reasons)</p>
<p>See <a href="https://github.com/pulp/pulpcore/blob/main/pulpcore/app/tasks/importer.py#L396" class="external">https://github.com/pulp/pulpcore/blob/main/pulpcore/app/tasks/importer.py#L396</a></p> Pulp - Backport #9662 (CLOSED - CURRENTRELEASE): Backport #9660 "django update broke pulpimport f...https://pulp.plan.io/issues/96622022-01-06T17:46:06Zttereshcttereshc@redhat.com
<p>Django addressed a security issue involving filepaths in a way that broke how pulpimport was using Storage:</p>
<p>In 3.14, the following failure in <code>pulp_rpm.tests.functional.api.test_pulpimport.ParallelImportTestCase testMethod=test_clean_import</code> :</p>
<pre><code class="text syntaxhl" data-language="text">E pulp_smash.pulp3.bindings.PulpTaskError: (PulpTaskError(...), "Pulp task failed (Detected path traversal attempt in '/var/lib/pulp/media/artifact/d4/89a5ea552e5ea595976e39f891fe249e95d8eb40cbd7f50a46c0126a7072ab')")
</code></pre>
<p>Against core/main, the same test hangs.</p>
<p>The problem is that core/import builds a full-path to send to Storage.save(), which used to "work" but is now Not Allowed (for perfectly good security-reasons)</p>
<p>See <a href="https://github.com/pulp/pulpcore/blob/main/pulpcore/app/tasks/importer.py#L396" class="external">https://github.com/pulp/pulpcore/blob/main/pulpcore/app/tasks/importer.py#L396</a></p> Pulp - Issue #9654 (CLOSED - CURRENTRELEASE): backport 9642 to 3.17.1: Migration 081 was incompat...https://pulp.plan.io/issues/96542021-12-21T15:05:31Zdkliban@redhat.comRPM Support - Backport #9650 (CLOSED - CURRENTRELEASE): Backport #9636 'FileNotFoundError: [Errno...https://pulp.plan.io/issues/96502021-12-20T16:06:20Zttereshcttereshc@redhat.com
<p>Pulp expects custom metadata files to have a specific format for filenames: -.</p>
<p>During pulp-2to3-migration, users get <code>FileNotFoundError: [Errno 2] No such file or directory: ' '</code> because the filename is only a checksum :/, which we remove to determine the filename, so the path we detect becomes an empty string.</p>
<pre><code>Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 478, in migrate_repo_distributor
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: pulp2dist, repo_version)
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repository.py", line 91, in migrate_to_pulp3
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: publish(repo_version.pk, checksum_types=checksum_types, sqlite_metadata=sqlite)
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 344, in publish
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: publication_data.populate()
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 253, in populate
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: self.repomdrecords = self.prepare_metadata_files(main_content)
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 99, in prepare_metadata_files
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: with open(path, "wb") as new_file:
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: FileNotFoundError: [Errno 2] No such file or directory: ''
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: pulp: rq.worker:INFO: 27738@1002a1103081001.xxx.com: c0d58c5a-9ff1-4d40-bbb3-9c24fdf0fdb4
Nov 22 21:31:34 1002a1103081001 pulpcore-resource-manager: pulp: rq.worker:INFO: resource-manager: 43f43c1a-d09e-46bd-99fe-73b6e2ee397c
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-3: pulp: rq.worker:INFO: 27740@1002a1103081001.xxx.com: Job OK (b25f3fa2-401c-425c-92eb-b49a61415617)
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-2: pulp: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=XXX-Red_Hat_Ansible_Engine_2_7_RPMs_for_Red_Hat_Enterprise_Linux_7_Server_x86_64, version=1
</code></pre> RPM Support - Issue #9627 (MODIFIED): publish fails on MD5-checksummed repos, on FIPShttps://pulp.plan.io/issues/96272021-12-09T18:59:35Zggainey
<p>See associated BZ for details, reproducer</p> RPM Support - Test #9622 (MODIFIED): Add a repo signed using 'sha' as alias for 'sha1'https://pulp.plan.io/issues/96222021-12-08T19:00:00Zggainey
<p>'sha' support exists in the wild, is the same as 'sha1', and has broken us several times now, Let's make it possible to write tests for it.</p> Migration Plugin - Backport #9612 (MODIFIED): Backport #8968 "'NoneType' object has no attribute ...https://pulp.plan.io/issues/96122021-12-07T18:13:47Zttereshcttereshc@redhat.com
<p>Backtrace:</p>
<pre>
"error"=>
{"traceback"=>
" File \"/usr/lib/python3.6/site-packages/rq/worker.py\", line 936, in perform_job\n" +
" rv = job.perform()\n" +
" File \"/usr/lib/python3.6/site-packages/rq/job.py\", line 684, in perform\n" +
" self._result = self._execute()\n" +
" File \"/usr/lib/python3.6/site-packages/rq/job.py\", line 690, in _execute\n" +
" return self.func(*self.args, **self.kwargs)\n" +
" File \"/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py\", line 76, in migrate_from_pulp2\n" +
" pre_migrate_all_without_content(plan)\n" +
" File \"/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py\", line 493, in pre_migrate_all_without_content\n" +
" pre_migrate_importer(repo_id, importer_types)\n" +
" File \"/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py\", line 601, in pre_migrate_importer\n" +
" importer.pulp3_remote.delete()\n",
"description"=>"'NoneType' object has no attribute 'delete'"},
"worker"=>"/pulp/api/v3/workers/fc6ba1d6-ddc6-494d-9385-2368544a09ef/",
</pre>
<p>This happened on Katello 3.18.3 which uses:</p>
<pre>
pulp-2to3-migration (0.11.1)
pulp-certguard (1.0.3)
pulp-container (2.1.2)
pulp-deb (2.7.0)
pulp-file (1.3.0)
pulp-rpm (3.10.0)
pulpcore (3.7.6)
</pre> Container Support - Backport #9600 (CLOSED - CURRENTRELEASE): Backport 9586 to pulp_container 2.5https://pulp.plan.io/issues/96002021-12-02T20:22:29Znewswangerd
<p>We'd like to request that 9586 be backported to pulp_container 2.5 so that it can be used in galaxy_ng 4.3</p> Pulp - Issue #9595 (CLOSED - CURRENTRELEASE): HEAD requests on the artefacts from S3 storage reci...https://pulp.plan.io/issues/95952021-12-01T13:01:52Zipanova@redhat.comipanova@redhat.com
<pre><code>(pulp) [vagrant@pulp3-source-fedora34 _scripts]$ http https://pulp3-source-fedora34.fluffy.example.com/pulp/api/v3/distributions/file/file/9034d885-babb-4d19-81b3-091bd9f63a7f/
HTTP/1.1 200 OK
Access-Control-Expose-Headers: Correlation-ID
Allow: GET, PUT, PATCH, DELETE, HEAD, OPTIONS
Connection: keep-alive
Content-Length: 398
Content-Type: application/json
Correlation-ID: c5d10e5eaf7c40518b04130e3a2b22d3
Date: Wed, 01 Dec 2021 12:41:05 GMT
Referrer-Policy: same-origin
Server: nginx/1.20.1
Strict-Transport-Security: max-age=15768000
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
{
"base_path": "vewtf",
"base_url": "https://pulp3-source-fedora34.fluffy.example.com/pulp/content/vewtf/",
"content_guard": null,
"name": "cizwr",
"publication": "/pulp/api/v3/publications/file/file/10c34224-83ff-40b5-a47e-1453a13cbc88/",
"pulp_created": "2021-12-01T12:34:22.111939Z",
"pulp_href": "/pulp/api/v3/distributions/file/file/9034d885-babb-4d19-81b3-091bd9f63a7f/",
"pulp_labels": {},
"repository": null
}
(pulp) [vagrant@pulp3-source-fedora34 _scripts]$ http https://pulp3-source-fedora34.fluffy.example.com/pulp/content/vewtf/
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 343
Content-Type: text/html; charset=utf-8
Date: Wed, 01 Dec 2021 12:41:14 GMT
Server: nginx/1.20.1
Strict-Transport-Security: max-age=15768000
X-PULP-CACHE: HIT
<!DOCTYPE html>
<html>
<body>
<ul>
<li><a href="PULP_MANIFEST">PULP_MANIFEST</a></li>
<li><a href="test_upload.txt">test_upload.txt</a></li>
</ul>
</body>
</html>
(pulp) [vagrant@pulp3-source-fedora34 _scripts]$ http --follow HEAD https://pulp3-source-fedora34.fluffy.example.com/pulp/content/vewtf/test_upload.txt
HTTP/1.1 403 Forbidden
Accept-Ranges: bytes
Content-Length: 0
Content-Security-Policy: block-all-mixed-content
Date: Wed, 01 Dec 2021 12:41:20 GMT
Server: MinIO
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin, Accept-Encoding
X-Amz-Request-Id: 16BCA1FED0CD6B9E
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block
(pulp) [vagrant@pulp3-source-fedora34 _scripts]$ http --follow GET https://pulp3-source-fedora34.fluffy.example.com/pulp/content/vewtf/test_upload.txt
HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Disposition: attachment;filename=test_upload.txt
Content-Length: 11
Content-Security-Policy: block-all-mixed-content
Content-Type: text/plain
Date: Wed, 01 Dec 2021 12:41:28 GMT
ETag: "eef16594e73fc257de8125c7f1727a95"
Last-Modified: Wed, 01 Dec 2021 12:34:12 GMT
Server: MinIO
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
X-Amz-Request-Id: 16BCA200B53E2618
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block
rzwdbspfbe
(pulp) [vagrant@pulp3-source-fedora34 _scripts]$
</code></pre>
<p>Presigned URLs allow one type of HTTP request method, which is defined at their creation time. By default, Boto3 creates presigned URLs that <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-presigned-urls.html#presigned-urls" class="external">permit only the HTTP GET method</a> however, the request method can be <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-presigned-urls.html#using-presigned-urls-to-perform-other-s3-operations" class="external">specified</a>. This is exposed in <code>django-storages</code> <code>S3Boto3Storage.url</code> as <a href="https://github.com/jschneier/django-storages/blob/master/storages/backends/s3boto3.py#L567" class="external">http_method</a>.</p> Pulp - Issue #9590 (MODIFIED): Pulp CI badges are no longer validhttps://pulp.plan.io/issues/95902021-11-30T15:50:22Zlmjachky
<p>Current Pulp CI status badges are no longer valid after merging the commit <a href="https://github.com/pulp/pulpcore/commit/dae72fa404de50b347d877c89c1a269937ab27b0#diff-b803fcb7f17ed9235f1e5cb1fcd2f5d3b2838429d4368ae4c57ce4436577f03fL13-L15" class="external">https://github.com/pulp/pulpcore/commit/dae72fa404de50b347d877c89c1a269937ab27b0#diff-b803fcb7f17ed9235f1e5cb1fcd2f5d3b2838429d4368ae4c57ce4436577f03fL13-L15</a> (we did not want to run the CI pipeline once again after merging changes).</p>
<p>Removing status badges from all repositories should be sufficient to resolve the problem. Having only Pulp Nightly CI/CD badges available is good enough (<a href="https://github.com/pulp/pulp-ci" class="external">https://github.com/pulp/pulp-ci</a>).</p> Pulp - Issue #9588 (CLOSED - CURRENTRELEASE): RBAC: Groups cannot use creation_hookshttps://pulp.plan.io/issues/95882021-11-26T08:28:47Zmdellweg
<p>As <code>Group</code> is not a pulpcore model, it does not inherit from <code>DjangoLifecycleMixin</code> and <code>AutoAddObjPermsMixin</code>. Therefore, the <code>creation_hooks</code> are not executed.</p>
<p>Possible solution: Create a proxy model that does not add another database table.</p> Maven Plugin - Issue #8678 (MODIFIED): Provide 'view_name' warning when using the all in one cont...https://pulp.plan.io/issues/86782021-04-30T19:17:03Zgerrod
<p>A user from pulp-dev was worried about this warning message appearing after changing the admin password to the all in one container. Pretty sure it's harmless, but it can scare users thinking that their installation isn't correct.</p>
<p><code>pulpcore.app.serializers.base:WARNING: Please provide either 'view_name' or 'view_name_pattern' for DetailRelatedField on _call_with_frames_removed.</code></p> Pulp - Issue #8610 (CLOSED - CURRENTRELEASE): PulpImporter assumes tempfiles can always go to /tmphttps://pulp.plan.io/issues/86102021-04-21T19:44:37Zggainey
<p>importer.pulp_import uses tempfile.TemporaryDirectory() in places like this:</p>
<p><a href="https://github.com/pulp/pulpcore/blob/master/pulpcore/app/tasks/importer.py#L118" class="external">https://github.com/pulp/pulpcore/blob/master/pulpcore/app/tasks/importer.py#L118</a></p>
<p>If your /tmp is small, and your export is Large, this can cause Bad Things to happen.</p>
<p>We should perhas set dir= to the workers work-directory?</p>