Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-05-17T23:14:44ZPulp
Planio RPM Support - Issue #8769 (CLOSED - WORKSFORME): Missing left parenthesis error when creating con...https://pulp.plan.io/issues/87692021-05-17T23:14:44Zciokan
<p>I get this error when trying to upload a file to the rpm package:</p>
<pre><code>pulp_worker_1 | pulp [f0af99ab768d4acfadf171acf9efb553]: rq.worker:ERROR: Traceback (most recent call last):
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
pulp_worker_1 | return self.cursor.execute(sql, params)
pulp_worker_1 | psycopg2.errors.InvalidTextRepresentation: malformed record literal: ""
pulp_worker_1 | LINE 1: ...uuid, 'test', '0', '0.2.59', '1', 'x86_64', '', 'efc6c...
pulp_worker_1 | ^
pulp_worker_1 | DETAIL: Missing left parenthesis.
pulp_worker_1 |
pulp_worker_1 |
pulp_worker_1 | The above exception was the direct cause of the following exception:
pulp_worker_1 |
pulp_worker_1 | Traceback (most recent call last):
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/rq/worker.py", line 1013, in perform_job
pulp_worker_1 | rv = job.perform()
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/rq/job.py", line 709, in perform
pulp_worker_1 | self._result = self._execute()
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/rq/job.py", line 732, in _execute
pulp_worker_1 | result = self.func(*self.args, **self.kwargs)
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/pulpcore/app/tasks/base.py", line 35, in general_create
pulp_worker_1 | serializer.save()
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/rest_framework/serializers.py", line 205, in save
pulp_worker_1 | self.instance = self.create(validated_data)
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/pulpcore/plugin/serializers/content.py", line 45, in create
pulp_worker_1 | content = super().create(validated_data)
pulp_worker_1 | File "/usr/lib64/python3.9/contextlib.py", line 79, in inner
pulp_worker_1 | return func(*args, **kwds)
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/pulpcore/app/serializers/content.py", line 60, in create
pulp_worker_1 | content = self.Meta.model.objects.create(**validated_data)
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/manager.py", line 82, in manager_method
pulp_worker_1 | return getattr(self.get_queryset(), name)(*args, **kwargs)
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 422, in create
pulp_worker_1 | obj.save(force_insert=True, using=self.db)
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/pulpcore/app/models/base.py", line 149, in save
pulp_worker_1 | return super().save(*args, **kwargs)
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/django_lifecycle/mixins.py", line 134, in save
pulp_worker_1 | save(*args, **kwargs)
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/base.py", line 743, in save
pulp_worker_1 | self.save_base(using=using, force_insert=force_insert,
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/base.py", line 780, in save_base
pulp_worker_1 | updated = self._save_table(
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/base.py", line 873, in _save_table
pulp_worker_1 | result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/base.py", line 910, in _do_insert
pulp_worker_1 | return manager._insert([self], fields=fields, return_id=update_pk,
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/manager.py", line 82, in manager_method
pulp_worker_1 | return getattr(self.get_queryset(), name)(*args, **kwargs)
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 1186, in _insert
pulp_worker_1 | return query.get_compiler(using=using).execute_sql(return_id)
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1377, in execute_sql
pulp_worker_1 | cursor.execute(sql, params)
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 67, in execute
pulp_worker_1 | return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
pulp_worker_1 | return executor(sql, params, many, context)
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
pulp_worker_1 | return self.cursor.execute(sql, params)
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/django/db/utils.py", line 89, in __exit__
pulp_worker_1 | raise dj_exc_value.with_traceback(traceback) from exc_value
pulp_worker_1 | File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
pulp_worker_1 | return self.cursor.execute(sql, params)
pulp_worker_1 | django.db.utils.DataError: malformed record literal: ""
pulp_worker_1 | LINE 1: ...uuid, 'test', '0', '0.2.59', '1', 'x86_64', '', 'efc6c...
pulp_worker_1 | ^
pulp_worker_1 | DETAIL: Missing left parenthesis.
pulp_worker_1 |
</code></pre>
<p>The error propagates from this db query:</p>
<pre><code>INSERT INTO "rpm_package"
("content_ptr_id",
"name",
"epoch",
"version",
"release",
"arch",
"evr",
"pkgId",
"checksum_type",
"summary",
"description",
"url",
"changelogs",
"files",
"requires",
"provides",
"conflicts",
"obsoletes",
"suggests",
"enhances",
"recommends",
"supplements",
"location_base",
"location_href",
"rpm_buildhost",
"rpm_group",
"rpm_license",
"rpm_packager",
"rpm_sourcerpm",
"rpm_vendor",
"rpm_header_start",
"rpm_header_end",
"size_archive",
"size_installed",
"size_package",
"time_build",
"time_file",
"is_modular")
VALUES ('340746f2-bcd3-4140-9510-c7b23a8ec124' :: uuid,
'test',
'0',
'0.2.59',
'1',
'x86_64',
'',
'efc6c3ce265e4b46fcb0d0176f731ed3f3b39d5bf5acef095888189a93a78c0d',
'sha256',
'test DNS Proxy',
'test DNS Proxy',
'https://test.com',
'[]',
'[["", "/usr/bin/", "test"]]',
'[]',
'[["test", "EQ", "0", "0.2.59", "1", false]]',
'[]',
'[]',
'[]',
'[]',
'[]',
'[]',
'',
'dnsadblock_0.2.59_linux_amd64.rpm',
'ciokan',
'Development/Tools',
'',
'test <office@test.com>',
'test-0.2.59-1.src.rpm',
'test Inc',
272,
1804,
9003008,
9003008,
3572127,
1591340775,
1621291690,
FALSE)
</code></pre>
<p>...and it complaints about a missing left parenthesis on the 7th param which is empty (<code>evr</code> column).</p>
<p>The file was uploaded using a bash script:</p>
<pre><code>task_url=$(http --form POST "$BASE_ADDR/pulp/api/v3/content/rpm/packages/" "$CREDS" file@"$file" relative_path="$pkg" | jq -r '.task')
</code></pre>
<p>Postgres 11.1
pulpcore==3.12.2
pulp-rpm==3.10.0
pulp-deb==2.12.0</p> RPM Support - Issue #8492 (CLOSED - WORKSFORME): Alma 8 repositories cannot be synced without err...https://pulp.plan.io/issues/84922021-04-01T10:06:43Zbrimioulle
<p>When syncing Alma 8 AppStream, BaseOS and Kickstart repositories on a foreman/katello the packages are downloaded but in the end following error is thrown:</p>
<p>"AttributeError: 'Repository' object has no attribute 'repo_id'\n",</p>
<p>Exception:
Katello::Errors::PulpError: PLP0000: 'Repository' object has no attribute 'repo_id'</p> RPM Support - Issue #8433 (CLOSED - WORKSFORME): Issues synchronizing repositoryhttps://pulp.plan.io/issues/84332021-03-22T10:52:22Zkarlism
<p>Hello,</p>
<p>I'm having following issues while trying to synchronize repository:</p>
<pre><code>
# /usr/local/bin/pulp rpm repository create --name=centos_8_3_base
{
"pulp_href": "/pulp/api/v3/repositories/rpm/rpm/de4b4a2d-4c3e-4f40-b225-ef6e9cdbea93/",
"pulp_created": "2021-03-22T10:37:29.513788Z",
"versions_href": "/pulp/api/v3/repositories/rpm/rpm/de4b4a2d-4c3e-4f40-b225-ef6e9cdbea93/versions/",
"latest_version_href": "/pulp/api/v3/repositories/rpm/rpm/de4b4a2d-4c3e-4f40-b225-ef6e9cdbea93/versions/0/",
"name": "centos_8_3_base",
"description": null,
"remote": null,
"metadata_signing_service": null,
"retain_package_versions": 0
}
# /usr/local/bin/pulp rpm remote create --name=centos_8_3_base --url=http://mirror.centos.org/centos/8.3.2011/BaseOS/x86_64/os/
{
"pulp_href": "/pulp/api/v3/remotes/rpm/rpm/136aef71-a90b-4e04-a593-1d1aaeb5db89/",
"pulp_created": "2021-03-22T10:38:01.007355Z",
"name": "centos_8_3_base",
"url": "http://mirror.centos.org/centos/8.3.2011/BaseOS/x86_64/os/",
"ca_cert": null,
"client_cert": null,
"client_key": null,
"tls_validation": true,
"proxy_url": null,
"username": null,
"password": null,
"pulp_last_updated": "2021-03-22T10:38:01.007391Z",
"download_concurrency": 10,
"policy": "immediate",
"total_timeout": null,
"connect_timeout": null,
"sock_connect_timeout": null,
"sock_read_timeout": null,
"sles_auth_token": null
}
# /usr/local/bin/pulp rpm repository sync --name=centos_8_3_base --remote=centos_8_3_base
Started background task /pulp/api/v3/tasks/60f50a37-44b1-47b4-852f-2ed8cba569a6/
.......................................................................Error: Task /pulp/api/v3/tasks/60f50a37-44b1-47b4-852f-2ed8cba569a6/ failed: 'malformed record literal: ""
LINE 1: ....3.8', '6.module_el8.3.0+604+ab7bf9cc', 'x86_64', '', 'e25f6...
^
DETAIL: Missing left parenthesis.
'
</code></pre>
<p>Following errors in logs:</p>
<pre><code>Mar 22 10:40:06 hostname gunicorn[23229]: pulp [e32eba41066f4576a18540dd19cc772a]: 127.0.0.1 - admin [22/Mar/2021:10:40:06 +0000] "GET /pulp/api/v3/tasks/60f50a37-44b1-47b4-852f-2ed8cba569a6/ HTTP/1.0" 200 1478 "-" "python-requests/2.25.1"
Mar 22 10:40:07 hostname rq[23370]: pulp [82b62bafeb7f4cce9f1a1eb06c9b4905]: rq.worker:ERROR: Traceback (most recent call last):
Mar 22 10:40:07 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 22 10:40:07 hostname rq[23370]: return self.cursor.execute(sql, params)
Mar 22 10:40:07 hostname rq[23370]: psycopg2.errors.InvalidTextRepresentation: malformed record literal: ""
Mar 22 10:40:07 hostname rq[23370]: LINE 1: ....3.8', '6.module_el8.3.0+604+ab7bf9cc', 'x86_64', '', 'e25f6...
Mar 22 10:40:07 hostname rq[23370]: ^
Mar 22 10:40:07 hostname rq[23370]: DETAIL: Missing left parenthesis.
Mar 22 10:40:07 hostname rq[23370]: The above exception was the direct cause of the following exception:
Mar 22 10:40:07 hostname rq[23370]: Traceback (most recent call last):
Mar 22 10:40:07 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 975, in perform_job
Mar 22 10:40:07 hostname rq[23370]: rv = job.perform()
Mar 22 10:40:07 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 696, in perform
Mar 22 10:40:07 hostname rq[23370]: self._result = self._execute()
Mar 22 10:40:07 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 719, in _execute
Mar 22 10:40:07 hostname rq[23370]: return self.func(*self.args, **self.kwargs)
Mar 22 10:40:07 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 257, in synchronize
Mar 22 10:40:07 hostname rq[23370]: dv.create()
Mar 22 10:40:07 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 148, in create
Mar 22 10:40:07 hostname rq[23370]: loop.run_until_complete(pipeline)
Mar 22 10:40:07 hostname rq[23370]: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Mar 22 10:40:07 hostname rq[23370]: return future.result()
Mar 22 10:40:07 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Mar 22 10:40:07 hostname rq[23370]: await asyncio.gather(*futures)
Mar 22 10:40:07 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
Mar 22 10:40:07 hostname rq[23370]: await self.run()
Mar 22 10:40:07 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/content_stages.py", line 95, in run
Mar 22 10:40:07 hostname rq[23370]: d_content.content.save()
Mar 22 10:40:07 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/pulpcore/app/models/base.py", line 115, in save
Mar 22 10:40:07 hostname rq[23370]: return super().save(*args, **kwargs)
Mar 22 10:40:07 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django_lifecycle/mixins.py", line 129, in save
Mar 22 10:40:07 hostname rq[23370]: save(*args, **kwargs)
Mar 22 10:40:07 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 744, in save
Mar 22 10:40:08 hostname rq[23370]: force_update=force_update, update_fields=update_fields)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 782, in save_base
Mar 22 10:40:08 hostname rq[23370]: force_update, using, update_fields,
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 873, in _save_table
Mar 22 10:40:08 hostname rq[23370]: result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 911, in _do_insert
Mar 22 10:40:08 hostname rq[23370]: using=using, raw=raw)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
Mar 22 10:40:08 hostname rq[23370]: return getattr(self.get_queryset(), name)(*args, **kwargs)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 1186, in _insert
Mar 22 10:40:08 hostname rq[23370]: return query.get_compiler(using=using).execute_sql(return_id)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1377, in execute_sql
Mar 22 10:40:08 hostname rq[23370]: cursor.execute(sql, params)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
Mar 22 10:40:08 hostname rq[23370]: return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
Mar 22 10:40:08 hostname rq[23370]: return executor(sql, params, many, context)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 22 10:40:08 hostname rq[23370]: return self.cursor.execute(sql, params)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
Mar 22 10:40:08 hostname rq[23370]: raise dj_exc_value.with_traceback(traceback) from exc_value
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 22 10:40:08 hostname rq[23370]: return self.cursor.execute(sql, params)
Mar 22 10:40:08 hostname rq[23370]: django.db.utils.DataError: malformed record literal: ""
Mar 22 10:40:08 hostname rq[23370]: LINE 1: ....3.8', '6.module_el8.3.0+604+ab7bf9cc', 'x86_64', '', 'e25f6...
Mar 22 10:40:08 hostname rq[23370]: ^
Mar 22 10:40:08 hostname rq[23370]: DETAIL: Missing left parenthesis.
Mar 22 10:40:08 hostname rq[23370]: Traceback (most recent call last):
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 22 10:40:08 hostname rq[23370]: return self.cursor.execute(sql, params)
Mar 22 10:40:08 hostname rq[23370]: psycopg2.errors.InvalidTextRepresentation: malformed record literal: ""
Mar 22 10:40:08 hostname rq[23370]: LINE 1: ....3.8', '6.module_el8.3.0+604+ab7bf9cc', 'x86_64', '', 'e25f6...
Mar 22 10:40:08 hostname rq[23370]: ^
Mar 22 10:40:08 hostname rq[23370]: DETAIL: Missing left parenthesis.
Mar 22 10:40:08 hostname rq[23370]: The above exception was the direct cause of the following exception:
Mar 22 10:40:08 hostname rq[23370]: Traceback (most recent call last):
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 975, in perform_job
Mar 22 10:40:08 hostname rq[23370]: rv = job.perform()
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 696, in perform
Mar 22 10:40:08 hostname rq[23370]: self._result = self._execute()
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 719, in _execute
Mar 22 10:40:08 hostname rq[23370]: return self.func(*self.args, **self.kwargs)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 257, in synchronize
Mar 22 10:40:08 hostname rq[23370]: dv.create()
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 148, in create
Mar 22 10:40:08 hostname rq[23370]: loop.run_until_complete(pipeline)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Mar 22 10:40:08 hostname rq[23370]: return future.result()
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Mar 22 10:40:08 hostname rq[23370]: await asyncio.gather(*futures)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
Mar 22 10:40:08 hostname rq[23370]: await self.run()
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/content_stages.py", line 95, in run
Mar 22 10:40:08 hostname rq[23370]: d_content.content.save()
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/pulpcore/app/models/base.py", line 115, in save
Mar 22 10:40:08 hostname rq[23370]: return super().save(*args, **kwargs)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django_lifecycle/mixins.py", line 129, in save
Mar 22 10:40:08 hostname rq[23370]: save(*args, **kwargs)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 744, in save
Mar 22 10:40:08 hostname rq[23370]: force_update=force_update, update_fields=update_fields)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 782, in save_base
Mar 22 10:40:08 hostname rq[23370]: force_update, using, update_fields,
Mar 22 10:40:08 hostname gunicorn[23229]: pulp [6e4108df743c4f6dafcd1411234e9250]: 127.0.0.1 - admin [22/Mar/2021:10:40:08 +0000] "GET /pulp/api/v3/tasks/60f50a37-44b1-47b4-852f-2ed8cba569a6/ HTTP/1.0" 200 4947 "-" "python-requests/2.25.1"
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 873, in _save_table
Mar 22 10:40:08 hostname rq[23370]: result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 911, in _do_insert
Mar 22 10:40:08 hostname rq[23370]: using=using, raw=raw)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
Mar 22 10:40:08 hostname rq[23370]: return getattr(self.get_queryset(), name)(*args, **kwargs)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 1186, in _insert
Mar 22 10:40:08 hostname rq[23370]: return query.get_compiler(using=using).execute_sql(return_id)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1377, in execute_sql
Mar 22 10:40:08 hostname rq[23370]: cursor.execute(sql, params)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
Mar 22 10:40:08 hostname rq[23370]: return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
Mar 22 10:40:08 hostname rq[23370]: return executor(sql, params, many, context)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 22 10:40:08 hostname rq[23370]: return self.cursor.execute(sql, params)
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
Mar 22 10:40:08 hostname rq[23370]: raise dj_exc_value.with_traceback(traceback) from exc_value
Mar 22 10:40:08 hostname rq[23370]: File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 22 10:40:08 hostname rq[23370]: return self.cursor.execute(sql, params)
Mar 22 10:40:08 hostname rq[23370]: django.db.utils.DataError: malformed record literal: ""
Mar 22 10:40:08 hostname rq[23370]: LINE 1: ....3.8', '6.module_el8.3.0+604+ab7bf9cc', 'x86_64', '', 'e25f6...
Mar 22 10:40:08 hostname rq[23370]: ^
Mar 22 10:40:08 hostname rq[23370]: DETAIL: Missing left parenthesis.
Mar 22 10:40:08 hostname rq[23370]: pulp [None]: rq.worker:INFO: 23370@hostname.sns.sec: 5a36c8da-85e4-41b3-9636-59d281645352
Mar 22 10:40:10 hostname rq[23370]: pulp [None]: rq.worker:INFO: 23370@hostname.sns.sec: Job OK (5a36c8da-85e4-41b3-9636-59d281645352
</code></pre>
<p>It is VM running Oracle Linux 8.3, also tried it with version 3.11.1 with the same issue:</p>
<pre><code># /usr/local/bin/pulp status
{
"versions": [
{
"component": "pulpcore",
"version": "3.9.1"
},
{
"component": "pulp_rpm",
"version": "3.9.0"
}
],
"online_workers": [
{
"pulp_created": "2021-03-22T07:12:16.765141Z",
"pulp_href": "/pulp/api/v3/workers/b3a0a25a-0f50-42e6-8a39-1b68226992e3/",
"name": "23370@hostname.sns.sec",
"last_heartbeat": "2021-03-22T10:48:30.806559Z"
},
{
"pulp_created": "2021-03-22T07:12:10.076036Z",
"pulp_href": "/pulp/api/v3/workers/8c65bc91-19e1-4e30-ba6a-05beaa4ca573/",
"name": "resource-manager",
"last_heartbeat": "2021-03-22T10:48:34.517400Z"
},
{
"pulp_created": "2021-03-22T07:12:17.186151Z",
"pulp_href": "/pulp/api/v3/workers/cefd5028-dab6-4a2b-a593-b7a50bf3bba1/",
"name": "23509@hostname.sns.sec",
"last_heartbeat": "2021-03-22T10:48:57.177400Z"
}
],
"online_content_apps": [
{
"name": "23136@hostname",
"last_heartbeat": "2021-03-22T10:48:54.277865Z"
},
{
"name": "23192@hostname",
"last_heartbeat": "2021-03-22T10:48:55.040598Z"
},
{
"name": "23154@hostname",
"last_heartbeat": "2021-03-22T10:48:55.045353Z"
},
{
"name": "23155@hostname",
"last_heartbeat": "2021-03-22T10:48:55.080843Z"
},
{
"name": "23138@hostname",
"last_heartbeat": "2021-03-22T10:48:55.111686Z"
},
{
"name": "23200@hostname",
"last_heartbeat": "2021-03-22T10:48:55.528796Z"
},
{
"name": "23172@hostname",
"last_heartbeat": "2021-03-22T10:48:55.567173Z"
},
{
"name": "23199@hostname",
"last_heartbeat": "2021-03-22T10:48:56.001181Z"
}
],
"database_connection": {
"connected": true
},
"redis_connection": {
"connected": true
},
"storage": {
"total": 85851357184,
"used": 3087319040,
"free": 78358970368
}
}
</code></pre>
<p>Any help in solving this issue would be appreciated. Same issue happens also with other RPM repositories and I've verfied that URLs are correct and accessible from host where Pulp is running.</p> RPM Support - Issue #8287 (CLOSED - WORKSFORME): Rhel8 remote redhat repo showing Malformed repos...https://pulp.plan.io/issues/82872021-02-22T07:42:32Zmidhuhk
<p>Remote repo:
appstream-rhel8</p>
<p>data": {
"reason": "metadata is missing for some packages in filelists.xml and in other.xml"
},
"description": "Malformed repository: metadata is missing for some packages in filelists.xml and in other.xml",</p> RPM Support - Issue #8100 (CLOSED - WORKSFORME): Error when publishing repository Red Hat Enterpr...https://pulp.plan.io/issues/81002021-01-15T06:55:08Zpieta
<pre><code>Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: pulp: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6Server-189772, version=1
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: pulp: rq.worker:ERROR: Traceback (most recent call last):
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: rv = job.perform()
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: self._result = self._execute()
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: return self.func(*self.args, **self.kwargs)
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 283, in publish
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: publication_data.populate()
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 209, in populate
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: self.repomdrecords = self.prepare_metadata_files(main_content)
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 89, in prepare_metadata_files
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: current_file = content_artifact.artifact.file.file
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/django/db/models/fields/files.py", line 43, in _get_file
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: self._file = self.storage.open(self.name, 'rb')
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/django/core/files/storage.py", line 36, in open
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: return self._open(name, mode)
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/django/core/files/storage.py", line 224, in _open
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: return File(open(self.path(name), mode))
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/pulp/media/artifact/5b/290c0e700df6f4b4669d9e7366f6aa4b6817f96e1dcb9b81284409ae640972'
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: Traceback (most recent call last):
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: rv = job.perform()
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: self._result = self._execute()
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: return self.func(*self.args, **self.kwargs)
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 283, in publish
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: publication_data.populate()
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 209, in populate
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: self.repomdrecords = self.prepare_metadata_files(main_content)
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 89, in prepare_metadata_files
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: current_file = content_artifact.artifact.file.file
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/django/db/models/fields/files.py", line 43, in _get_file
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: self._file = self.storage.open(self.name, 'rb')
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/django/core/files/storage.py", line 36, in open
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: return self._open(name, mode)
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/django/core/files/storage.py", line 224, in _open
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: return File(open(self.path(name), mode))
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/pulp/media/artifact/5b/290c0e700df6f4b4669d9e7366f6aa4b6817f96e1dcb9b81284409ae640972'
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: pulp: rq.worker:INFO: 32205@oplrepo-cpd-dmz.corpnet.pl: cb6fefb5-5347-4d56-a933-c9716438a13c
Jan 14 12:10:07 oplrepo-cpd-dmz pulpcore-worker-3: pulp: rq.worker:INFO: 32205@oplrepo-cpd-dmz.corpnet.pl: Job OK (cb6fefb5-5347-4d56-a933-c9716438a13c)
Jan 14 12:10:08 oplrepo-cpd-dmz pulpcore-api: - - [14/Jan/2021:11:10:08 +0000] "GET /pulp/api/v3/tasks/ed9ab0b6-1878-4f3e-ba36-552963eb92eb/ HTTP/1.1" 200 1957 "-" "OpenAPI-Generator/3.7.1/ruby"
Jan 14 12:10:08 oplrepo-cpd-dmz pulpcore-resource-manager: pulp: rq.worker:INFO: resource-manager: e780fd48-487e-4bde-b6ad-f46eae938da8
Jan 14 12:10:08 oplrepo-cpd-dmz pulpcore-api: - - [14/Jan/2021:11:10:08 +0000] "PATCH /pulp/api/v3/distributions/rpm/rpm/1d113bbc-48b9-4897-b3e8-ae5e18546b6a/ HTTP/1.1" 202 67 "-" "OpenAPI-Generator/3.7.0/ruby"
Jan 14 12:10:08 oplrepo-cpd-dmz pulpcore-api: - - [14/Jan/2021:11:10:08 +0000] "GET /pulp/api/v3/tasks/dd30cc46-38fd-4ae3-8935-6e94fc967400/ HTTP/1.1" 200 401 "-" "OpenAPI-Generator/3.7.1/ruby"
Jan 14 12:10:08 oplrepo-cpd-dmz pulpcore-worker-1: pulp: rq.worker:INFO: 32146@oplrepo-cpd-dmz.corpnet.pl: dd30cc46-38fd-4ae3-8935-6e94fc967400
Jan 14 12:10:08 oplrepo-cpd-dmz pulpcore-resource-manager: pulp: rq.worker:INFO: resource-manager: Job OK (e780fd48-487e-4bde-b6ad-f46eae938da8)
Jan 14 12:10:09 oplrepo-cpd-dmz pulpcore-api: - - [14/Jan/2021:11:10:09 +0000] "GET /pulp/api/v3/tasks/dd30cc46-38fd-4ae3-8935-6e94fc967400/ HTTP/1.1" 200 482 "-" "OpenAPI-Generator/3.7.1/ruby"
Jan 14 12:10:09 oplrepo-cpd-dmz pulpcore-worker-1: pulp: rq.worker:INFO: 32146@oplrepo-cpd-dmz.corpnet.pl: Job OK (dd30cc46-38fd-4ae3-8935-6e94fc967400)
Jan 14 12:10:09 oplrepo-cpd-dmz pulpcore-worker-1: pulp: rq.worker:INFO: 32146@oplrepo-cpd-dmz.corpnet.pl: f33e4dc9-4dea-4487-80c8-7f705a0c4571
Jan 14 12:10:09 oplrepo-cpd-dmz pulpcore-worker-1: pulp: rq.worker:INFO: 32146@oplrepo-cpd-dmz.corpnet.pl: Job OK (f33e4dc9-4dea-4487-80c8-7f705a0c4571)
Jan 14 12:10:09 oplrepo-cpd-dmz pulpcore-api: - - [14/Jan/2021:11:10:09 +0000] "GET /pulp/api/v3/tasks/dd30cc46-38fd-4ae3-8935-6e94fc967400/ HTTP/1.1" 200 509 "-" "OpenAPI-Generator/3.7.1/ruby"
Jan 14 12:10:09 oplrepo-cpd-dmz pulpcore-resource-manager: pulp: rq.worker:INFO: resource-manager: cc602b56-95da-449d-ab86-f9d16580a5d4
Jan 14 12:10:09 oplrepo-cpd-dmz pulpcore-api: - - [14/Jan/2021:11:10:09 +0000] "PATCH /pulp/api/v3/distributions/rpm/rpm/1d113bbc-48b9-4897-b3e8-ae5e18546b6a/ HTTP/1.1" 202 67 "-" "OpenAPI-Generator/3.7.0/ruby"
Jan 14 12:10:09 oplrepo-cpd-dmz pulpcore-worker-4: pulp: rq.worker:INFO: 32227@oplrepo-cpd-dmz.corpnet.pl: 5733c987-6ebb-4b23-a759-68efa2ced7e4
Jan 14 12:10:09 oplrepo-cpd-dmz pulpcore-resource-manager: pulp: rq.worker:INFO: resource-manager: Job OK (cc602b56-95da-449d-ab86-f9d16580a5d4)
Jan 14 12:10:10 oplrepo-cpd-dmz pulpcore-worker-4: pulp: rq.worker:INFO: 32227@oplrepo-cpd-dmz.corpnet.pl: Job OK (5733c987-6ebb-4b23-a759-68efa2ced7e4)
Jan 14 12:10:10 oplrepo-cpd-dmz pulpcore-worker-4: pulp: rq.worker:INFO: 32227@oplrepo-cpd-dmz.corpnet.pl: b2df0b58-9ce4-4a6f-ab02-8c4cf0732ebe
Jan 14 12:10:10 oplrepo-cpd-dmz pulpcore-worker-4: pulp: rq.worker:INFO: 32227@oplrepo-cpd-dmz.corpnet.pl: Job OK (b2df0b58-9ce4-4a6f-ab02-8c4cf0732ebe)
Jan 14 12:10:40 oplrepo-cpd-dmz pulpcore-api: [2021-01-14 12:10:40 +0100] [1995] [CRITICAL] WORKER TIMEOUT (pid:2058)
Jan 14 12:10:42 oplrepo-cpd-dmz pulpcore-api: [2021-01-14 12:10:42 +0100] [12482] [INFO] Booting worker with pid: 12482
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-resource-manager: pulp: rq.worker:INFO: resource-manager: 31df0561-106d-4edc-ba58-eaacf58bf71e
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-api: - - [14/Jan/2021:11:10:47 +0000] "POST /pulp/api/v3/publications/rpm/rpm/ HTTP/1.1" 202 67 "-" "OpenAPI-Generator/3.7.0/ruby"
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: pulp: rq.worker:INFO: 32227@oplrepo-cpd-dmz.corpnet.pl: 004d6d1e-02a3-4805-9ba1-4379be2a620d
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-resource-manager: pulp: rq.worker:INFO: resource-manager: Job OK (31df0561-106d-4edc-ba58-eaacf58bf71e)
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-api: - - [14/Jan/2021:11:10:47 +0000] "GET /pulp/api/v3/tasks/004d6d1e-02a3-4805-9ba1-4379be2a620d/ HTTP/1.1" 200 449 "-" "OpenAPI-Generator/3.7.1/ruby"
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: pulp: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6Server-189772, version=1
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: pulp: rq.worker:ERROR: Traceback (most recent call last):
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: rv = job.perform()
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: self._result = self._execute()
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: return self.func(*self.args, **self.kwargs)
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 283, in publish
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: publication_data.populate()
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 209, in populate
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: self.repomdrecords = self.prepare_metadata_files(main_content)
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 89, in prepare_metadata_files
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: current_file = content_artifact.artifact.file.file
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/django/db/models/fields/files.py", line 43, in _get_file
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: self._file = self.storage.open(self.name, 'rb')
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/django/core/files/storage.py", line 36, in open
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: return self._open(name, mode)
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/django/core/files/storage.py", line 224, in _open
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: return File(open(self.path(name), mode))
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/pulp/media/artifact/5b/290c0e700df6f4b4669d9e7366f6aa4b6817f96e1dcb9b81284409ae640972'
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: Traceback (most recent call last):
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: rv = job.perform()
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: self._result = self._execute()
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: return self.func(*self.args, **self.kwargs)
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 283, in publish
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: publication_data.populate()
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 209, in populate
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: self.repomdrecords = self.prepare_metadata_files(main_content)
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 89, in prepare_metadata_files
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: current_file = content_artifact.artifact.file.file
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/django/db/models/fields/files.py", line 43, in _get_file
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: self._file = self.storage.open(self.name, 'rb')
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/django/core/files/storage.py", line 36, in open
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: return self._open(name, mode)
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/django/core/files/storage.py", line 224, in _open
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: return File(open(self.path(name), mode))
Jan 14 12:10:47 oplrepo-cpd-dmz pulpcore-worker-4: FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/pulp/media/artifact/5b/290c0e700df6f4b4669d9e7366f6aa4b6817f96e1dcb9b81284409ae640972'
</code></pre> RPM Support - Issue #7786 (CLOSED - WORKSFORME): Pulp_rpm sync reports "completed" but does not u...https://pulp.plan.io/issues/77862020-11-04T07:51:35ZAant
<p>This is a copy of <a href="https://pulp.plan.io/issues/7535?next_issue_id=7534" class="external">https://pulp.plan.io/issues/7535?next_issue_id=7534</a>. The issue persists.
Even if I delete and recreate the publication and distribution after synchronization I still experience this issue.
This is a very severe bug as it makes me believe that my systems are up to date while they are not.</p>
<a name="What-would-be-the-proper-workflow-to-resynchronise-a-repo-after-it-is-created"></a>
<h2 >What would be the proper workflow to resynchronise a repo after it is created?<a href="#What-would-be-the-proper-workflow-to-resynchronise-a-repo-after-it-is-created" class="wiki-anchor">¶</a></h2>
<p>I have set up Pulpcore v3.6.3 with Pulp_rpm v3.6.2 - later updated both to 3.7.0.
I created the following remote:</p>
<pre><code>{
"pulp_href": "/pulp/api/v3/remotes/rpm/rpm/d8fb524c-e73b-4afc-ba2e-05eea7f5b6a8/",
"pulp_created": "2020-08-18T08:47:19.183836Z",
"name": "centos8_base",
"url": "http://mirror.centos.org/centos-8/8/BaseOS/x86_64/os/",
"ca_cert": null,
"client_cert": null,
"client_key": null,
"tls_validation": true,
"proxy_url": "http://<proxy server>",
"username": null,
"password": null,
"pulp_last_updated": "2020-08-18T08:47:19.183855Z",
"download_concurrency": 10,
"policy": "immediate",
"sles_auth_token": null
}
</code></pre>
<p>and this repository:</p>
<pre><code>{
"pulp_href": "/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/",
"pulp_created": "2020-08-18T08:47:17.704104Z",
"versions_href": "/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/versions/",
"latest_version_href": "/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/versions/1/",
"name": "centos8_base",
"description": null,
"remote": null,
"metadata_signing_service": null,
"retain_package_versions": 0
}
</code></pre>
<p>If I sync, it reports success:</p>
<pre><code>http --auth admin --auth-type basic :9000/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/sync/ remote=/pulp/api/v3/remotes/rpm/rpm/d8fb524c-e73b-4afc-ba2e-05eea7f5b6a8/
http: password for admin@localhost:9000:
HTTP/1.1 202 Accepted
Allow: POST, OPTIONS
Connection: close
Content-Length: 67
Content-Type: application/json
Date: Mon, 21 Sep 2020 08:10:11 GMT
Server: gunicorn/20.0.4
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"task": "/pulp/api/v3/tasks/655d2e7b-2783-4496-a851-3bcd8d5e442e/"
}
</code></pre>
<p>Let's take a closer look at the task:</p>
<pre><code> http --auth admin --auth-type basic :9000/pulp/api/v3/tasks/655d2e7b-2783-4496-a851-3bcd8d5e442e/
http: password for admin@localhost:9000:
HTTP/1.1 200 OK
Allow: GET, PATCH, DELETE, HEAD, OPTIONS
Connection: close
Content-Length: 743
Content-Type: application/json
Date: Mon, 21 Sep 2020 08:10:29 GMT
Server: gunicorn/20.0.4
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"child_tasks": [],
"created_resources": [],
"error": null,
"finished_at": "2020-09-21T08:10:11.585792Z",
"name": "pulp_rpm.app.tasks.synchronizing.synchronize",
"parent_task": null,
"progress_reports": [
{
"code": "optimizing.sync",
"done": 1,
"message": "Optimizing Sync",
"state": "completed",
"suffix": null,
"total": null
}
],
"pulp_created": "2020-09-21T08:10:11.257796Z",
"pulp_href": "/pulp/api/v3/tasks/655d2e7b-2783-4496-a851-3bcd8d5e442e/",
"reserved_resources_record": [
"/pulp/api/v3/remotes/rpm/rpm/d8fb524c-e73b-4afc-ba2e-05eea7f5b6a8/",
"/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/"
],
"started_at": "2020-09-21T08:10:11.507912Z",
"state": "completed",
"task_group": null,
"worker": "/pulp/api/v3/workers/80ca382e-fdec-4eef-94b4-6c1c67433511/"
}
</code></pre>
<p>Let's use this repo on the same machine:</p>
<pre><code>cat /etc/yum.repos.d/CentOS-Base.repo
[Base]
name=CentOS-$releasever - Base
enabled=1
baseurl=https://<myurl>/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
</code></pre>
<p>Now if I do <code>yum check-update</code> it does not find any package to update.
If I change <code>baseurl=</code> to <code>http://mirror.centos.org/centos-8/8/BaseOS/x86_64/os/</code> (which is the same url I have in the remote) and I do <code>yum check-update</code>, then it finds 28 outdated packages. How is it possible?</p> RPM Support - Issue #7535 (CLOSED - WORKSFORME): Pulp_rpm sync reports "completed" but does not u...https://pulp.plan.io/issues/75352020-09-21T08:28:31ZAant
<p>I have set up Pulpcore v3.6.3 with Pulp_rpm v3.6.2.
I created the following remote:</p>
<pre><code>{
"pulp_href": "/pulp/api/v3/remotes/rpm/rpm/d8fb524c-e73b-4afc-ba2e-05eea7f5b6a8/",
"pulp_created": "2020-08-18T08:47:19.183836Z",
"name": "centos8_base",
"url": "http://mirror.centos.org/centos-8/8/BaseOS/x86_64/os/",
"ca_cert": null,
"client_cert": null,
"client_key": null,
"tls_validation": true,
"proxy_url": "http://<proxy server>",
"username": null,
"password": null,
"pulp_last_updated": "2020-08-18T08:47:19.183855Z",
"download_concurrency": 10,
"policy": "immediate",
"sles_auth_token": null
}
</code></pre>
<p>and this repository:</p>
<pre><code>{
"pulp_href": "/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/",
"pulp_created": "2020-08-18T08:47:17.704104Z",
"versions_href": "/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/versions/",
"latest_version_href": "/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/versions/1/",
"name": "centos8_base",
"description": null,
"remote": null,
"metadata_signing_service": null,
"retain_package_versions": 0
}
</code></pre>
<p>If I sync, it reports success:</p>
<pre><code>http --auth admin --auth-type basic :9000/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/sync/ remote=/pulp/api/v3/remotes/rpm/rpm/d8fb524c-e73b-4afc-ba2e-05eea7f5b6a8/
http: password for admin@localhost:9000:
HTTP/1.1 202 Accepted
Allow: POST, OPTIONS
Connection: close
Content-Length: 67
Content-Type: application/json
Date: Mon, 21 Sep 2020 08:10:11 GMT
Server: gunicorn/20.0.4
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"task": "/pulp/api/v3/tasks/655d2e7b-2783-4496-a851-3bcd8d5e442e/"
}
</code></pre>
<p>Let's take a closer look at the task:</p>
<pre><code> http --auth admin --auth-type basic :9000/pulp/api/v3/tasks/655d2e7b-2783-4496-a851-3bcd8d5e442e/
http: password for admin@localhost:9000:
HTTP/1.1 200 OK
Allow: GET, PATCH, DELETE, HEAD, OPTIONS
Connection: close
Content-Length: 743
Content-Type: application/json
Date: Mon, 21 Sep 2020 08:10:29 GMT
Server: gunicorn/20.0.4
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"child_tasks": [],
"created_resources": [],
"error": null,
"finished_at": "2020-09-21T08:10:11.585792Z",
"name": "pulp_rpm.app.tasks.synchronizing.synchronize",
"parent_task": null,
"progress_reports": [
{
"code": "optimizing.sync",
"done": 1,
"message": "Optimizing Sync",
"state": "completed",
"suffix": null,
"total": null
}
],
"pulp_created": "2020-09-21T08:10:11.257796Z",
"pulp_href": "/pulp/api/v3/tasks/655d2e7b-2783-4496-a851-3bcd8d5e442e/",
"reserved_resources_record": [
"/pulp/api/v3/remotes/rpm/rpm/d8fb524c-e73b-4afc-ba2e-05eea7f5b6a8/",
"/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/"
],
"started_at": "2020-09-21T08:10:11.507912Z",
"state": "completed",
"task_group": null,
"worker": "/pulp/api/v3/workers/80ca382e-fdec-4eef-94b4-6c1c67433511/"
}
</code></pre>
<p>Let's use this repo on the same machine:</p>
<pre><code>cat /etc/yum.repos.d/CentOS-Base.repo
[Base]
name=CentOS-$releasever - Base
enabled=1
baseurl=https://<myurl>/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
</code></pre>
<p>Now if I do <code>yum check-update</code> it does not find any package to update.
If I change <code>baseurl=</code> to <code>http://mirror.centos.org/centos-8/8/BaseOS/x86_64/os/</code> (which is the same url I have in the remote) and I do <code>yum check-update</code>, then it finds 28 outdated packages. How is it possible?</p> RPM Support - Issue #7298 (CLOSED - WORKSFORME): Sync into the imported repository fails for comp...https://pulp.plan.io/issues/72982020-08-10T16:47:44Zttereshcttereshc@redhat.com
<p>I used Centos8 BaseOS kicstart repo, hopefully any smaller repo with comps.xml will work.</p>
<p>To reproduce:</p>
<ol>
<li>Create a repo, sync it, and export it.</li>
<li>Import a repo (I imported it to a different pulp instance, probably plcean and importing into the same one will work the same way).</li>
<li>Sync from the same remote URL.</li>
</ol>
<pre><code>Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: pulp: rq.worker:ERROR: ValueError: Cannot create repository version. More than one rpm.packagecategory content with the duplicate values for id., More than one rpm.packagegroup content with the duplicate values for id., More than one rpm.packageenvironment content with the duplicate values for id.
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: Traceback (most recent call last):
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py", line 934, in perform_job
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: rv = job.perform()
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 686, in perform
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: self._result = self._execute()
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 692, in _execute
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: return self.func(*self.args, **self.kwargs)
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: File "/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/synchronizing.py", line 208, in synchronize
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: dv.create()
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/declarative_version.py", line 148, in create
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: loop.run_until_complete(pipeline)
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: File "/home/vagrant/devel/pulpcore/pulpcore/app/models/repository.py", line 790, in __exit__
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: repository.finalize_new_version(self)
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: File "/home/vagrant/devel/pulp_rpm/pulp_rpm/app/models/repository.py", line 173, in finalize_new_version
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: validate_repo_version(new_version)
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/repo_version_utils.py", line 140, in validate_repo_version
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: validate_duplicate_content(version)
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/repo_version_utils.py", line 110, in validate_duplicate_content
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: _("Cannot create repository version. {msg}").format(msg=", ".join(error_messages))
Aug 10 16:29:41 pulp3-source-fedora31.rhgoose.example.com rq[55125]: ValueError: Cannot create repository version. More than one rpm.packagecategory content with the duplicate values for id., More than one rpm.packagegroup content with the duplicate values for id., More than one rpm.packageenvironment content with the duplicate values for id.
</code></pre> RPM Support - Issue #7146 (CLOSED - WORKSFORME): Pulp_installer failure (version check) when tryi...https://pulp.plan.io/issues/71462020-07-15T11:20:50Zswisscom
<p>Dear support team,
I tried installing a new Pulp server on a CentOS 7 fresh server. I have added the pulp-rpm vars in my playbook file and downloaded the necessary Ansible role. The Ansible run fails with the following error :</p>
<pre><code class="text syntaxhl" data-language="text">TASK [pulp_common : Run pip-compile to check pulpcore/plugin compatibility] ***************************************************************************************************************************************
[WARNING]: conditional statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ failed_condition | default("compatibility.rc != 0") }}
fatal: [localhost]: FAILED! => {"changed": false, "cmd": ["/usr/local/lib/pulp/bin/pip-compile"], "delta": "0:00:03.277298", "end": "2020-07-15 12:47:44.915505", "failed_when_result": true, "msg": "non-zero return code", "rc": 2, "start": "2020-07-15 12:47:41.638207", "stderr": "Could not find a version that matches pulpcore<3.5,==3.5.0,>=3.4 (from -r requirements.in (line 1))\nTried: 3.0.0, 3.0.0, 3.0.1, 3.0.1, 3.1.0, 3.1.0, 3.1.1, 3.1.1, 3.2.0, 3.2.0, 3.2.1, 3.2.1, 3.3.0, 3.3.0, 3.3.1, 3.3.1, 3.4.0, 3.4.0, 3.4.1, 3.4.1, 3.5.0, 3.5.0\nSkipped pre-versions: 3.0.0a1.dev3, 3.0.0a1.dev3, 3.0.0a1.dev4, 3.0.0a1.dev4, 3.0.0a1.dev5, 3.0.0a1.dev5, 3.0.0a1, 3.0.0a1, 3.0.0a2, 3.0.0a2, 3.0.0a3, 3.0.0a3, 3.0.0a4, 3.0.0a4, 3.0.0a5, 3.0.0a5, 3.0.0a6, 3.0.0a6, 3.0.0a7, 3.0.0a7, 3.0.0a8, 3.0.0a8, 3.0.0a9, 3.0.0a9, 3.0.0a10, 3.0.0a10, 3.0.0a11, 3.0.0a11, 3.0.0a12, 3.0.0a12, 3.0.0a13, 3.0.0a13, 3.0.0a14, 3.0.0a14, 3.0.0a15, 3.0.0a15, 3.0.0a16, 3.0.0a16, 3.0.0a17, 3.0.0a17, 3.0.0a18, 3.0.0a18, 3.0.0a19, 3.0.0a19, 3.0.0a20, 3.0.0a20, 3.0.0a21, 3.0.0a21, 3.0.0a22, 3.0.0a22, 3.0.0a23, 3.0.0a23, 3.0.0a24, 3.0.0a24, 3.0.0a25, 3.0.0a25, 3.0.0a26, 3.0.0a26, 3.0.0a27, 3.0.0a27, 3.0.0a28, 3.0.0a28, 3.0.0a29, 3.0.0a29, 3.0.0b1, 3.0.0b1, 3.0.0b2, 3.0.0b2, 3.0.0b3, 3.0.0b3, 3.0.0b4, 3.0.0b4, 3.0.0b5, 3.0.0b5, 3.0.0b6, 3.0.0b6, 3.0.0b7, 3.0.0b7, 3.0.0b8, 3.0.0b8, 3.0.0b9, 3.0.0b9, 3.0.0b10, 3.0.0b10, 3.0.0b11, 3.0.0b11, 3.0.0b12, 3.0.0b12, 3.0.0b13, 3.0.0b13, 3.0.0b14, 3.0.0b14, 3.0.0b15, 3.0.0b15, 3.0.0b16, 3.0.0b16, 3.0.0b17, 3.0.0b18, 3.0.0b18, 3.0.0b19, 3.0.0b19, 3.0.0b20, 3.0.0b20, 3.0.0b21, 3.0.0b21, 3.0.0b22, 3.0.0b22, 3.0.0b23, 3.0.0b23, 3.0.0rc1, 3.0.0rc1, 3.0.0rc2, 3.0.0rc2, 3.0.0rc3, 3.0.0rc3, 3.0.0rc4, 3.0.0rc4, 3.0.0rc5, 3.0.0rc5, 3.0.0rc6, 3.0.0rc6, 3.0.0rc7, 3.0.0rc7, 3.0.0rc8, 3.0.0rc8, 3.0.0rc9, 3.0.0rc9\nThere are incompatible versions in the resolved dependencies:\n pulpcore==3.5.0 (from -r requirements.in (line 1))\n pulpcore<3.5,>=3.4 (from pulp-rpm==3.4.1->-r requirements.in (line 2))", "stderr_lines": ["Could not find a version that matches pulpcore<3.5,==3.5.0,>=3.4 (from -r requirements.in (line 1))", "Tried: 3.0.0, 3.0.0, 3.0.1, 3.0.1, 3.1.0, 3.1.0, 3.1.1, 3.1.1, 3.2.0, 3.2.0, 3.2.1, 3.2.1, 3.3.0, 3.3.0, 3.3.1, 3.3.1, 3.4.0, 3.4.0, 3.4.1, 3.4.1, 3.5.0, 3.5.0", "Skipped pre-versions: 3.0.0a1.dev3, 3.0.0a1.dev3, 3.0.0a1.dev4, 3.0.0a1.dev4, 3.0.0a1.dev5, 3.0.0a1.dev5, 3.0.0a1, 3.0.0a1, 3.0.0a2, 3.0.0a2, 3.0.0a3, 3.0.0a3, 3.0.0a4, 3.0.0a4, 3.0.0a5, 3.0.0a5, 3.0.0a6, 3.0.0a6, 3.0.0a7, 3.0.0a7, 3.0.0a8, 3.0.0a8, 3.0.0a9, 3.0.0a9, 3.0.0a10, 3.0.0a10, 3.0.0a11, 3.0.0a11, 3.0.0a12, 3.0.0a12, 3.0.0a13, 3.0.0a13, 3.0.0a14, 3.0.0a14, 3.0.0a15, 3.0.0a15, 3.0.0a16, 3.0.0a16, 3.0.0a17, 3.0.0a17, 3.0.0a18, 3.0.0a18, 3.0.0a19, 3.0.0a19, 3.0.0a20, 3.0.0a20, 3.0.0a21, 3.0.0a21, 3.0.0a22, 3.0.0a22, 3.0.0a23, 3.0.0a23, 3.0.0a24, 3.0.0a24, 3.0.0a25, 3.0.0a25, 3.0.0a26, 3.0.0a26, 3.0.0a27, 3.0.0a27, 3.0.0a28, 3.0.0a28, 3.0.0a29, 3.0.0a29, 3.0.0b1, 3.0.0b1, 3.0.0b2, 3.0.0b2, 3.0.0b3, 3.0.0b3, 3.0.0b4, 3.0.0b4, 3.0.0b5, 3.0.0b5, 3.0.0b6, 3.0.0b6, 3.0.0b7, 3.0.0b7, 3.0.0b8, 3.0.0b8, 3.0.0b9, 3.0.0b9, 3.0.0b10, 3.0.0b10, 3.0.0b11, 3.0.0b11, 3.0.0b12, 3.0.0b12, 3.0.0b13, 3.0.0b13, 3.0.0b14, 3.0.0b14, 3.0.0b15, 3.0.0b15, 3.0.0b16, 3.0.0b16, 3.0.0b17, 3.0.0b18, 3.0.0b18, 3.0.0b19, 3.0.0b19, 3.0.0b20, 3.0.0b20, 3.0.0b21, 3.0.0b21, 3.0.0b22, 3.0.0b22, 3.0.0b23, 3.0.0b23, 3.0.0rc1, 3.0.0rc1, 3.0.0rc2, 3.0.0rc2, 3.0.0rc3, 3.0.0rc3, 3.0.0rc4, 3.0.0rc4, 3.0.0rc5, 3.0.0rc5, 3.0.0rc6, 3.0.0rc6, 3.0.0rc7, 3.0.0rc7, 3.0.0rc8, 3.0.0rc8, 3.0.0rc9, 3.0.0rc9", "There are incompatible versions in the resolved dependencies:", " pulpcore==3.5.0 (from -r requirements.in (line 1))", " pulpcore<3.5,>=3.4 (from pulp-rpm==3.4.1->-r requirements.in (line 2))"], "stdout": "", "stdout_lines": []}
PLAY RECAP ********************************************************************************************************************************************************************************************************
localhost : ok=48 changed=0 unreachable=0 failed=1 skipped=31 rescued=0 ignored=0
</code></pre>
<p>Here is the content of my Ansible playbook :</p>
<pre><code class="text syntaxhl" data-language="text">---
- hosts: all
vars:
pulp_install_plugins:
pulp-rpm:
prereq_role: "pulp.pulp_rpm_prerequisites"
pre_tasks:
# The version string below is the highest of all those in roles' metadata:
# "min_ansible_version". It needs to be kept manually up-to-date.
- name: Verify Ansible meets min required version
assert:
that: "ansible_version.full is version_compare('2.8', '>=')"
msg: >
"You must update Ansible to at least 2.8 to use this version of Pulp 3 Installer."
roles:
- pulp_all_services
environment:
DJANGO_SETTINGS_MODULE: pulpcore.app.settings
</code></pre>
<p>Prior to running the playbook, I downloaded the Ansible role as described in the RPM plugin documentation :</p>
<pre><code class="text syntaxhl" data-language="text">ansible-galaxy install -r requirements.yml
ansible-galaxy install pulp.pulp_rpm_prerequisites -p ./roles/
ansible-playbook playbooks/example-use/playbook.yml
</code></pre>
<p>I could install Pulp by commenting pulp-rpm vars in the playbook file, and I was able to install the prlp-rpm plugin manually (I had to install a lot more rpm packages than the ones described in <a href="https://pulp-rpm.readthedocs.io/en/3.4/installation.html" class="external">https://pulp-rpm.readthedocs.io/en/3.4/installation.html</a> otherwise the "pip install pulp-rpm" command was failing).</p>
<p>Would it be possible to have a look at the Pulp installer ?
Thanks a lot</p> RPM Support - Issue #7141 (CLOSED - WORKSFORME): lazy sync does not properly handle upstream repo...https://pulp.plan.io/issues/71412020-07-14T19:19:55Zjsherril@redhat.comjsherril@redhat.com
<p>Say you have two repos that contain the same rpm, but at different paths:</p>
<p>os
/Packages/f/foo.rpm</p>
<p>ks
/Packages/foo.rpm</p>
<p>Now you sync them both using 'on_demand' , but lets say the os repo gets the unit imported first. The rpm unit gets created with a relativepath of:</p>
<p>/Packages/f/foo.rpm</p>
<p>and then a lazy_catalog_content entry gets created with a url of:
<a href="https://server.example.com/os//Packages/f/foo.rpm" class="external">https://server.example.com/os//Packages/f/foo.rpm</a></p>
<p>This is all correct, now the unit gets processsed for the ks repo. It correctly reuses the same unit, but then creates a 2nd lazy_catalog_content entry with a url of:
<a href="https://server.example.com/ks/Packages/f/foo.rpm" class="external">https://server.example.com/ks/Packages/f/foo.rpm</a></p>
<p>Its using the relativepath of the rpm unit to build the lazy_catalog_content's url attribute. In reality this looks like:</p>
<pre><code>> db.lazy_content_catalog.find({"path": {$regex: '.*libXxf86vm\-devel\-1\.1\.4\-9\.el8\.i686\.rpm'}})
{ "_id" : ObjectId("5f07ee48cc531034cce38acc"), "_ns" : "lazy_content_catalog", "path" : "/var/lib/pulp/content/units/rpm/8a/cd9d02545dff8fab381aaa6185a778a26cacbec1585bcd8f7b2f6509f254a2/libXxf86vm-devel-1.1.4-9.el8.i686.rpm", "importer_id" : "5f07ed47cc53103b7b1f02c9", "unit_id" : "305ec066-9d0f-46a7-a198-6b966218a40e", "unit_type_id" : "rpm", "url" : "https://cdn.redhat.com/content/dist/rhel8/8.2/x86_64/appstream/kickstart/Packages/libXxf86vm-devel-1.1.4-9.el8.i686.rpm", "checksum" : "e375334723b40b39a407d243d1dab859a6edf1b2b383faa68c257c1afb399e2f", "checksum_algorithm" : "sha256", "revision" : 1, "data" : { } }
{ "_id" : ObjectId("5f07ef17cc531034b8afd793"), "_ns" : "lazy_content_catalog", "path" : "/var/lib/pulp/content/units/rpm/8a/cd9d02545dff8fab381aaa6185a778a26cacbec1585bcd8f7b2f6509f254a2/libXxf86vm-devel-1.1.4-9.el8.i686.rpm", "importer_id" : "5f07ed0dcc53103b7b1f02b5", "unit_id" : "305ec066-9d0f-46a7-a198-6b966218a40e", "unit_type_id" : "rpm", "url" : "https://cdn.redhat.com/content/dist/rhel8/8/x86_64/appstream/os/Packages/libXxf86vm-devel-1.1.4-9.el8.i686.rpm", "checksum" : "e375334723b40b39a407d243d1dab859a6edf1b2b383faa68c257c1afb399e2f", "checksum_algorithm" : "sha256", "revision" : 1, "data" : { } }
</code></pre>
<p>Directions to reproduce:</p>
<ol>
<li>Sync the rhel 8 base os repo using on_demand</li>
<li>Sync the rhel 8 kickstart repo using on_demand</li>
</ol>
<p>attempt to fetch each rpm from the kickstart repo or base os repo (maybe a random assortment of each)</p>
<p>Results, you will get a lot of 404s from the streamer app:</p>
<p>Jul 13 17:19:35 dhcp-8-30-46 pulp_streamer: pulp.streamer.server:INFO: Download failed [404]: <a href="https://cdn.redhat.com/content/dist/rhel8/8/x86_64/appstream/os/Packages/texlive-luatex85-20180414-14.el8.noarch.rpm" class="external">https://cdn.redhat.com/content/dist/rhel8/8/x86_64/appstream/os/Packages/texlive-luatex85-20180414-14.el8.noarch.rpm</a></p>
<p>This is because its using the wrong relative path when fetching rpms from the kickstart repo. Its non-deterministic as to which lazy_content_catalogue entry it will pick, so some will get a 404 and some won't. Re-trying to download an rpm again, may result it in working.</p> RPM Support - Issue #7130 (CLOSED - WORKSFORME): Orphan cleanup removes kickstart contenthttps://pulp.plan.io/issues/71302020-07-14T11:37:57Zdaviddavis
<p>I haven't tested this out but looking <a href="https://github.com/pulp/pulpcore/blob/f3147c20c0c188026b92c41055af10a9ed9f58af/pulpcore/app/tasks/orphan.py#L17-L19" class="external">at the orphan cleanup code</a> it seems likely that orphan cleanup will remove content that belongs to a distribution tree since this content is not tied directly to a RepositoryVersion.</p> RPM Support - Issue #7000 (CLOSED - WORKSFORME): Impossible to update a comps.xmlhttps://pulp.plan.io/issues/70002020-06-18T12:35:48Zmichele.santuari@wuerth-phoenix.com
<p>As a user, I want to update an existing comps.xml by adding a new rpm.</p>
<p>The steps that I did are:</p>
<ol>
<li>add the rpm telegraf to this existing comps.xml:</li>
</ol>
<pre><code class="xml syntaxhl" data-language="xml"><span class="cp"><?xml version="1.0" encoding="UTF-8"?></span>
<span class="cp"><!DOCTYPE comps PUBLIC "-//CentOS//DTD Comps info//EN" "comps.dtd"></span>
<span class="nt"><comps></span>
<span class="nt"><group></span>
<span class="nt"><id></span>test<span class="nt"></id></span>
<span class="nt"><name></span>Test Pulp Comps<span class="nt"></name></span>
<span class="nt"><packagelist></span>
<span class="nt"><packagereq</span> <span class="na">type=</span><span class="s">"mandatory"</span><span class="nt">></span>neteye-agent<span class="nt"></packagereq></span>
<span class="nt"><packagereq</span> <span class="na">type=</span><span class="s">"mandatory"</span><span class="nt">></span>telegraf<span class="nt"></packagereq></span>
<span class="nt"></packagelist></span>
<span class="nt"></group></span>
<span class="nt"></comps></span>
</code></pre>
<ol>
<li>Upload the "new" comps.xml and publish the repo:</li>
</ol>
<pre><code class="bash syntaxhl" data-language="bash">pulp-admin rpm repo uploads comps <span class="nt">--repo-id</span><span class="o">=</span>extras <span class="nt">--file</span> /root/comps/comps.xml
pulp-admin rpm repo publish run <span class="nt">--repo-id</span><span class="o">=</span>extras
</code></pre>
<ol>
<li>List the content on the group and telegraf is not listed.</li>
</ol>
<pre><code class="bash syntaxhl" data-language="bash">ulp-admin rpm repo content group <span class="nt">--repo-id</span><span class="o">=</span>extras
Conditional Package Names:
Default Package Names:
Description: <span class="nb">test </span>Packages
Id: <span class="nb">test
</span>Mandatory Package Names: neteye-agent
Name: <span class="nb">test
</span>Optional Package Names:
User Visible: True
</code></pre> RPM Support - Issue #6945 (CLOSED - WORKSFORME): Sync error: Response payload is not completedhttps://pulp.plan.io/issues/69452020-06-10T08:24:57Zttereshcttereshc@redhat.com
<p>On Pulp 3.3.2 during sync the followin error occurs.</p>
<p>Repo: rhel-8.1-for-x86_64-appstream-debug-rpms
Error: Response payload is not completed</p> RPM Support - Issue #6944 (CLOSED - WORKSFORME): Sync error: No declared artifact with relative p...https://pulp.plan.io/issues/69442020-06-10T08:23:15Zttereshcttereshc@redhat.com
<p>Tested on pulp_rpm 3.3.2.
Sync issue with the following RH repos.</p>
<p>rhel-server-rhscl-7.7-eus-rpms<br>
No declared artifact with relative path "repodata/productid" for content "<RepoMetadataFile: pk=d453c6c2-ed45-49be-b4c8-40191d26521d>"</p>
<p>rhel-8.2-for-x86_64-appstream-rpms<br>
No declared artifact with relative path "repodata/a22b89e3-765e-45bf-97f0-60e53435f3e5" for content "<RepoMetadataFile: pk=66acc8be-c241-44de-a393-79967e724920>"</p>
<p>rhel-8.1-for-x86_64-appstream-rpms<br>
No declared artifact with relative path "repodata/ccbdfec4-421c-4cfe-86aa-887da978e8ac" for content "<RepoMetadataFile: pk=2682dc8e-4c92-45cb-acc5-d609efd71e37>"</p>
<p>rhel-server-rhscl-7.7-eus-rpms<br>
No declared artifact with relative path "repodata/productid" for content "<RepoMetadataFile: pk=d453c6c2-ed45-49be-b4c8-40191d26521d>"</p>
<p>rhel-8.2-for-x86_64-appstream-rpms<br>
No declared artifact with relative path "repodata/a22b89e3-765e-45bf-97f0-60e53435f3e5" for content "<RepoMetadataFile: pk=66acc8be-c241-44de-a393-79967e724920>"</p>
<p>rhel-8.1-for-x86_64-appstream-rpms<br>
No declared artifact with relative path "repodata/ccbdfec4-421c-4cfe-86aa-887da978e8ac" for content "<RepoMetadataFile: pk=2682dc8e-4c92-45cb-acc5-d609efd71e37>"</p> RPM Support - Issue #6731 (CLOSED - WORKSFORME): WARNING: Please provide either 'view_name' or 'v...https://pulp.plan.io/issues/67312020-05-14T09:55:51Zdkliban@redhat.com