Actions
Issue #7836
closedTask #7960: FIPS and support for ALLOWED_CONTENT_CHECKSUMS
Import fails when ArtifactResource.json has blank checksums
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 87
Quarter:
Q4-2020
Description
The export delivers empty checksums as empty strings:
{
"file": "artifact/9c/3f67ddd806d47c09d563ef94b622cd5a1736552b2f5bfb4c71f98cebb14729",
"size": 1859,
"md5": "",
"sha1": "",
"sha224": "",
"sha256": "9c3f67ddd806d47c09d563ef94b622cd5a1736552b2f5bfb4c71f98cebb14729",
"sha384": "",
"sha512": "8b53cbae3f6b504981bc44d9653bc63dd21ded38ed48a168b50d515d745df13444ec0e5bddc53ed662f767d0905571eb754e4e4daa591c33ec6850bda1fb769e"
},
At import, this fails with an error like this one :
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: pulp: pulpcore.app.tasks.importer:INFO: ...Importing resource ArtifactResource.
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: pulp: rq.worker:ERROR: Traceback (most recent call last):
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: return self.cursor.execute(sql, params)
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "core_artifact_sha384_key"
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: DETAIL: Key (sha384)=() already exists.
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: The above exception was the direct cause of the following exception:
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: Traceback (most recent call last):
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/rq/worker.py", line 936, in perform_job
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: rv = job.perform()
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/rq/job.py", line 684, in perform
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: self._result = self._execute()
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/rq/job.py", line 690, in _execute
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: return self.func(*self.args, **self.kwargs)
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/home/vagrant/devel/pulpcore/pulpcore/app/tasks/importer.py", line 339, in pulp_import
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: ar_result = _import_file(os.path.join(temp_dir, ARTIFACT_FILE), ArtifactResource)
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/home/vagrant/devel/pulpcore/pulpcore/app/tasks/importer.py", line 61, in _import_file
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: return resource.import_data(data, raise_errors=True)
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/import_export/resources.py", line 745, in import_data
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: return self.import_data_inner(dataset, dry_run, raise_errors, using_transactions, collect_failed_rows, **kwargs)
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/import_export/resources.py", line 792, in import_data_inner
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: raise row_result.errors[-1].error
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/import_export/resources.py", line 672, in import_row
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: self.save_instance(instance, using_transactions, dry_run)
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/import_export/resources.py", line 450, in save_instance
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: instance.save()
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/home/vagrant/devel/pulpcore/pulpcore/app/models/content.py", line 140, in save
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: super().save(*args, **kwargs)
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django_lifecycle/mixins.py", line 129, in save
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: save(*args, **kwargs)
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django/db/models/base.py", line 743, in save
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: self.save_base(using=using, force_insert=force_insert,
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django/db/models/base.py", line 780, in save_base
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: updated = self._save_table(
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django/db/models/base.py", line 873, in _save_table
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django/db/models/base.py", line 910, in _do_insert
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: return manager._insert([self], fields=fields, return_id=update_pk,
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django/db/models/manager.py", line 82, in manager_method
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: return getattr(self.get_queryset(), name)(*args, **kwargs)
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django/db/models/query.py", line 1186, in _insert
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: return query.get_compiler(using=using).execute_sql(return_id)
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django/db/models/sql/compiler.py", line 1377, in execute_sql
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: cursor.execute(sql, params)
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django/db/backends/utils.py", line 67, in execute
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: return executor(sql, params, many, context)
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: return self.cursor.execute(sql, params)
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django/db/utils.py", line 89, in __exit__
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: raise dj_exc_value.with_traceback(traceback) from exc_value
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
Nov 14 20:38:12 pulp3-source-fedora32.padre-fedora.example.com rq[309851]: return self.cursor.execute(sql, params)
Related issues
Actions
Fix import/export bug when sha384 or sha512 are null
fixes #7836