Pulp

When testing fix for https://pulp.plan.io/issues/8700 (which actually works, thx!), I rebuild our pulp rpm package and tried to sync centos 8 stream "appstream" repository. However sync failed with following traceback:

May  7 12:58:41 pulp3-dev02 rq[2259857]: Traceback (most recent call last):
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/rq/worker.py", line 1008, in perform_job
May  7 12:58:41 pulp3-dev02 rq[2259857]:    rv = job.perform()
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/rq/job.py", line 706, in perform
May  7 12:58:41 pulp3-dev02 rq[2259857]:    self._result = self._execute()
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/rq/job.py", line 729, in _execute
May  7 12:58:41 pulp3-dev02 rq[2259857]:    result = self.func(*self.args, **self.kwargs)
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 270, in synchronize
May  7 12:58:41 pulp3-dev02 rq[2259857]:    dv.create()
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 149, in create
May  7 12:58:41 pulp3-dev02 rq[2259857]:    loop.run_until_complete(pipeline)
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
May  7 12:58:41 pulp3-dev02 rq[2259857]:    return future.result()
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
May  7 12:58:41 pulp3-dev02 rq[2259857]:    await asyncio.gather(*futures)
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
May  7 12:58:41 pulp3-dev02 rq[2259857]:    await self.run()
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 241, in run
May  7 12:58:41 pulp3-dev02 rq[2259857]:    d_artifact.artifact for d_artifact in da_to_save
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulpcore/app/models/content.py", line 82, in bulk_get_or_create
May  7 12:58:41 pulp3-dev02 rq[2259857]:    return super().bulk_create(objs, batch_size=batch_size)
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
May  7 12:58:41 pulp3-dev02 rq[2259857]:    return getattr(self.get_queryset(), name)(*args, **kwargs)
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/django/db/models/query.py", line 468, in bulk_create
May  7 12:58:41 pulp3-dev02 rq[2259857]:    self._batched_insert(objs_with_pk, fields, batch_size, ignore_conflicts=ignore_conflicts)
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/django/db/models/query.py", line 1204, in _batched_insert
May  7 12:58:41 pulp3-dev02 rq[2259857]:    ignore_conflicts=ignore_conflicts,
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/django/db/models/query.py", line 1186, in _insert
May  7 12:58:41 pulp3-dev02 rq[2259857]:    return query.get_compiler(using=using).execute_sql(return_id)
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1376, in execute_sql
May  7 12:58:41 pulp3-dev02 rq[2259857]:    for sql, params in self.as_sql():
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/django_readonly_field/compiler.py", line 31, in as_sql
May  7 12:58:41 pulp3-dev02 rq[2259857]:    return super(ReadonlySQLCompilerMixin, self).as_sql()
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1320, in as_sql
May  7 12:58:41 pulp3-dev02 rq[2259857]:    for obj in self.query.objs
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1320, in <listcomp>
May  7 12:58:41 pulp3-dev02 rq[2259857]:    for obj in self.query.objs
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1319, in <listcomp>
May  7 12:58:41 pulp3-dev02 rq[2259857]:    [self.prepare_value(field, self.pre_save_val(field, obj)) for field in fields]
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1270, in pre_save_val
May  7 12:58:41 pulp3-dev02 rq[2259857]:    return field.pre_save(obj, add=True)
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulpcore/app/models/fields.py", line 68, in pre_save
May  7 12:58:41 pulp3-dev02 rq[2259857]:    return super().pre_save(model_instance, add)
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/django/db/models/fields/files.py", line 289, in pre_save
May  7 12:58:41 pulp3-dev02 rq[2259857]:    file.save(file.name, file.file, save=False)
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/django/db/models/fields/files.py", line 87, in save
May  7 12:58:41 pulp3-dev02 rq[2259857]:    name = self.field.generate_filename(self.instance, name)
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/django/db/models/fields/files.py", line 303, in generate_filename
May  7 12:58:41 pulp3-dev02 rq[2259857]:    filename = validate_file_name(filename)
May  7 12:58:41 pulp3-dev02 rq[2259857]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/django/core/files/utils.py", line 8, in validate_file_name
May  7 12:58:41 pulp3-dev02 rq[2259857]:    raise SuspiciousFileOperation("File name '%s' includes path elements" % name)
May  7 12:58:41 pulp3-dev02 rq[2259857]: django.core.exceptions.SuspiciousFileOperation: File name '/mnt/tmp/pulp/2259857@pulp3-dev02.int.na.intgdc.com/a67d1c18-523a-4fa9-b2f8-56bef773ca69/tmpkbcud43d' includes path elements

After digging and troubleshooting, our old and new "all-in-one" pulp rpm, I found following difference:

diff -urp old/opt/gdc/pulp3/lib/python3.6/site-packages/django/__init__.py new/opt/gdc/pulp3/lib/python3.6/site-packages/django/__init__.py
--- old/opt/gdc/pulp3/lib/python3.6/site-packages/django/__init__.py    2021-05-07 13:29:30.819399222 +0200
+++ new/opt/gdc/pulp3/lib/python3.6/site-packages/django/__init__.py    2021-05-07 13:29:42.689538260 +0200
@@ -1,6 +1,6 @@
 from django.utils.version import get_version

-VERSION = (2, 2, 20, 'final', 0)
+VERSION = (2, 2, 22, 'final', 0)

 __version__ = get_version(VERSION)

There are quite a lot of changes related to SuspiciousFileOperation between 2.2.20 and 2.2.22 , so I assume some of this change broke current pulpcore. Please note that we use S3 for artifact storage and have following in our settings.py:

[root@pulp3-dev02:~] cat /etc/pulp/settings.py
SECRET_KEY = 'secret'
CONTENT_ORIGIN = 'http://localhost:24816'

WORKING_DIRECTORY = '/mnt/tmp/pulp'
FILE_UPLOAD_TEMP_DIR = '/mnt/tmp/pulp'

ALLOWED_CONTENT_CHECKSUMS = ["sha1", "sha224", "sha256", "sha384", "sha512"]

# S3 based storage
MEDIA_ROOT = ''
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
AWS_ACCESS_KEY_ID = 'secret'
AWS_SECRET_ACCESS_KEY = 'secret'
AWS_STORAGE_BUCKET_NAME = 'secret'
AWS_DEFAULT_ACL = "@none None"
S3_USE_SIGV4 = True
AWS_S3_SIGNATURE_VERSION = "s3v4"
AWS_S3_ADDRESSING_STYLE = "path"
AWS_S3_REGION_NAME = "secret"

DATABASES = {
...
}

When I downgraded django from 2.2.22 to 2.2.20, everything works like a charm.

Please let me know if you need more information, thank you!

Regards, Adam