Project

Profile

Help

Issue #8245

closed

Distribution tree parsing fails if is_layered configuration parameter is present

Added by ttereshc about 3 years ago. Updated about 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 91
Quarter:
Q1-2021

Description

Try to sync or migrate a distribution tree which has is_layered set in its .treeinfo and it would lead to this error:

Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: Traceback (most recent call last):
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 975, in perform_job
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: rv = job.perform()
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 696, in perform
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: self._result = self._execute()
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 719, in _execute
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: return self.func(*self.args, **self.kwargs)
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/tasks/migrate.py", line 81, in migrate_from_pulp2
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: migrate_content(plan, skip_corrupted=skip_corrupted)
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/migration.py", line 47, in migrate_content
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: plugin.migrator.migrate_content_to_pulp3(skip_corrupted=skip_corrupted)
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 145, in migrate_content_to_pulp3
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: loop.run_until_complete(dm.create())
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: return future.result()
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/plugin/content.py", line 90, in create
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: await pipeline
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: await asyncio.gather(*futures)
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/api.py", line 43, in __call__
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: await self.run()
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/content_stages.py", line 95, in run
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: d_content.content.save()
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/home/vagrant/devel/pulpcore/pulpcore/app/models/base.py", line 149, in save
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: return super().save(*args, **kwargs)
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django_lifecycle/mixins.py", line 129, in save
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: save(*args, **kwargs)
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 744, in save
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: force_update=force_update, update_fields=update_fields)
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 782, in save_base
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: force_update, using, update_fields,
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 873, in _save_table
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 911, in _do_insert
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: using=using, raw=raw)
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: return getattr(self.get_queryset(), name)(*args, **kwargs)
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py", line 1186, in _insert
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: return query.get_compiler(using=using).execute_sql(return_id)
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1376, in execute_sql
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: for sql, params in self.as_sql():
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django_readonly_field/compiler.py", line 31, in as_sql
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: return super(ReadonlySQLCompilerMixin, self).as_sql()
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1320, in as_sql
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: for obj in self.query.objs
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1320, in <listcomp>
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: for obj in self.query.objs
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1319, in <listcomp>
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: [self.prepare_value(field, self.pre_save_val(field, obj)) for field in fields]
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1260, in prepare_value
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: value = field.get_db_prep_save(value, connection=self.connection)
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/fields/__init__.py", line 793, in get_db_prep_save
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: return self.get_db_prep_value(value, connection=connection, prepared=False)
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/fields/__init__.py", line 788, in get_db_prep_value
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: value = self.get_prep_value(value)
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/fields/__init__.py", line 1024, in get_prep_value
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: return self.to_python(value)
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/fields/__init__.py", line 1017, in to_python
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: params={'value': value},
Feb 11 18:54:31 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com rq[2338]: django.core.exceptions.ValidationError: ["'true' value must be either True or False."]

We are not using the main parser in pulp_rpm because we need to do a lot of data rejuggling ourselves. Main parser in productmd turned out to work fine, it's the one we use is not supposed to convert filed to their correct types.

A solution would be to handle this configuration parameter separately.


Related issues

Related to RPM Support - Backport #8374: backport 8245 to pulp-rpm 3.9CLOSED - CURRENTRELEASEttereshc

Actions
Actions #1

Updated by pulpbot about 3 years ago

  • Status changed from ASSIGNED to POST
Actions #3

Updated by dalley about 3 years ago

  • Priority changed from Normal to High
  • Triaged changed from No to Yes
Actions #4

Updated by rchan about 3 years ago

  • Sprint changed from Sprint 90 to Sprint 91

Added by ttereshc about 3 years ago

Revision da54c1df | View on GitHub

Fix parsing of is_layered configuration parameter for Distribution Trees

closes #8245 https://pulp.plan.io/issues/8245

Added by ggainey about 3 years ago

Revision c055198d | View on GitHub

Fix remaining path where is_layered could be a string.

re #8245

Actions #5

Updated by ttereshc about 3 years ago

  • Status changed from POST to MODIFIED
Actions #6

Updated by ttereshc about 3 years ago

Actions #7

Updated by dalley about 3 years ago

  • Sprint/Milestone set to 3.10.0
Actions #8

Updated by dalley about 3 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF