Issue #4485: Failure when attempting to use single request upload for a rpm already present in pulp - RPM Support - Pulp

Actions

Send by e-mail Copy link

Issue #4485

closed

Failure when attempting to use single request upload for a rpm already present in pulp

Added by kersom about 5 years ago. Updated over 4 years ago.

Status:

CLOSED - CURRENTRELEASE

Priority:

Normal

Assignee:

ppicka

Sprint/Milestone:

Start date:

Due date:

Estimated time:

Severity:

2. Medium

Version:

Platform Release:

OS:

Triaged:

Yes

Groomed:

Sprint Candidate:

Tags:

Sprint:

Sprint 49

Quarter:

Description

1 - Create a repository
2 - Upload a RPM using single request upload to the previous repo.
3 - Attempt to upload - single request, the same RPM again to the same repo.

Traceback:

("Task report /pulp/api/v3/tasks/16/ contains a error: {'code': None, "
 "'description': 'duplicate key value violates unique constraint "
 '"pulp_app_contentartifact_content_id_relative_path_c7af024d_uniq"\\nDETAIL:  '
 'Key (content_id, relative_path)=(3, '
 'b0f0bb58be244393cc565e8ee5ef0ad36884d8ba8eec74542ff47d299a34c1) already '
 "exists.\\n', 'traceback': '  File "
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py", line 799, '
 'in perform_job\\n    rv = job.perform()\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 600, in '
 'perform\\n    self._result = self._execute()\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 606, in '
 '_execute\\n    return self.func(*self.args, **self.kwargs)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/upload.py", '
 'line 31, in one_shot_upload\\n    relative_path=filename\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/manager.py", '
 'line 82, in manager_method\\n    return getattr(self.get_queryset(), '
 'name)(*args, **kwargs)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/query.py", '
 'line 413, in create\\n    obj.save(force_insert=True, using=self.db)\\n  '
 'File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
 'line 718, in save\\n    force_update=force_update, '
 'update_fields=update_fields)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
 'line 748, in save_base\\n    updated = self._save_table(raw, cls, '
 'force_insert, force_update, using, update_fields)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
 'line 831, in _save_table\\n    result = self._do_insert(cls._base_manager, '
 'using, fields, update_pk, raw)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
 'line 869, in _do_insert\\n    using=using, raw=raw)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/manager.py", '
 'line 82, in manager_method\\n    return getattr(self.get_queryset(), '
 'name)(*args, **kwargs)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/query.py", '
 'line 1136, in _insert\\n    return '
 'query.get_compiler(using=using).execute_sql(return_id)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/sql/compiler.py", '
 'line 1289, in execute_sql\\n    cursor.execute(sql, params)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
 'line 68, in execute\\n    return self._execute_with_wrappers(sql, params, '
 'many=False, executor=self._execute)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
 'line 77, in _execute_with_wrappers\\n    return executor(sql, params, many, '
 'context)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
 'line 85, in _execute\\n    return self.cursor.execute(sql, params)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/utils.py", line '
 '89, in __exit__\\n    raise dj_exc_value.with_traceback(traceback) from '
 'exc_value\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
 "line 85, in _execute\\n    return self.cursor.execute(sql, params)\\n'}\n"
 "Full task report: {'_href': '/pulp/api/v3/tasks/16/', '_created': "
 "'2018-12-20T21:54:36.049068Z', 'job_id': "
 "'7a1e07a1-5cce-41bf-9ffd-4e71ddd2ddc3', 'state': 'failed', 'name': "
 "'pulp_rpm.app.upload.one_shot_upload', 'started_at': "
 "'2018-12-20T21:54:36.124283Z', 'finished_at': '2018-12-20T21:54:36.163181Z', "
 "'non_fatal_errors': [], 'error': {'code': None, 'description': 'duplicate "
 'key value violates unique constraint '
 '"pulp_app_contentartifact_content_id_relative_path_c7af024d_uniq"\\nDETAIL:  '
 'Key (content_id, relative_path)=(3, '
 'b0f0bb58be244393cc565e8ee5ef0ad36884d8ba8eec74542ff47d299a34c1) already '
 "exists.\\n', 'traceback': '  File "
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py", line 799, '
 'in perform_job\\n    rv = job.perform()\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 600, in '
 'perform\\n    self._result = self._execute()\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 606, in '
 '_execute\\n    return self.func(*self.args, **self.kwargs)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/upload.py", '
 'line 31, in one_shot_upload\\n    relative_path=filename\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/manager.py", '
 'line 82, in manager_method\\n    return getattr(self.get_queryset(), '
 'name)(*args, **kwargs)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/query.py", '
 'line 413, in create\\n    obj.save(force_insert=True, using=self.db)\\n  '
 'File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
 'line 718, in save\\n    force_update=force_update, '
 'update_fields=update_fields)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
 'line 748, in save_base\\n    updated = self._save_table(raw, cls, '
 'force_insert, force_update, using, update_fields)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
 'line 831, in _save_table\\n    result = self._do_insert(cls._base_manager, '
 'using, fields, update_pk, raw)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
 'line 869, in _do_insert\\n    using=using, raw=raw)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/manager.py", '
 'line 82, in manager_method\\n    return getattr(self.get_queryset(), '
 'name)(*args, **kwargs)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/query.py", '
 'line 1136, in _insert\\n    return '
 'query.get_compiler(using=using).execute_sql(return_id)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/sql/compiler.py", '
 'line 1289, in execute_sql\\n    cursor.execute(sql, params)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
 'line 68, in execute\\n    return self._execute_with_wrappers(sql, params, '
 'many=False, executor=self._execute)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
 'line 77, in _execute_with_wrappers\\n    return executor(sql, params, many, '
 'context)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
 'line 85, in _execute\\n    return self.cursor.execute(sql, params)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/utils.py", line '
 '89, in __exit__\\n    raise dj_exc_value.with_traceback(traceback) from '
 'exc_value\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
 "line 85, in _execute\\n    return self.cursor.execute(sql, params)\\n'}, "
 "'worker': '/pulp/api/v3/workers/1/', 'parent': None, 'spawned_tasks': [], "
 "'progress_reports': [], 'created_resources': []}",
 {'_created': '2018-12-20T21:54:36.049068Z',
  '_href': '/pulp/api/v3/tasks/16/',
  'created_resources': [],
  'error': {'code': None,
            'description': 'duplicate key value violates unique constraint '
                           '"pulp_app_contentartifact_content_id_relative_path_c7af024d_uniq"\n'
                           'DETAIL:  Key (content_id, relative_path)=(3, '
                           'b0f0bb58be244393cc565e8ee5ef0ad36884d8ba8eec74542ff47d299a34c1) '
                           'already exists.\n',
            'traceback': '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py", '
                         'line 799, in perform_job\n'
                         '    rv = job.perform()\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", '
                         'line 600, in perform\n'
                         '    self._result = self._execute()\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", '
                         'line 606, in _execute\n'
                         '    return self.func(*self.args, **self.kwargs)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/upload.py", '
                         'line 31, in one_shot_upload\n'
                         '    relative_path=filename\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/manager.py", '
                         'line 82, in manager_method\n'
                         '    return getattr(self.get_queryset(), name)(*args, '
                         '**kwargs)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/query.py", '
                         'line 413, in create\n'
                         '    obj.save(force_insert=True, using=self.db)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
                         'line 718, in save\n'
                         '    force_update=force_update, '
                         'update_fields=update_fields)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
                         'line 748, in save_base\n'
                         '    updated = self._save_table(raw, cls, '
                         'force_insert, force_update, using, update_fields)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
                         'line 831, in _save_table\n'
                         '    result = self._do_insert(cls._base_manager, '
                         'using, fields, update_pk, raw)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
                         'line 869, in _do_insert\n'
                         '    using=using, raw=raw)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/manager.py", '
                         'line 82, in manager_method\n'
                         '    return getattr(self.get_queryset(), name)(*args, '
                         '**kwargs)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/query.py", '
                         'line 1136, in _insert\n'
                         '    return '
                         'query.get_compiler(using=using).execute_sql(return_id)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/sql/compiler.py", '
                         'line 1289, in execute_sql\n'
                         '    cursor.execute(sql, params)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
                         'line 68, in execute\n'
                         '    return self._execute_with_wrappers(sql, params, '
                         'many=False, executor=self._execute)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
                         'line 77, in _execute_with_wrappers\n'
                         '    return executor(sql, params, many, context)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
                         'line 85, in _execute\n'
                         '    return self.cursor.execute(sql, params)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/utils.py", '
                         'line 89, in __exit__\n'
                         '    raise dj_exc_value.with_traceback(traceback) '
                         'from exc_value\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
                         'line 85, in _execute\n'
                         '    return self.cursor.execute(sql, params)\n'},
  'finished_at': '2018-12-20T21:54:36.163181Z',
  'job_id': '7a1e07a1-5cce-41bf-9ffd-4e71ddd2ddc3',
  'name': 'pulp_rpm.app.upload.one_shot_upload',
  'non_fatal_errors': [],
  'parent': None,
  'progress_reports': [],
  'spawned_tasks': [],
  'started_at': '2018-12-20T21:54:36.124283Z',
  'state': 'failed',
  'worker': '/pulp/api/v3/workers/1/'})

Pulp 3 - Installation from source FEB/27.

Related issues

Actions

Copy link

Updated by kersom about 5 years ago

Description updated (diff)

Actions

Copy link

Updated by kersom about 5 years ago

Related to Story #4087: As a user, I can upload an rpm file with a single request added

Actions

Copy link

Updated by daviddavis about 5 years ago

Project changed from Pulp to RPM Support

Actions

Copy link

Updated by ppicka about 5 years ago

Status changed from NEW to ASSIGNED
Assignee set to ppicka

Actions

Copy link

Updated by ttereshc about 5 years ago

Triaged changed from No to Yes
Sprint set to Sprint 49

Added by ppicka about 5 years ago

Revision 5185efcc | View on GitHub

One shot package duplicate check

Check for package as already exists in pulp. One line of doc string removed as not used.

closes: #4485 https://pulp.plan.io/issues/4485

Signed-off-by: Pavel Picka ppicka@redhat.com

Actions

Copy link

Updated by ppicka about 5 years ago

Status changed from ASSIGNED to POST

https://github.com/pulp/pulp_rpm/pull/1292

Actions

Copy link

Updated by ppicka about 5 years ago

Status changed from POST to MODIFIED

Applied in changeset 5185efcc318f7acafc2bb5d511100e0db8e2b649.

Actions

Copy link

Updated by bherring about 5 years ago

Copied to Test #4536: Failure when attempting to use single request upload for a rpm already present in pulp added

Actions

Copy link

Updated by kersom about 5 years ago

After new code was added this is the traceback when using single request upload for a file already in Pulp.

'Task report /pulp/api/v3/tasks/4e7f2a48-18d7-47a1-910e-15cb1b4e2144/ '
 "contains a error: {'code': None, 'description': 'RPM package "
 "bear-4.1-1.noarch.rpm already exists.', 'traceback': '  File "
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py", line 799, '
 'in perform_job\\n    rv = job.perform()\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 600, in '
 'perform\\n    self._result = self._execute()\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 606, in '
 '_execute\\n    return self.func(*self.args, **self.kwargs)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/upload.py", '
 "line 29, in one_shot_upload\\n    raise OSError(\\'RPM package {} already "
 "exists.\\'.format(pkg.filename))\\n'}\n"
 "Full task report: {'_href': "
 "'/pulp/api/v3/tasks/4e7f2a48-18d7-47a1-910e-15cb1b4e2144/', '_created': "
 "'2018-12-21T04:18:54.417596Z', 'state': 'failed', 'name': "
 "'pulp_rpm.app.upload.one_shot_upload', 'started_at': "
 "'2018-12-21T04:18:54.508100Z', 'finished_at': '2018-12-21T04:18:54.536285Z', "
 "'non_fatal_errors': [], 'error': {'code': None, 'description': 'RPM package "
 "bear-4.1-1.noarch.rpm already exists.', 'traceback': '  File "
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py", line 799, '
 'in perform_job\\n    rv = job.perform()\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 600, in '
 'perform\\n    self._result = self._execute()\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 606, in '
 '_execute\\n    return self.func(*self.args, **self.kwargs)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/upload.py", '
 "line 29, in one_shot_upload\\n    raise OSError(\\'RPM package {} already "
 "exists.\\'.format(pkg.filename))\\n'}, 'worker': "
 "'/pulp/api/v3/workers/4e514866-3a11-4e32-a03e-a9019207e764/', 'parent': "
 "None, 'spawned_tasks': [], 'progress_reports': [], 'created_resources': []}",
 {'_created': '2018-12-21T04:18:54.417596Z',
  '_href': '/pulp/api/v3/tasks/4e7f2a48-18d7-47a1-910e-15cb1b4e2144/',
  'created_resources': [],
  'error': {'code': None,
            'description': 'RPM package bear-4.1-1.noarch.rpm already exists.',
            'traceback': '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py", '
                         'line 799, in perform_job\n'
                         '    rv = job.perform()\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", '
                         'line 600, in perform\n'
                         '    self._result = self._execute()\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", '
                         'line 606, in _execute\n'
                         '    return self.func(*self.args, **self.kwargs)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/upload.py", '
                         'line 29, in one_shot_upload\n'
                         "    raise OSError('RPM package {} already "
                         "exists.'.format(pkg.filename))\n"},
  'finished_at': '2018-12-21T04:18:54.536285Z',
  'name': 'pulp_rpm.app.upload.one_shot_upload',
  'non_fatal_errors': [],
  'parent': None,
  'progress_reports': [],
  'spawned_tasks': [],
  'started_at': '2018-12-21T04:18:54.508100Z',
  'state': 'failed',
  'worker': '/pulp/api/v3/workers/4e514866-3a11-4e32-a03e-a9019207e764/'})

In other cases when Pulp is dealing with duplicate files a 400 HTTP error is raised, and a message error is sent back.

See: https://pulp.plan.io/issues/3436

Any reason for this one behave different?

Actions

Copy link

#10