Project

Profile

Help

Issue #4485

Failure when attempting to use single request upload for a rpm already present in pulp

Added by kersom 10 months ago. Updated about 5 hours ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Severity:
2. Medium
Version:
Platform Release:
Blocks Release:
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 49

Description

1 - Create a repository
2 - Upload a RPM using single request upload to the previous repo.
3 - Attempt to upload - single request, the same RPM again to the same repo.

Traceback:

("Task report /pulp/api/v3/tasks/16/ contains a error: {'code': None, " 
 "'description': 'duplicate key value violates unique constraint " 
 '"pulp_app_contentartifact_content_id_relative_path_c7af024d_uniq"\\nDETAIL:  '
 'Key (content_id, relative_path)=(3, '
 'b0f0bb58be244393cc565e8ee5ef0ad36884d8ba8eec74542ff47d299a34c1) already '
 "exists.\\n', 'traceback': '  File " 
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py", line 799, '
 'in perform_job\\n    rv = job.perform()\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 600, in '
 'perform\\n    self._result = self._execute()\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 606, in '
 '_execute\\n    return self.func(*self.args, **self.kwargs)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/upload.py", '
 'line 31, in one_shot_upload\\n    relative_path=filename\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/manager.py", '
 'line 82, in manager_method\\n    return getattr(self.get_queryset(), '
 'name)(*args, **kwargs)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/query.py", '
 'line 413, in create\\n    obj.save(force_insert=True, using=self.db)\\n  '
 'File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
 'line 718, in save\\n    force_update=force_update, '
 'update_fields=update_fields)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
 'line 748, in save_base\\n    updated = self._save_table(raw, cls, '
 'force_insert, force_update, using, update_fields)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
 'line 831, in _save_table\\n    result = self._do_insert(cls._base_manager, '
 'using, fields, update_pk, raw)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
 'line 869, in _do_insert\\n    using=using, raw=raw)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/manager.py", '
 'line 82, in manager_method\\n    return getattr(self.get_queryset(), '
 'name)(*args, **kwargs)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/query.py", '
 'line 1136, in _insert\\n    return '
 'query.get_compiler(using=using).execute_sql(return_id)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/sql/compiler.py", '
 'line 1289, in execute_sql\\n    cursor.execute(sql, params)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
 'line 68, in execute\\n    return self._execute_with_wrappers(sql, params, '
 'many=False, executor=self._execute)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
 'line 77, in _execute_with_wrappers\\n    return executor(sql, params, many, '
 'context)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
 'line 85, in _execute\\n    return self.cursor.execute(sql, params)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/utils.py", line '
 '89, in __exit__\\n    raise dj_exc_value.with_traceback(traceback) from '
 'exc_value\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
 "line 85, in _execute\\n    return self.cursor.execute(sql, params)\\n'}\n" 
 "Full task report: {'_href': '/pulp/api/v3/tasks/16/', '_created': " 
 "'2018-12-20T21:54:36.049068Z', 'job_id': " 
 "'7a1e07a1-5cce-41bf-9ffd-4e71ddd2ddc3', 'state': 'failed', 'name': " 
 "'pulp_rpm.app.upload.one_shot_upload', 'started_at': " 
 "'2018-12-20T21:54:36.124283Z', 'finished_at': '2018-12-20T21:54:36.163181Z', " 
 "'non_fatal_errors': [], 'error': {'code': None, 'description': 'duplicate " 
 'key value violates unique constraint '
 '"pulp_app_contentartifact_content_id_relative_path_c7af024d_uniq"\\nDETAIL:  '
 'Key (content_id, relative_path)=(3, '
 'b0f0bb58be244393cc565e8ee5ef0ad36884d8ba8eec74542ff47d299a34c1) already '
 "exists.\\n', 'traceback': '  File " 
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py", line 799, '
 'in perform_job\\n    rv = job.perform()\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 600, in '
 'perform\\n    self._result = self._execute()\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 606, in '
 '_execute\\n    return self.func(*self.args, **self.kwargs)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/upload.py", '
 'line 31, in one_shot_upload\\n    relative_path=filename\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/manager.py", '
 'line 82, in manager_method\\n    return getattr(self.get_queryset(), '
 'name)(*args, **kwargs)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/query.py", '
 'line 413, in create\\n    obj.save(force_insert=True, using=self.db)\\n  '
 'File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
 'line 718, in save\\n    force_update=force_update, '
 'update_fields=update_fields)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
 'line 748, in save_base\\n    updated = self._save_table(raw, cls, '
 'force_insert, force_update, using, update_fields)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
 'line 831, in _save_table\\n    result = self._do_insert(cls._base_manager, '
 'using, fields, update_pk, raw)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
 'line 869, in _do_insert\\n    using=using, raw=raw)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/manager.py", '
 'line 82, in manager_method\\n    return getattr(self.get_queryset(), '
 'name)(*args, **kwargs)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/query.py", '
 'line 1136, in _insert\\n    return '
 'query.get_compiler(using=using).execute_sql(return_id)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/sql/compiler.py", '
 'line 1289, in execute_sql\\n    cursor.execute(sql, params)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
 'line 68, in execute\\n    return self._execute_with_wrappers(sql, params, '
 'many=False, executor=self._execute)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
 'line 77, in _execute_with_wrappers\\n    return executor(sql, params, many, '
 'context)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
 'line 85, in _execute\\n    return self.cursor.execute(sql, params)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/utils.py", line '
 '89, in __exit__\\n    raise dj_exc_value.with_traceback(traceback) from '
 'exc_value\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
 "line 85, in _execute\\n    return self.cursor.execute(sql, params)\\n'}, " 
 "'worker': '/pulp/api/v3/workers/1/', 'parent': None, 'spawned_tasks': [], " 
 "'progress_reports': [], 'created_resources': []}",
 {'_created': '2018-12-20T21:54:36.049068Z',
  '_href': '/pulp/api/v3/tasks/16/',
  'created_resources': [],
  'error': {'code': None,
            'description': 'duplicate key value violates unique constraint '
                           '"pulp_app_contentartifact_content_id_relative_path_c7af024d_uniq"\n'
                           'DETAIL:  Key (content_id, relative_path)=(3, '
                           'b0f0bb58be244393cc565e8ee5ef0ad36884d8ba8eec74542ff47d299a34c1) '
                           'already exists.\n',
            'traceback': '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py", '
                         'line 799, in perform_job\n'
                         '    rv = job.perform()\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", '
                         'line 600, in perform\n'
                         '    self._result = self._execute()\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", '
                         'line 606, in _execute\n'
                         '    return self.func(*self.args, **self.kwargs)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/upload.py", '
                         'line 31, in one_shot_upload\n'
                         '    relative_path=filename\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/manager.py", '
                         'line 82, in manager_method\n'
                         '    return getattr(self.get_queryset(), name)(*args, '
                         '**kwargs)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/query.py", '
                         'line 413, in create\n'
                         '    obj.save(force_insert=True, using=self.db)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
                         'line 718, in save\n'
                         '    force_update=force_update, '
                         'update_fields=update_fields)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
                         'line 748, in save_base\n'
                         '    updated = self._save_table(raw, cls, '
                         'force_insert, force_update, using, update_fields)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
                         'line 831, in _save_table\n'
                         '    result = self._do_insert(cls._base_manager, '
                         'using, fields, update_pk, raw)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", '
                         'line 869, in _do_insert\n'
                         '    using=using, raw=raw)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/manager.py", '
                         'line 82, in manager_method\n'
                         '    return getattr(self.get_queryset(), name)(*args, '
                         '**kwargs)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/query.py", '
                         'line 1136, in _insert\n'
                         '    return '
                         'query.get_compiler(using=using).execute_sql(return_id)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/sql/compiler.py", '
                         'line 1289, in execute_sql\n'
                         '    cursor.execute(sql, params)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
                         'line 68, in execute\n'
                         '    return self._execute_with_wrappers(sql, params, '
                         'many=False, executor=self._execute)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
                         'line 77, in _execute_with_wrappers\n'
                         '    return executor(sql, params, many, context)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
                         'line 85, in _execute\n'
                         '    return self.cursor.execute(sql, params)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/utils.py", '
                         'line 89, in __exit__\n'
                         '    raise dj_exc_value.with_traceback(traceback) '
                         'from exc_value\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/backends/utils.py", '
                         'line 85, in _execute\n'
                         '    return self.cursor.execute(sql, params)\n'},
  'finished_at': '2018-12-20T21:54:36.163181Z',
  'job_id': '7a1e07a1-5cce-41bf-9ffd-4e71ddd2ddc3',
  'name': 'pulp_rpm.app.upload.one_shot_upload',
  'non_fatal_errors': [],
  'parent': None,
  'progress_reports': [],
  'spawned_tasks': [],
  'started_at': '2018-12-20T21:54:36.124283Z',
  'state': 'failed',
  'worker': '/pulp/api/v3/workers/1/'})

Pulp 3 - Installation from source FEB/27.


Related issues

Related to RPM Support - Story #4087: As a user, I can upload an rpm file with a single request CLOSED - CURRENTRELEASE Actions
Copied to RPM Support - Test #4536: Failure when attempting to use single request upload for a rpm already present in pulp CLOSED - COMPLETE Actions

Associated revisions

Revision 5185efcc View on GitHub
Added by ppicka 9 months ago

One shot package duplicate check

Check for package as already exists in pulp.
One line of doc string removed as not used.

closes: #4485
https://pulp.plan.io/issues/4485

Signed-off-by: Pavel Picka <>

History

#1 Updated by kersom 10 months ago

  • Description updated (diff)

#2 Updated by kersom 10 months ago

  • Related to Story #4087: As a user, I can upload an rpm file with a single request added

#3 Updated by daviddavis 10 months ago

  • Project changed from Pulp to RPM Support

#4 Updated by ppicka 10 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ppicka

#5 Updated by ttereshc 10 months ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 49

#6 Updated by ppicka 9 months ago

  • Status changed from ASSIGNED to POST

#7 Updated by ppicka 9 months ago

  • Status changed from POST to MODIFIED

#8 Updated by bherring 9 months ago

  • Copied to Test #4536: Failure when attempting to use single request upload for a rpm already present in pulp added

#9 Updated by kersom 8 months ago

After new code was added this is the traceback when using single request upload for a file already in Pulp.

'Task report /pulp/api/v3/tasks/4e7f2a48-18d7-47a1-910e-15cb1b4e2144/ '
 "contains a error: {'code': None, 'description': 'RPM package " 
 "bear-4.1-1.noarch.rpm already exists.', 'traceback': '  File " 
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py", line 799, '
 'in perform_job\\n    rv = job.perform()\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 600, in '
 'perform\\n    self._result = self._execute()\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 606, in '
 '_execute\\n    return self.func(*self.args, **self.kwargs)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/upload.py", '
 "line 29, in one_shot_upload\\n    raise OSError(\\'RPM package {} already " 
 "exists.\\'.format(pkg.filename))\\n'}\n" 
 "Full task report: {'_href': " 
 "'/pulp/api/v3/tasks/4e7f2a48-18d7-47a1-910e-15cb1b4e2144/', '_created': " 
 "'2018-12-21T04:18:54.417596Z', 'state': 'failed', 'name': " 
 "'pulp_rpm.app.upload.one_shot_upload', 'started_at': " 
 "'2018-12-21T04:18:54.508100Z', 'finished_at': '2018-12-21T04:18:54.536285Z', " 
 "'non_fatal_errors': [], 'error': {'code': None, 'description': 'RPM package " 
 "bear-4.1-1.noarch.rpm already exists.', 'traceback': '  File " 
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py", line 799, '
 'in perform_job\\n    rv = job.perform()\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 600, in '
 'perform\\n    self._result = self._execute()\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 606, in '
 '_execute\\n    return self.func(*self.args, **self.kwargs)\\n  File '
 '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/upload.py", '
 "line 29, in one_shot_upload\\n    raise OSError(\\'RPM package {} already " 
 "exists.\\'.format(pkg.filename))\\n'}, 'worker': " 
 "'/pulp/api/v3/workers/4e514866-3a11-4e32-a03e-a9019207e764/', 'parent': " 
 "None, 'spawned_tasks': [], 'progress_reports': [], 'created_resources': []}",
 {'_created': '2018-12-21T04:18:54.417596Z',
  '_href': '/pulp/api/v3/tasks/4e7f2a48-18d7-47a1-910e-15cb1b4e2144/',
  'created_resources': [],
  'error': {'code': None,
            'description': 'RPM package bear-4.1-1.noarch.rpm already exists.',
            'traceback': '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py", '
                         'line 799, in perform_job\n'
                         '    rv = job.perform()\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", '
                         'line 600, in perform\n'
                         '    self._result = self._execute()\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", '
                         'line 606, in _execute\n'
                         '    return self.func(*self.args, **self.kwargs)\n'
                         '  File '
                         '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/upload.py", '
                         'line 29, in one_shot_upload\n'
                         "    raise OSError('RPM package {} already " 
                         "exists.'.format(pkg.filename))\n"},
  'finished_at': '2018-12-21T04:18:54.536285Z',
  'name': 'pulp_rpm.app.upload.one_shot_upload',
  'non_fatal_errors': [],
  'parent': None,
  'progress_reports': [],
  'spawned_tasks': [],
  'started_at': '2018-12-21T04:18:54.508100Z',
  'state': 'failed',
  'worker': '/pulp/api/v3/workers/4e514866-3a11-4e32-a03e-a9019207e764/'})

In other cases when Pulp is dealing with duplicate files a 400 HTTP error is raised, and a message error is sent back.

See: https://pulp.plan.io/issues/3436

Any reason for this one behave different?

#10 Updated by bmbouter 8 months ago

  • Tags deleted (Pulp 3)

#11 Updated by ttereshc about 5 hours ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF