Project

Profile

Help

Issue #2190

closed

Unit is associated with the repo before it is copied to the final location

Added by ttereshc over 7 years ago. Updated about 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
2.10.1
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 7
Quarter:

Description

If there will be some issue during copy of the rpms to final location during sync (as it was in my case),
sync won't notice it and task will be completed without any visible indication for the user (except logs).
Publish will succeed as well and create broken symlinks which is a problem since download policy is immediate in my example.

Here unit is associated to the repo and only after that unit is copied to its final location.
The exceptions are caught and logged here, so if you do not look at logs you won't notice the problem.

To reproduce just raise some exception right after unit association which is done here.
Then create repo with immediate download policy and sync it.

Observe no failures and associated units in repo.

$ pulp-admin rpm repo create --feed https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm/ --repo-id zoo  --relative-url zoo
Successfully created repository [zoo]

$ pulp-admin rpm repo sync run --repo-id zoo
+----------------------------------------------------------------------+
                     Synchronizing Repository [zoo]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       32/32 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[\]
... completed

Importing package groups/categories...
[-]
... completed

Cleaning duplicate packages...
[-]
... completed

Task Succeeded

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[==================================================] 100%
32 of 32 items
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[==================================================] 100%
4 of 4 items
... completed

Publishing Comps file
[==================================================] 100%
4 of 4 items
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Generating HTML files
... skipped

Publishing files to web
[-]
... completed

Writing Listings File
[-]
... completed

Task Succeeded
$ pulp-admin repo list
+----------------------------------------------------------------------+
                              Repositories
+----------------------------------------------------------------------+

Id:                  zoo
Display Name:        None
Description:         None
Content Unit Counts: 
  Erratum:           4
  Package Category:  1
  Package Group:     2
  Package Langpacks: 1
  Rpm:               32

Logs

...
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) Listener error on event: succeeded
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) Traceback (most recent call last):
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280)   File "/home/user1/devel/pulp/server/pulp/server/content/sources/event.py", line 39, in __call__
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280)     listener.on_event(self)
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280)   File "/home/user1/devel/pulp/server/pulp/server/content/sources/event.py", line 135, in on_event
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280)     method(event.request)
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280)   File "/home/user1/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/repomd/alternate.py", line 119, in on_succeeded
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280)     self.content_listener.download_succeeded(report)
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280)   File "/home/user1/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/listener.py", line 204, in download_succeeded
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280)     added_unit.safe_import_content(report.destination)
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280)   File "/home/user1/devel/pulp/server/pulp/server/db/model/__init__.py", line 920, in safe_import_content
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280)     self.import_content(path, location)
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280)   File "/home/user1/devel/pulp/server/pulp/server/db/model/__init__.py", line 894, in import_content
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280)     storage.put(self, path, location)
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280)   File "/home/user1/devel/pulp/server/pulp/server/content/storage.py", line 122, in put
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280)     raise Exception('bla-bla')
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) Exception: bla-bla
...

Related issues

Blocks RPM Support - Issue #1823: RPMs partially downloadedCLOSED - CURRENTRELEASEttereshcActions
Actions #1

Updated by amacdona@redhat.com over 7 years ago

  • Priority changed from Normal to High
  • Severity changed from 2. Medium to 3. High
  • Triaged changed from No to Yes
Actions #2

Updated by ttereshc over 7 years ago

Actions #3

Updated by ttereshc over 7 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ttereshc

Added by ttereshc over 7 years ago

Revision 8e2bfe71 | View on GitHub

Associate unit after it is placed to its final location

closes #2190 https://pulp.plan.io/issues/2190

Actions #4

Updated by ttereshc over 7 years ago

  • Status changed from ASSIGNED to POST
Actions #5

Updated by ttereshc over 7 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions #6

Updated by ttereshc over 7 years ago

  • Sprint/Milestone set to 25
  • Platform Release set to 2.9.4

Added by ttereshc over 7 years ago

Revision e6e71c8f | View on GitHub

Fix unit association if unit was downloaded for the second time

re #2190 https://pulp.plan.io/issues/2190

Actions #7

Updated by semyers over 7 years ago

  • Platform Release deleted (2.9.4)

We may not be doing a 2.9.4 release, so I'm clearing the target release field for these to make sure they don't get lost in the next bugfix release cycle.

Actions #8

Updated by semyers over 7 years ago

  • Platform Release set to 2.10.1
Actions #9

Updated by semyers over 7 years ago

  • Status changed from MODIFIED to 5
Actions #10

Updated by semyers over 7 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #11

Updated by bmbouter about 6 years ago

  • Sprint set to Sprint 7
Actions #12

Updated by bmbouter about 6 years ago

  • Sprint/Milestone deleted (25)
Actions #13

Updated by bmbouter about 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF