Issue #2190
closedUnit is associated with the repo before it is copied to the final location
Description
If there will be some issue during copy of the rpms to final location during sync (as it was in my case),
sync won't notice it and task will be completed without any visible indication for the user (except logs).
Publish will succeed as well and create broken symlinks which is a problem since download policy is immediate in my example.
Here unit is associated to the repo and only after that unit is copied to its final location.
The exceptions are caught and logged here, so if you do not look at logs you won't notice the problem.
To reproduce just raise some exception right after unit association which is done here.
Then create repo with immediate download policy and sync it.
Observe no failures and associated units in repo.
$ pulp-admin rpm repo create --feed https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm/ --repo-id zoo --relative-url zoo
Successfully created repository [zoo]
$ pulp-admin rpm repo sync run --repo-id zoo
+----------------------------------------------------------------------+
Synchronizing Repository [zoo]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[|]
... completed
Downloading repository content...
[==================================================] 100%
RPMs: 32/32 items
Delta RPMs: 0/0 items
... completed
Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed
Importing errata...
[\]
... completed
Importing package groups/categories...
[-]
... completed
Cleaning duplicate packages...
[-]
... completed
Task Succeeded
Initializing repo metadata
[-]
... completed
Publishing Distribution files
[-]
... completed
Publishing RPMs
[==================================================] 100%
32 of 32 items
... completed
Publishing Delta RPMs
... skipped
Publishing Errata
[==================================================] 100%
4 of 4 items
... completed
Publishing Comps file
[==================================================] 100%
4 of 4 items
... completed
Publishing Metadata.
[-]
... completed
Closing repo metadata
[-]
... completed
Generating sqlite files
... skipped
Generating HTML files
... skipped
Publishing files to web
[-]
... completed
Writing Listings File
[-]
... completed
Task Succeeded
$ pulp-admin repo list
+----------------------------------------------------------------------+
Repositories
+----------------------------------------------------------------------+
Id: zoo
Display Name: None
Description: None
Content Unit Counts:
Erratum: 4
Package Category: 1
Package Group: 2
Package Langpacks: 1
Rpm: 32
Logs
...
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) Listener error on event: succeeded
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) Traceback (most recent call last):
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) File "/home/user1/devel/pulp/server/pulp/server/content/sources/event.py", line 39, in __call__
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) listener.on_event(self)
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) File "/home/user1/devel/pulp/server/pulp/server/content/sources/event.py", line 135, in on_event
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) method(event.request)
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) File "/home/user1/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/repomd/alternate.py", line 119, in on_succeeded
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) self.content_listener.download_succeeded(report)
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) File "/home/user1/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/listener.py", line 204, in download_succeeded
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) added_unit.safe_import_content(report.destination)
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) File "/home/user1/devel/pulp/server/pulp/server/db/model/__init__.py", line 920, in safe_import_content
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) self.import_content(path, location)
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) File "/home/user1/devel/pulp/server/pulp/server/db/model/__init__.py", line 894, in import_content
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) storage.put(self, path, location)
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) File "/home/user1/devel/pulp/server/pulp/server/content/storage.py", line 122, in put
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) raise Exception('bla-bla')
Aug 19 15:41:21 localhost pulp[17574]: pulp.server.content.sources.event:ERROR: (17574-37280) Exception: bla-bla
...
Related issues
Associate unit after it is placed to its final location
closes #2190 https://pulp.plan.io/issues/2190