RPM Support

I think at this point 1st rpm is just replaced with 2nd uploaded.
i was not able to reproduce the situation:

$ pulp-admin rpm repo uploads rpm --repo-id test2 --file /home/ipanova/mouse-0.1.12-1.noarch.rpm

+----------------------------------------------------------------------+
                              Unit Upload
+----------------------------------------------------------------------+

Extracting necessary metadata for each request...
[==================================================] 100%
Analyzing: mouse-0.1.12-1.noarch.rpm
... completed

Creating upload requests on the server...
[==================================================] 100%
Initializing: mouse-0.1.12-1.noarch.rpm
... completed

Starting upload of selected units. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
canceled entirely using the cancel command.

Uploading: mouse-0.1.12-1.noarch.rpm
[==================================================] 100%
2476/2476 bytes
... completed

Importing into the repository...
This command may be exited via ctrl+c without affecting the request.

[\]
Running...

Task Succeeded

Deleting the upload request...
... completed

[ipanova@ina ~]$ pulp-admin rpm repo uploads rpm --repo-id test2 --file /home/ipanova/mouse-0.1.12-1.noarch.rpm

+----------------------------------------------------------------------+
                              Unit Upload
+----------------------------------------------------------------------+

Extracting necessary metadata for each request...
[==================================================] 100%
Analyzing: mouse-0.1.12-1.noarch.rpm
... completed

Creating upload requests on the server...
[==================================================] 100%
Initializing: mouse-0.1.12-1.noarch.rpm
... completed

Starting upload of selected units. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
canceled entirely using the cancel command.

Uploading: mouse-0.1.12-1.noarch.rpm
[==================================================] 100%
2476/2476 bytes
... completed

Importing into the repository...
This command may be exited via ctrl+c without affecting the request.

[\]
Running...

Task Succeeded

Deleting the upload request...
... completed

Once the rpm conversion to mongoengine is done it will raise the ValidationError because there are new uniqueness constraints that prevent the second save() from occurring.

Because of how this issue is affecting satellite and for the fact that its a 2.8 regression, Can we bump the priority on this so that we can get this addressed in 2.8.

What is correct/expected behavior here? I don't get any exception after uploading same unit 2x in 2.7 neither in 2.8. What code should be returned upon failure?

This appears to be a problem only with the rendering of output to the user. The correct behavior seems to be happening, followed by an ugly message on screen.

Also, I haven't seen any indication that this is currently impacting katello or satellite. It was moved to 2.8.1 in collaboration with that team. If there is new information in that regard, please add it here.

Partially(?) fixed here, PR

I hit an issue related to this today. I created a puppet repo, then uploaded a puppet module, then deleted it. I wanted the module back again so I went to upload it, and it errored out.

I was testing this as part of an unrelated bug, but I think this is a common use case.

I agree it's a common use case. Until the bug is fixed, one workaround for the workflow you described is to do an orphan cleanup after the delete and before the second upload.

A different method to hit this bug which I hit today:

• sync puppet forge in one repo to get all puppet modules
• wait some time and upload the puppet manifests you want manually to a different puppet repo so you can have the modules you need while the long-running sync completes

If the unit you are uploading has already been created via the other sync but not yet associated to the repo, you'll get this error. The workaround is to wait for the completion of the sync process.

This is a general problem related to mongoengine conversion. The save_unit() method in the conduits caught duplicate key constraint violations. When caught (by the conduit), the unit in the DB was updated and associated to the repository. Some of the mongoengine converted plugins are missing try/catch NotUniqueError logic. The danger of ignoring this is that the unit already stored in the DB is not associated to the repository as expected and in some cases fails silently. I think this needs to be aligned to 2.8.0.

I agree the mongoengine conversion causes the plugins to create-and-save a unit directly within plugin code, so we're not in a good position to fix this all in one place. Anywhere we do want it fixed, we could override the save() method to catch DuplicateKeyErrors and return the existing object. We could introduce that in the inheritance tree of RPM's mongoengine models which is probably best. I'm not sure we want all ContentUnits to catch DuplicateKeyErrors in all cases.

Another option is to put Try/Catch exceptions around the save() calls throughout the plugin code, which does not seem as good as ^.

Oh and I don't have a comment on 2.8.0 versus 2.8.1.