RPM Support

This is not a problem on model hydration. I'm not familiar enough with the internals of the validation on hydration/save to speak for why this is the case, but the problem is that we cannot ever save any unit with a null value in a field MongoEngine has declared as ``required`` (from my brief investigation).

I think that this will not have a huge effect on users (they can still do most activities, just not sync RHEL5 distributions) so we can push this to a z release without too much damage. However, this is very likely the tip of the iceburg when it comes to data validation on hydration and on save. We modify documents outside of MongoEngine all over the place and if any of those fail to match the validation requires of our MongoEngine models it will crash and burn when MongoEngine attempts to hydrate them, or when it attempts to save them.

I know we have that script to hydrate and save, but it clearly hasn't even been used on all repositories synced from the CDN, let alone other repositories out there.

+1 to moving this to 2.8.1

Regarding the validation and hydration issues we need to make sure that the test scripts are identified in the release notes, upgrade notes, and probably some proactive mail on pulp-list.

It appears that pulp < 2.8 stores the "variant" as null in the database for a case like rhel 5.10, where there is no variant. Mongoengine seems to not support the concept of having a null value in the database at all, but rather removes or omits the key when its value is None.

Related discussion: https://github.com/MongoEngine/mongoengine/issues/734

This whole model definitely deserves a re-think. But to address this immediate bug, we might be able to make a simple custom field that gets us back on the road.

[root@qe-blade-03 ~]# rpm -qa |grep pulp-server
pulp-server-2.8.3-0.1.beta.git.44.70215c3.el7.noarch

[root@pulp ~]# pulp-admin rpm repo create --feed https://cdn.redhat.com/content/dist/rhel/rhui/server/5/5.11/x86_64/kickstart/ --feed-ca-cert cdn-ca.crt --feed-cert cdn.crt --feed-key cdn.key --repo-id rhel511-ks
Successfully created repository [rhel511-ks]

[root@pulp ~]#
[root@pulp ~]#
[root@pulp ~]# pulp-admin rpm repo sync run --repo-id rhel511-ks
--------------------------------------------------------------------
Synchronizing Repository [rhel511-ks]
--------------------------------------------------------------------

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[\]
... completed

Downloading repository content...
[-]
[==================================================] 100%
RPMs: 3372/3372 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[================================================= ] 98%
Distributions: 107/109 items

[==================================================] 100%
Distributions: 109/109 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed

Cleaning duplicate packages...
[-]
... completed

Task Succeeded

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[==================================================] 100%
3372 of 3372 items
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[-]
... completed

Publishing Comps file
[-]
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[|]
... completed

Writing Listings File
[-]
... completed

Task Succeeded

verified
https://pulp.plan.io/issues/1768#note-20