https://pulp.plan.io/https://pulp.plan.io/favicon.ico2020-05-25T16:31:17ZPulpRPM Support - Issue #6815: Not all advisory models are defined for import/exporthttps://pulp.plan.io/issues/6815?journal_id=569582020-05-25T16:31:17Zttereshcttereshc@redhat.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/56958/diff?detail_id=57543">diff</a>)</li></ul> RPM Support - Issue #6815: Not all advisory models are defined for import/exporthttps://pulp.plan.io/issues/6815?journal_id=571962020-05-28T15:11:02Zggainey
<ul><li><strong>Parent issue</strong> set to <i>#6134</i></li></ul> RPM Support - Issue #6815: Not all advisory models are defined for import/exporthttps://pulp.plan.io/issues/6815?journal_id=571992020-05-28T15:14:35Zggainey
<ul><li><strong>Triaged</strong> changed from <i>No</i> to <i>Yes</i></li></ul> RPM Support - Issue #6815: Not all advisory models are defined for import/exporthttps://pulp.plan.io/issues/6815?journal_id=577122020-06-07T12:14:22Zggainey
<ul></ul><p>@ttereshenc - A number of these have no 'natural key' supported at the DB level via uniqueness-constraints. I can work around that, but I think it might just be an oversight.</p>
<ul>
<li>
<strong>UpdateCollection</strong>: is 'name' expected to be unique? And what about not-null - as currently defined, all fields except pulp_id/created are allowed to be null - is that right?</li>
<li>
<strong>UpdateCollectionPackage</strong>: is 'sum,sum_type' expected to be unique? sum_type can be null - is that right? What about 'filename' or 'name'?</li>
<li>
<strong>UpdateReference</strong>: is 'href,ref_type' expected to be unique?</li>
</ul>
<p>Once I have answers to these, I can make the necessary changes and then make them all exportable.</p> RPM Support - Issue #6815: Not all advisory models are defined for import/exporthttps://pulp.plan.io/issues/6815?journal_id=578132020-06-09T09:11:51Zttereshcttereshc@redhat.com
<ul></ul><p>Yes, all those auxiliary models have no uniqueness constraint and are created/removed for every advisory.
<code>digest</code> on the UpdateRecord model reflects all content of advisory, including those models.</p>
<blockquote>
<ul>
<li>
<strong>UpdateCollection</strong>: is 'name' expected to be unique? And what about not-null - as currently defined, all fields except pulp_id/created are allowed to be null - is that right?</li>
</ul>
</blockquote>
<p>In theory, it should be unique within an advisory with a specific id. It's not checked/enforced in pulp3 at the moment but we have a story filed.
At a DB level they are not required to be unique.
As for the null constraints, advisory can have anything/nothing, you know ;) I can check why we chose <code>null</code> for the <code>name</code> as well, probably <code>name</code> is always present but it definitely can be an empty string. It also depends how createrepo_c works with it. Let me know if it's critical and I can dig deeper.</p>
<blockquote>
<ul>
<li>
<strong>UpdateCollectionPackage</strong>: is 'sum,sum_type' expected to be unique? sum_type can be null - is that right? What about 'filename' or 'name'?</li>
</ul>
</blockquote>
<p><code>sum</code> and <code>sum_type</code> are often absent. <code>filename</code> is potentially a good candidate. <code>name</code> is not enough, should be all NEVRA then.
FWIW, if we choose the path of introducing uniqueness constraints, I think it should be for all 3 models. It also affects sync code significantly and requires data migration which needs to remove duplicated entries and create correct relations for all affected advisories.</p>
<blockquote>
<ul>
<li>
<strong>UpdateReference</strong>: is 'href,ref_type' expected to be unique?</li>
</ul>
</blockquote>
<p>Probably within the same advisory. I don't think there is any guarantee that it's this way and they can also be absent.</p> RPM Support - Issue #6815: Not all advisory models are defined for import/exporthttps://pulp.plan.io/issues/6815?journal_id=595472020-07-15T14:13:53Zggainey
<ul><li><strong>Status</strong> changed from <i>NEW</i> to <i>ASSIGNED</i></li><li><strong>Assignee</strong> set to <i>ggainey</i></li><li><strong>Sprint</strong> set to <i>Sprint 77</i></li></ul> RPM Support - Issue #6815: Not all advisory models are defined for import/exporthttps://pulp.plan.io/issues/6815?journal_id=597322020-07-21T14:06:42Zggainey
<ul></ul><p>There are two issues with the current <a href="https://github.com/pulp/pulp_rpm/blob/master/pulp_rpm/app/models/advisory.py#L229" class="external">UpdateCollection model</a>:</p>
<ul>
<li>name is nullable. Because all nulls are unique in postgres, this means that you could have two UpdateCollections null/update-id-1 and null/update-id-1, and the would be considered 'unique'. If a collection can truly be unnamed, we need to make "not-named' be an empty string instead of null.</li>
<li>update_collection is a many-to-many relationship instead of a foreign-key relationship. This looks like just an error tha needs to be fixed.</li>
</ul>
<p>Fixing this in existing databases will require:</p>
<ol>
<li>for all rpm_updatecollection where name is null, set name = ""</li>
<li>create a foreign-key column, update_record_id, in rpm_updatecollection</li>
<li>find all existing update-collections that are related to exactly one update-record</li>
<li>for all elements found in step-2:
<ol>
<li>fill in update_record_id-column with the update-record-id they are associated with</li>
<li>remove these entries from rpm_updatecollection_update_record</li>
</ol>
</li>
<li>for all rpm_updatecollection rows where update-collection-id is still null:
<ol>
<li>if none, break</li>
<li>for each UpdateCollection, find all the UpdateRecords it is associated with</li>
<li>create a duplicate UpdateCollection (and associated UpdateCollectionPackages), assigned each to a specific UpdateRecord</li>
<li>remove records from rpm_updatecollection_update_record for the originating UpdateRecord</li>
</ol>
</li>
<li>if rpm_updatecollection_update_record is not empty - ERROR</li>
<li>update rpm_updatecollection.name to NOT be nullable</li>
<li>set a uniqueness-constraint on rpm_updatecollection of (name, update-record-id)</li>
</ol>
<p>Definitely a Fun migration...</p>
<p>A thought : if the API doesn't expose creating/linking updatecollection to updaterecords directly, then we just have to understand how advisory-upload/create and sync use this model - the data may not be as broken as the db-schema allows it to be. It would let us skip steps 5-6 above.</p> RPM Support - Issue #6815: Not all advisory models are defined for import/exporthttps://pulp.plan.io/issues/6815?journal_id=598142020-07-22T19:38:41Zggainey
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-11 priority-7 priority-high2 closed" href="/issues/7195">Task #7195</a>: Enforce correct uniqueness for UpdateCollection at the Model level</i> added</li></ul> RPM Support - Issue #6815: Not all advisory models are defined for import/exporthttps://pulp.plan.io/issues/6815?journal_id=600952020-07-27T22:06:26Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 77</i> to <i>Sprint 78</i></li></ul> RPM Support - Issue #6815: Not all advisory models are defined for import/exporthttps://pulp.plan.io/issues/6815?journal_id=605372020-08-07T14:34:25Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 78</i> to <i>Sprint 79</i></li></ul> RPM Support - Issue #6815: Not all advisory models are defined for import/exporthttps://pulp.plan.io/issues/6815?journal_id=608122020-08-13T17:17:10Zpulpbot
<ul><li><strong>Status</strong> changed from <i>ASSIGNED</i> to <i>POST</i></li></ul><p>PR: <a href="https://github.com/pulp/pulp_rpm/pull/1811" class="external">https://github.com/pulp/pulp_rpm/pull/1811</a></p> RPM Support - Issue #6815: Not all advisory models are defined for import/exporthttps://pulp.plan.io/issues/6815?journal_id=609212020-08-13T22:11:46Zggainey
<ul><li><strong>Status</strong> changed from <i>POST</i> to <i>MODIFIED</i></li></ul><p>Applied in changeset <a class="changeset" title="Add remaining RPM-entities to import/export. Includes UpdateReference, UpdateCollection, and Upd..." href="https://pulp.plan.io/projects/pulp_rpm/repository/9/revisions/db0a7926eba0c402271f2b45a2298447b35a6f3d">db0a7926eba0c402271f2b45a2298447b35a6f3d</a>.</p> RPM Support - Issue #6815: Not all advisory models are defined for import/exporthttps://pulp.plan.io/issues/6815?journal_id=609462020-08-17T16:26:38Zttereshcttereshc@redhat.com
<ul><li><strong>Sprint/Milestone</strong> set to <i>3.6.0</i></li></ul> RPM Support - Issue #6815: Not all advisory models are defined for import/exporthttps://pulp.plan.io/issues/6815?journal_id=609572020-08-17T17:29:56Zpulpbot
<ul><li><strong>Status</strong> changed from <i>MODIFIED</i> to <i>CLOSED - CURRENTRELEASE</i></li></ul>