https://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-02-23T15:47:27ZPulpRPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=679532021-02-23T15:47:27Zfao89
<ul><li><strong>Project</strong> changed from <i>Pulp</i> to <i>RPM Support</i></li></ul> RPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=679562021-02-23T15:50:07Zipanova@redhat.comipanova@redhat.com
<ul><li><strong>Tags</strong> <i>Pulp 2</i> added</li></ul> RPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=679572021-02-23T15:52:41Zggainey
<ul></ul><p>@midhuhk - this is a Pulp2 error-msg. Can you tell us what version of Pulp2 you're using? Also, can you tell us the URL you're using to sync rhel8-appstream? Thanks</p> RPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=679632021-02-23T16:49:06Zggainey
<ul></ul><p>Running on pulp-2.21.5, the following script sync'd rhel8-base and rhel8-appstream from cdn.redhat.com successfully:</p>
<pre><code>#!/bin/bash -v
BASE='rhel8-baseos'
STREAM='rhel8-appstream'
REMOTE1='http://cdn.redhat.com/content/dist/rhel8/8/x86_64/baseos/os/'
REMOTE2='http://cdn.redhat.com/content/dist/rhel8/8/x86_64/appstream/os/'
DEST='destination'
pulp-admin rpm repo create --repo-id=$BASE --relative-url=$BASE --feed=$REMOTE1 \
--download-policy on_demand \
--generate-sqlite false --repoview false \
--feed-ca-cert /home/vagrant/devel/pulp_startup/CDN_cert/redhat-uep.pem \
--feed-cert /home/vagrant/devel/pulp_startup/CDN_cert/cdn.crt \
--feed-key /home/vagrant/devel/pulp_startup/CDN_cert/cdn.key
pulp-admin rpm repo sync run --repo-id=$BASE
pulp-admin rpm repo create --repo-id=$STREAM --relative-url=$STREAM --feed=$REMOTE2 \
--download-policy on_demand \
--generate-sqlite false --repoview false \
--feed-ca-cert /home/vagrant/devel/pulp_startup/CDN_cert/redhat-uep.pem \
--feed-cert /home/vagrant/devel/pulp_startup/CDN_cert/cdn.crt \
--feed-key /home/vagrant/devel/pulp_startup/CDN_cert/cdn.key
pulp-admin rpm repo sync run --repo-id=$STREAM
pulp-admin rpm repo create --repo-id=$DEST
pulp-admin repo list
</code></pre>
<p>If the problem persists, you might want to open a support ticket with Red Hat - it's more likely a problem with bad data on CDN, and there isn't anything Pulp can do about that.</p> RPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=679972021-02-25T10:38:56Zmidhuhk
<ul></ul><p>Pulp version used : 2.21.3-1.el7.
I see issue only with appstream repo . Base repo works fine.
Case has already been opened with redhat . Just want to make sure on pulp side as well</p>
<p>link : <a href="https://cdn.redhat.com/content/dist/rhel8/8/x86_64/appstream/os/" class="external">https://cdn.redhat.com/content/dist/rhel8/8/x86_64/appstream/os/</a></p> RPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=680062021-02-25T16:27:01Zdalleydalley@redhat.com
<ul><li><strong>Triaged</strong> changed from <i>No</i> to <i>Yes</i></li></ul> RPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=680282021-03-01T06:20:15Zmidhuhk
<ul></ul><p>Redhat confirmed CDN is fine and validated by syncing with satellite in lab setup and it was fine . But with pulp sync , its still showing same error for me . Any suggestions ??</p> RPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=680952021-03-02T16:45:42Zggainey
<ul></ul><p>midhuhk wrote:</p>
<blockquote>
<p>Redhat confirmed CDN is fine and validated by syncing with satellite in lab setup and it was fine . But with pulp sync , its still showing same error for me . Any suggestions ??</p>
</blockquote>
<p>On CentOS7 on 2.21.3, I can't recreate - whatever the problem is, it's not Pulp.</p>
<p>I'm leaning towards "bad repo on a specific akamai node". Syncing "from a lab machine" won't help if that's the case, because you don't know what CDN node you're getting the content from.</p>
<p>If you ping cdn.redhat.com <em>from the pulp2 box</em>, what Ip/machine does it return? I'll see if I can coerce my reproducer into using that specific node for its feed and see what happens.</p> RPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=681262021-03-03T08:21:41Zmidhuhk
<ul></ul><p>cdn.redhat.com is connected via proxy like all other repo . For failed repo sync ,destination IP is pointing to 95.101.92.251</p> RPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=681302021-03-03T13:14:23Zggainey
<ul></ul><p>midhuhk wrote:</p>
<blockquote>
<p>cdn.redhat.com is connected via proxy like all other repo . For failed repo sync ,destination IP is pointing to 95.101.92.251</p>
</blockquote>
<p>Well, there;s another idea down the tubes. I linked cdn.redhat.com to that specific IP/akamai-node and sync'd rhel8-appstream with no problem (other than it being slow due to coming from Far Away).</p>
<p>Somewhere there is a cache of filelists.xml and other.xml, that is broken, that your sync is finding. Is your proxy perchance a caching proxy?</p>
<p>The repo-metadata appears to be ok, we've had no other reports of pulp2 having problems with rhel8-appstream. I'm out of ideas, I'm afraid. :(</p> RPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=682642021-03-05T10:22:07Zmidhuhk
<ul></ul><p>Deleted the repo and created again , but still issue perisist . Looks like corrupted data still exist some where even after deleting and recreating the repo.</p> RPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=683912021-03-08T12:54:20Zaarvelobt
<ul></ul><p>I'm also seeing the same thing with pulp version 2.21.5. I also opened a case with Redhat and they confirmed that their repo data is correct. We too use a proxy but we disable caching.</p> RPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=684302021-03-09T12:10:39Zmidhuhk
<ul></ul><p>it looks like issue is with pulp . We tried to set retain-old-count to a high value (100) and sync works fine with this setting .Before the value set was retain-old-count =2 .</p>
<p><a class="user active" href="https://pulp.plan.io/users/14796">ggainey</a> , Can you set retain-old-count to 2 in your setup and see if its working or not ?</p> RPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=684752021-03-09T21:37:58Zggainey
<ul></ul><p>midhuhk wrote:</p>
<blockquote>
<p>it looks like issue is with pulp . We tried to set retain-old-count to a high value (100) and sync works fine with this setting .Before the value set was retain-old-count =2 .</p>
<p><a class="user active" href="https://pulp.plan.io/users/14796">ggainey</a> , Can you set retain-old-count to 2 in your setup and see if its working or not ?</p>
</blockquote>
<p>Ah! That was the missing piece, I can now reproduce the problem with this script:</p>
<pre><code>#!/bin/bash -v
STREAM='rhel8-appstream'
REMOTE2='http://cdn.redhat.com/content/dist/rhel8/8/x86_64/appstream/os/'
pulp-admin rpm repo create --repo-id=$STREAM --relative-url=$STREAM --feed=$REMOTE2 \
--retain-old-count 2 \
--download-policy on_demand \
--generate-sqlite false --repoview false \
--feed-ca-cert /home/vagrant/devel/pulp_startup/CDN_cert/redhat-uep.pem \
--feed-cert /home/vagrant/devel/pulp_startup/CDN_cert/cdn.crt \
--feed-key /home/vagrant/devel/pulp_startup/CDN_cert/cdn.key
pulp-admin rpm repo sync run --repo-id=$STREAM
pulp-admin repo list
</code></pre>
<p>The code involved in retain-old-counts hasn't changed in two years, so I'm not sure what's going on here. Maybe something about the shape of rhel8-appstream collides with the "only keep so many versions" logic?</p>
<p>For the moment, the workaround is "don't use retain-old-versions on rhel8-appstream repo". Given where Pulp2 is in its lifecycle, I can't guarantee we'll get any further than that, I'm afraid.</p> RPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=709332021-05-18T15:26:49Zggainey
<ul></ul><p>Came back to this issue and started trying to figure out what is/was going on. However, my reproducer from <a class="issue tracker-3 status-8 priority-6 priority-default closed" title="Story: As a user I can rely on node having 100% unit test coverage and no regression in integration test... (CLOSED - NOTABUG)" href="https://pulp.plan.io/issues/14">#14</a> now runs successfully. My pulp2 hasn't changed (it's 2.21.5). @midhuhk , @aarvelobt , do you still see this happening?</p> RPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=721652021-06-21T02:13:13Zdalleydalley@redhat.com
<ul></ul><p>Have you seen this continue happening?</p> RPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=721962021-06-22T08:14:07Zmidhuhk
<ul></ul><p>Thank you for checking . Yes it works fine now with retain-old-count =2 . Pulp version is same here and not sure what caused it .</p> RPM Support - Issue #8287: Rhel8 remote redhat repo showing Malformed repository: metadata is missing for some packages in filelists.xml and in other.xmlhttps://pulp.plan.io/issues/8287?journal_id=722072021-06-22T11:25:18Zggainey
<ul><li><strong>Status</strong> changed from <i>NEW</i> to <i>CLOSED - WORKSFORME</i></li></ul><p>@midhuhk Well, good, I guess? I'm going to close this as "WORKSFORME" - if the problem crops up again, add the log entries and reopen and we'll revisit. Thanks for checking back!</p>