Actions
Task #2668
closed
Test Pulp Syncing Several Repos with Large Metadata
Added by bmbouter about 7 years ago. Updated about 5 years ago.
Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:
0%
Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:
Description
A user reported that Pulp may not sync some repos due to large metadata even with the fix in https://github.com/pulp/pulp_rpm/pull/1018
Katello ticket 2487 has a link to the repo that failed to sync:
https://fedorapeople.org/groups/katello/releases/yum/foreman/1.13/el6/x86_64/
Jboss package from ticket 723 should be available from brew:
https://brewweb.engineering.redhat.com/brew/packageinfo?packageID=36912
You can also try to sync full JBEAP latest compose (I do not have pulp installed anywhere right now, but I think this should also fail):
http://download-node-02.eng.bos.redhat.com/devel/candidates/JBEAP/composing/latest-JBEAP-7.0-RHEL-7/compose/Server/x86_64/os/
Updated by pthomas@redhat.com about 7 years ago
[root@ibm-x3550m3-13 ~]# rpm -qa |grep pulp |sort
pulp-admin-client-2.12.2-0.1.beta.el7.noarch
pulp-docker-admin-extensions-2.3.0-1.el7.noarch
pulp-docker-plugins-2.3.0-1.el7.noarch
pulp-ostree-admin-extensions-1.2.1-0.1.beta.el7.noarch
pulp-ostree-plugins-1.2.1-0.1.beta.el7.noarch
pulp-puppet-admin-extensions-2.12.2-0.1.beta.el7.noarch
pulp-puppet-plugins-2.12.2-0.1.beta.el7.noarch
pulp-python-admin-extensions-1.1.3-1.el7.noarch
pulp-python-plugins-1.1.3-1.el7.noarch
pulp-rpm-admin-extensions-2.12.2-0.1.beta.el7.noarch
pulp-rpm-plugins-2.12.2-0.1.beta.el7.noarch
pulp-selinux-2.12.2-0.1.beta.el7.noarch
pulp-server-2.12.2-0.1.beta.el7.noarch
python-isodate-0.5.0-4.pulp.el7.noarch
python-kombu-3.0.33-6.pulp.el7.noarch
python-pulp-bindings-2.12.2-0.1.beta.el7.noarch
python-pulp-client-lib-2.12.2-0.1.beta.el7.noarch
python-pulp-common-2.12.2-0.1.beta.el7.noarch
python-pulp-docker-common-2.3.0-1.el7.noarch
python-pulp-oid_validation-2.12.2-0.1.beta.el7.noarch
python-pulp-ostree-common-1.2.1-0.1.beta.el7.noarch
python-pulp-puppet-common-2.12.2-0.1.beta.el7.noarch
python-pulp-python-common-1.1.3-1.el7.noarch
python-pulp-repoauth-2.12.2-0.1.beta.el7.noarch
python-pulp-rpm-common-2.12.2-0.1.beta.el7.noarch
python-pulp-streamer-2.12.2-0.1.beta.el7.noarch
[root@ibm-x3550m3-13 ~]#
[root@ibm-x3550m3-13 ~]# pulp-admin rpm repo create --repo-id foreman --feed https://fedorapeople.org/groups/katello/releases/yum/foreman/1.13/el6/x86_64/
Successfully created repository [foreman]
[root@ibm-x3550m3-13 ~]# pulp-admin rpm repo sync run --repo-id foreman
+----------------------------------------------------------------------+
Synchronizing Repository [foreman]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[|]
... completed
Downloading repository content...
[-]
[==================================================] 100%
RPMs: 344/344 items
Delta RPMs: 0/0 items
... completed
Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed
Importing errata...
[-]
... completed
Importing package groups/categories...
[-]
... completed
Cleaning duplicate packages...
[-]
... completed
Task Succeeded
Initializing repo metadata
[-]
... completed
Publishing Distribution files
[-]
... completed
Publishing RPMs
[==================================================] 100%
344 of 344 items
... completed
Publishing Delta RPMs
... skipped
Publishing Errata
[\]
... completed
Publishing Comps file
[-]
... completed
Publishing Metadata.
[-]
... completed
Closing repo metadata
[-]
... completed
Generating sqlite files
... skipped
Generating HTML files
... skipped
Publishing files to web
[-]
... completed
Writing Listings File
[-]
... completed
Task Succeeded
[root@ibm-x3550m3-13 ~]# pulp-admin rpm repo create --repo-id issue2668 --feed http://download-node-02.eng.bos.redhat.com/devel/candidates/JBEAP/composing/latest-JBEAP-7.0-RHEL-7/compose/Server/x86_64/os/
Successfully created repository [issue2668]
[root@ibm-x3550m3-13 ~]# pulp-admin repo sync run --repo-id issue2668
Usage: pulp-admin [SUB_SECTION, ..] COMMAND
Description: list repositories and manage repo groups
Available Sections:
group - repository group commands
history - show sync and publish history
tasks - list and cancel tasks related to a specific repository
Available Commands:
download - queues a full download of all missing units in a repository that is using a
"background" or "on_demand" download policy.
list - lists repositories on the Pulp server
[root@ibm-x3550m3-13 ~]# pulp-admin rpm repo sync run --repo-id issue2668
+----------------------------------------------------------------------+
Synchronizing Repository [issue2668]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[|]
... completed
Downloading repository content...
[-]
[==================================================] 100%
RPMs: 296/296 items
Delta RPMs: 0/0 items
... completed
Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed
Importing errata...
[-]
... completed
Importing package groups/categories...
[-]
... completed
Cleaning duplicate packages...
[-]
... completed
Task Succeeded
Initializing repo metadata
[-]
... completed
Publishing Distribution files
[-]
... completed
Publishing RPMs
[==================================================] 100%
296 of 296 items
... completed
Publishing Delta RPMs
... skipped
Publishing Errata
[-]
... completed
Publishing Comps file
[==================================================] 100%
2 of 2 items
... completed
Publishing Metadata.
[-]
... completed
Closing repo metadata
[-]
... completed
Generating sqlite files
... skipped
Generating HTML files
... skipped
Publishing files to web
[\]
... completed
Writing Listings File
[-]
... completed
Task Succeeded
Updated by bmbouter about 7 years ago
Thank you for testing this.
Can a test also be done that uploads a few of the jbossas-javadocs rpms? Those are from here: https://brewweb.engineering.redhat.com/brew/packageinfo?packageID=36912
With that third piece I would consider this done.
Updated by pthomas@redhat.com about 7 years ago
[root@hp-dl380pgen8-02-vm-12 ~]# pulp-admin rpm repo uploads rpm --repo-id upload --dir test/ -vvv
+----------------------------------------------------------------------+
Unit Upload
+----------------------------------------------------------------------+
Extracting necessary metadata for each request...
[==================================================] 100%
Analyzing: jbossas-javadocs-7.5.14-2.Final_redhat_2.1.ep6.el7.noarch.rpm
... completed
Files to be uploaded:
jbossas-javadocs-7.5.14-1.Final_redhat_1.1.ep6.el5.noarch.rpm
jbossas-javadocs-7.5.14-2.Final_redhat_2.1.ep6.el6.noarch.rpm
jbossas-javadocs-7.5.14-2.Final_redhat_2.1.ep6.el7.noarch.rpm
Creating upload requests on the server...
[==================================================] 100%
Initializing: jbossas-javadocs-7.5.14-2.Final_redhat_2.1.ep6.el7.noarch.rpm
... completed
Starting upload of selected units. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
canceled entirely using the cancel command.
Uploading: jbossas-javadocs-7.5.14-1.Final_redhat_1.1.ep6.el5.noarch.rpm
[==================================================] 100%
49558304/49558304 bytes
... completed
Importing into the repository...
This command may be exited via ctrl+c without affecting the request.
[/]
Running...
Task Succeeded
Deleting the upload request...
... completed
Uploading: jbossas-javadocs-7.5.14-2.Final_redhat_2.1.ep6.el6.noarch.rpm
[==================================================] 100%
36352104/36352104 bytes
... completed
Importing into the repository...
This command may be exited via ctrl+c without affecting the request.
[/]
Running...
Task Succeeded
Deleting the upload request...
... completed
Uploading: jbossas-javadocs-7.5.14-2.Final_redhat_2.1.ep6.el7.noarch.rpm
[==================================================] 100%
31275636/31275636 bytes
... completed
Importing into the repository...
This command may be exited via ctrl+c without affecting the request.
[|]
Running...
Task Succeeded
Deleting the upload request...
... completed
Updated by bmbouter about 5 years ago
- Status changed from NEW to CLOSED - WONTFIX
Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.
Actions
.