Issue #2280: Error is not reported correctly when size verification for DRPM unit fails - RPM Support - Pulp

Actions

Send by e-mail Copy link

Issue #2280

closed

Error is not reported correctly when size verification for DRPM unit fails

Added by ttereshc over 7 years ago. Updated about 5 years ago.

Status:

CLOSED - WORKSFORME

Priority:

Normal

Assignee:

mansari

Sprint/Milestone:

Start date:

Due date:

Estimated time:

Severity:

2. Medium

Version:

Platform Release:

OS:

Triaged:

Yes

Groomed:

Sprint Candidate:

Tags:

Easy Fix, Pulp 2

Sprint:

Quarter:

Description

Create and sync DRPM repo with `validate` option enabled for importer and the feed for which size verification of DRPMs will fail, for now it is https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/

Tested on Pulp 2.9.3.

$ pulp-admin rpm repo create --repo-id drpm  --feed  https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/  --validate true

Sync task will just succeed, in the logs there will be a traceback.

$ pulp-admin rpm repo sync run --repo-id drpm
+----------------------------------------------------------------------+
                    Synchronizing Repository [drpm]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[\]
... completed

Downloading repository content...
[-]
[==================================================] 100%
RPMs:       5/5 items
Delta RPMs: 0/4 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed

Cleaning duplicate packages...
[-]
... completed

Task Succeeded

Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512) 'DRPM' object has no attribute 'name'
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512) Traceback (most recent call last):
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512)   File "/usr/lib/python2.7/site-packages/nectar/downloaders/base.py", line 145, in _fire_event_to_listener
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512)     event_listener_callback(*args, **kwargs)
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512)   File "/home/user1/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/listener.py", line 194, in download_succeeded
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512)     super(RPMListener, self).download_succeeded(report)
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512)   File "/home/user1/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/listener.py", line 81, in download_succeeded
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512)     self._verify_size(unit, report)
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512)   File "/home/user1/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/listener.py", line 120, in _verify_size
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512)     constants.NAME: unit.name,
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512) AttributeError: 'DRPM' object has no attribute 'name'

Expected result, example of the same error reporting for srpm:

$ pulp-admin rpm repo sync run --repo-id srpm
+----------------------------------------------------------------------+
                    Synchronizing Repository [srpm]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[\]
... completed

Downloading repository content...
[-]
[==================================================] 100%
RPMs:       3/3 items
Delta RPMs: 0/0 items

... completed

Individual package errors encountered during sync:

Package: test-srpm01
Error: The size did not match the value specified in the
repository metadata.

Package: test-srpm03
Error: The size did not match the value specified in the
repository metadata.

Package: test-srpm02
Error: The size did not match the value specified in the
repository metadata.

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[\]
... completed

Cleaning duplicate packages...
[-]
... completed

Task Succeeded

JFYI, the importer `--validate` option is enabled by default in Katello https://github.com/Katello/katello/pull/4747/files

Actions

Copy link

Updated by ttereshc over 7 years ago

Description updated (diff)

Actions

Copy link

Updated by pcreech over 7 years ago

Triaged changed from No to Yes

Actions

Copy link

Updated by mansari almost 7 years ago

Unable to reproduce the issue.

pulp-admin rpm repo sync run --repo-id drpm
+----------------------------------------------------------------------+
                    Synchronizing Repository [drpm]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       5/5 items
Delta RPMs: 4/4 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed

Cleaning duplicate packages...
[-]
... completed

Task Succeeded

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[==================================================] 100%
5 of 5 items
... completed

Publishing Delta RPMs
[==================================================] 100%
4 of 4 items
... completed

Publishing Errata
[-]
... completed

Publishing Comps file
[-]
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Generating HTML files
... skipped

Publishing files to web
[-]
... completed

Writing Listings File
[-]
... completed

Task Succeeded

[vagrant@dev ~]$ journalctl -r
-- Logs begin at Wed 2017-05-31 16:39:46 UTC, end at Tue 2057-07-24 18:13:16 UTC. --
Jun 01 20:41:33 dev.example.com audit[20898]: USER_CMD pid=20898 uid=1001 auid=1001 ses=12 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='cwd="/home/vagrant/devel/pulp_puppet" cmd=73797374656D63
-- Reboot --
Jun 02 18:40:21 dev.example.com pulp[13828]: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[5edfe9d8-17a9-4c08-a6a5-de3213720663] succeeded in 0.00381578100132s: None
Jun 02 18:40:21 dev.example.com pulp[13828]: celery.worker.job:INFO: Task pulp.server.managers.repo.publish.publish[6ac47295-6221-404e-ac45-a25d144a50c7] succeeded in 0.335323383002s: {'exception': None, 'repo_i
Jun 02 18:40:21 dev.example.com pulp[13828]: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[72b9450e-4c23-42e0-8d94-8c16ff1c3f8f] succeeded in 0.0381245940007s: None
Jun 02 18:40:21 dev.example.com pulp[13800]: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[999d411d-ce88-4a34-8bca-aa495b1fdad3] succeeded in 0.0387191960035s: None
Jun 02 18:40:21 dev.example.com pulp[13828]: celery.worker.job:INFO: Task pulp.server.managers.repo.sync.sync[e001b643-7047-4384-9b1b-8e012e347924] succeeded in 9.487843471s: <pulp.server.async.tasks.TaskResult 
Jun 02 18:40:21 dev.example.com pulp[13828]: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[5edfe9d8-17a9-4c08-a6a5-de3213720663]
Jun 02 18:40:21 dev.example.com pulp[13828]: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.publish.publish[6ac47295-6221-404e-ac45-a25d144a50c7]
Jun 02 18:40:21 dev.example.com pulp[13800]: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[999d411d-ce88-4a34-8bca-aa495b1fdad3]
Jun 02 18:40:21 dev.example.com pulp[14074]: kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS
Jun 02 18:40:21 dev.example.com pulp[14074]: pulp_rpm.plugins.importers.yum.sync:INFO: [e001b643] Sync complete.
Jun 02 18:40:20 dev.example.com pulp[14074]: nectar.downloaders.threaded:INFO: Download failed: Download of https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/treeinfo failed with code 404: Not Found
Jun 02 18:40:20 dev.example.com pulp[14074]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): repos.fedorapeople.org
Jun 02 18:40:19 dev.example.com pulp[14074]: nectar.downloaders.threaded:INFO: Download failed: Download of https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/.treeinfo failed with code 404: Not Found
Jun 02 18:40:19 dev.example.com pulp[14074]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): repos.fedorapeople.org
Jun 02 18:40:19 dev.example.com pulp[14074]: pulp_rpm.plugins.importers.yum.sync:INFO: [e001b643] Downloading additional units.
Jun 02 18:40:18 dev.example.com pulp[14074]: nectar.downloaders.threaded:INFO: Download succeeded: https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/drpms/test-alpha-1.1-2_1.1-3.noarch.drpm.
Jun 02 18:40:18 dev.example.com pulp[14074]: nectar.downloaders.threaded:INFO: Download succeeded: https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/drpms/test-alpha-1.1-3_1.2-1.noarch.drpm.
Jun 02 18:40:18 dev.example.com pulp[14074]: nectar.downloaders.threaded:INFO: Download succeeded: https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/drpms/test-alpha-1.1-1_1.1-2.noarch.drpm.
Jun 02 18:40:18 dev.example.com pulp[14074]: nectar.downloaders.threaded:INFO: Download succeeded: https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/drpms/test-alpha-1.2-1_1.2-2.noarch.drpm.
Jun 02 18:40:18 dev.example.com pulp[14074]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (4): repos.fedorapeople.org
Jun 02 18:40:18 dev.example.com pulp[14074]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (3): repos.fedorapeople.org
Jun 02 18:40:18 dev.example.com pulp[14074]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (2): repos.fedorapeople.org
Jun 02 18:40:18 dev.example.com pulp[14074]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): repos.fedorapeople.org
Jun 02 18:40:18 dev.example.com pulp[14074]: pulp_rpm.plugins.importers.yum.sync:INFO: [e001b643] Downloading 4 DRPMs.
Jun 02 18:40:18 dev.example.com pulp[14074]: pulp_rpm.plugins.importers.yum.repomd.alternate:INFO: [e001b643] The content container reported: {'downloads': {'___/primary/___': {'total_failed': 0, 'total_succeede
Jun 02 18:40:17 dev.example.com pulp[14074]: nectar.downloaders.threaded:INFO: Download succeeded: https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/test-alpha-1.1-1.noarch.rpm.
Jun 02 18:40:17 dev.example.com pulp[14074]: nectar.downloaders.threaded:INFO: Download succeeded: https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/test-alpha-1.2-2.noarch.rpm.
Jun 02 18:40:17 dev.example.com pulp[14074]: nectar.downloaders.threaded:INFO: Download succeeded: https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/test-alpha-1.2-1.noarch.rpm.
Jun 02 18:40:17 dev.example.com pulp[14074]: nectar.downloaders.threaded:INFO: Download succeeded: https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/test-alpha-1.1-3.noarch.rpm.
Jun 02 18:40:17 dev.example.com pulp[14074]: nectar.downloaders.threaded:INFO: Download succeeded: https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/test-alpha-1.1-2.noarch.rpm.
Jun 02 18:40:17 dev.example.com pulp[14074]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (5): repos.fedorapeople.org
Jun 02 18:40:17 dev.example.com pulp[14074]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (4): repos.fedorapeople.org
Jun 02 18:40:17 dev.example.com pulp[14074]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (3): repos.fedorapeople.org
Jun 02 18:40:17 dev.example.com pulp[14074]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (2): repos.fedorapeople.org
Jun 02 18:40:17 dev.example.com pulp[14074]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): repos.fedorapeople.org
Jun 02 18:40:17 dev.example.com pulp[14074]: pulp_rpm.plugins.importers.yum.sync:INFO: [e001b643] Downloading 5 RPMs.
Jun 02 18:40:17 dev.example.com pulp[14074]: pulp_rpm.plugins.importers.yum.sync:INFO: [e001b643] Determining which units need to be downloaded.
Jun 02 18:40:17 dev.example.com pulp[14074]: pulp_rpm.plugins.importers.yum.sync:INFO: [e001b643] Generating metadata databases.
Jun 02 18:40:16 dev.example.com pulp[14074]: nectar.downloaders.threaded:INFO: Download succeeded: https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/repodata/0e494af06cb44255ea21222c9c0e12891208b07128c0647b
Jun 02 18:40:16 dev.example.com pulp[14074]: nectar.downloaders.threaded:INFO: Download succeeded: https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/repodata/86b066bdc7f165d3a2add30190204afb71771a5fa528a81b
Jun 02 18:40:16 dev.example.com pulp[14074]: nectar.downloaders.threaded:INFO: Download succeeded: https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/repodata/82848c3367be1eed14de66ddc59548118ed3ea239f88d531
Jun 02 18:40:16 dev.example.com pulp[14074]: nectar.downloaders.threaded:INFO: Download succeeded: https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/repodata/1ab3e8a4570da8ef09c7fdd20db8e6fc5f3f7c08083ad808
Jun 02 18:40:16 dev.example.com pulp[14074]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (4): repos.fedorapeople.org
Jun 02 18:40:16 dev.example.com pulp[14074]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (3): repos.fedorapeople.org
Jun 02 18:40:16 dev.example.com pulp[14074]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (2): repos.fedorapeople.org
Jun 02 18:40:16 dev.example.com pulp[14074]: pulp_rpm.plugins.importers.yum.sync:INFO: [e001b643] Downloading metadata files.
Jun 02 18:40:16 dev.example.com pulp[14074]: pulp_rpm.plugins.importers.yum.sync:INFO: [e001b643] Parsing metadata.
Jun 02 18:40:15 dev.example.com pulp[14074]: nectar.downloaders.threaded:INFO: Download succeeded: https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/repodata/repomd.xml.
Jun 02 18:40:15 dev.example.com pulp[14074]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): repos.fedorapeople.org
Jun 02 18:40:15 dev.example.com pulp[14074]: pulp_rpm.plugins.importers.yum.sync:INFO: [e001b643] Downloading metadata from https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/.

Actions

Copy link

Updated by bmbouter almost 7 years ago

The output you paste show 4 DRPMs being sync'd and 4 DRPMs being published so I think that is enough to close the issue as WORKSFORME. Thanks for investigating mansari

Actions

Copy link