Issue #2439: No data available in publish step (OSError: [Errno 61] No data available) - Pulp

Actions

Send by e-mail Copy link

Issue #2439

closed

No data available in publish step (OSError: [Errno 61] No data available)

Added by vaclav.adamec@suchy-zleb.cz over 7 years ago. Updated about 5 years ago.

Status:

CLOSED - CURRENTRELEASE

Priority:

Urgent

Assignee:

dkliban@redhat.com

Category:

Sprint/Milestone:

Start date:

Due date:

Estimated time:

Severity:

4. Urgent

Version:

2.10.2

Platform Release:

2.11.1

OS:

CentOS 7

Triaged:

Yes

Groomed:

Sprint Candidate:

Tags:

Pulp 2

Sprint:

Sprint 11

Quarter:

Description

Hi, I'm getting this error for some repozitories, not sure what's the reason If update to 2.10.2 or any issue on Artifactory side (as seems that affected are only these repos, at least most of them). Can you please help/show direction why this happened ?

pulp-admin rpm repo sync run --repo-id=custom-repo-7-x86_64-live
+----------------------------------------------------------------------+
           Synchronizing Repository [custom-repo-7-x86_64-live]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[\]
... completed

Downloading repository content...
[-]
[==================================================] 100%
RPMs:       0/0 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed

Cleaning duplicate packages...
[/]
... completed

Task Succeeded

Task Failed

[Errno 61] No data available

Log:

<pre>
Nov 23 12:29:01 pulp-server pulp[10516]: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading 0 RPMs.
Nov 23 12:29:03 pulp-server pulp[10516]: pulp_rpm.plugins.importers.yum.repomd.alternate:INFO: The content container reported: {'downloads': {}, 'total_sources': 0} for base URL: https://artifactory-master.custom/artifactory/yum-local/custom-repo/7/x86_64/
Nov 23 12:29:03 pulp-server pulp[10516]: pulp_rpm.plugins.importers.yum.purge:INFO: Removing missing units.
Nov 23 12:29:04 pulp-server pulp[10516]: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading additional units.
Nov 23 12:29:04 pulp-server pulp[10516]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): artifactory-master.custom
Nov 23 12:29:05 pulp-server pulp[10516]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): artifactory-master.custom
Nov 23 12:29:25 pulp-server pulp[10516]: pulp_rpm.plugins.importers.yum.sync:INFO: Sync complete.
Nov 23 12:29:25 pulp-server pulp[10337]: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[6fe1befa-19dc-46da-91c9-a875bb0e78e9]
Nov 23 12:29:25 pulp-server pulp[10375]: celery.worker.job:INFO: Task pulp.server.managers.repo.sync.sync[e83473e0-4aaf-4ed0-93a5-bf9d619147c4] succeeded in 37.9289886756s: <pulp.server.async.tasks.TaskResult object at 0x4377850>
Nov 23 12:29:25 pulp-server pulp[10375]: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[275a1307-c7f4-40b5-bf7b-5e2b5da00086] succeeded in 0.0216982029378s: None
Nov 23 12:29:25 pulp-server pulp[10375]: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.publish.publish[11aae4e6-88aa-46f2-93b0-0581da1847c3]
Nov 23 12:29:25 pulp-server pulp[10337]: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[6fe1befa-19dc-46da-91c9-a875bb0e78e9] succeeded in 0.0661013638601s: None
Nov 23 12:29:25 pulp-server pulp[10375]: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[10467dd3-7761-4cdb-89ce-43f0a6647b8f]
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608) Exception caught from plugin during publish for repo [custom-repo-7-x86_64-live]
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608) Traceback (most recent call last):
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 1115, in _do_publish
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)     publish_report = publish_repo(transfer_repo, conduit, call_config)
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 673, in wrap_f
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)     return f(*args, **kwargs)
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 174, in publish_repo
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)     return self._publisher.process_lifecycle()
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 566, in process_lifecycle
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)     super(PluginStep, self).process_lifecycle()
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 163, in process_lifecycle
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)     step.process()
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 253, in process
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)     self._process_block()
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 297, in _process_block
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)     self.process_main()
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 906, in process_main
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)     selinux.restorecon(timestamp_master_dir.encode('utf-8'), recursive=True)
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)   File "/usr/lib64/python2.7/site-packages/selinux/__init__.py", line 88, in restorecon
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608)     status, oldcontext = lgetfilecon(path)
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.controllers.repository:ERROR: (10516-16608) OSError: [Errno 61] No data available
Nov 23 12:29:26 pulp-server pulp[10516]: pulp.server.async.tasks:INFO: Task failed : [11aae4e6-88aa-46f2-93b0-0581da1847c3]
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608) Task pulp.server.managers.repo.publish.publish[11aae4e6-88aa-46f2-93b0-0581da1847c3] raised unexpected: OSError(61, 'No data available')
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608) Traceback (most recent call last):
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)     R = retval = fun(*args, **kwargs)
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 488, in __call__
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)     return super(Task, self).__call__(*args, **kwargs)
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in __call__
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)     return super(PulpTask, self).__call__(*args, **kwargs)
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)     return self.run(*args, **kwargs)
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 971, in publish
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)     result = check_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 1063, in check_publish
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)     result = _do_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 1115, in _do_publish
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)     publish_report = publish_repo(transfer_repo, conduit, call_config)
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 673, in wrap_f
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)     return f(*args, **kwargs)
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 174, in publish_repo
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)     return self._publisher.process_lifecycle()
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 566, in process_lifecycle
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)     super(PluginStep, self).process_lifecycle()
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 163, in process_lifecycle
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)     step.process()
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 253, in process
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)     self._process_block()
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 297, in _process_block
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)     self.process_main()
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 906, in process_main
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)     selinux.restorecon(timestamp_master_dir.encode('utf-8'), recursive=True)
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)   File "/usr/lib64/python2.7/site-packages/selinux/__init__.py", line 88, in restorecon
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608)     status, oldcontext = lgetfilecon(path)
Nov 23 12:29:26 pulp-server pulp[10375]: celery.worker.job:ERROR: (10375-16608) OSError: [Errno 61] No data available

Actions

Copy link

Updated by vaclav.adamec@suchy-zleb.cz over 7 years ago

pulp-admin repo list --repo-id=custom-repo-7-x86_64-live --details

+----------------------------------------------------------------------+
                              Repositories
+----------------------------------------------------------------------+

Id:                   custom-repo-7-x86_64-live
Display Name:         Custom repo 7/x86_64 (live)
Description:          Custom repo 7/x86_64 (live)
Content Unit Counts:
  Rpm: 183
Notes:
Scratchpad:
  Checksum Type: sha1
Importers:
  Config:
    Basic Auth Password: *****
    Basic Auth Username: xxxxx
    Feed:                https://artifactory-master.custom/artifactory/yum-l
                         ocal/custom-repo/7/x86_64
    Remove Missing:      True
    Type Skip List:      erratum
  Id:               yum_importer
  Importer Type Id: yum_importer
  Last Sync:        2016-11-23T11:49:24Z
  Repo Id:          custom-repo-7-x86_64-live
  Scratchpad:
    Previous Skip List:
    Repomd Revision:    0
Distributors:
  Auto Publish:         True
  Config:
    Checksum Type: sha1
    Gpgkey:        -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v2.0.14
 -----END PGP PUBLIC KEY BLOCK-----

    Http:          True
    Https:         False
    Relative URL:  custom-repo/live/7/x86_64
    Skip:          erratum
  Distributor Type Id:  yum_distributor
  Id:                   yum_distributor
  Last Override Config:
  Last Publish:         None
  Last Updated:         2016-11-23T11:52:26Z
  Repo Id:              custom-repo-7-x86_64-live
  Scratchpad:
  Auto Publish:         False
  Config:
    Http:         True
    Https:        False
    Relative URL: custom-repo/live/7/x86_64
    Skip:         erratum
  Distributor Type Id:  export_distributor
  Id:                   export_distributor
  Last Override Config:
  Last Publish:         None
  Last Updated:         2016-11-23T11:52:26Z
  Repo Id:              custom-repo-7-x86_64-live
  Scratchpad:

Actions

Copy link

Updated by vaclav.adamec@suchy-zleb.cz over 7 years ago

Seems that if I enable selinux than it's ok :-)

Actions

Copy link

Updated by dkliban@redhat.com over 7 years ago

Status changed from NEW to POST
Assignee set to dkliban@redhat.com
Priority changed from Normal to Urgent
Sprint/Milestone set to 29
Triaged changed from No to Yes

https://github.com/pulp/pulp/pull/2861/

Added by dkliban@redhat.com over 7 years ago

Revision 15d32052 | View on GitHub

Adds exception handling for when SELinux is disabled on EL7

On EL7, restorecon calls out to lgetfilecon which raises an exception when SELinux security context labels are not available.

https://www.gnu.org/software/gnulib/manual/html_node/lgetfilecon.html

closes #2439 https://pulp.plan.io/issues/2439

Added by dkliban@redhat.com over 7 years ago

Revision 15d32052 | View on GitHub

Adds exception handling for when SELinux is disabled on EL7

On EL7, restorecon calls out to lgetfilecon which raises an exception when SELinux security context labels are not available.

https://www.gnu.org/software/gnulib/manual/html_node/lgetfilecon.html

closes #2439 https://pulp.plan.io/issues/2439

Actions

Copy link

Updated by dkliban@redhat.com over 7 years ago

Status changed from POST to MODIFIED

Applied in changeset pulp|15d320525fcf0a09b4762d54687e22f16e5b8f40.

Actions

Copy link

Updated by semyers over 7 years ago

Platform Release set to 2.10.3

Actions

Copy link

Updated by semyers over 7 years ago

Platform Release changed from 2.10.3 to 2.11.1

Actions

Copy link

Updated by semyers over 7 years ago

Status changed from MODIFIED to 5

Actions

Copy link

Updated by pthomas@redhat.com over 7 years ago

Status changed from 5 to 6

verified

[root@qe-blade-05 ~]# pulp-admin rpm repo create --repo-id rpm --feed https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-signed/
Successfully created repository [rpm]

[root@qe-blade-05 ~]# getenforce
Disabled

[root@qe-blade-05 ~]# pulp-admin rpm repo sync run --repo-id rpm
+----------------------------------------------------------------------+
                     Synchronizing Repository [rpm]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       32/32 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed

Cleaning duplicate packages...
[-]
... completed

Task Succeeded

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[==================================================] 100%
32 of 32 items
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[==================================================] 100%
4 of 4 items
... completed

Publishing Comps file
[==================================================] 100%
4 of 4 items
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Generating HTML files
... skipped

Publishing files to web
[-]
... completed

Writing Listings File
[-]
... completed

Task Succeeded

Actions

Copy link