Actions
Issue #2657
closedPublishing iso_rsync_distributor fails when iso_distributor is published with serve_https==False
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.12.1
Platform Release:
2.12.2
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 17
Quarter:
Description
When ISO repo in Pulp is not published to https path, publishing iso_rsync_distributor fails due to trying to copy from a non-existing https path.
Reproducing steps:
1. Create an ISO repo which only serves via http
$ curl -k -X POST https://10.66.136.68/pulp/api/v2/repositories/ -H "Content-Type: application/json" -H "Authorization: Basic YWRtaW46YWRtaW4=" -d '{"display_name": "test-repo23", "description": "test repo", "distributors": [{"distributor_id": "iso_distributor", "distributor_type_id": "iso_distributor", "distributor_config": {"serve_https": false, "serve_http": true, "relative_url": "content/dist/test-repo23"}}, {"distributor_id": "cdn_distributor", "distributor_type_id": "iso_rsync_distributor", "distributor_config": {"remote": {"auth_type": "publickey", "ssh_user": "cdn", "host": "10.66.136.68", "root": "/home/cdn/", "ssh_identity_file": "/var/www/.ssh/id_rsa"}, "predistributor_id": "iso_distributor"}}], "notes": {"rsync_repodata": "True", "_repo-type": "iso-repo", "relative_url": "content/dist/test-repo23"}, "importer_type_id": "iso_importer", "id": "test-repo23"}'
{"scratchpad": {}, "display_name": "test-repo23", "description": "test repo", "last_unit_added": null, "notes": {"rsync_repodata": "True", "_repo-type": "iso-repo", "relative_url": "content/dist/test-repo23"}, "last_unit_removed": null, "content_unit_counts": {}, "_ns": "repos", "_id": {"$oid": "58d3856020ed2e2185ee575b"}, "id": "test-repo23", "_href": "/pulp/api/v2/repositories/test-repo23/"}
2. Check distributors for created repo
db.repo_distributors.find({repo_id:"test-repo23"})
/* 0 */
{
"_id" : ObjectId("58d3856020ed2e2185ee575d"),
"repo_id" : "test-repo23",
"distributor_id" : "iso_distributor",
"distributor_type_id" : "iso_distributor",
"config" : {
"serve_https" : false,
"relative_url" : "content/dist/test-repo23",
"serve_http" : true
},
"auto_publish" : false,
"last_updated" : ISODate("2017-03-23T08:20:48.592Z"),
"last_override_config" : {},
"scratchpad" : {},
"_ns" : "repo_distributors",
"last_publish" : ISODate("2017-03-23T08:22:15.719Z")
}
/* 1 */
{
"_id" : ObjectId("58d3856020ed2e2185ee575e"),
"repo_id" : "test-repo23",
"distributor_id" : "cdn_distributor",
"distributor_type_id" : "iso_rsync_distributor",
"config" : {
"remote" : {
"auth_type" : "publickey",
"ssh_user" : "cdn",
"host" : "10.66.136.68",
"root" : "/home/cdn/",
"ssh_identity_file" : "/var/www/.ssh/id_rsa"
},
"predistributor_id" : "iso_distributor"
},
"auto_publish" : false,
"last_updated" : ISODate("2017-03-23T08:20:48.595Z"),
"last_override_config" : {},
"scratchpad" : {},
"_ns" : "repo_distributors"
}
3. Upload an ISO to this repo
# pulp-admin iso repo uploads upload --repo-id test-repo23 --file 3.iso
+----------------------------------------------------------------------+
Unit Upload
+----------------------------------------------------------------------+
...
[\]
Running...
Task Succeeded
Deleting the upload request...
... completed
4. Run ISO publish
# pulp-admin iso repo publish run --repo-id test-repo23
+----------------------------------------------------------------------+
Publishing Repository [test-repo18]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
The repository was successfully published.
Task Succeeded
4. Check ISO repo in Pulp
# ll /var/lib/pulp/published/http/isos/content/dist/test-repo23/
total 4
lrwxrwxrwx. 1 apache apache 97 Mar 23 16:22 3.iso -> /var/lib/pulp/content/units/iso/da/3ce6e699362190cb518be82ea132c67a38dda031afe8d79011ee1a2d2cf377
-rw-r--r--. 1 apache apache 80 Mar 23 16:22 PULP_MANIFEST
# ll /var/lib/pulp/published/https/isos/content/dist/test-repo23/
ls: cannot access /var/lib/pulp/published/https/isos/content/dist/test-repo23/: No such file or directory
Content is only published to http path as expected.
5. Run Rsycn publish
$ curl -k -X POST https://10.66.136.68/pulp/api/v2/repositories/test-repo23/actions/publish/ -H "Content-Type: application/json" -H "Authorization: Basic YWRtaW46YWRtaW4=" -d '{"id": "cdn_distributor", "override_config": {}}'
The publish fails. Error log in Pulp server:
Mar 23 16:23:10 dhcp-136-68 pulp: pulp.plugins.rsync.publish:INFO: (8098-62688) ['rsync', '-rtKOzi', '--files-from', u'/var/cache/pulp/reserved_resource_worker-1@dhcp-136-68.nay.redhat.com/bd0c988e-0062-4556-9974-2c0c3427cb7b/f2360d71-7352-4abc-a967-f2bcaf1aa7a6', '--relative', '-e', u'ssh -l cdn -i /var/www/.ssh/id_rsa -o "StrictHostKeyChecking no" -o "UserKnownHostsFile /dev/null" -S /tmp/rsync_distributor-%r@%h:%p -o "ControlMaster auto" -o "ControlPersist 10"', '--copy-links', u'/var/lib/pulp/published/https/isos/content/dist/test-repo23', u'cdn@10.66.136.68:/home/cdn/content/dist/test-repo23']
Mar 23 16:23:10 dhcp-136-68 pulp: pulp.plugins.rsync.publish:INFO: (8098-62688) rsync: link_stat "/var/lib/pulp/published/https/isos/content/dist/test-repo23/PULP_MANIFEST" failed: No such file or directory (2)
Mar 23 16:23:10 dhcp-136-68 pulp: pulp.plugins.rsync.publish:INFO: (8098-62688) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1052) [sender=3.0.9]
Mar 23 16:23:10 dhcp-136-68 pulp: pulp.plugins.rsync.publish:INFO: (8098-62688)
Mar 23 16:23:10 dhcp-136-68 pulp: pulp.plugins.rsync.publish:ERROR: (8098-62688) ['rsync', '-rtKOzi', '--files-from', u'/var/cache/pulp/reserved_resource_worker-1@dhcp-136-68.nay.redhat.com/bd0c988e-0062-4556-9974-2c0c3427cb7b/f2360d71-7352-4abc-a967-f2bcaf1aa7a6', '--relative', '-e', u'ssh -l cdn -i /var/www/.ssh/id_rsa -o "StrictHostKeyChecking no" -o "UserKnownHostsFile /dev/null" -S /tmp/rsync_distributor-%r@%h:%p -o "ControlMaster auto" -o "ControlPersist 10"', '--copy-links', u'/var/lib/pulp/published/https/isos/content/dist/test-repo23', u'cdn@10.66.136.68:/home/cdn/content/dist/test-repo23']
Mar 23 16:23:10 dhcp-136-68 pulp: pulp.plugins.rsync.publish:ERROR: (8098-62688) rsync: link_stat "/var/lib/pulp/published/https/isos/content/dist/test-repo23/PULP_MANIFEST" failed: No such file or directory (2)
Mar 23 16:23:10 dhcp-136-68 pulp: pulp.plugins.rsync.publish:ERROR: (8098-62688) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1052) [sender=3.0.9]
Mar 23 16:23:10 dhcp-136-68 pulp: pulp.plugins.rsync.publish:ERROR: (8098-62688)
Updated by ipanova@redhat.com over 7 years ago
The issue is options name
def _get_root_publish_dir(self):
"""
Returns the publish directory path for the predistributor.
:return: absolute path to the master publish directory of predistributor
:rtype: str
"""
if self.predistributor["config"].get("http", False):
return constants.ISO_HTTP_DIR
else:
return constants.ISO_HTTPS_DIR
We need to change code to look for 'serve_http'
Updated by ipanova@redhat.com over 7 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to ipanova@redhat.com
Updated by bizhang over 7 years ago
- Sprint/Milestone set to 36
- Triaged changed from No to Yes
Added by ipanova@redhat.com over 7 years ago
Updated by ipanova@redhat.com over 7 years ago
- Status changed from ASSIGNED to POST
Updated by ipanova@redhat.com over 7 years ago
- Status changed from POST to MODIFIED
Applied in changeset 4878b2e3df1903f99bd319b04c2321b17d47feb9.
Updated by Ichimonji10 over 7 years ago
Verified against Pulp 2.12 and 2.13 nightlies on F24, F25 and RHEL 7. For reference, here's the packages installed on one of my test systems:
[root@rhel-7-3-pulp-2-12 ~]# rpm -qa | grep -i pulp | sort
pulp-admin-client-2.12.3-0.1.alpha.git.1202.52400ed.el7.noarch
pulp-docker-admin-extensions-2.3.1-0.1.alpha.git.5.052c506.el7.noarch
pulp-docker-plugins-2.3.1-0.1.alpha.git.5.052c506.el7.noarch
pulp-ostree-admin-extensions-1.2.2-0.1.alpha.git.3.809be44.el7.noarch
pulp-ostree-plugins-1.2.2-0.1.alpha.git.3.809be44.el7.noarch
pulp-puppet-admin-extensions-2.12.3-0.1.alpha.git.7.89c545a.el7.noarch
pulp-puppet-plugins-2.12.3-0.1.alpha.git.7.89c545a.el7.noarch
pulp-python-admin-extensions-1.1.4-0.1.alpha.git.31.36b75e3.el7.noarch
pulp-python-plugins-1.1.4-0.1.alpha.git.31.36b75e3.el7.noarch
pulp-rpm-admin-extensions-2.12.3-0.1.alpha.git.20.5e18bff.el7.noarch
pulp-rpm-plugins-2.12.3-0.1.alpha.git.20.5e18bff.el7.noarch
pulp-selinux-2.12.3-0.1.alpha.git.1202.52400ed.el7.noarch
pulp-server-2.12.3-0.1.alpha.git.1202.52400ed.el7.noarch
python-isodate-0.5.0-4.pulp.el7.noarch
python-kombu-3.0.33-6.pulp.el7.noarch
python-pulp-bindings-2.12.3-0.1.alpha.git.1202.52400ed.el7.noarch
python-pulp-client-lib-2.12.3-0.1.alpha.git.1202.52400ed.el7.noarch
python-pulp-common-2.12.3-0.1.alpha.git.1202.52400ed.el7.noarch
python-pulp-docker-common-2.3.1-0.1.alpha.git.5.052c506.el7.noarch
python-pulp-oid_validation-2.12.3-0.1.alpha.git.1202.52400ed.el7.noarch
python-pulp-ostree-common-1.2.2-0.1.alpha.git.3.809be44.el7.noarch
python-pulp-puppet-common-2.12.3-0.1.alpha.git.7.89c545a.el7.noarch
python-pulp-python-common-1.1.4-0.1.alpha.git.31.36b75e3.el7.noarch
python-pulp-repoauth-2.12.3-0.1.alpha.git.1202.52400ed.el7.noarch
python-pulp-rpm-common-2.12.3-0.1.alpha.git.20.5e18bff.el7.noarch
python-pulp-streamer-2.12.3-0.1.alpha.git.1202.52400ed.el7.noarch
Updated by bizhang over 7 years ago
- Status changed from 5 to CLOSED - CURRENTRELEASE
Actions
Fix iso distributor option config name.
closes #2657 https://pulp.plan.io/issues/2657