Project

Profile

Help

Issue #2657

closed

Publishing iso_rsync_distributor fails when iso_distributor is published with serve_https==False

Added by yuzheng about 7 years ago. Updated about 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.12.1
Platform Release:
2.12.2
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 17
Quarter:

Description

When ISO repo in Pulp is not published to https path, publishing iso_rsync_distributor fails due to trying to copy from a non-existing https path.

Reproducing steps:

1. Create an ISO repo which only serves via http
$ curl -k -X POST https://10.66.136.68/pulp/api/v2/repositories/ -H "Content-Type: application/json" -H  "Authorization: Basic YWRtaW46YWRtaW4=" -d '{"display_name": "test-repo23", "description": "test repo", "distributors": [{"distributor_id": "iso_distributor", "distributor_type_id": "iso_distributor", "distributor_config": {"serve_https": false, "serve_http": true, "relative_url": "content/dist/test-repo23"}}, {"distributor_id": "cdn_distributor", "distributor_type_id": "iso_rsync_distributor", "distributor_config": {"remote": {"auth_type": "publickey", "ssh_user": "cdn", "host": "10.66.136.68", "root": "/home/cdn/", "ssh_identity_file": "/var/www/.ssh/id_rsa"}, "predistributor_id": "iso_distributor"}}], "notes": {"rsync_repodata": "True", "_repo-type": "iso-repo", "relative_url": "content/dist/test-repo23"}, "importer_type_id": "iso_importer", "id": "test-repo23"}'
{"scratchpad": {}, "display_name": "test-repo23", "description": "test repo", "last_unit_added": null, "notes": {"rsync_repodata": "True", "_repo-type": "iso-repo", "relative_url": "content/dist/test-repo23"}, "last_unit_removed": null, "content_unit_counts": {}, "_ns": "repos", "_id": {"$oid": "58d3856020ed2e2185ee575b"}, "id": "test-repo23", "_href": "/pulp/api/v2/repositories/test-repo23/"}

2. Check distributors for created repo
db.repo_distributors.find({repo_id:"test-repo23"})
/* 0 */
{
    "_id" : ObjectId("58d3856020ed2e2185ee575d"),
    "repo_id" : "test-repo23",
    "distributor_id" : "iso_distributor",
    "distributor_type_id" : "iso_distributor",
    "config" : {
        "serve_https" : false,
        "relative_url" : "content/dist/test-repo23",
        "serve_http" : true
    },
    "auto_publish" : false,
    "last_updated" : ISODate("2017-03-23T08:20:48.592Z"),
    "last_override_config" : {},
    "scratchpad" : {},
    "_ns" : "repo_distributors",
    "last_publish" : ISODate("2017-03-23T08:22:15.719Z")
}

/* 1 */
{
    "_id" : ObjectId("58d3856020ed2e2185ee575e"),
    "repo_id" : "test-repo23",
    "distributor_id" : "cdn_distributor",
    "distributor_type_id" : "iso_rsync_distributor",
    "config" : {
        "remote" : {
            "auth_type" : "publickey",
            "ssh_user" : "cdn",
            "host" : "10.66.136.68",
            "root" : "/home/cdn/",
            "ssh_identity_file" : "/var/www/.ssh/id_rsa"
        },
        "predistributor_id" : "iso_distributor"
    },
    "auto_publish" : false,
    "last_updated" : ISODate("2017-03-23T08:20:48.595Z"),
    "last_override_config" : {},
    "scratchpad" : {},
    "_ns" : "repo_distributors"
}

3. Upload an ISO to this repo
# pulp-admin iso repo uploads upload --repo-id test-repo23 --file 3.iso
+----------------------------------------------------------------------+
                              Unit Upload
+----------------------------------------------------------------------+

...
[\]
Running...

Task Succeeded

Deleting the upload request...
... completed

4. Run ISO publish
# pulp-admin iso repo publish run --repo-id test-repo23
+----------------------------------------------------------------------+
                  Publishing Repository [test-repo18]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

The repository was successfully published.

Task Succeeded

4. Check ISO repo in Pulp
# ll /var/lib/pulp/published/http/isos/content/dist/test-repo23/
total 4
lrwxrwxrwx. 1 apache apache 97 Mar 23 16:22 3.iso -> /var/lib/pulp/content/units/iso/da/3ce6e699362190cb518be82ea132c67a38dda031afe8d79011ee1a2d2cf377
-rw-r--r--. 1 apache apache 80 Mar 23 16:22 PULP_MANIFEST

# ll /var/lib/pulp/published/https/isos/content/dist/test-repo23/
ls: cannot access /var/lib/pulp/published/https/isos/content/dist/test-repo23/: No such file or directory

Content is only published to http path as expected.

5. Run Rsycn publish
$ curl -k -X POST https://10.66.136.68/pulp/api/v2/repositories/test-repo23/actions/publish/ -H "Content-Type: application/json" -H  "Authorization: Basic YWRtaW46YWRtaW4=" -d '{"id": "cdn_distributor", "override_config": {}}'

The publish fails. Error log in Pulp server:
Mar 23 16:23:10 dhcp-136-68 pulp: pulp.plugins.rsync.publish:INFO: (8098-62688) ['rsync', '-rtKOzi', '--files-from', u'/var/cache/pulp/reserved_resource_worker-1@dhcp-136-68.nay.redhat.com/bd0c988e-0062-4556-9974-2c0c3427cb7b/f2360d71-7352-4abc-a967-f2bcaf1aa7a6', '--relative', '-e', u'ssh -l cdn -i /var/www/.ssh/id_rsa -o "StrictHostKeyChecking no" -o "UserKnownHostsFile /dev/null" -S /tmp/rsync_distributor-%r@%h:%p -o "ControlMaster auto" -o "ControlPersist 10"', '--copy-links', u'/var/lib/pulp/published/https/isos/content/dist/test-repo23', u'cdn@10.66.136.68:/home/cdn/content/dist/test-repo23']
Mar 23 16:23:10 dhcp-136-68 pulp: pulp.plugins.rsync.publish:INFO: (8098-62688) rsync: link_stat "/var/lib/pulp/published/https/isos/content/dist/test-repo23/PULP_MANIFEST" failed: No such file or directory (2)
Mar 23 16:23:10 dhcp-136-68 pulp: pulp.plugins.rsync.publish:INFO: (8098-62688) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1052) [sender=3.0.9]
Mar 23 16:23:10 dhcp-136-68 pulp: pulp.plugins.rsync.publish:INFO: (8098-62688)
Mar 23 16:23:10 dhcp-136-68 pulp: pulp.plugins.rsync.publish:ERROR: (8098-62688) ['rsync', '-rtKOzi', '--files-from', u'/var/cache/pulp/reserved_resource_worker-1@dhcp-136-68.nay.redhat.com/bd0c988e-0062-4556-9974-2c0c3427cb7b/f2360d71-7352-4abc-a967-f2bcaf1aa7a6', '--relative', '-e', u'ssh -l cdn -i /var/www/.ssh/id_rsa -o "StrictHostKeyChecking no" -o "UserKnownHostsFile /dev/null" -S /tmp/rsync_distributor-%r@%h:%p -o "ControlMaster auto" -o "ControlPersist 10"', '--copy-links', u'/var/lib/pulp/published/https/isos/content/dist/test-repo23', u'cdn@10.66.136.68:/home/cdn/content/dist/test-repo23']
Mar 23 16:23:10 dhcp-136-68 pulp: pulp.plugins.rsync.publish:ERROR: (8098-62688) rsync: link_stat "/var/lib/pulp/published/https/isos/content/dist/test-repo23/PULP_MANIFEST" failed: No such file or directory (2)
Mar 23 16:23:10 dhcp-136-68 pulp: pulp.plugins.rsync.publish:ERROR: (8098-62688) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1052) [sender=3.0.9]
Mar 23 16:23:10 dhcp-136-68 pulp: pulp.plugins.rsync.publish:ERROR: (8098-62688)

Also available in: Atom PDF