Issue #4725

Debian lazy sync virt-install fails w/ a 404 error, yet it's on the remote's CDN

Added by bmbouter over 2 years ago. Updated over 1 year ago.

Start date:
Due date:
Estimated time:
2. Medium
Platform Release:
Sprint Candidate:


mdellweg and I both are experiencing this issue with the latest source checkouts of pulpcore, pulpcore-plugin and the pulp_deb. Using pulp_lift you can easily get into this setup.

Then sync+publish w/ a script like:

# Sync Repository
export REPOSITORY=$(http POST :24817/pulp/api/v3/repositories/ name=debian_stretch | jq -r '._href')
export REMOTE=$(http POST :24817/pulp/api/v3/remotes/deb/apt/ name=debian_stretch policy=streamed url="" distributions=stretch components=main architectures=amd64 sync_udebs=True sync_installer=True | jq -r '._href')
export SYNC_TASK=$(http POST :24817${REMOTE}sync/ repository=${REPOSITORY} | jq -r '.task')
# wait for task to finish
export VERSION=$(http :24817${SYNC_TASK} | jq -r '.created_resources[0]')

# Publish Repository
export PUBLISHER=$(http POST :24817/pulp/api/v3/publishers/deb/verbatim/ name=verbatim_stretch | jq -r '._href')
export PUBLISH_TASK=$(http POST :24817${PUBLISHER}publish/ repository=${REPOSITORY} | jq -r '.task')
# wait for task to finish
export PUBLICATION=$(http :24817${PUBLISH_TASK} | jq -r '.created_resources[0]')
http POST :24817/pulp/api/v3/distributions/ name=verbatim_stretch base_path=verbatim_stretch publication=${PUBLICATION}

You'll need your local ports forwarding from all local interfaces to the pulplift VM for ports 24816 and 24817. You can do this with something like:

vagrant ssh pulp3-source-fedora29 -- -R 29437:localhost:29437 -R 29438:localhost:29438 -L *:24816: -L *:24817:

Then virt-install with a command like:

virt-install     \
    --name deb-pulp-install \
    --disk size=10 \
    --memory 1024 \
    --connect qemu:///system \
    --os-variant debian9 \
    --location \
    --initrd-inject preseed.cfg

Eventually your system will fail with an error that looks like:

It's requesting the URL:

which should exist because this URL exists:

deb_install-1.png (15.7 KB) deb_install-1.png bmbouter, 04/23/2019 06:57 PM


#1 Updated by bmbouter over 2 years ago

In debugging the pulp-streamer the request that shows up here on the call that returns the 404 it is showing up as: 'verbatim_stretch/pool/main/b/busybox/busybox_1.22.0-19%2Bb3_amd64.deb'

That is strange because the console output shows it having the '+' on it. Because it's not unquoted it's failing to match on the ContentArtifact.relative_path here

#2 Updated by bmbouter over 2 years ago

I confirmed that this PR fix does resolve the issue, but I'm still labelling it as a WIP:

I'm going to look around a bit more though because I don't think the unquote should be necessary...

#3 Updated by bmbouter over 2 years ago

  • Status changed from ASSIGNED to POST

I enabled access logging and reran the virt-install and I can see the URL that is failing is being requested as:

GET /pulp/content/verbatim_stretch/pool/main/b/busybox/busybox_1.22.0-19%2Bb3_amd64.deb

In debugging I can see that both request.path and request.raw_path are both /pulp/content/verbatim_stretch/pool/main/b/busybox/busybox_1.22.0-19%2Bb3_amd64.deb which means aiohttp is failing to unquote like it should. I filed this bug to resolve that.

I moved this to POST because we can merge the unquote workaround now and I also filed to remove it before the GA release. The PR is here:

#4 Updated by over 2 years ago

  • Triaged changed from No to Yes

#5 Updated by bmbouter about 2 years ago

  • Status changed from POST to NEW

#6 Updated by fao89 about 2 years ago

  • Sprint/Milestone set to 3.0.0

#7 Updated by fao89 about 2 years ago

I also got this error when I was testing kickstarts installing CentOS 7

#8 Updated by bmbouter about 2 years ago

  • Assignee deleted (bmbouter)

#9 Updated by bmbouter about 2 years ago

  • Sprint/Milestone deleted (3.0.0)

Removing from 3.0 blocker because there won't be a code change Pulp can make to fix it. Also yarl has fixes it, but they have not yet released that fix.

#10 Updated by bmbouter over 1 year ago

  • Status changed from NEW to CLOSED - CURRENTRELEASE

This has been resolved by yarl 1.4.0 which is released and shows the fix in its changelog here: "Fix quoting of plus in path by pure python version (#339)"

Please register to edit this issue

Also available in: Atom PDF