Project

Profile

Help

Issue #5655

closed

RPM repo remote URL without trailing slash causes 404

Added by iballou about 5 years ago. Updated over 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 61
Quarter:

Description

I was trying to sync EPEL 7 with the remote URL as https://dl.fedoraproject.org/pub/epel/7/x86_64. When syncing, Pulp chopped the 'x86_64' off:

"404, message='Not Found', url=URL('https://dl.fedoraproject.org/pub/epel/7/repodata/repomd.xml')"},

When I added the trailing slash and updated the remote, the syncing worked.

Actions #1

Updated by ttereshc about 5 years ago

  • Project changed from RPM Support to Pulp
Actions #2

Updated by ttereshc about 5 years ago

  • Project changed from Pulp to RPM Support
  • Triaged changed from No to Yes
  • Sprint set to Sprint 61
Actions #3

Updated by fao89 about 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to fao89
Actions #4

Updated by fao89 about 5 years ago

In [1]: url_slash = "https://dl.fedoraproject.org/pub/epel/7/x86_64/"                                                                                                                                                                         

In [2]: url = "https://dl.fedoraproject.org/pub/epel/7/x86_64"                                                                                                                                                                                

In [3]: from urllib.parse import urljoin                                                                                                                                                                                                      

In [4]: repomd = "repodata/repomd.xml"                                                                                                                                                                                                        

In [5]: urljoin(url_slash, repomd)                                                                                                                                                                                                            
Out[5]: 'https://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/repomd.xml'

In [6]: urljoin(url, repomd)                                                                                                                                                                                                                  
Out[6]: 'https://dl.fedoraproject.org/pub/epel/7/repodata/repomd.xml'
Actions #5

Updated by fao89 about 5 years ago

Signature: urljoin(base, url, allow_fragments=True)
Source:   
def urljoin(base, url, allow_fragments=True):
    """Join a base URL and a possibly relative URL to form an absolute
    interpretation of the latter."""
    if not base:
        return url
    if not url:
        return base

    base, url, _coerce_result = _coerce_args(base, url)
    bscheme, bnetloc, bpath, bparams, bquery, bfragment = \
            urlparse(base, '', allow_fragments)
    scheme, netloc, path, params, query, fragment = \
            urlparse(url, bscheme, allow_fragments)

    if scheme != bscheme or scheme not in uses_relative:
        return _coerce_result(url)
    if scheme in uses_netloc:
        if netloc:
            return _coerce_result(urlunparse((scheme, netloc, path,
                                              params, query, fragment)))
        netloc = bnetloc

    if not path and not params:
        path = bpath
        params = bparams
        if not query:
            query = bquery
        return _coerce_result(urlunparse((scheme, netloc, path,
                                          params, query, fragment)))

    base_parts = bpath.split('/')
    if base_parts[-1] != '':
        # the last item is not a directory, so will not be taken into account
        # in resolving the relative path
        del base_parts[-1]

    # for rfc3986, ignore all base path should the first character be root.
    if path[:1] == '/':
        segments = path.split('/')
    else:
        segments = base_parts + path.split('/')
        # filter out elements that would cause redundant slashes on re-joining
        # the resolved_path
        segments[1:-1] = filter(None, segments[1:-1])

    resolved_path = []

    for seg in segments:
        if seg == '..':
            try:
                resolved_path.pop()
            except IndexError:
                # ignore any .. segments that would otherwise cause an IndexError
                # when popped from resolved_path if resolving for rfc3986
                pass
        elif seg == '.':
            continue
        else:
            resolved_path.append(seg)

    if segments[-1] in ('.', '..'):
        # do some post-processing here. if the last segment was a relative dir,
        # then we need to append the trailing '/'
        resolved_path.append('')

    return _coerce_result(urlunparse((scheme, netloc, '/'.join(
        resolved_path) or '/', params, query, fragment)))
File:      ~/.pyenv/versions/3.7.3/lib/python3.7/urllib/parse.py
Type:      function

Added by Fabricio Aguiar about 5 years ago

Revision f696e13e | View on GitHub

Fix 404: repo remote URL is without trailing slash

https://pulp.plan.io/issues/5655 closes #5655

Actions #6

Updated by fao89 about 5 years ago

  • Status changed from ASSIGNED to POST
Actions #7

Updated by Anonymous about 5 years ago

  • Status changed from POST to MODIFIED
Actions #8

Updated by ttereshc almost 5 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions #9

Updated by ggainey over 4 years ago

  • Tags Katello added
  • Tags deleted (Katello-P2)

Also available in: Atom PDF