Project

Profile

Help

Issue #2077

publish fails if Django Template Syntax in changelog

Added by st0ne over 5 years ago. Updated almost 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Urgent
Assignee:
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Master
Platform Release:
2.9.2
OS:
RHEL 7
Triaged:
Yes
Groomed:
No
Sprint Candidate:
Yes
Tags:
Pulp 2
Sprint:
Sprint 6
Quarter:

Description

The CentOS 5 rpm package python-suds-0.4.1-2.el5.noarch.rpm contains a django template syntax like comment in its changelog.

changelog snippet

...
* Mon Dec 08 2008 jortel <jortel@redhat.com> - 0.3.4-1
- Static (automatic) Import.bind('http://schemas.xmlsoap.org/soap/encoding/')
- Basic ws-security with {{{UsernameToken}}} and clear-text password only.
- Add support for ''sparse'' soap headers via passing dictionary
- Add support for arbitrary user defined soap headers
...

When i try to publish the CentOS 5 repository it exits with the following error:

...
Publishing RPMs
[|]

Task Failed

Could not parse the remainder: '{UsernameToken' from '{UsernameToken'
test.py (1.38 KB) test.py Suggested change fdobrovo, 07/27/2016 02:36 PM

Related issues

Related to RPM Support - Issue #2135: Repo publish error Unclosed tags: endverbatim CLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Has duplicate Pulp - Issue #2079: Fails to sync EPEL 7.x repository with Pulp 2.9.0CLOSED - DUPLICATE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision cba1edd4 View on GitHub
Added by ttereshc over 5 years ago

Escape Django template variables in metadata during publish

closes #2077 https://pulp.plan.io/issues/2077

Revision c42d1d26 View on GitHub
Added by ttereshc over 5 years ago

Backport verbatim tag from Django 1.5

re #2077 https://pulp.plan.io/issues/2077

Revision c42d1d26 View on GitHub
Added by ttereshc over 5 years ago

Backport verbatim tag from Django 1.5

re #2077 https://pulp.plan.io/issues/2077

History

#1 Updated by amacdona@redhat.com over 5 years ago

  • Priority changed from Normal to High
  • Severity changed from 2. Medium to 3. High
  • Triaged changed from No to Yes

#3 Updated by mhrivnak over 5 years ago

  • Sprint Candidate changed from No to Yes

#4 Updated by mhrivnak over 5 years ago

  • Sprint/Milestone set to 24

#5 Updated by mhrivnak over 5 years ago

  • Has duplicate Issue #2079: Fails to sync EPEL 7.x repository with Pulp 2.9.0 added

#6 Updated by mhrivnak over 5 years ago

I did the original implementation of template rendering during publish, so I think this is a good opportunity for someone else to work on it and get familiar with it.

Reading what django has to offer, this could be difficult. There is no way to "escape" characters in a template.

One option, that's a bit ugly, is to use the "templatetag" keyword in the templates. https://docs.djangoproject.com/en/1.8/ref/templates/builtins/#templatetag

In that case, we would for example replace any occurrence of "{{" with "{% templatetag openvariable %}". The string values are mapped internally in django here:

https://github.com/django/django/blob/fff5dbe59ca629c295480693f045f03537858eee/django/template/defaulttags.py#L403

A better option would probably be to switch to jinja2 as a template engine, which supports escaping: http://jinja.pocoo.org/docs/dev/templates/#escaping

The current template syntax we've used from django's engine is 100% compatible with jinja2. The challenges would be dealing with django 1.4, which doesn't have the swappable engine concept that 1.8 has, and writing a migration to modify all of the existing templates in the DB.

#7 Updated by fdobrovo over 5 years ago

And what about using verbatim? https://docs.djangoproject.com/en/1.8/ref/templates/builtins/#verbatim

I also included a snippet how I think it could be done.

#8 Updated by fdobrovo over 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to fdobrovo

#9 Updated by Anonymous over 5 years ago

fdobrovo wrote:

And what about using verbatim? https://docs.djangoproject.com/en/1.8/ref/templates/builtins/#verbatim

I also included a snippet how I think it could be done.

Seems like this still present on 2.9.1

#10 Updated by fdobrovo over 5 years ago

  • Status changed from ASSIGNED to POST

#11 Updated by ttereshc over 5 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#12 Updated by ttereshc over 5 years ago

  • Platform Release set to 2.9.2

#13 Updated by semyers over 5 years ago

  • Project changed from Pulp to RPM Support
  • Category deleted (15)

#14 Updated by semyers over 5 years ago

  • Status changed from MODIFIED to 5

#15 Updated by ttereshc over 5 years ago

  • Status changed from 5 to ASSIGNED
  • Assignee changed from fdobrovo to ttereshc

#16 Updated by ttereshc over 5 years ago

  • Status changed from ASSIGNED to 5
  • Assignee changed from ttereshc to fdobrovo

#17 Updated by ttereshc over 5 years ago

  • Related to Issue #2135: Repo publish error Unclosed tags: endverbatim added

#18 Updated by semyers over 5 years ago

  • Priority changed from High to Urgent

#19 Updated by semyers over 5 years ago

While this fix worked, it's been found to be incomplete due to issues tracked in https://pulp.plan.io/issues/2135 (related). That fix is currently in review, but it probably makes sense to wait for #2135 to be MODIFIED and included in a new build before testing both.

#20 Updated by pthomas@redhat.com over 5 years ago

  • Status changed from 5 to 6

verified
[root@tigger ~]# rpm -qa pulp-server
pulp-server-2.9.2-0.3.beta.el7.noarch
[root@tigger ~]#

[root@tigger ~]# pulp-admin rpm repo sync run --repo-id centos5
+----------------------------------------------------------------------+
                   Synchronizing Repository [centos5]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[|]
... completed

Downloading repository content...
[-]
[==================================================] 100%
RPMs:       0/0 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[\]
... completed

Cleaning duplicate packages...
[\]
... completed

Task Succeeded

Copying files
[-]
... completed

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[\]
... completed

Publishing RPMs
[/]
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[\]
... completed

Publishing Comps file
[==================================================] 100%
117 of 117 items
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Generating HTML files
... skipped

Publishing files to web
[|]
... completed

Writing Listings File
[-]
... completed

Task Succeeded

[root@tigger ~]# 
[root@tigger ~]# 
[root@tigger ~]# pulp-admin rpm repo content rpm --repo-id centos5 --str-eq name=python-suds
Arch:         noarch
Buildhost:    builder10.centos.org
Checksum:     190c190dea16a25cdc25dbea4b7b8cd63cc060fb
Checksumtype: sha1
Description:  The suds project is a python soap web services client lib.  Suds
              leverages python meta programming to provide an intuitive API for
              consuming web services.  Objectification of types defined in the
              WSDL is provided without class generation.  Programmers rarely
              need to read the WSDL since services and WSDL based objects can be
              easily inspected.
Epoch:        0
Filename:     python-suds-0.4.1-2.el5.noarch.rpm
License:      LGPLv3+
Name:         python-suds
Provides:     python-suds = 0.4.1-2.el5-0
Release:      2.el5
Requires:     python >= 2.4-0, rpmlib(CompressedFileNames) <= 3.0.4-1-0,
              python(abi) = 2.4-0, rpmlib(PayloadFilesHavePrefix) <= 4.0-1-0
Version:      0.4.1

[root@tigger ~]# 

#21 Updated by semyers over 5 years ago

  • Status changed from 6 to CLOSED - CURRENTRELEASE

#23 Updated by bmbouter almost 4 years ago

  • Sprint set to Sprint 6

#24 Updated by bmbouter almost 4 years ago

  • Sprint/Milestone deleted (24)

#25 Updated by bmbouter almost 3 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF