Project

Profile

Help

Test #4357

Pulp 2.19 master pulled epel django instead of pulp 2 django

Added by bherring 9 months ago. Updated 6 months ago.

Status:
CLOSED - COMPLETE
Priority:
Urgent
Assignee:
Category:
-
Sprint/Milestone:
-
Version:
Platform Release:
Tags:
Pulp 2
Sprint:

Description

Problem

When running nightly rhel76-fips testing, there were 500 errors [0] on jenkins tests since 1/23 and issues in journalctl with django [1].

After talking with asmacdo and daviddavis, it appears this may be a django being pulled from EPEL and not from our Pulp 2 repos on build [2][3]

Testing

It was determined this was the result of EPEL django precedence over the pulp.repo version that was built. This issue has been seen before.

Forced installation of the python2-django-1.11.17-1 from the fedorapeople repo [3] and restart of services (or with a hammer of shutdown now -r) resulted in a functioning pulp install.

[root@rhel76-fips ~]# journalctl -f
-- Logs begin at Mon 2019-01-28 12:33:13 EST. --
Jan 28 12:34:38 rhel76-fips pulp[5046]: kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424) Internal Server Error: /pulp/api/v2/status/
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424) Traceback (most recent call last):
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424)   File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 131, in get_response
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424)     response = middleware_method(request, response)
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424)   File "/usr/lib/python2.7/site-packages/django/middleware/http.py", line 23, in process_response
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424)     set_response_etag(response)
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424)   File "/usr/lib/python2.7/site-packages/django/utils/cache.py", line 109, in set_response_etag
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424)     response['ETag'] = quote_etag(hashlib.md5(response.content).hexdigest())
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424) ValueError: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips

Actions

Pulp-dev determined this issue as a low priority issue where documentation exercise may happen as a result of this issue.

For QE, this is a critical breakage of CI as the ansible install fails on FIPS installation and puts the box into a non-testable state. QE will be opening a related Urgent test tracker to fix the issue in the pulp-ci installer to fix mission critical work.

References

[0] - https://paste.fedoraproject.org/paste/bPRtYPa4ZCLShrZ7VUMqMQ
[1] - https://pulp-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/Pulp%202%20-%20Master/job/pulp-2-master-dev-rhel7-fips/228/
[2] - https://paste.fedoraproject.org/paste/6eHTS1Uc3NkC0hohvKe-TA [from the affected 2.19a1-nightly box under test
[3] - 2.11.17-1 https://repos.fedorapeople.org/repos/pulp/pulp/v2/testing/automation/2.19/stage/7Server/x86_64/
[4] - https://wiki.centos.org/PackageManagement/Yum/Priorities


Related issues

Related to Pulp - Test #4359: 2.18.1 Testing CLOSED - COMPLETE Actions
Copied from Pulp - Issue #4354: Pulp 2.19 master pulled epel django instead of pulp 2 django CLOSED - WONTFIX Actions

History

#1 Updated by bherring 9 months ago

  • Copied from Issue #4354: Pulp 2.19 master pulled epel django instead of pulp 2 django added

#2 Updated by bherring 9 months ago

  • Description updated (diff)
  • Status changed from NEW to ASSIGNED

#3 Updated by bherring 9 months ago

Notes

Need to decrease the priority (or have a value so default of 99 is not used) for pulp.repo [4].

Target fix currently is [5].

Additional References:

[4] - https://docs.ansible.com/ansible/latest/modules/yum_repository_module.html
[5] - https://github.com/pulp/pulp-ci/blob/master/ci/ansible/roles/pulp/tasks/main.yaml

#4 Updated by bherring 9 months ago

#5 Updated by bherring 9 months ago

  • Description updated (diff)

#6 Updated by bherring 9 months ago

Update

@Raga, if it helps, the change to our ansible-playbooks will look something like this in pulp_server --> role: -pulp --> task/main.yaml

- name: Setup Pulp nightly repository
  template:
    src: yum_repo.j2
    dest: /etc/yum.repos.d/{{ item.key }}.repo
  with_dict:
    pulp:
      name: Pulp Project repository
      baseurl: "https://repos.fedorapeople.org/pulp/pulp/testing/automation/{{ pulp_version }}/stage/{% if ansible_distribution == 'Fedora' %}fedora-{% endif %}$releasever/$basearch/" 
      gpgcheck: 0
      priority: 50
  when: pulp_build == "nightly" 

- name: Setup Pulp nightly koji repository
  template:
    src: yum_repo.j2
    dest: /etc/yum.repos.d/{{ item.key }}.repo
  with_dict:
    pulp:
      name: Pulp Project repository
      baseurl: "http://koji.katello.org/releases/yum/pulp-nightly/pulp/{% if ansible_distribution == 'Fedora' %}fedora-{% endif %}$releasever/$basearch/" 
      gpgcheck: 0
      priority: 50
  when: pulp_build == "nightly_koji" 

- name: Setup Pulp beta or stable repository
  template:
    src: yum_repo.j2
    dest: /etc/yum.repos.d/{{ item.key }}.repo
  with_dict:
    pulp:
      name: Pulp Project repository
      baseurl: "https://repos.fedorapeople.org/pulp/pulp/{{ pulp_build }}/{{ pulp_version }}/{% if ansible_distribution == 'Fedora' %}fedora-{% endif %}$releasever/$basearch/" 
      gpgcheck: 0
      priority: 50
  when: pulp_build == "beta" or pulp_build == "stable" 

- name: check fips state
  shell: cat /proc/sys/crypto/fips_enabled
  register: fips_state

- name: install yum-plugin-priorities
  package:
    name: yum-plugin-priorities
    state: present
  when: fips_state.stdout == '1'

Will be checking in later after 2.18.1.b1

#7 Updated by bherring 8 months ago

I mis-typed the PR, so it didn't auto-add: https://github.com/pulp/pulp-ci/pull/636

#8 Updated by bherring 8 months ago

  • Status changed from ASSIGNED to MODIFIED

#9 Updated by bherring 8 months ago

  • Status changed from MODIFIED to CLOSED - COMPLETE

#10 Updated by bmbouter 6 months ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF