Test #4357: Pulp 2.19 master pulled epel django instead of pulp 2 django - Pulp

Actions

Send by e-mail Copy link

Test #4357

closed

Pulp 2.19 master pulled epel django instead of pulp 2 django

Added by bherring about 5 years ago. Updated about 5 years ago.

Status:

CLOSED - COMPLETE

Priority:

Urgent

Assignee:

bherring

Category:

Sprint/Milestone:

Version:

Platform Release:

Tags:

Pulp 2

Sprint:

Quarter:

Description

Problem¶

When running nightly rhel76-fips testing, there were 500 errors [0] on jenkins tests since 1/23 and issues in journalctl with django [1].

After talking with asmacdo and daviddavis, it appears this may be a django being pulled from EPEL and not from our Pulp 2 repos on build [2][3]

Testing¶

It was determined this was the result of EPEL django precedence over the pulp.repo version that was built. This issue has been seen before.

Forced installation of the python2-django-1.11.17-1 from the fedorapeople repo [3] and restart of services (or with a hammer of shutdown now -r) resulted in a functioning pulp install.

[root@rhel76-fips ~]# journalctl -f
-- Logs begin at Mon 2019-01-28 12:33:13 EST. --
Jan 28 12:34:38 rhel76-fips pulp[5046]: kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424) Internal Server Error: /pulp/api/v2/status/
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424) Traceback (most recent call last):
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424)   File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 131, in get_response
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424)     response = middleware_method(request, response)
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424)   File "/usr/lib/python2.7/site-packages/django/middleware/http.py", line 23, in process_response
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424)     set_response_etag(response)
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424)   File "/usr/lib/python2.7/site-packages/django/utils/cache.py", line 109, in set_response_etag
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424)     response['ETag'] = quote_etag(hashlib.md5(response.content).hexdigest())
Jan 28 12:34:38 rhel76-fips pulp[5046]: django.request:ERROR: (5046-27424) ValueError: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips

Actions¶

Pulp-dev determined this issue as a low priority issue where documentation exercise may happen as a result of this issue.

For QE, this is a critical breakage of CI as the ansible install fails on FIPS installation and puts the box into a non-testable state. QE will be opening a related Urgent test tracker to fix the issue in the pulp-ci installer to fix mission critical work.

References¶

[0] - https://paste.fedoraproject.org/paste/bPRtYPa4ZCLShrZ7VUMqMQ
[1] - https://pulp-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/Pulp%202%20-%20Master/job/pulp-2-master-dev-rhel7-fips/228/
[2] - https://paste.fedoraproject.org/paste/6eHTS1Uc3NkC0hohvKe-TA [from the affected 2.19a1-nightly box under test
[3] - 2.11.17-1 https://repos.fedorapeople.org/repos/pulp/pulp/v2/testing/automation/2.19/stage/7Server/x86_64/
[4] - https://wiki.centos.org/PackageManagement/Yum/Priorities

Related issues

Actions

Copy link

Updated by bherring about 5 years ago

Copied from Issue #4354: Pulp 2.19 master pulled epel django instead of pulp 2 django added

Actions

Copy link

Updated by bherring about 5 years ago

Description updated (diff)
Status changed from NEW to ASSIGNED

Actions

Copy link

Updated by bherring about 5 years ago

Notes¶

Need to decrease the priority (or have a value so default of 99 is not used) for pulp.repo [4].

Target fix currently is [5].

Additional References:¶

[4] - https://docs.ansible.com/ansible/latest/modules/yum_repository_module.html
[5] - https://github.com/pulp/pulp-ci/blob/master/ci/ansible/roles/pulp/tasks/main.yaml

Actions

Copy link

Updated by bherring about 5 years ago

Related to Test #4359: 2.18.1 Testing added

Actions

Copy link

Updated by bherring about 5 years ago

Description updated (diff)

Actions

Copy link

Updated by bherring about 5 years ago

Update¶

@Raga, if it helps, the change to our ansible-playbooks will look something like this in pulp_server --> role: -pulp --> task/main.yaml

- name: Setup Pulp nightly repository
  template:
    src: yum_repo.j2
    dest: /etc/yum.repos.d/{{ item.key }}.repo
  with_dict:
    pulp:
      name: Pulp Project repository
      baseurl: "https://repos.fedorapeople.org/pulp/pulp/testing/automation/{{ pulp_version }}/stage/{% if ansible_distribution == 'Fedora' %}fedora-{% endif %}$releasever/$basearch/"
      gpgcheck: 0
      priority: 50
  when: pulp_build == "nightly"

- name: Setup Pulp nightly koji repository
  template:
    src: yum_repo.j2
    dest: /etc/yum.repos.d/{{ item.key }}.repo
  with_dict:
    pulp:
      name: Pulp Project repository
      baseurl: "http://koji.katello.org/releases/yum/pulp-nightly/pulp/{% if ansible_distribution == 'Fedora' %}fedora-{% endif %}$releasever/$basearch/"
      gpgcheck: 0
      priority: 50
  when: pulp_build == "nightly_koji"

- name: Setup Pulp beta or stable repository
  template:
    src: yum_repo.j2
    dest: /etc/yum.repos.d/{{ item.key }}.repo
  with_dict:
    pulp:
      name: Pulp Project repository
      baseurl: "https://repos.fedorapeople.org/pulp/pulp/{{ pulp_build }}/{{ pulp_version }}/{% if ansible_distribution == 'Fedora' %}fedora-{% endif %}$releasever/$basearch/"
      gpgcheck: 0
      priority: 50
  when: pulp_build == "beta" or pulp_build == "stable"

- name: check fips state
  shell: cat /proc/sys/crypto/fips_enabled
  register: fips_state

- name: install yum-plugin-priorities
  package:
    name: yum-plugin-priorities
    state: present
  when: fips_state.stdout == '1'

Will be checking in later after 2.18.1.b1

Actions

Copy link