Project

Profile

Help

Issue #2929

closed

Node sync is failing with unexpected error repo_id=None

Added by dsouza.rohan06 almost 5 years ago. Updated about 3 years ago.

Status:
CLOSED - NOTABUG
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
2.13.3
Platform Release:
OS:
CentOS 7
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

Hello,

I am facing this issue wherein when doing a node sync, it throws me an error:

[root@pulpserver ~]# pulp-admin node sync run --node-id child-1
Warning: this Nodes command is deprecated as the Nodes functionality will be
removed from Pulp 3.0.

This command may be exited via ctrl+c without affecting the request.

Error occurred during synchronization, check the child node logs for details

--------------------------------------------------------------------
Child Node Synchronization
--------------------------------------------------------------------

Repository:
Action: Pending
Content Sources:
Downloads:
Id: myown
Units:
Added: 0
Removed: 0
Updated: 0

--------------------------------------------------------------------
The following [1] errors were reported
--------------------------------------------------------------------

- 01: An unexpected error occurred. repo_id=None

A verbose output can be found here: http://paste.ubuntu.com/25155828/

Please note that I have set verify = false in nodes.conf, admin.conf as well as consumer.conf. Hence, I do not see how I should be getting this error. Tried restarting httpd , goferd etc. but to no avail.

I can find these errors in journalctl and /var/log/messages:

Jul 23 23:27:37 pulp-slave-server pulp: pulp.server.webservices.middleware.exception:INFO: Authentication with username None failed: invalid SSL certificate.
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - synchronization failed
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - Traceback (most recent call last):
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - File "/usr/lib/python2.7/site-packages/pulp_node/handlers/strategies.py", line 99, in synchronize
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - validator.validate(request.bindings)
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - File "/usr/lib/python2.7/site-packages/pulp_node/handlers/validation.py", line 38, in validate
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - self.report.errors.extend(self._validate_plugins(bindings))
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - File "/usr/lib/python2.7/site-packages/pulp_node/handlers/validation.py", line 54, in _validate_plugins
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - child = ChildServer()
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - File "/usr/lib/python2.7/site-packages/pulp_node/handlers/validation.py", line 72, in init
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - self.importers = self._importers()
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - File "/usr/lib/python2.7/site-packages/pulp_node/handlers/validation.py", line 83, in _importers
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - http = bindings.server_info.get_importers()
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - File "/usr/lib/python2.7/site-packages/pulp/bindings/server_info.py", line 31, in get_importers
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - return self.server.GET (path)
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - File "/usr/lib/python2.7/site-packages/pulp/bindings/server.py", line 92, in GET
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - return self._request('GET', path, queries, ignore_prefix=ignore_prefix)
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - File "/usr/lib/python2.7/site-packages/pulp/bindings/server.py", line 166, in _request
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - self._handle_exceptions(response_code, response_body)
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - File "/usr/lib/python2.7/site-packages/pulp/bindings/server.py", line 207, in _handle_exceptions
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - raise code_class_mappings[response_code](response_body)
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.strategies:112 - PermissionsException: RequestException: GET request on /pulp/api/v2/plugins/importers/ failed with 401 - Authentication with username None failed: invalid SSL certificate.
Jul 23 23:27:37 pulp-slave-server goferd: [ERROR][worker-0] pulp_node.handlers.handler:96 - An unexpected error occurred. repo_id=None

[root@pulpserver ~]# rpm -qa pulp-server
pulp-server-2.13.3-1.el7.noarch

On doing a little research, I found out that quite a few users have already reported the bug here in previous versions of pulp: https://pulp.plan.io/issues/528

Please help me fix the issue at the earliest.

Regards,
Rohan Dsouza

Actions #1

Updated by dsouza.rohan06 almost 5 years ago

Forgot to mention about oauth. oauth keys and secrets are connect on the parent nodes and the child nodes. Also, a typo above, verify_ssl = false is set in nodes.conf

Actions #2

Updated by mhrivnak almost 5 years ago

  • Version changed from 2.13.1 to 2.13.3
Actions #3

Updated by ttereshc almost 5 years ago

  • Tags deleted (Known Issue for Release Notes)
Actions #4

Updated by bizhang almost 5 years ago

Rohan,
Can you validate that your nodes.conf is not running into this issue: https://pulp.plan.io/issues/528#note-10
If that's not the problem, could you share your server.conf and nodes.conf?

Actions #5

Updated by dsouza.rohan06 almost 5 years ago

Hi,

I'm pretty sure there is no issues with my config. However, please find details as requested.

server.conf (On Parent node)

[oauth]
enabled: true
oauth_key: ab3cd9j4ks73hf7g
oauth_secret: qwedsa

nodes.conf (On Child node)

[main]

  1. ca_path: /etc/pki/tls/certs/ca-bundle.crt
  2. node_certificate: /etc/pki/pulp/nodes/node.crt
    verify_ssl = False
  1. The oauth configuration used to authenticate to this node
  2. user_id - The pulp user ID

[oauth]
user_id: admin

  1. The oauth configuration used to authenticate to the parent node
  2. key - The oauth key
  3. secret - The oauth secret
  4. user_id - The pulp user ID

[parent_oauth]
key: ab3cd9j4ks73hf7g
secret: qwedsa
user_id: admin

Kindly help me fix this issue.

Regards,
Rohan Dsouza

Actions #6

Updated by dsouza.rohan06 almost 5 years ago

Also, just an FYI, selinux and firewalld on both nodes are disabled.

Actions #7

Updated by dsouza.rohan06 almost 5 years ago

Guys, please ignore. I forgot to mention the oauth settings in my child nodes server.conf file. Its working now.

Actions #8

Updated by bmbouter almost 5 years ago

  • Status changed from NEW to CLOSED - NOTABUG

Thanks for the followup. I'm glad it's working for you. I'm closing as NOTABUG.

Actions #9

Updated by bmbouter about 3 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF