Issue #340
closed
Missing resource error in pulp.log if you deletes the consumer without re-registering it
Description
Clone for Master
++ This bug was initially created as a clone of Bug #876234 ++
Description of problem:
If you remove the consumer rpms and consumer related files, Missing resource error appears in the pulp.log
Version-Release number of selected component (if applicable):
[root@preethi ~]# rpm -qa pulp-rpm-server
pulp-rpm-server-2.0.6-0.2.beta.noarch
How reproducible:
Steps to Reproduce:
1. Install the consumer-client packages and register a client to a pulp server
2. Remove the consumer-client rpms
3. Notice the error in pulp.log
Actual results:
Expected results:
Additional info:
From pulp.log
2012-11-13 10:32:08,447 16913:139677436737280: pulp.server.webservices.middleware.exception:ERROR: exception:46 Missing resource(s): consumer=preethi
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/middleware/exception.py", line 44, in call
return self.app(environ, start_response)
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/middleware/postponed.py", line 39, in call
return self.app(environ, start_response)
File "/usr/lib/python2.7/site-packages/web/application.py", line 277, in wsgi
result = self.handle_with_processors()
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py", line 26, in _handle_with_processors
return process(self.processors)
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py", line 23, in process
return p(lambda : process(processors))
File "/usr/lib/python2.7/site-packages/web/application.py", line 555, in processor
return handler()
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py", line 23, in <lambda>
return p(lambda : process(processors))
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py", line 23, in process
return p(lambda : process(processors))
File "/usr/lib/python2.7/site-packages/web/application.py", line 570, in processor
result = handler()
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py", line 23, in <lambda>
return p(lambda : process(processors))
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py", line 25, in process
return self.handle()
File "/usr/lib/python2.7/site-packages/web/application.py", line 228, in handle
return self._delegate(fn, self.fvars, args)
File "/usr/lib/python2.7/site-packages/web/application.py", line 411, in _delegate
return f()
File "/usr/lib/python2.7/site-packages/web/application.py", line 419, in <lambda>
f = lambda: self._delegate_sub_application(pat, what)
File "/usr/lib/python2.7/site-packages/web/application.py", line 444, in _delegate_sub_application
return app.handle_with_processors()
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py", line 26, in _handle_with_processors
return process(self.processors)
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py", line 23, in process
return p(lambda : process(processors))
File "/usr/lib/python2.7/site-packages/web/application.py", line 555, in processor
return handler()
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py", line 23, in <lambda>
return p(lambda : process(processors))
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py", line 23, in process
return p(lambda : process(processors))
File "/usr/lib/python2.7/site-packages/web/application.py", line 570, in processor
result = handler()
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py", line 23, in <lambda>
return p(lambda : process(processors))
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py", line 25, in process
return self.handle()
File "/usr/lib/python2.7/site-packages/web/application.py", line 228, in handle
return self._delegate(fn, self.fvars, args)
File "/usr/lib/python2.7/site-packages/web/application.py", line 394, in _delegate
return handle_class(f)
File "/usr/lib/python2.7/site-packages/web/application.py", line 385, in handle_class
return tocall(*args)
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/controllers/decorators.py", line 231, in _auth_decorator
value = method(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/controllers/consumers.py", line 484, in POST
result = execution.execute_sync_created(self, call_request, link)
File "/usr/lib/python2.7/site-packages/pulp/server/dispatch/task.py", line 123, in _run
result = call(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/pulp/server/managers/consumer/profile.py", line 43, in create
return self.update(consumer_id, content_type, profile)
File "/usr/lib/python2.7/site-packages/pulp/server/managers/consumer/profile.py", line 57, in update
manager.get_consumer(consumer_id)
File "/usr/lib/python2.7/site-packages/pulp/server/managers/consumer/cud.py", line 202, in get_consumer
raise MissingResource(consumer=id)
MissingResource: Missing resource(s): consumer=preethi
+ This bug was cloned from Bugzilla Bug #876677 +
Updated by jason.dobies@redhat.com over 9 years ago
Jeff - I'd like to know what's happening here. This is weird. It's a pretty annoying problem if a consumer disappearing (which she kinda simulated here, but I suspect the agent is still running so it's not exactly like the consumer box up and died) and the server continues to log errors like this. I'm aligning to 2.0 Beta to investigate. Ping me when you know what's happening.
+ This comment was cloned from Bugzilla #876677 comment 1 +
Updated by jortel@redhat.com over 9 years ago
***** Bug 876234 has been marked as a duplicate of this bug. ***
+ This comment was cloned from Bugzilla #876677 comment 2 +
Updated by jortel@redhat.com over 9 years ago
This bug describes a condition whereby the consumer is unregistered from pulp but a valid consumer certifiate still exists on the consumer. The agent is still installed and running and because valid consumer certificates still exist on the box, it continues to send profile updates as scheduled. The agent detects registration/unregistration based on the existance of a valid consumer certificate.
Under normal circumstances, the certificate is deleted locally when the consumer is unregistered using pulp-consumer. If the consumer is unregistered using pulp-admin, notification of the unregistration is sent to the agent which deletes the certificate.
There are a few abnormal scenerios that I can think of that can produce this condition. The condition being that the consumer not longer exists in the server but the agent thinks it's still registered.
1. The server is reinstalled or DB is manually cleared.
2. Consumer is unregistered using pulp-admin and connectivity between the
server and qpid broker cannot be established. Short term, this causes
a condition where the agent continues to report package profiles but
the consumer no longer exits. Once the connection to the broken
is established, the condition is resolved. However, if httpd is restarted
before the connection is established, the message to the agent will never
be sent and the condition will never be resolved.
3. Consumer is unregistered using pulp-admin and connectivity between the
agent and qpid broker cannot be established. Short term, this causes
a condition where the agent continues to report package profiles but
the consumer no longer exits. Once the connection to the broken
is established, the condition is resolved.
4. A deleted consumer certificate reappears as the result of a restored backup
on the consumer.
5. User manually installs consumer certificate in some other way.
Since interruptions in the communication path between the server and consumer are unavoidable, we need to assume that this condition can exist for at least short durations. Here are some possible solutions:
1. Don't raise ResourceMissing when reporting profiles. Bad idea.
2. Have agent query to see if consumer exists before sending the profile.
This approach subject to race conditions so not very solid.
3. Implement CRLs on the server. When consumer is unregistered, revoke the
consumer certificate and REST calls to report package profile are rejected
at the front door and don't show up as ResourceMissing in the logs. Thinking
of this solution raises the issue of unregistered consumers continuing to
consume content from protected repos when this condition exists.
4. TBD
+ This comment was cloned from Bugzilla #876677 comment 3 +
Updated by jason.dobies@redhat.com over 9 years ago
Given this is an exceptional case she ran into, I'm punting this off of 2.0. There's not an easy solution so I'd rather take some time to step back and handle these sorts of consumer incongruities as its own story.
+ This comment was cloned from Bugzilla #876677 comment 4 +
Updated by bmbouter over 5 years ago
- Status changed from NEW to CLOSED - WONTFIX
Updated by bmbouter over 5 years ago
Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.
.