Issue #2134
closed
Updating a repo without specifying checksum_type causes KeyError
Description
From bugzilla:
On updating a repo without touching checksum_type attribute, pulp raises an unexpected KeyError:
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) Task pulp.server.tasks.repository.distributor_update[36edf787-70ca-437a-b840-a3aebba3fd4e] raised unexpected: KeyError(u'checksum_type',)
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) Traceback (most recent call last):
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) R = retval = fun(*args, **kwargs)
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in __call__
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) return super(Task, self).__call__(*args, **kwargs)
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in __call__
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) return super(PulpTask, self).__call__(*args, **kwargs)
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) return self.run(*args, **kwargs)
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) File "/usr/lib/python2.7/site-packages/pulp/server/controllers/distributor.py", line 211, in update
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) distributor.config.pop(k)
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) File "/usr/lib/python2.7/site-packages/mongoengine/base/datastructures.py", line 73, in pop
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) return super(BaseDict, self).pop(*args, **kwargs)
Jul 15 10:50:32 rhsm-qe-3 pulp: celery.worker.job:ERROR: (4746-12352) KeyError: u'checksum_type'
Things seem to go wrong here: https://github.com/pulp/pulp/blob/2.8-dev/server/pulp/server/controllers/distributor.py#L211
Related issues
Updated by ipanova@redhat.com over 7 years ago
- Status changed from NEW to ASSIGNED
Updated by amacdona@redhat.com over 7 years ago
Did not triage because we are looking for more info on replication.
Updated by dalley over 7 years ago
Granted, there is no information here as to what command, specifically, caused the error for this user, but I have found a way to reproduce it this error message.
pulp-admin rpm repo update --repo-id zoo --checksum-type=
The above code will cause the same error if "checksum_type" is not previously defined in the distributor config.
I'm not sufficiently familiar with Pulp's CLI to know if that's supposed to be valid syntax or not, but in any case, I believe that is what the user is doing and what they meant by "not specifying the 'checksum_type'".
The root cause is that the config being passed into update(..., config=None, ...) contains keys not included in the distributor config. Thus when the code does this:
for k, v in config.iteritems():
if v is None:
distributor.config.pop(k)
else:
distributor.config[k] = v
"v" evaluates to None, and the key "k" is not already present in the distributor config, and so it will error when it attempts to "pop" the key off.
Updated by fdobrovo over 7 years ago
- Tags Easy Fix added
If that so the only thing needed is to convert.
distributor.config.pop(k) -> distributor.config.pop(k, None)
Updated by fdobrovo over 7 years ago
- Status changed from ASSIGNED to POST
The title should more likely be "Removing option from distributor config which was not set before causes KeyError"
Updated by amacdona@redhat.com over 7 years ago
Added by fdobrovo over 7 years ago
Added by fdobrovo over 7 years ago
Revision a4038809 | View on GitHub
Fix crash from removing missing option from dist conf
Fixed crash that arise when we try to remove an option from distributor confing which was not before set. This ends up in KeyError because this option was not in config dict. This was fixed and also added TestCase for it.
Summary:
- Fixed pulp.server.controllers.distributor.update to ignore missing options from distributor config
- Added TestCase for this behaviour TestUpdate.test_remove_none_values_non_existing()
Updated by fdobrovo over 7 years ago
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
Applied in changeset pulp:pulp|a4038809ff82ae7a02cfa5c8e9d89deeab77ea35.
Updated by elyezer over 7 years ago
- Status changed from 5 to 6
I have verified this on a Pulp 2.9.3b1 installation.
Step to verify:
1 pulp-admin login -u admin -p admin
2 pulp-admin rpm repo create --repo-id zoo
3 pulp-admin rpm repo update --repo-id zoo --checksum-type= # update with a null checksum-type
4 pulp-admin rpm repo list --repo-id zoo --details
5 pulp-admin rpm repo update --repo-id zoo --checksum-type=sha256 # update to a valid checksum-type
6 pulp-admin rpm repo list --repo-id zoo --details
7 pulp-admin rpm repo update --repo-id zoo --checksum-type= # update back to a null checksum-type
8 pulp-admin rpm repo list --repo-id zoo --details
Updated by elyezer over 7 years ago
An automated test was written to cover this issue [1]. That will ensure that a regression does not happen.
Updated by semyers over 7 years ago
- Status changed from 6 to CLOSED - CURRENTRELEASE
Updated by ipanova@redhat.com over 7 years ago
- Has duplicate Issue #2076: distributor update fails trying to remove already-missing key added
.
Fix crash from removing missing option from dist conf
Fixed crash that arise when we try to remove an option from distributor confing which was not before set. This ends up in KeyError because this option was not in config dict. This was fixed and also added TestCase for it.
Summary:
closes #2134 https://pulp.plan.io/issues/2134