Pulp

PR available at: https://github.com/pulp/pulp/pull/1443

+ This comment was cloned from Bugzilla #1175512 comment 1 +

Merged to 2.6-dev -> master

+ This comment was cloned from Bugzilla #1175512 comment 2 +

fixed in pulp 2.6.0-0.2.beta

+ This comment was cloned from Bugzilla #1175512 comment 3 +

How do you verify this bz?

Will verifying
https://bugzilla.redhat.com/show_bug.cgi?id=1182335 verify this bz as well?

+ This comment was cloned from Bugzilla #1175512 comment 4 +

rpm -qa pulp-server

pulp-server-2.6.0-0.5.beta.el6.noarch

grep bind_ip /etc/mongodb.conf

bind_ip = ip-XXX.internal
#bind_ip = localhost

in /etc/pulp/server.conf

[database]
#name: pulp_database
name: fake_db
seeds: ip-XXX.internal:27017

mongo --host `hostname`

MongoDB shell version: 2.4.12
connecting to: ip-XXX.internal:27017/test

use fake_db

switched to db fake_db

db.getCollectionNames()

[
"consumer_bindings",
"consumers",
"content_types",
"event_listeners",
"migration_trackers",
"permissions",
"repo_content_units",
"repo_distributors",
"repo_importers",
"repos",
"roles",
"system.indexes",
"units_distribution",
"units_drpm",
"units_erratum",
"units_iso",
"units_node",
"units_package_category",
"units_package_environment",
"units_package_group",
"units_puppet_module",
"units_repository",
"units_rpm",
"units_srpm",
"units_yum_repo_metadata_file",
"users"
]

exit

bye

pulp-admin -u admin -p admin rpm repo create --repo-id nemo

Successfully created repository [nemo]

pulp-admin -u admin -p admin rpm repo publish run --repo-id nemo

--------------------------------------------------------------------
Publishing Repository [nemo]
--------------------------------------------------------------------

This command may be exited via ctrl+c without affecting the request.

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[-]
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[-]
... completed

Publishing Comps file
[-]
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[-]
... completed

Writing Listings File
[-]
... completed

Task Succeeded

mongo --host `hostname`

MongoDB shell version: 2.4.12
connecting to: ip-XXX.internal:27017/test

use fake_db

switched to db fake_db

db.repos

db.repos

db.repos.find()

{ "_id" : ObjectId("54d367f95a0f2b48405b9245"), "scratchpad" : { }, "display_name" : "nemo", "description" : null, "last_unit_added" : null, "notes" : { "_repo-type" : "rpm-repo" }, "last_unit_removed" : null, "content_unit_counts" : { }, "_ns" : "repos", "id" : "nemo" }

exit

bye

+ This comment was cloned from Bugzilla #1175512 comment 5 +

In addition I run some automation tests, for example:

nosetests -vs tests/general_tests/test_01_log_in.py

test_login (tests.general_tests.test_01_log_in.SimpleTestLogin) ... ok
test_wrong_login (tests.general_tests.test_01_log_in.SimpleTestLogin) ... ok

----------------------------------------------------------------------
Ran 2 tests in 1.214s

OK

+ This comment was cloned from Bugzilla #1175512 comment 6 +

The tricky thing about this is that the bug is present, but it only will show itself in a future version of pulp (ie: what is currently on master). The commit that introduces it on master is 39d332158ad8aa9b2d09c7f9ad8d71d7e3a8fa93

Attached is a reproducer script (reproduce_1175512.py). Here's how to use it:

1) delete all databases from mongodb except the one named 'local'. The 'local' database is used by mongodb itself. Show your databases using 'show databases'.

2) run pulp-manage-db

3) show your databases again, you should see pulp_database has been added (assuming you have a default server.conf).

4) run the attached reproducer script `python reproduce_1175512.py`

5) show the databases again. If you see any change from step 3 then fail the bug.

6) list the collections on pulp_database. You should see a collection named 'test_reproducer' was added. If that is not present fail the bug.

7) Manually delete the 'test_reproducer' collection from pulp_database to clean up from the reproducer script. Alternatively you can drop your pulp_database and rerun pulp-manage-db.

8) If you got this far then VERIFY the fix.

+ This comment was cloned from Bugzilla #1175512 comment 8 +

stopped all pulp services

mongo

MongoDB shell version: 2.4.12
connecting to: test

show databases

fake_db 0.203125GB
local 0.078125GB

use fake_db

switched to db fake_db

db.dropDatabase()

{ "dropped" : "fake_db", "ok" : 1 }

show databases

local 0.078125GB

service qpidd start

Starting Qpid AMQP daemon: [ OK ]

sudo -u apache pulp-manage-db

Mongo database for connection is version 2.4.12
Loading content types.
Loading type descriptors [nodes.json, rpm_support.json, puppet.json, iso_support.json]
Parsing type descriptors
Validating type descriptor syntactic integrity
Validating type descriptor semantic integrity
Updating the database with types [repository, node, distribution, drpm, erratum, package_group, package_category, package_environment, rpm, srpm, yum_repo_metadata_file, puppet_module, iso]
/usr/lib/python2.6/site-packages/pulp/server/db/connection.py:162: DeprecationWarning: The safe parameter is deprecated. Please use write concern options instead.
return method(*args, **kwargs)
Content types loaded.
Ensuring the admin role and user are in place.
Mongo database for connection is version 2.4.12
Admin role and user are in place.
Beginning database migrations.
Applying pulp.server.db.migrations version 1
Migration to pulp.server.db.migrations version 1 complete.
Applying pulp.server.db.migrations version 2
Migration to pulp.server.db.migrations version 2 complete.
Applying pulp.server.db.migrations version 3
Migration to pulp.server.db.migrations version 3 complete.
Applying pulp.server.db.migrations version 4
regenerating content unit counts for 0 repositories
Migration to pulp.server.db.migrations version 4 complete.
Applying pulp.server.db.migrations version 5
Migration to pulp.server.db.migrations version 5 complete.
Applying pulp.server.db.migrations version 6
Migration to pulp.server.db.migrations version 6 complete.
Applying pulp.server.db.migrations version 7
Migration to pulp.server.db.migrations version 7 complete.
Applying pulp.server.db.migrations version 8
Migration to pulp.server.db.migrations version 8 complete.
Applying pulp.server.db.migrations version 9
Migration to pulp.server.db.migrations version 9 complete.
Applying pulp.server.db.migrations version 10
Migration to pulp.server.db.migrations version 10 complete.
Applying pulp.server.db.migrations version 11
Migration to pulp.server.db.migrations version 11 complete.
Applying pulp.server.db.migrations version 12
Migration to pulp.server.db.migrations version 12 complete.
Applying pulp_puppet.plugins.migrations version 1
Migrated puppet modules to include checksum
Migration to pulp_puppet.plugins.migrations version 1 complete.
Applying pulp_puppet.plugins.migrations version 2
Migration to pulp_puppet.plugins.migrations version 2 complete.
Applying pulp_rpm.plugins.migrations version 1
Export distributor migration for rpm repositories started
Export distributor migration for rpm repositories finished
Migration to pulp_rpm.plugins.migrations version 1 complete.
Applying pulp_rpm.plugins.migrations version 2
Migrated rpms to include rpm metadata
Migration to pulp_rpm.plugins.migrations version 2 complete.
Applying pulp_rpm.plugins.migrations version 3
Migration to pulp_rpm.plugins.migrations version 3 complete.
Applying pulp_rpm.plugins.migrations version 4
Migration to pulp_rpm.plugins.migrations version 4 complete.
Applying pulp_rpm.plugins.migrations version 5
Migrated rpms to include rpm changelog and filelist metadata
Migration to pulp_rpm.plugins.migrations version 5 complete.
Applying pulp_rpm.plugins.migrations version 6
Migration to pulp_rpm.plugins.migrations version 6 complete.
Applying pulp_rpm.plugins.migrations version 7
No yum repositories found to inventory custom metadata on
Migration to pulp_rpm.plugins.migrations version 7 complete.
Applying pulp_rpm.plugins.migrations version 8
Migration to pulp_rpm.plugins.migrations version 8 complete.
Applying pulp_rpm.plugins.migrations version 9
Migration to pulp_rpm.plugins.migrations version 9 complete.
Applying pulp_rpm.plugins.migrations version 10
Migration to pulp_rpm.plugins.migrations version 10 complete.
Applying pulp_rpm.plugins.migrations version 11
Migration to pulp_rpm.plugins.migrations version 11 complete.
Applying pulp_rpm.plugins.migrations version 12
Migration to pulp_rpm.plugins.migrations version 12 complete.
Applying pulp_rpm.plugins.migrations version 13
Migration to pulp_rpm.plugins.migrations version 13 complete.
Applying pulp_rpm.plugins.migrations version 14
Migration to pulp_rpm.plugins.migrations version 14 complete.
Applying pulp_rpm.plugins.migrations version 15
examining distribution units
Migration to pulp_rpm.plugins.migrations version 15 complete.
Applying pulp_rpm.plugins.migrations version 16
Loaded plugin yum_distributor for types: rpm,srpm,drpm,erratum,package_group,package_category,distribution,yum_repo_metadata_file
Loaded plugin export_distributor for types: rpm,srpm,drpm,erratum,distribution,package_category,package_group
Loaded plugin iso_distributor for types: iso
Loaded plugin puppet_distributor for types: puppet_module
Loaded plugin puppet_file_distributor for types: puppet_module
Loaded plugin puppet_install_distributor for types: puppet_module
Loaded plugin group_export_distributor for types: rpm,srpm,drpm,erratum,distribution,package_category,package_group
Loaded plugin nodes_http_importer for types: node,repository
Loaded plugin yum_importer for types: distribution,drpm,erratum,package_group,package_category,rpm,srpm,yum_repo_metadata_file,package_environment
Loaded plugin iso_importer for types: iso
Loaded plugin puppet_importer for types: puppet_module
Loaded plugin yum_profiler for types: rpm,erratum
Loaded plugin puppet_whole_repo_profiler for types: puppet_module
Loaded plugin yum for types: rpm
Loaded plugin rhui for types: rpm
Migration to pulp_rpm.plugins.migrations version 16 complete.
Applying pulp_rpm.plugins.migrations version 17
Migration to pulp_rpm.plugins.migrations version 17 complete.
Applying pulp_rpm.plugins.migrations version 18
Migration to pulp_rpm.plugins.migrations version 18 complete.
Database migrations complete.

mongo

MongoDB shell version: 2.4.12
connecting to: test

show databases

local 0.078125GB
pulp_database 0.203125GB

use pulp_database

switched to db pulp_database

show collections

consumer_bindings
consumers
content_types
event_listeners
migration_trackers
permissions
repo_content_units
repo_distributors
repo_importers
repos
roles
system.indexes
units_distribution
units_drpm
units_erratum
units_iso
units_node
units_package_category
units_package_environment
units_package_group
units_puppet_module
units_repository
units_rpm
units_srpm
units_yum_repo_metadata_file
users

python ./BZ1175512.py

/usr/lib64/python2.6/site-packages/pymongo/collection.py:266: DeprecationWarning: The safe parameter is deprecated. Please use write concern options instead.
return self.insert(to_save, manipulate, safe, check_keys, **kwargs)

mongo

MongoDB shell version: 2.4.12
connecting to: test

show databases

local 0.078125GB
pulp_database 0.203125GB

use pulp_database

switched to db pulp_database

show collections

consumer_bindings
consumers
content_types
event_listeners
migration_trackers
permissions
repo_content_units
repo_distributors
repo_importers
repos
roles
system.indexes
test_reproducer <----------it's here
units_distribution
units_drpm
units_erratum
units_iso
units_node
units_package_category
units_package_environment
units_package_group
units_puppet_module
units_repository
units_rpm
units_srpm
units_yum_repo_metadata_file
users

Cleaning UP

db.dropDatabase()

{ "dropped" : "pulp_database", "ok" : 1 }

exit

bye

sudo -u apache pulp-manage-db

Mongo database for connection is version 2.4.12
Loading content types.
Loading type descriptors [nodes.json, rpm_support.json, puppet.json, iso_support.json]
Parsing type descriptors
Validating type descriptor syntactic integrity
Validating type descriptor semantic integrity
Updating the database with types [repository, node, distribution, drpm, erratum, package_group, package_category, package_environment, rpm, srpm, yum_repo_metadata_file, puppet_module, iso]
/usr/lib/python2.6/site-packages/pulp/server/db/connection.py:162: DeprecationWarning: The safe parameter is deprecated. Please use write concern options instead.
return method(*args, **kwargs)
Content types loaded.
Ensuring the admin role and user are in place.
Mongo database for connection is version 2.4.12
Admin role and user are in place.
Beginning database migrations.
Applying pulp.server.db.migrations version 1
Migration to pulp.server.db.migrations version 1 complete.
Applying pulp.server.db.migrations version 2
Migration to pulp.server.db.migrations version 2 complete.
Applying pulp.server.db.migrations version 3
Migration to pulp.server.db.migrations version 3 complete.
Applying pulp.server.db.migrations version 4
regenerating content unit counts for 0 repositories
Migration to pulp.server.db.migrations version 4 complete.
Applying pulp.server.db.migrations version 5
Migration to pulp.server.db.migrations version 5 complete.
Applying pulp.server.db.migrations version 6
Migration to pulp.server.db.migrations version 6 complete.
Applying pulp.server.db.migrations version 7
Migration to pulp.server.db.migrations version 7 complete.
Applying pulp.server.db.migrations version 8
Migration to pulp.server.db.migrations version 8 complete.
Applying pulp.server.db.migrations version 9
Migration to pulp.server.db.migrations version 9 complete.
Applying pulp.server.db.migrations version 10
Migration to pulp.server.db.migrations version 10 complete.
Applying pulp.server.db.migrations version 11
Migration to pulp.server.db.migrations version 11 complete.
Applying pulp.server.db.migrations version 12
Migration to pulp.server.db.migrations version 12 complete.
Applying pulp_puppet.plugins.migrations version 1
Migrated puppet modules to include checksum
Migration to pulp_puppet.plugins.migrations version 1 complete.
Applying pulp_puppet.plugins.migrations version 2
Migration to pulp_puppet.plugins.migrations version 2 complete.
Applying pulp_rpm.plugins.migrations version 1
Export distributor migration for rpm repositories started
Export distributor migration for rpm repositories finished
Migration to pulp_rpm.plugins.migrations version 1 complete.
Applying pulp_rpm.plugins.migrations version 2
Migrated rpms to include rpm metadata
Migration to pulp_rpm.plugins.migrations version 2 complete.
Applying pulp_rpm.plugins.migrations version 3
Migration to pulp_rpm.plugins.migrations version 3 complete.
Applying pulp_rpm.plugins.migrations version 4
Migration to pulp_rpm.plugins.migrations version 4 complete.
Applying pulp_rpm.plugins.migrations version 5
Migrated rpms to include rpm changelog and filelist metadata
Migration to pulp_rpm.plugins.migrations version 5 complete.
Applying pulp_rpm.plugins.migrations version 6
Migration to pulp_rpm.plugins.migrations version 6 complete.
Applying pulp_rpm.plugins.migrations version 7
No yum repositories found to inventory custom metadata on
Migration to pulp_rpm.plugins.migrations version 7 complete.
Applying pulp_rpm.plugins.migrations version 8
Migration to pulp_rpm.plugins.migrations version 8 complete.
Applying pulp_rpm.plugins.migrations version 9
Migration to pulp_rpm.plugins.migrations version 9 complete.
Applying pulp_rpm.plugins.migrations version 10
Migration to pulp_rpm.plugins.migrations version 10 complete.
Applying pulp_rpm.plugins.migrations version 11
Migration to pulp_rpm.plugins.migrations version 11 complete.
Applying pulp_rpm.plugins.migrations version 12
Migration to pulp_rpm.plugins.migrations version 12 complete.
Applying pulp_rpm.plugins.migrations version 13
Migration to pulp_rpm.plugins.migrations version 13 complete.
Applying pulp_rpm.plugins.migrations version 14
Migration to pulp_rpm.plugins.migrations version 14 complete.
Applying pulp_rpm.plugins.migrations version 15
examining distribution units
Migration to pulp_rpm.plugins.migrations version 15 complete.
Applying pulp_rpm.plugins.migrations version 16
Loaded plugin yum_distributor for types: rpm,srpm,drpm,erratum,package_group,package_category,distribution,yum_repo_metadata_file
Loaded plugin export_distributor for types: rpm,srpm,drpm,erratum,distribution,package_category,package_group
Loaded plugin iso_distributor for types: iso
Loaded plugin puppet_distributor for types: puppet_module
Loaded plugin puppet_file_distributor for types: puppet_module
Loaded plugin puppet_install_distributor for types: puppet_module
Loaded plugin group_export_distributor for types: rpm,srpm,drpm,erratum,distribution,package_category,package_group
Loaded plugin nodes_http_importer for types: node,repository
Loaded plugin yum_importer for types: distribution,drpm,erratum,package_group,package_category,rpm,srpm,yum_repo_metadata_file,package_environment
Loaded plugin iso_importer for types: iso
Loaded plugin puppet_importer for types: puppet_module
Loaded plugin yum_profiler for types: rpm,erratum
Loaded plugin puppet_whole_repo_profiler for types: puppet_module
Loaded plugin yum for types: rpm
Loaded plugin rhui for types: rpm
Migration to pulp_rpm.plugins.migrations version 16 complete.
Applying pulp_rpm.plugins.migrations version 17
Migration to pulp_rpm.plugins.migrations version 17 complete.
Applying pulp_rpm.plugins.migrations version 18
Migration to pulp_rpm.plugins.migrations version 18 complete.
Database migrations complete.

mongo

MongoDB shell version: 2.4.12
connecting to: test

show databases

local 0.078125GB
pulp_database 0.203125GB

use pulp_database

switched to db pulp_database

show collections

consumer_bindings
consumers
content_types
event_listeners
migration_trackers
permissions
repo_content_units
repo_distributors
repo_importers
repos
roles
system.indexes
units_distribution
units_drpm
units_erratum
units_iso
units_node
units_package_category
units_package_environment
units_package_group
units_puppet_module
units_repository
units_rpm
units_srpm
units_yum_repo_metadata_file
users

no test reproducer

+ This comment was cloned from Bugzilla #1175512 comment 9 +