Story #2962


Story #3821: As a user, I can migrate from Pulp 2 to Pulp 3

As an User I would like to migrate in-place from Pulp2 to Pulp3

Added by bizhang over 6 years ago. Updated almost 3 years ago.

Start date:
Due date:
% Done:


Estimated time:
Platform Release:
Sprint Candidate:


As a user I need some way of migrating my data from Pulp2 to Pulp3 in place.

I need :
1. custom queries for mongoexport to only export only necessary data. Here is a good starting point for the pulp2 data that is needed:
2. some way of validating my exported pulp2 data to make sure there are no surprises when importing it into a psql database. Attached is a dumped pulp3 schema that the data should conform to.
3. an estimate of how long the migration process is expected to take, so I can schedule downtime accordingly.
4. instructions on rolling back in case something goes wrong

From the community survey, /var/lib/pulp varied from 2GB to 2.8TB. The mean size (after trimming max and min values) is 233GB with a standard deviation of 198. Our strategy should not include any touching or cleanup for /var/lib/pulp during the migration. We should plan on leaving the Pulp2 symlinks in place without a cleanup until we have a confirmed successful migration to Pulp 3, since doing such during migration would be too time insensitive and destroy any chance of rolling back a bad migration. Removing symlinks should be painless after a successful migration to Pulp 3. A script to remove symlinks is outside of the scope of this story.

Test Plan:

1. stop mongo and run mongoexport with custom queries
2. Validate the dumped data conforms with the pulp3 schema
3. Import data into postgres
4. uninstall pulp2, install pulp3
5. Publish in pulp3
6. See that content is being served from the correct location
7. uninstall pulp3
8. install pulp2
9 ensure that pulp2 is still functional


pulp3.schema (46.5 KB) pulp3.schema bizhang, 08/07/2017 10:58 PM

Related issues

Related to Pulp - Refactor #2116: Migrate mongo collections to postgres tables.CLOSED - WONTFIX

Actions #1

Updated by pcreech over 6 years ago

  • Groomed changed from No to Yes
Actions #2

Updated by bizhang over 6 years ago

  • Description updated (diff)
Actions #3

Updated by bizhang over 6 years ago

  • Sprint Candidate changed from No to Yes
Actions #4

Updated by over 6 years ago

  • Related to Refactor #2116: Migrate mongo collections to postgres tables. added
Actions #5

Updated by rchan about 6 years ago

  • Sprint Candidate deleted (Yes)
Actions #6

Updated by over 5 years ago

  • Parent issue set to #3821
  • Sprint Candidate set to No
Actions #7

Updated by bmbouter almost 5 years ago

  • Tags deleted (Pulp 3)
Actions #8

Updated by ttereshc about 4 years ago

  • Project changed from Pulp to Migration Plugin
Actions #9

Updated by ttereshc almost 3 years ago

  • Status changed from NEW to CLOSED - WONTFIX

This plan is outdated. Pulp 2to3 migration plugin uses hardlinks if possible, and doesn't modify any pulp2 data. Pulp 2 can be cleaned up separately after the migration is complete.

Also available in: Atom PDF