Migration Plugin

While running migration I run into error message like:

Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:   File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:     rv = job.perform()
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:   File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:     self._result = self._execute()
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:   File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:     return self.func(*self.args, **self.kwargs)
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:   File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 139, in migrate_from_pulp2
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:     migrate_content(plan)
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:   File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 36, in migrate_content
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:     plugin.migrator.migrate_content_to_pulp3()
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:   File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 140, in migrate_content_to_pulp3
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:     loop.run_until_complete(dm.create())
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:   File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:     return future.result()
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:   File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 88, in create
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:     await pipeline
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:   File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:     await asyncio.gather(*futures)
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:   File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:     await self.run()
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:   File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 169, in run
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:     self.migrate_to_pulp3(cmodel, ctype)
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:   File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 365, in migrate_to_pulp3
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:     downloaded=pulp2content.downloaded
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:   File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 125, in create_artifact
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:     expected_size=expected_size)
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:   File "/usr/lib/python3.6/site-packages/pulpcore/app/models/content.py", line 266, in init_and_validate
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]:     with open(file, "rb") as f:
Oct 26 06:40:46 lxserv2285.smhi.se rq[3185274]: FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/pulp/content/units/rpm/a9/8836abda4102abc7cf2f9f179a71f77cd3132599117a695380ec2568382548/libusnic_verbs-utils-1.1.0.237-2.el7.x86_64.rpm'

The file is indeed missing, but it is present in my pulp2 db and marked 'downloaded'. So to fix it I run a repo sync in pulp2 with '--force-full' and then rerun the migration from scratch. Only to once again run into the same error for some other file. And after doing this a couple of times the same filenames start to re-appear, files I have made certain are indeed present before starting the migration.

So it seems that the migration is actually deleting files from my pulp2 content dir, and deleting it before it tries to import it. My understanding was that the migration can be run in parallell while still using pulp2 in production, which makes this behaviour very odd. Have I misunderstood something?

This is on an rpm based installation on RHEL8:

python3-pulpcore-3.7.1-3.el8.noarch
python3-pulp-rpm-3.7.0-1.el8.noarch
python3-pulp-2to3-migration-0.5.0-1.el8.noarch