RPM Support

Before trying to fix it, I suggest to approach centos folks and ask for the use case of such stripped .treeinfo files. Do they merge them all at some point?
It would be also good to check with productmd folks, if it's a valid use according to specs.

We have a test system where 2to3 migration fails with:

No declared artifact with relative path \"images/boot.iso\" for content \"<DistributionTree: pk=f4651a50-5f7a-49a9-8fe5-3247a82362f1>\"

Is it meaningful that this fails on images/boot.iso and not on .treeinfo? (I know nothing about distribution trees...)

I queried for the DistributionTree that is throwing the error, and found that it comes from a AlmaLinux 8 repo. (Just posting this in case that is useful information.)

is this (https://community.theforeman.org/t/yum-repository-synchronization-error/26527) related?

I believe the root cause is as described, the uniqueness constraint is not enough for CentOS and AlmaLinux repos.
It does not work because of those 2 factors:

• each repo which is supposed to be an addon or variant is a standalone repo which has its own partial .treeinfo.
• the .treeinfo files are sometimes generated at the same second, so the build_timestamp does not help in such cases.

As of yesterday/today, these repos have conflicts and I experimented with them:

• CentOS8 HA and CenOS8 Power Tools (both are partial .treeinfos, aka no images associated or present in metadata)
  http://mirror.centos.org/centos/8-stream/HighAvailability/x86_64/os/.treeinfo
  http://mirror.centos.org/centos/8-stream/PowerTools/x86_64/os/.treeinfo

• Alma8 BaseOS and Alma8 AppStream (BaseOS is the one with images, AppStream has a partial .treeinfo)
  https://repo.almalinux.org/almalinux/8/BaseOS/x86_64/os/.treeinfo
  https://repo.almalinux.org/almalinux/8/AppStream/x86_64/os/.treeinfo

On the main branch (basically, current pulpcore 3.17 and pulp_rpm 3.16):

• I synced those repositories and in different order.
  • CentOS8 HA and CenOS8 Power Tools both will get the .treeinfo from the one which is synced the last, and no error(!) :(
  • same for Alma8 BaseOS and Alma8 AppStream

On the pulpcore 3.7 and pulp_rpm 3.11 (they correspond to Katello 3.18):

• I synced those repositories and in different order.
  • CentOS8 HA and CenOS8 Power Tools both get merged into one DistributionTree object but have the .treeinfo from the one which is synced the last, and no error(!) :(
  • if I sync Alma8 AppStream first, and then Alma8 BaseOS, I get "'DistributionTree' object has no attribute 'filename'
• Synced same repos in pulp2 and ran pulp-2to3-migration (order is not controlled)
  • CentOS8 HA and CenOS8 Power Tools get merged into one DistributionTree object but have the .treeinfo from the one which is migrated the last, and no error(!) :(
  • Alma8 BaseOS and Alma8 AppStream gave me No declared artifact with relative path \"images/boot.iso\" for content \"<DistributionTree: pk=f4651a50-5f7a-49a9-8fe5-3247a82362f1>\"

I could not reproduce the No declared artifact with relative path ".treeinfo" one but I believe that the root cause is the same and it's a result of a different order and/or something being removed in between runs as well.

I'm preparing a patch for the main branch with a database migration. This is not backportable and won't help those on pulp_rpm 3.11.

I'm trying to think of some solution for Katello users.
So far I have only a very hacky idea:

• Check the variants or the whole .treeinfo and if it differs, tweak the build_timestamp, basically force the timestamps to differ, so they become 2 different objects in the database.
  We do not preserve the original timestamp but everything else works.

Any thoughts or ideas are welcome.