pulp-installer CI doesn't properly test upgrading RPM packages
pulp_common : Add pulpcore RPM repositories doesn't replace the old repo URL when users specify a new one
pulp_pkg_repo. This should be properly fixed, but this bug is written for a CI workaround.
The katello repo had "latest" in its URL, but the foreman repos have pulpcore minor releases like "3.6" in their URL. This triggers the problem when a user upgrades, or the CI upgrades.