Project

Profile

Help

Task #1993

closed

Clean up our distribution packaging by moving it to a dedicated repository and building with copr

Added by jcline@redhat.com almost 8 years ago. Updated almost 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

40%

Estimated time:
Platform Release:
master
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Sprint:
Sprint 7
Quarter:

Description

Our distribution packaging files are spread across all our repositories. This is painful.

This task tracks the effort to move everything packaging-related into `pulp/packaging` on GitHub.


Related issues

Related to Pulp - Task #2155: Investigate (and maybe fix) copr on-demand repodata generationCLOSED - NOTABUGsemyers

Actions
Blocks Pulp - Story #1852: Move Pulp to a Software Collection on EL6CLOSED - WONTFIX

Actions
Actions #1

Updated by jcline@redhat.com almost 8 years ago

  • Blocks Story #1852: Move Pulp to a Software Collection on EL6 added
Actions #2

Updated by jcline@redhat.com almost 8 years ago

  • Sprint/Milestone set to 22
Actions #3

Updated by mhrivnak almost 8 years ago

  • Sprint/Milestone changed from 22 to 23
Actions #4

Updated by semyers almost 8 years ago

  • Assignee changed from jcline@redhat.com to semyers

jcline's done most of the work on this, but it's been transitioning on to my plate more and more. I'll be owning it entirely starting next week, so I might as well pick up the ticket now.

Actions #5

Updated by mhrivnak almost 8 years ago

  • Sprint/Milestone changed from 23 to 24
Actions #6

Updated by semyers over 7 years ago

Moving the pulp 2.10 release back a week meant it made sense to prioritize #2086 before this. I haven't spent significant time on this since Friday, but at this point the only thing that's still not working right is setting the python setup.py versions correctly based on the release config and pushing the tag prior building the SRPMs. I should have that fixed up later today for some test builds of 2.10 releases.

Actions #7

Updated by semyers over 7 years ago

I was able to run a 2.10 build in copr today using the new scripts, and get a working pulp install using just the copr repo in fedora 24. The 'kobo' package was not installed despite being listed in the spec file, indicating the packaging repo likely needs an update for pulp 2.10 specs. Once that's taken care of, pulp 2.10 installs from should be working fine.

Added by semyers over 7 years ago

Revision e00a852e | View on GitHub

docs mention the COPR projects, no more groupinstalls

groupinstall change: re #1515 https://pulp.plan.io/issues/1515

copr stuff: re #1993 https://pulp.plan.io/issues/1993

Added by semyers over 7 years ago

Revision e00a852e | View on GitHub

docs mention the COPR projects, no more groupinstalls

groupinstall change: re #1515 https://pulp.plan.io/issues/1515

copr stuff: re #1993 https://pulp.plan.io/issues/1993

Added by semyers over 7 years ago

Revision 698a34a6 | View on GitHub

Remove references to copr

I sadly admit defeat. This removes references to COPR added in yesterday's PR, but leaves in the changes that remove references to using groupinstall.

this change partially reverts e00a852ef94f2d8d8737956ef130eb03b576091a

re #1993 https://pulp.plan.io/issues/1993

Added by semyers over 7 years ago

Revision 698a34a6 | View on GitHub

Remove references to copr

I sadly admit defeat. This removes references to COPR added in yesterday's PR, but leaves in the changes that remove references to using groupinstall.

this change partially reverts e00a852ef94f2d8d8737956ef130eb03b576091a

re #1993 https://pulp.plan.io/issues/1993

Added by semyers over 7 years ago

Revision 4f1b1806 | View on GitHub

Remove references to copr

I sadly admit defeat. This removes references to COPR added in yesterday's PR, but leaves in the changes that remove references to using groupinstall.

this change partially reverts e00a852ef94f2d8d8737956ef130eb03b576091a

re #1993 https://pulp.plan.io/issues/1993

Added by semyers over 7 years ago

Revision 4f1b1806 | View on GitHub

Remove references to copr

I sadly admit defeat. This removes references to COPR added in yesterday's PR, but leaves in the changes that remove references to using groupinstall.

this change partially reverts e00a852ef94f2d8d8737956ef130eb03b576091a

re #1993 https://pulp.plan.io/issues/1993

Actions #8

Updated by semyers over 7 years ago

  • Subject changed from Clean up our distribution packaging by moving it to a dedicated repository to Clean up our distribution packaging by moving it to a dedicated repository and building with copr
  • Sprint/Milestone deleted (24)

Updated the subject to match what I'm actually doing with this ticket. Moving the spec files into a dedicated repo per-dist is inexorable from the effort to build pulp in copr, and the effort to build in copr is stalled.

Stuff that's we can do with the new packaging scripts:
- pull down sources and build SRPMs
- tag both the source repo and spec repo with the modified sources (modified to change the version strings)
- build those SRPMs in corp or a local mock root
- being able to tag the specfile repo separately from the source repos means we finally have a way to see exactly what plugins were released for a given version of pulp in a distribution: Just look at the spec files for the tagged version.

It's subtle, but a result of this effort is that anyone with a copr can build pulp packages. Since anyone can get access to copr, this means that anyone can build pulp packages (woot).

Stuff that doesn't work:
- The packages. :( I did a test install of our copr build in fedora23, and while all the normal services (http apps, plus workers) started and appeared to log success, aiming pulp-smash at the install revealed a lot of failures. This was late in the release process for 2.10.0, so rather than dig into those failures at all, I made the decision to effectively give up and began building 2.10 using the "old" process.
- copr appears to support comps.xml, but doesn't include it when generating repodata. This isn't really a problem since we're looking to get rid of comps (#1515), and the docs for 2.10 have indeed been updated to list packages explicitly and not use package groups (at least for platform).
- on-demand repodata generation in copr doesn't work, but really should. It automatically regenerates repodata after every build successfully, but we don't want this for "stable" releases. Rather, we want to ensure all packages build successfully before regenerating repo metadata so that users get all package updates in a single atomic repo update. copr supports this, but the functionality appears to simply be broken.

I'm leaving this assigned to me, as I'll continue to tinker with this, but I'm pulling it off of this sprint. A discussion among the team leads confirms that while we do want to do this, it can (and perhaps should, in hindsight) wait for pulp 3.

Actions #9

Updated by semyers over 7 years ago

  • Tags Pulp 3 added

The bulk of the copr work is being broken out into separate tasks. This one, as written, is probably done, since we do indeed have a packaging repo with spec files in it for pulp, its plugins, and its deps. I'd still like to clean up my devel branch related to this, which adds a lot of copr workflow bits that we'll want, so I think I'll call this done once that branch gets merged.

Actions #10

Updated by semyers over 7 years ago

  • Related to Task #2155: Investigate (and maybe fix) copr on-demand repodata generation added
Actions #11

Updated by mhrivnak over 7 years ago

  • Sprint/Milestone set to 24
Actions #12

Updated by mhrivnak over 7 years ago

  • Sprint/Milestone changed from 24 to 25

Added by semyers over 7 years ago

Revision 6c0c54ec | View on GitHub

rename pulp-devel to pulp-dev, bring in EVR and copr support

This is a megatrash commit, a snapshot of the work I've done.

Once this is reviewed and merged (and I see what's left...), I'll put together redmine tasks tracking the remaining work for "complete" copr integration.

closes #1993

Actions #13

Updated by semyers over 7 years ago

  • Status changed from ASSIGNED to CLOSED - CURRENTRELEASE
  • Platform Release set to master

Apologies for not moving this to POST:
https://github.com/pulp/devel/pull/2

The devel repo isn't yet known to redmine, but the referenced PR is included on 3.0-dev in that repo:
https://github.com/pulp/devel/tree/3.0-dev

Since there's also no release structure for the devel repo, I couldn't come up with a better resolution than closing the ticket against the "master" target release.

Actions #14

Updated by bmbouter about 6 years ago

  • Sprint set to Sprint 7
Actions #15

Updated by bmbouter about 6 years ago

  • Sprint/Milestone deleted (25)
Actions #16

Updated by daviddavis about 5 years ago

  • Sprint/Milestone set to 3.0.0
Actions #17

Updated by bmbouter almost 5 years ago

  • Tags deleted (Pulp 3)

Also available in: Atom PDF