Project

Profile

Help

Story #95

As a user, I can manage ostree content units by branch

Added by jortel@redhat.com almost 6 years ago. Updated over 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Target Release - OSTree:
1.0.0
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
February 2015
Quarter:

Description

Users wants pulp to model ostree content units where each unit is a branch head instead of a snaphot of the entire repository. They also want us to only publish objects associated to those commits and not just links to the entire backing repository. Red Hat composed repositories will include a version field in the commit metadata. Users want the commit metadata inventoried with each content unit and displayed in our CLI. They also want us to chnage what is published to only include links into objects/ that are referenced by branch head commits. Currently we are simply linking to the entire objects/ directory which exposes too much.

Details

  • Update the type definition to model each unit as a branch head. The unit key will be something like: {remote_id:<id>, branch_id:<path>, commit_id:<hash>}.
  • After pulling from a remote repository, add a content unit per branch head instead of one unit representing a snapshot of the entire repository.
  • Include the metadata contained in the commit with the unit metadata. OSTree repositories composed by Red Hat will contain a version.
  • Update the CLI to display this metadata.
  • Update publishing to perform the following steps:
  • Create a new ostree repository
  • Add a file in the refs/ for each content unit.
  • Do a local-pull from the backing ostree repository to create hard links to the objects.

Deliverables

  • OSTree repositories contain a unit per branch that includes the commit metadata.
  • Published ostree repositories include a file in refs/heads per content unit.
  • Published ostree repositories objects/ contains hard links to only those objects referenced by the branch head commits.
  • The ostree CLI displays commit metadata for each unit.

History

#1 Updated by jortel@redhat.com almost 6 years ago

  • Description updated (diff)

#2 Updated by jortel@redhat.com almost 6 years ago

  • Sprint/Milestone deleted (9)

#3 Updated by jortel@redhat.com almost 6 years ago

  • Tracker changed from Issue to Story

#5 Updated by cduryee almost 6 years ago

  • Sprint/Milestone set to 9

#6 Updated by jortel@redhat.com almost 6 years ago

  • Assignee set to jortel@redhat.com

#7 Updated by rbarlow almost 6 years ago

  • Project changed from Pulp to OSTree Support

#8 Updated by jortel@redhat.com over 5 years ago

libostree support for pull-local we'd planned to use for publishing needs work: https://bugzilla.redhat.com/show_bug.cgi?id=1185106. We'll likely need to shell out to ostree instead :(

Eg:

ostree --repo=repo-clone pull-local repo fcc7dc55bebb8f154c7c527c908349c4513165791ff3e010603b0915c48f08c2

Filed this RFE after discussing with Colin Walters: https://bugzilla.redhat.com/show_bug.cgi?id=1185106

#9 Updated by bmbouter over 5 years ago

It sounds like we may have few alternatives, but what can be done about potential orphan processes if the task process suddenly gets killed with SIGKILL? With the 2.6.0 release plugin writers are told not to use subprocess. There is some talk of this in the 2.6.0 release notes

Perhaps the ostree will auto-exit after some time, like a timeout? Or maybe we can call subprocess in such a way that when the parent exits the child will also exit. Can we do that?

#10 Updated by jortel@redhat.com over 5 years ago

If we decide to shell out, I think this command is fast enough to run safely as a non-deamon child process without concern of orphans. That said, I really don't like the idea of it. We should still consider just using libostree without the additional API. The downside is that when newer features/constructs get introduced into ostree (and ostree repositories), we may need to patch pulp's use of the lib. When discussing with Colin, he referenced a new thing called "deltas" they are working on. I will discusss with Colin further to better understand the impact of this decision.

Example:

[root@icarus fedora-atomic]# ostree --repo=repo-clone init --mode=archive-z2
[root@icarus fedora-atomic]# time ostree --repo=repo-clone pull-local repo fedora-atomic/rawhide/x86_64/docker-host
Enumerating objects...

pull: 17537/17537 scanned, 17537 objects copied
Writing 1 refs

real 0m0.321s
user 0m0.249s
sys 0m0.388s

#11 Updated by bmbouter over 5 years ago

@jortel: As long as the shell calls are guaranteed to exit on their own I think its fine. It sounds like they will. My main concern is orphan processes being leftover when cancel becomes a SIGKILL with 2.6.0.

#12 Updated by mhrivnak over 5 years ago

  • Sprint/Milestone deleted (9)

#13 Updated by bmbouter over 5 years ago

  • Tags Sprint Candidate added

#14 Updated by mhrivnak over 5 years ago

  • Sprint/Milestone set to 12

#15 Updated by jortel@redhat.com over 5 years ago

  • Status changed from NEW to ASSIGNED
  • Start date changed from 01/07/2015 to 02/16/2015

#16 Updated by jortel@redhat.com over 5 years ago

  • % Done changed from 0 to 10

#17 Updated by jortel@redhat.com over 5 years ago

  • % Done changed from 10 to 100

#18 Updated by jortel@redhat.com over 5 years ago

  • Status changed from ASSIGNED to POST

#19 Updated by bmbouter over 5 years ago

  • Tags deleted (Sprint Candidate)

#20 Updated by jortel@redhat.com over 5 years ago

  • Status changed from POST to MODIFIED

#21 Updated by jortel@redhat.com over 5 years ago

  • Assignee deleted (jortel@redhat.com)

#22 Updated by dkliban@redhat.com over 5 years ago

  • Target Release - OSTree set to master

#23 Updated by dkliban@redhat.com over 5 years ago

  • Status changed from MODIFIED to 5

#24 Updated by bmbouter about 5 years ago

  • Assignee set to jortel@redhat.com
  • Groomed set to No
  • Sprint Candidate set to No

This was missing assignee so I added it.

#25 Updated by rbarlow over 4 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
  • Target Release - OSTree changed from master to 1.0.0

#26 Updated by bmbouter over 2 years ago

  • Sprint set to February 2015

#27 Updated by bmbouter over 2 years ago

  • Sprint/Milestone deleted (12)

#28 Updated by bmbouter over 1 year ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF