Story #95
closedAs a user, I can manage ostree content units by branch
100%
Description
Users wants pulp to model ostree content units where each unit is a branch head instead of a snaphot of the entire repository. They also want us to only publish objects associated to those commits and not just links to the entire backing repository. Red Hat composed repositories will include a version field in the commit metadata. Users want the commit metadata inventoried with each content unit and displayed in our CLI. They also want us to chnage what is published to only include links into objects/ that are referenced by branch head commits. Currently we are simply linking to the entire objects/ directory which exposes too much.
Details¶
- Update the type definition to model each unit as a branch head. The unit key will be something like: {remote_id:<id>, branch_id:<path>, commit_id:<hash>}.
- After pulling from a remote repository, add a content unit per branch head instead of one unit representing a snapshot of the entire repository.
- Include the metadata contained in the commit with the unit metadata. OSTree repositories composed by Red Hat will contain a version.
- Update the CLI to display this metadata.
- Update publishing to perform the following steps:
- Create a new ostree repository
- Add a file in the refs/ for each content unit.
- Do a local-pull from the backing ostree repository to create hard links to the objects.
Deliverables¶
- OSTree repositories contain a unit per branch that includes the commit metadata.
- Published ostree repositories include a file in refs/heads per content unit.
- Published ostree repositories objects/ contains hard links to only those objects referenced by the branch head commits.
- The ostree CLI displays commit metadata for each unit.
Updated by jortel@redhat.com over 9 years ago
- Tracker changed from Issue to Story
Updated by jortel@redhat.com about 9 years ago
libostree support for pull-local we'd planned to use for publishing needs work: https://bugzilla.redhat.com/show_bug.cgi?id=1185106. We'll likely need to shell out to ostree instead :(
Eg:
ostree --repo=repo-clone pull-local repo fcc7dc55bebb8f154c7c527c908349c4513165791ff3e010603b0915c48f08c2
Filed this RFE after discussing with Colin Walters: https://bugzilla.redhat.com/show_bug.cgi?id=1185106
Updated by bmbouter about 9 years ago
It sounds like we may have few alternatives, but what can be done about potential orphan processes if the task process suddenly gets killed with SIGKILL? With the 2.6.0 release plugin writers are told not to use subprocess. There is some talk of this in the 2.6.0 release notes
Perhaps the ostree will auto-exit after some time, like a timeout? Or maybe we can call subprocess in such a way that when the parent exits the child will also exit. Can we do that?
Updated by jortel@redhat.com about 9 years ago
If we decide to shell out, I think this command is fast enough to run safely as a non-deamon child process without concern of orphans. That said, I really don't like the idea of it. We should still consider just using libostree without the additional API. The downside is that when newer features/constructs get introduced into ostree (and ostree repositories), we may need to patch pulp's use of the lib. When discussing with Colin, he referenced a new thing called "deltas" they are working on. I will discusss with Colin further to better understand the impact of this decision.
Example:
[root@icarus fedora-atomic]# ostree --repo=repo-clone init --mode=archive-z2
[root@icarus fedora-atomic]# time ostree --repo=repo-clone pull-local repo fedora-atomic/rawhide/x86_64/docker-host
Enumerating objects...
pull: 17537/17537 scanned, 17537 objects copied
Writing 1 refs
real 0m0.321s
user 0m0.249s
sys 0m0.388s
Updated by bmbouter about 9 years ago
@jortel: As long as the shell calls are guaranteed to exit on their own I think its fine. It sounds like they will. My main concern is orphan processes being leftover when cancel becomes a SIGKILL with 2.6.0.
Updated by jortel@redhat.com about 9 years ago
- Status changed from NEW to ASSIGNED
- Start date changed from 01/07/2015 to 02/16/2015
Updated by jortel@redhat.com about 9 years ago
- Status changed from ASSIGNED to POST
Updated by jortel@redhat.com about 9 years ago
- Status changed from POST to MODIFIED
Updated by jortel@redhat.com about 9 years ago
- Assignee deleted (
jortel@redhat.com)
Updated by dkliban@redhat.com almost 9 years ago
- Target Release - OSTree set to master
Updated by dkliban@redhat.com almost 9 years ago
- Status changed from MODIFIED to 5
Updated by bmbouter over 8 years ago
- Assignee set to jortel@redhat.com
- Groomed set to No
- Sprint Candidate set to No
This was missing assignee so I added it.
Updated by rbarlow about 8 years ago
- Status changed from 5 to CLOSED - CURRENTRELEASE
- Target Release - OSTree changed from master to 1.0.0