Project

Profile

Help

Maintenance: Planio will be observing a scheduled maintenance window this Tuesday, November 5, 2024 from 03:00 UTC until 06:30 UTC to perform urgent network maintenance in our primary data center. Your Planio account will be unavailable during this maintenance window.

Issue #2717

closed

Issue #2701: Crane documentation doesn't answer critical installation and usage questions

Config file's "data_dir" default value doesn't work with Pulp

Added by Ichimonji10 over 7 years ago. Updated over 5 years ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version - Crane:
Platform Release:
Target Release - Crane:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Documentation, Pulp 2
Sprint:
Quarter:

Description

During its normal operation, Crane searches for and JSON "redirect files" that define certain information about the Docker images published by Pulp. The "data_dir" option controls where Crane looks for these files. By default, Cranne looks in /var/lib/crane/metadata/ By default, Pulp places these files in:

  • /var/lib/pulp/published/docker/v1/app/
  • /var/lib/pulp/published/docker/v2/app/

This is a pain in the butt. Can Crane's data_dir option default to /var/lib/pulp/published/docker/?

Actions #1

Updated by ttereshc over 7 years ago

@ipanova will comment on the issue, we will triage it next time.

Actions #2

Updated by ipanova@redhat.com over 7 years ago

by default Pulp places published files under /v/l/p/published/docker , you are correct. One of he reason why we do not hardcode the path as you suggest is, in the distributor config of the docker you can change the publish directory of docker content.

http://docs.pulpproject.org/plugins/pulp_docker/tech-reference/distributor.html#supported-keys

So the publish dir can be changed if you specify that in the distributor config.

If you scroll down here https://github.com/pulp/crane/blob/master/docs/index.rst#configuration until the Note, you will see that it is clearly said 'The path specified in data_dir should be a shared mount point between Crane and Pulp' For me it is quite straightforward, but maybe not for the newcomer. what i can do is to write even more detailed and clear note that would mention in addition that you need to mount volume with /var/lib/pulp/published to /var/lib/crane/metadata, or mount it somewhere else and then symlink.

Does that satisfy your needs and questions?

Actions #3

Updated by Ichimonji10 over 7 years ago

One of he reason why we do not hardcode the path as you suggest is, in the distributor config of the docker you can change the publish directory of docker content.

Sure. It's totally possible to change the configuration of each pulp_docker distributor so that metadata files are published to /var/lib/crane/metadata. But wouldn't it be nice if users didn't have to go through that extra step to make Pulp and Crane work together?

what i can do is to write even more detailed and clear note

Good documentation is always nice, but it's beside the point. What I'm asking for is for Crane to work out of the box, and documentation doesn't do that.

Actions #4

Updated by Ichimonji10 over 7 years ago

As per a discussion on IRC, the following scenario was proposed: What if a user had changed Pulp's working directory from /var/lib/pulp to /foo/bar/? In that case, if Crane's default data_dir value is /var/lib/pulp/published/docker, then Crane will break.

That's true. But it's an absurd argument. If a user takes steps to change how Pulp works, then they should also be prepared to fix the breakage that occurs as a result of those steps. The possibility of a non-standard Pulp deployment doesn't relieve us of our obligation to make Crane deployments work out of the box when reasonably possible.

The counter-factual above is like saying "a user can re-compile the kernel so that /etc is re-mounted at /etcetera, so instead of letting Crane look for its configuration file at /etc/crane.conf by default, we will require that the user specify the full path to Crane's configuration file at every start-up." If a user re-compiles the kernel in this manner, then that user should be prepared to fix the breakage that occurs as a result of those steps. The possibility of a non-standard kernel deployment doesn't relieve us of our obligation to make Crane deployments work out of the box when reasonably possible.

Actions #5

Updated by Ichimonji10 over 7 years ago

As per some extended discussions on IRC, I've been informed that the primary use case for Crane is to have it installed on a separate system from the rest of Pulp. In those deployments, the /var/lib/pulp/published/docker/ directory wouldn't even exist. In light of this, I totally see the rationale for letting Crane search for metadata in /var/lib/crane/metadata/ by default. Better to search in an empty directory that's guaranteed to exist than to search in a directory that doesn't exist.

At the very least, I would love to see some improvements to the documentation. Crane is broken by default, and the documentation needs to make a big deal out of that fact. (Heck, if RPM packages can be configured to print out post-installation warnings, that would be nice too.) I also think that it needs to more clearly explain the rationale for why Crane is broken by default: Crane and Pulp are intended to be installed on different systems.

On a more goal-oriented note: can we give more importance to the use case of having Crane and Pulp installed on the same system? It's the simplest use case that I can think of. I like the idea of making life easy for a user who's installing Crane for the first time and becoming familiar with how it works.

Maybe a good solution is to create a symlink pointing from /var/lib/crane/metadata/default to /var/lib/pulp/published/docker/. If Crane and Pulp are on the same system, then everything will work just fine. And if Crane and Pulp are on different systems, then an NFS mount can just be slapped on top of /var/lib/crane/metadata/, thus shadowing the symlink. This approach lets us have our cake and eat it too.

Actions #6

Updated by ipanova@redhat.com over 7 years ago

@ichi
I am for improvements in documentation which i suggested to do in comment #2. Plus we could highlight the details and pitfalls with different usecases of where crane is installed. I am against though to create any kind of symlinks, i think it should up to user what and where to symlink and or mount.

Actions #7

Updated by ttereshc over 7 years ago

  • Triaged changed from No to Yes
  • Tags Documentation added
Actions #8

Updated by ipanova@redhat.com over 7 years ago

  • Parent issue set to #2701
Actions #9

Updated by bmbouter over 5 years ago

  • Status changed from NEW to CLOSED - WONTFIX
Actions #10

Updated by bmbouter over 5 years ago

Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.

Actions #11

Updated by bmbouter over 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF