Pulp

I believe webservers have the same issues, users can request a url like /foo/ which would be a directory on the filesystem foo and expect to receive data. They do this with the 'index' convention where the webserver typically looks for a file called index with some type of file extension.

One option is to leave Pulp as-is and just have the Exporter export with ^ convention. At export time though it would be more complicated to perform.

Another option is to find some way to prevent Pulp from making a Publication that can't be exported. Some sort of validation that ensures you don't have /foo and /foo/bar both in a Publication.

What sort of adjustment to PublishedArtifact and PublishedMetadata would improve this situation? Maybe the DB should distinguish directories? Or is it better to leave as-is, and handle at export time?

Another case to consider is plugins that don't have publications (as plugins have the option to distribute repo versions directly like we do in pulp_ansible). Perhaps one option is to have plugins validate repo versions.

daviddavis wrote:

Another case to consider is plugins that don't have publications (as plugins have the option to distribute repo versions directly like we do in pulp_ansible). Perhaps one option is to have plugins validate repo versions.

That's a good point...

I agree with bmbouter's approach.

I will offer some other suggestions/ideas:

1. A filesystem is a hierarchical database. The Windows registry (also a hierarchical database) encountered a kind-of-similar issue decades ago. Their approach applied to this situation would be to create a file named like:

foo/.default

That gets created whenever foo is both a file and a directory. The registry is only accessed via their APIs, so they could do that easily though.

However, that would prevent package managers from accessing the filesystem directly at the original paths and reading foo, correct? Or any 3rd party webservers (hosting for package managers) would have to interpret the filepath.

2. You could also store the real path to the "default" file in a filesystem's folder's extended attribute (user_xattr), but that would still have the same problem.

I think we are trying to make a rather "esoteric" use case work if we map file/directory name conflicts to index or .default or similar.

What should happen if there is already a file/directory named index or .default? We are just shifting the problem one directory layer down. (Of course we could use another mapping in this case)

Additionally, we might want to enable directory listings in the content app in the future. Theoretically, http:/content.app/foo/ (returning the directory listing) could behave differently than http:/content.app/foo (returning the foo content). But that would be very surprising (usually web servers deliver a redirection foo -> foo/ and that's what I would expect the content app to do as well).

I think that we should regard this use case as invalid. We have three options where to do this:
- repo version (as proposed by @daviddavis, i.e. it could be part of the upcoming repo version validation)
- publication
- export

IMHO, we should let publication fail given the strange URL scheme such a publication exhibits when being distributed.
But what is the point of allowing a repo version that can't be published? That's why I tend to already let validation fail. And, as noted above by @daviddavis, this also solves the problem for plugins that have no publication.

Last week we discussed this problem and came to the conclusion that we need to add this validation to two places:

1. Repo version creation
2. Publication

In order to handle the case where the plugin doesn't have publications, we need to check at repo version creation time. We also need to check at publication time though because published artifacts may clash with content relative paths. For example, suppose I create a file content with relative path PULP_MANIFEST/test.iso. That will create problems for file publications.

I think we should also just fail if a overlap is detected. And I think plugins control both of these places though so we'll probably need to create a method or some util that they can reuse.

If this all sounds right, I'll update the description and get this issue groomed by next week.