Need a file-system integrity report for /var/lib/pulp
In certain cases due to historical issues with Satellite or on-going problems during content manipulation, there are situations where various inconsistencies can exist with the files contained within /var/lib/pulp
We have customers who are going to start deploying the new 'repair' facilities in this feature we are adding:
- [RFE] Allow Pulp to verify/repair corrupted packages in a repository
with the addition of the repair side of this feature we need a way to identify the following conditions:
- Missing RPMs from /var/lib/pulp/content
- Corrupt/NOT OK md5sums on any unit in /var/lib/pulp/content
- invalid repositories contained within /var/lib/pulp/published where the yum metadata points at sylinks that are missing
- missing or broken symlinks for published repositories for Content Views
May add more criteria to check but in order to restore confidence in the integrity of /var/lib/pulp, we need to be able to report on the state of this sub-directory.
Runtimes to generate this report are expected to be very long but this should not be a blocker for the implementation
#1 Updated by firstname.lastname@example.org 2 months ago
I imagine this would be a stand alone tool (script) that runs on each satellite/capsule and writes a report. The tool should display progress when possible and write a file containing the report.
$ tool -h -s validate stored content file exists and match size/checksum when known. -b validate symlinks (find broken) -m validate that published metadata references valid symlinks -a validate all. -p restrict publishing validation to a specific directory. default: /var/lib/pulp/published -o path to generated report.
Some of the validation will require the tool to have content type specific knowledge so we need to determine which content types need to be supported. RPM has been specifically requested so let's start with that. The tool needs to be designed to support adding validation for additional content types as requested.
The tool should grab information about stale publishes and possibly qualify broken symlinks to reduce/eliminate false positives. Publishing is stale when the repository is sync'd after the last publish.
The report should have a heading for each test followed by summary and list of errors.
I think we should determine the expected data output and get the user to ack that it meets their needs before starting on the work. +1 to using json. Here is a half-proposal and a question:
"corrupted_content": ["/path/to/corrupted/file1", "/path/to/corrupted/file1"],
"broken_symlinks": ["/path/to/broken/symlink1", "/path/to/broken/symlink2"]
schema defines the schema version of this.
corrupted_content is a list of paths to corrupted content
broken_symlinks is a list of broken symlinks
I'm not sure what the output format should be for the "published metadata does not reference valid symlinks" part of the report. Can someone write an example of what a failure like this would be. Even a written example (not a json example would be good).
Please register to edit this issue