Pulp

+1 to minio https://hub.docker.com/r/minio/minio/ https://docs.min.io/docs/python-client-quickstart-guide.html

#!/usr/bin/env sh
export MINIO_ACCESS_KEY=AKIAIT2Z5TDYPX3ARJBA
export MINIO_SECRET_KEY=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
sudo mkdir /data
podman run -d -p 9000:9000 -e MINIO_ACCESS_KEY=$MINIO_ACCESS_KEY -e MINIO_SECRET_KEY=$MINIO_SECRET_KEY minio/minio server /data
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
sudo mv mc /usr/local/bin
sudo dnf install -y nc
while ! nc -z localhost 9000; do echo 'Wait minio to startup...' && sleep 0.1; done;
sleep 5
mc config host add s3 http://localhost:9000 AKIAIT2Z5TDYPX3ARJBA fqRvjWaPU5o0fCqQuUWbj9Fainj2pVZtBCiDiieS --api S3v4
mc config host rm local
mc mb s3/pulp3 --region eu-central-1
mc tree s3
workon pulp
sudo bash -c "echo AWS_ACCESS_KEY_ID = \'AKIAIT2Z5TDYPX3ARJBA\' >> /etc/pulp/settings.py"
sudo bash -c "echo AWS_SECRET_ACCESS_KEY = \'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\' >> /etc/pulp/settings.py"
sudo bash -c "echo AWS_STORAGE_BUCKET_NAME = \'pulp3\' >> /etc/pulp/settings.py"
sudo bash -c "echo AWS_DEFAULT_ACL = None >> /etc/pulp/settings.py"
sudo bash -c "echo S3_USE_SIGV4 = True >> /etc/pulp/settings.py"
sudo bash -c "echo AWS_S3_SIGNATURE_VERSION = \'s3v4\' >> /etc/pulp/settings.py"
sudo bash -c "echo AWS_S3_ADDRESSING_STYLE = \'path\' >> /etc/pulp/settings.py"
sudo bash -c "echo AWS_S3_REGION_NAME = \'eu-central-1\' >> /etc/pulp/settings.py"
sudo bash -c "echo DEFAULT_FILE_STORAGE = \'storages.backends.s3boto3.S3Boto3Storage\' >> /etc/pulp/settings.py"
sudo bash -c "echo MEDIA_ROOT = \'\' >> /etc/pulp/settings.py"
sudo bash -c "echo AWS_S3_ENDPOINT_URL = \'http:\/\/localhost:9000\' >> /etc/pulp/settings.py"
pip install django-storages[boto3] pytest
prestart
cat /etc/pulp/settings.py
cd devel/pulp_file/docs/_scripts
./docs_check_sync_publish.sh
mc tree s3/pulp3
cd ~/devel/pulp_file/
pytest pulp_file/tests/functional

Our regular CI currently runs against containers / pulp-operator

It is TBD whether it would be quicker to run against containers / pulp-operator, or to try to use ansible-pulp CI somehow.

#!/usr/bin/env sh
sudo dnf install -y docker
sudo groupadd docker && sudo gpasswd -a ${USER} docker && sudo systemctl restart docker
newgrp docker
workon pulp
pip install awscli django-storages[boto3] localstack pytest
aws configure set aws_access_key_id AKIAIT2Z5TDYPX3ARJBA
aws configure set aws_secret_access_key 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
aws configure set default.region eu-central-1
aws configure set s3.signature_version s3v4
aws configure set default.s3.addressing_style path
mkdir -p /tmp/localstack/data
export LOCALSTACK_SERVICES=s3
export DEFAULT_REGION="eu-central-1"
export DATA_DIR=/tmp/localstack/data
ENTRYPOINT=-d localstack start --docker
sleep 10
sudo bash -c "echo AWS_ACCESS_KEY_ID = \'AKIAIT2Z5TDYPX3ARJBA\' >> /etc/pulp/settings.py"
sudo bash -c "echo AWS_SECRET_ACCESS_KEY = \'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\' >> /etc/pulp/settings.py"
sudo bash -c "echo AWS_STORAGE_BUCKET_NAME = \'pulp3\' >> /etc/pulp/settings.py"
sudo bash -c "echo AWS_DEFAULT_ACL = None >> /etc/pulp/settings.py"
sudo bash -c "echo S3_USE_SIGV4 = True >> /etc/pulp/settings.py"
sudo bash -c "echo AWS_S3_SIGNATURE_VERSION = \'s3v4\' >> /etc/pulp/settings.py"
sudo bash -c "echo AWS_S3_ADDRESSING_STYLE = \'path\' >> /etc/pulp/settings.py"
sudo bash -c "echo AWS_S3_REGION_NAME = \'eu-central-1\' >> /etc/pulp/settings.py"
sudo bash -c "echo DEFAULT_FILE_STORAGE = \'storages.backends.s3boto3.S3Boto3Storage\' >> /etc/pulp/settings.py"
sudo bash -c "echo MEDIA_ROOT = \'\' >> /etc/pulp/settings.py"
sudo bash -c "echo AWS_S3_ENDPOINT_URL = \'http:\/\/localhost:4572\' >> /etc/pulp/settings.py"
prestart
aws --endpoint-url=http://localhost:4572 s3 mb s3://pulp3
aws --endpoint-url=http://localhost:4572 s3api list-buckets
cd devel/pulp_file/docs/_scripts
./docs_check_sync_publish.sh
aws --endpoint-url=http://localhost:4572 s3 ls s3://pulp3 --recursive
ls /tmp/localstack/data
cd ~/devel/pulp_file/
pytest pulp_file/tests/functional

https://github.com/ceph/cn cn is very limited, I could not find a way to set the region, signature version and addressing style. Even secret and access key we have to change in a hack-ish way: https://github.com/ceph/cn#note-that-the-object-gateway-tab-in-the-dashboard-is-not-enabled-yet-so-run-the-following-to-enable-rgw-dashboard