Task #5062
closed
Task #5069: Build pulp 3 containers
Create pulp_file container images automatically via CI
100%
Description
We need to extend the Travis CI so that whenever pulpcore gets built, it builds it from the actual source tree.
It needs to use the source trees for pulpcore-plugin & pulp_file as well, which are always on disk.
Stable builds (like 3.0.0 or 3.0.0rc5) can be handled via our Travis cronjobs or other means. I was told to look at our existing scripts for reference.
It will be up to other plugins which they wish to base on (nightly or stable.)
For pulpcore, .travis/install.sh needs to do the following:
build containers, with pulp_file
deploy k3s
deploy newly build containers / pulp-operator on top of k3s
And .travis/script.sh needs to do the following:
Run the tests, but against the container rather than locally.
(It is OK to drop codecov for now, to get this working.)
Also, it is OK to specify the source tree(s) only via absolute paths on disk, rather than relative paths like "../".
This task includes fixing the issue that when the CI builds an image and stores it in its docker cache, k3s does not use it. (pulp-operator.git has this issue, and now pulpcore.git will.) Multiple workarounds are possible (such as an image registry), but fixing it is preferred.
Updated by ipanova@redhat.com over 5 years ago
- Status changed from NEW to ASSIGNED
Updated by dkliban@redhat.com over 5 years ago
- Sprint changed from Sprint 55 to Sprint 56
Updated by mdepaulo@redhat.com over 5 years ago
- % Done changed from 0 to 10
Started implementing this as part of 5116:
https://github.com/pulp/pulpcore/pull/242
Added by Mike DePaulo over 5 years ago
Added by Mike DePaulo over 5 years ago
Revision 17ccb3ee | View on GitHub
containers: Build both stable and dev images
and make images far more configurable.
Implementation includes:
Build stable and master (latest) versions (tags) of both images.
Use an undefined plugins list for no plugins ather than the dummy pulpcore-plugin.
Use variables for the pulpcore & pulpcore-plugin pip installs
Use vars/vars.yml as an override for vars (including the images data structure.)
Start work on installing into the container, and limit the CI builds to only pulpcore & pulpcore-plugin stable.
fixes: #5116 Handle properly whether to build container images with pulpcore stable or dev (Actually does more than that.) https://pulp.plan.io/issues/5116
Includes some work on: re: #5062 Create pulpcore and pulp_file container images automatically via CI https://pulp.plan.io/issues/5062
Updated by mdepaulo@redhat.com over 5 years ago
- % Done changed from 10 to 20
Part of this was implemented in pulp-operator. It can now auto-publish its own pulp-operator image to quay.io.
https://github.com/pulp/pulp-operator/pull/15
Added by Mike DePaulo over 5 years ago
Revision 03532831 | View on GitHub
containers: Support pip install from Travis host dirs
Implementation includes: Copying the host dirs into the Docker build environment
Setting the Docker build "context" to the parent dir of the checked out pulpcore (So that it and all the other cloned repos can be accessed during docker build.)
Users must specify host dirs as relative to the parent dir of pulpcore, and begin them with "./". This triggers copying logic.
Templating the Dockerfile with jinja2, with some Ansible-bundled filters. (We've outgrown what env vars in the Dockerfile can do.)
Remove unused Dockerfile variable VERSION
.travis/post_script.sh: Build pulpcore & pulpcore-plugin from Travis host dirs
[noissue]
Partially implements: re: #5062 https://pulp.plan.io/issues/5062 Create pulpcore and pulp_file container images automatically via CI
Updated by mdepaulo@redhat.com about 5 years ago
- % Done changed from 20 to 70
Most of this is done.
It took very long because so many parts of the CI process needed to be updated. Including Pulp-Smash needing to support "sudo kubectl" instead of "ssh".
The outstanding tasks are:
1. Several functional tests are still failing.
2. Need to make the publishing of images happen, including like how python packages are published.
3. Need to update the python package and other publishing scripts for the containers.
Updated by mdepaulo@redhat.com about 5 years ago
Functional tests are now passing (thanks dkliban!)
It looks like there is no need to update the python package and other publishing scripts for the containers.
Need to make the publishing of images happen, including like how python packages are published.
Updated by mdepaulo@redhat.com about 5 years ago
- % Done changed from 70 to 100
The remaining work was moved into a new Epic:
https://pulp.plan.io/issues/5393
Updated by mdepaulo@redhat.com about 5 years ago
- Subject changed from Create pulpcore and pulp_file container images automatically via CI to Create pulp_file container images automatically via CI
- Description updated (diff)
Updated by mdepaulo@redhat.com about 5 years ago
- Status changed from ASSIGNED to 5
Updated by mdepaulo@redhat.com about 5 years ago
- Status changed from 5 to CLOSED - COMPLETE
.
Use Docker rather than k3s-bundled containerd
So that built & cached images can be used later during CI.
This means that the pulp-operator CI now actually tests the current version of the image it builds, rather than the version from quay.
re: #5062 Create pulpcore and pulp_file container images automatically via CI https://pulp.plan.io/issues/5062