Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-08-24T09:46:13ZPulp
Planio Pulp - Issue #9286 (NEW): Check failed during installation when using vault encrypted variablehttps://pulp.plan.io/issues/92862021-08-24T09:46:13Zbeenje
<p>I tried installing pulp using the Pulp 3 Ansible Installer playbook:</p>
<pre><code class="yaml syntaxhl" data-language="yaml"><span class="nn">---</span>
<span class="na">collections</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">pulp.pulp_installer</span>
<span class="na">version</span><span class="pi">:</span> <span class="s">3.14.4</span>
</code></pre>
<p>I encrypted the pulp_default_admin_password and secret_key in my inventory (using ansible-vault encrypt_string -n pulp_default_admin_password xxxxxx).
When running the playbook, 2 tasks failed:</p>
<pre><code class="yaml syntaxhl" data-language="yaml"><span class="s">ASK [pulp.pulp_installer.pulp_common</span> <span class="err">:</span> <span class="s">Check if required variables are set] ************************************************************************************</span>
<span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">w-v-pulp-0</span><span class="pi">]</span> <span class="s">=> (item=pulp_settings.content_origin) => {</span>
<span class="s">"__pulp_common_req_var"</span><span class="err">:</span> <span class="s2">"</span><span class="s">pulp_settings.content_origin"</span><span class="err">,</span>
<span class="s2">"</span><span class="s">ansible_loop_var"</span><span class="err">:</span> <span class="s2">"</span><span class="s">__pulp_common_req_var"</span><span class="err">,</span>
<span class="s2">"</span><span class="s">changed"</span><span class="err">:</span> <span class="no">false</span><span class="s">,</span>
<span class="s">"msg"</span><span class="err">:</span> <span class="s2">"</span><span class="s">All</span><span class="nv"> </span><span class="s">assertions</span><span class="nv"> </span><span class="s">passed"</span>
<span class="err">}</span>
<span class="na">fatal</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">w-v-pulp-0</span><span class="pi">]</span><span class="err">:</span> <span class="s">FAILED! => {"msg"</span><span class="err">:</span> <span class="s2">"</span><span class="s">The</span><span class="nv"> </span><span class="s">conditional</span><span class="nv"> </span><span class="s">check</span><span class="nv"> </span><span class="s">'pulp_settings.secret_key</span><span class="nv"> </span><span class="s">|</span><span class="nv"> </span><span class="s">default('',</span><span class="nv"> </span><span class="s">true)</span><span class="nv"> </span><span class="s">|</span><span class="nv"> </span><span class="s">length</span><span class="nv"> </span><span class="s">></span><span class="nv"> </span><span class="s">0'</span><span class="nv"> </span><span class="s">failed.</span><span class="nv"> </span><span class="s">The</span><span class="nv"> </span><span class="s">error</span><span class="nv"> </span><span class="s">was:</span><span class="nv"> </span><span class="s">Unexpected</span><span class="nv"> </span><span class="s">templating</span><span class="nv"> </span><span class="s">type</span><span class="nv"> </span><span class="s">error</span><span class="nv"> </span><span class="s">occurred</span><span class="nv"> </span><span class="s">on</span><span class="nv"> </span><span class="s">({%</span><span class="nv"> </span><span class="s">if</span><span class="nv"> </span><span class="s">pulp_settings.secret_key</span><span class="nv"> </span><span class="s">|</span><span class="nv"> </span><span class="s">default('',</span><span class="nv"> </span><span class="s">true)</span><span class="nv"> </span><span class="s">|</span><span class="nv"> </span><span class="s">length</span><span class="nv"> </span><span class="s">></span><span class="nv"> </span><span class="s">0</span><span class="nv"> </span><span class="s">%}</span><span class="nv"> </span><span class="s">True</span><span class="nv"> </span><span class="s">{%</span><span class="nv"> </span><span class="s">else</span><span class="nv"> </span><span class="s">%}</span><span class="nv"> </span><span class="s">False</span><span class="nv"> </span><span class="s">{%</span><span class="nv"> </span><span class="s">endif</span><span class="nv"> </span><span class="s">%}):</span><span class="nv"> </span><span class="s">object</span><span class="nv"> </span><span class="s">of</span><span class="nv"> </span><span class="s">type</span><span class="nv"> </span><span class="s">'AnsibleVaultEncryptedUnicode'</span><span class="nv"> </span><span class="s">has</span><span class="nv"> </span><span class="s">no</span><span class="nv"> </span><span class="s">len()"</span><span class="err">}</span>
<span class="s">TASK [pulp.pulp_installer.pulp_database_config</span> <span class="err">:</span> <span class="s">Check if required variables are set] ***************************************************************************</span>
<span class="na">fatal</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">w-v-pulp-0</span><span class="pi">]</span><span class="err">:</span> <span class="s">FAILED! => {"msg"</span><span class="err">:</span> <span class="s2">"</span><span class="s">The</span><span class="nv"> </span><span class="s">conditional</span><span class="nv"> </span><span class="s">check</span><span class="nv"> </span><span class="s">'pulp_default_admin_password</span><span class="nv"> </span><span class="s">|</span><span class="nv"> </span><span class="s">default('',</span><span class="nv"> </span><span class="s">true)</span><span class="nv"> </span><span class="s">|</span><span class="nv"> </span><span class="s">length</span><span class="nv"> </span><span class="s">></span><span class="nv"> </span><span class="s">0'</span><span class="nv"> </span><span class="s">failed.</span><span class="nv"> </span><span class="s">The</span><span class="nv"> </span><span class="s">error</span><span class="nv"> </span><span class="s">was:</span><span class="nv"> </span><span class="s">Unexpected</span><span class="nv"> </span><span class="s">templating</span><span class="nv"> </span><span class="s">type</span><span class="nv"> </span><span class="s">error</span><span class="nv"> </span><span class="s">occurred</span><span class="nv"> </span><span class="s">on</span><span class="nv"> </span><span class="s">({%</span><span class="nv"> </span><span class="s">if</span><span class="nv"> </span><span class="s">pulp_default_admin_password</span><span class="nv"> </span><span class="s">|</span><span class="nv"> </span><span class="s">default('',</span><span class="nv"> </span><span class="s">true)</span><span class="nv"> </span><span class="s">|</span><span class="nv"> </span><span class="s">length</span><span class="nv"> </span><span class="s">></span><span class="nv"> </span><span class="s">0</span><span class="nv"> </span><span class="s">%}</span><span class="nv"> </span><span class="s">True</span><span class="nv"> </span><span class="s">{%</span><span class="nv"> </span><span class="s">else</span><span class="nv"> </span><span class="s">%}</span><span class="nv"> </span><span class="s">False</span><span class="nv"> </span><span class="s">{%</span><span class="nv"> </span><span class="s">endif</span><span class="nv"> </span><span class="s">%}):</span><span class="nv"> </span><span class="s">object</span><span class="nv"> </span><span class="s">of</span><span class="nv"> </span><span class="s">type</span><span class="nv"> </span><span class="s">'AnsibleVaultEncryptedUnicode'</span><span class="nv"> </span><span class="s">has</span><span class="nv"> </span><span class="s">no</span><span class="nv"> </span><span class="s">len()"</span><span class="err">}</span>
</code></pre>
<p>I had to use plain strings to run the playbook.
It should be possible to use encrypted strings.</p> Pulp - Story #8086 (NEW): pulp_installer should use latest version of pip to install packageshttps://pulp.plan.io/issues/80862021-01-13T13:42:45Zdkliban@redhat.com
<p>The newer versions of pip include an improved dependency resolution mechanism. The pulp_installer needs a task added to upgrade pip before installing any pulp packages.</p> Pulp - Issue #8055 (NEW): When SELinux is enabled, pulp_installer relabels all the files in /var/...https://pulp.plan.io/issues/80552021-01-08T17:04:00Zdkliban@redhat.com
<p>The installer takes a long time to upgrade an existing system that's running with SELinux enabled because it always relabels /var/lib/pulp in the "Restore SELinux contexts on Pulp dirs that may exist" handler[0]. The list of the directories is here[1].</p>
<p>[0] <a href="https://github.com/pulp/pulp_installer/blob/3.9.0-1/roles/pulp_common/handlers/main.yml#L13-L21" class="external">https://github.com/pulp/pulp_installer/blob/3.9.0-1/roles/pulp_common/handlers/main.yml#L13-L21</a>
[1] <a href="https://github.com/pulp/pulp_installer/blob/3.9.0-1/roles/pulp_common/defaults/main.yml#L63-L66" class="external">https://github.com/pulp/pulp_installer/blob/3.9.0-1/roles/pulp_common/defaults/main.yml#L63-L66</a></p> Pulp - Issue #7993 (NEW): pulp_installer fails to create the database on EL7 when LANG=C.UTF-8https://pulp.plan.io/issues/79932020-12-11T18:14:24Zmdepaulo@redhat.com
<p>If the managed system is EL7 and has LANG=C.UTF-8, it fails.</p>
<p>This includes when the Vagrant host (Github Actions CI) has LANG=C.UTF-8, it bleeds over to the managed guest by Vagrant design:</p>
<pre><code>fatal: [pulp3-source-centos7]: FAILED! => {"changed": true, "cmd": ["/opt/rh/rh-postgresql96/root/bin/initdb", "-D", "/var/opt/rh/rh-postgresql96/lib/pgsql/data"], "delta": "0:00:08.709082", "end": "2020-12-09 03:28:35.519257", "msg": "non-zero return code", "rc": 1, "start": "2020-12-09 03:28:26.810175", "stderr": "FATAL: invalid input syntax for integer: \"NAMEDATALEN\"
child process exited with exit code 1
initdb: removing contents of data directory \"/var/opt/rh/rh-postgresql96/lib/pgsql/data\"", "stderr_lines": ["FATAL: invalid input syntax for integer: \"NAMEDATALEN\"", "child process exited with exit code 1", "initdb: removing contents of data directory \"/var/opt/rh/rh-postgresql96/lib/pgsql/data\""], "stdout": "The files belonging to this database system will be owned by user \"postgres\".
This user must also own the server process.
The database cluster will be initialized with locale \"C\".
The default database encoding has accordingly been set to \"SQL_ASCII\".
The default text search configuration will be set to \"english\".
Data page checksums are disabled.
fixing permissions on existing directory /var/opt/rh/rh-postgresql96/lib/pgsql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ", "stdout_lines": ["The files belonging to this database system will be owned by user \"postgres\".", "This user must also own the server process.", "", "The database cluster will be initialized with locale \"C\".", "The default database encoding has accordingly been set to \"SQL_ASCII\".", "The default text search configuration will be set to \"english\".", "", "Data page checksums are disabled.", "", "fixing permissions on existing directory /var/opt/rh/rh-postgresql96/lib/pgsql/data ... ok", "creating subdirectories ... ok", "selecting default max_connections ... 100", "selecting default shared_buffers ... 128MB", "selecting dynamic shared memory implementation ... posix", "creating configuration files ... ok", "running bootstrap script ... "]}
</code></pre>
<p>I have an incomplete fix here:</p>
<p><a href="https://gist.github.com/mikedep333/c70a1da4230af5da3daec545e304ffa2" class="external">https://gist.github.com/mikedep333/c70a1da4230af5da3daec545e304ffa2</a></p>
<p>(It's not working properly, LANG is still C.UTF-8 in the postgresql role when I added debug statements to said role.)</p>
<p>I will workaround this on Github Actions CI by changing LANG on the host.</p>
<p>The permanent fix will be to just upgrade CentOS 7 systems to PostgreSQL 10 instead.</p> Pulp - Task #7724 (NEW): Improve runtime of new installation of Pulphttps://pulp.plan.io/issues/77242020-10-20T14:06:47Zbmbouterbmbouter@redhat.com
<p>The request to make the installer go faster</p>
<pre><code>A tower standalone install with automation hub takes about ~40 mins. Which is almost more than double of a normal
Tower install. It seems the most of the time we spent is on pulp-common role. Is there anything we are planning to do
in terms of making it little faster (not running same tasks many time, which pulp common role does) ?
</code></pre> Pulp - Task #7668 (NEW): remove pid files from the systemd service fileshttps://pulp.plan.io/issues/76682020-10-07T17:05:32Zdkliban@redhat.com
<p>Systemd does not need explicitly defined pid files to keep track of the services. We should make a change the systemd service files similar to the change here: <a href="https://github.com/theforeman/puppet-pulpcore/commit/b3b7c133c513dd2c30b00a81e64b2bb33ca92397" class="external">https://github.com/theforeman/puppet-pulpcore/commit/b3b7c133c513dd2c30b00a81e64b2bb33ca92397</a></p> Pulp - Task #7638 (NEW): Fix ansible_python_interpreter issues in pulp_installerhttps://pulp.plan.io/issues/76382020-10-01T18:03:57Zmdepaulo@redhat.com
<p>There are 3 minor / potential issues pertaining to this.</p> Pulp - Task #7482 (NEW): pulp_installer change(s) for Recommended installation layouthttps://pulp.plan.io/issues/74822020-09-09T14:45:55Zmdepaulo@redhat.com
<p>See parent task.</p>
<p>We will just tell pulp_installer users to stop the services before upgrading, instead of the symlink. We will still perform the directory move though.</p> Pulp - Issue #7479 (NEW): pulp_installer source-upgrade CI is failing on pkg_resources.Contextual...https://pulp.plan.io/issues/74792020-09-09T13:50:14Zmdepaulo@redhat.com
<p>This occurs in debian-10 when running collect static after pulp_devel, not when running it after installing pulp via <code>pulp_all_services</code>.</p>
<p>It started occurring on daily CI and pull requests on 2020-09-8. I re-ran the overnight CI from the prior day (which originally succeeded), and when it did, it failed.</p>
<p>This error stands out as a dependency issue. Sometimes a package other than toml is listed though:</p>
<pre><code>pkg_resources.ContextualVersionConflict: (toml 0.10.1 (/usr/local/lib/pulp/lib/python3.7/site-packages), Requirement.parse('toml<=0.10.0'), {'dynaconf'})
</code></pre>
<p><a href="https://github.com/pulp/pulp_installer/runs/1091080108?check_suite_focus=true#step:7:1467" class="external">https://github.com/pulp/pulp_installer/runs/1091080108?check_suite_focus=true#step:7:1467</a></p>
<p>Full error:</p>
<pre><code> RUNNING HANDLER [pulp_common : Collect static content] *************************
Wednesday 09 September 2020 12:42:43 +0000 (0:00:02.148) 0:08:16.417 ***
fatal: [debian-10]: FAILED! => {
"changed": true,
"cmd": [
"/usr/local/lib/pulp/bin/django-admin",
"collectstatic",
"--noinput",
"--link"
],
"delta": "0:00:00.358256",
"end": "2020-09-09 12:42:43.925665",
"rc": 1,
"start": "2020-09-09 12:42:43.567409"
}
STDERR:
Traceback (most recent call last):
File "/usr/local/lib/pulp/bin/django-admin", line 8, in <module>
sys.exit(execute_from_command_line())
File "/usr/local/lib/pulp/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/usr/local/lib/pulp/lib/python3.7/site-packages/django/core/management/__init__.py", line 325, in execute
settings.INSTALLED_APPS
File "/usr/local/lib/pulp/lib/python3.7/site-packages/django/conf/__init__.py", line 79, in __getattr__
self._setup(name)
File "/usr/local/lib/pulp/lib/python3.7/site-packages/django/conf/__init__.py", line 66, in _setup
self._wrapped = Settings(settings_module)
File "/usr/local/lib/pulp/lib/python3.7/site-packages/django/conf/__init__.py", line 157, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/usr/local/lib/pulp/lib/python3.7/site-packages/pulpcore/app/settings.py", line 76, in <module>
plugin_app_config = entry_point.load()
File "/usr/local/lib/pulp/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2410, in load
self.require(*args, **kwargs)
File "/usr/local/lib/pulp/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2433, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/local/lib/pulp/lib/python3.7/site-packages/pkg_resources/__init__.py", line 791, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (toml 0.10.1 (/usr/local/lib/pulp/lib/python3.7/site-packages), Requirement.parse('toml<=0.10.0'), {'dynaconf'})
MSG:
non-zero return code
changed: [fedora-31]
</code></pre> Pulp - Task #7281 (NEW): Update docs to state that installer can only install one cluster at a timehttps://pulp.plan.io/issues/72812020-08-05T14:39:19Zdkliban@redhat.com
<p>The documentation needs to have a "Known limitations" section. One of the items should state that that the installer can only install one Pulp cluster at a time.</p> Pulp - Task #6904 (NEW): Document using https://pypi.org/project/pulpcore-releases/ for the insta...https://pulp.plan.io/issues/69042020-06-03T15:25:07Zbmbouterbmbouter@redhat.com
<p>The Pulp Dependency Checker is a great tool to show compatibility between a pulpcore version and various concerns.</p>
<p>We should do three things:</p>
<ol>
<li>
<p>Move the pdc tool to the pulp org.</p>
</li>
<li>
<p>Add a very obvious link to the pulp_installer docs recommending users to use the tool to determine pulpcore and plugin compatibility</p>
</li>
<li>
<p>Update the error message that the installer puts out when the pre-flight check fails. Have that error message point users to specifically check which plugins are compatible with the pulpcore version the installer is trying to install.</p>
</li>
</ol> Pulp - Task #6798 (NEW): Document the new guidelines for plugin installation logichttps://pulp.plan.io/issues/67982020-05-21T18:47:54Zmdepaulo@redhat.com
<p>There are 3 places they could be:</p>
<ol>
<li>A role in a separate git repo and on galaxy.</li>
<li>A separate role in the pulp_installer repo (pulp_rpm will be this.)</li>
<li>Conditional logic within the pulp_installer's other roles.</li>
</ol> Pulp - Task #6747 (NEW): Demo video for pulp_installerhttps://pulp.plan.io/issues/67472020-05-14T21:48:07Zfao89
<ul>
<li>Video should not have audio</li>
<li>
<a href="https://asciinema.org/" class="external">https://asciinema.org/</a> - records terminal output and can be embedded in our docs and in the README on github</li>
<li>include RPM and Container plugins</li>
</ul> Pulp - Story #6688 (NEW): pulp_installer: preflight check and system-wide packages are incompatiblehttps://pulp.plan.io/issues/66882020-05-08T14:40:15Zmdepaulo@redhat.com
<p>Part of the pre-flight check does not understand system-wide packages, but another part is still affected by them.</p>
<p>This leads to false positives (enforcements) in addition to false negatives in the preflight check.</p>
<p>We no longer need system-wide packages, so we should remove support for it, and migrate user installs off of it, as safely as possible.</p> Pulp - Issue #6274 (NEW): Problem: existing Pulp deployments with system-wide packages conflict w...https://pulp.plan.io/issues/62742020-03-03T19:51:59Zmdepaulo@redhat.com
<p>Note: This issue is specificaly about running ansible-pulp against existing deployments of Pulp.</p>
<p>Manual testing with <a href="https://quay.io/repository/pulp/pulp_rpm-ci-c7" class="external">centos 7 pulp_rpm images</a> reveals that we can modify /usr/local/lib/pulp/pyvenv.cfg to not use system-wide packages anymore, and then let the <code>pip install</code> commands fix the missing dependencies. So this is the approach that ansible-pulp should take when <code>pulp_use_system_wide_pkgs</code> is false (or perhaps whenever centos 7 is in use.)</p>