Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-07-28T11:43:11ZPulp
Planio Debian Support - Issue #9164 (CLOSED - CURRENTRELEASE): Dropping Python 3.6 and 3.7 from the 2.14...https://pulp.plan.io/issues/91642021-07-28T11:43:11Zquba42
<p>It looks like this was merged prematurely on my part.</p>
<p>Users have reported the following:</p>
<pre><code>I tried to update pulp_deb plugins using ansible but getting error as per below.
TASK [pulp.pulp_installer.pulp_common : Install prerequisites] ******************************************************************************************************************************************************************************
fatal: [pulptest008.karmalab.net]: FAILED! => {"changed": false, "msg": "No package matching 'rh-python38-python' found available, installed or updated", "rc": 126, "results": ["python-setuptools-0.9.8-7.el7.noarch providing python-setuptools is already installed", "No package matching 'rh-python38-python' found available, installed or updated"]}
</code></pre>
<p>Fresh installs are also suffering weird dependency effects where Python 3 is not being installed at all.</p> RPM Support - Issue #8700 (CLOSED - CURRENTRELEASE): CentOS 8 stream repositories fails to synchr...https://pulp.plan.io/issues/87002021-05-05T12:30:27Zadam.tkac@gooddata.com
<p>Hello pulp upstream,</p>
<p>we are regularly mirroring various CentOS stream repos, but recently "sync" tasks started to fail with following traceback:</p>
<pre><code class="text syntaxhl" data-language="text">May 5 12:20:22 pulp32 rq[2083851]: Traceback (most recent call last):
May 5 12:20:22 pulp32 rq[2083851]: File "/opt/gdc/pulp3/lib64/python3.6/site-packages/rq/worker.py", line 1008, in perform_job
May 5 12:20:22 pulp32 rq[2083851]: rv = job.perform()
May 5 12:20:22 pulp32 rq[2083851]: File "/opt/gdc/pulp3/lib64/python3.6/site-packages/rq/job.py", line 706, in perform
May 5 12:20:22 pulp32 rq[2083851]: self._result = self._execute()
May 5 12:20:22 pulp32 rq[2083851]: File "/opt/gdc/pulp3/lib64/python3.6/site-packages/rq/job.py", line 729, in _execute
May 5 12:20:22 pulp32 rq[2083851]: result = self.func(*self.args, **self.kwargs)
May 5 12:20:22 pulp32 rq[2083851]: File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 255, in synchronize
May 5 12:20:22 pulp32 rq[2083851]: dv.create()
May 5 12:20:22 pulp32 rq[2083851]: File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 149, in create
May 5 12:20:22 pulp32 rq[2083851]: loop.run_until_complete(pipeline)
May 5 12:20:22 pulp32 rq[2083851]: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
May 5 12:20:22 pulp32 rq[2083851]: return future.result()
May 5 12:20:22 pulp32 rq[2083851]: File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
May 5 12:20:22 pulp32 rq[2083851]: await asyncio.gather(*futures)
May 5 12:20:22 pulp32 rq[2083851]: File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
May 5 12:20:22 pulp32 rq[2083851]: await self.run()
May 5 12:20:22 pulp32 rq[2083851]: File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 439, in run
May 5 12:20:22 pulp32 rq[2083851]: await self.parse_modules_metadata()
May 5 12:20:22 pulp32 rq[2083851]: File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 495, in parse_modules_metadata
May 5 12:20:22 pulp32 rq[2083851]: modules_metadata_parser.parse()
May 5 12:20:22 pulp32 rq[2083851]: File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 747, in parse
May 5 12:20:22 pulp32 rq[2083851]: content = moduleyaml.read()
May 5 12:20:22 pulp32 rq[2083851]: File "/usr/lib64/python3.6/codecs.py", line 321, in decode
May 5 12:20:22 pulp32 rq[2083851]: (result, consumed) = self._buffer_decode(data, self.errors, final)
May 5 12:20:22 pulp32 rq[2083851]: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfd in position 0: invalid start byte
</code></pre>
<p>We are using following versions:</p>
<pre><code class="text syntaxhl" data-language="text">[root@pulp32:~] pulp status
{
"versions": [
{
"component": "core",
"version": "3.12.1"
},
{
"component": "rpm",
"version": "3.10.0"
},
{
"component": "file",
"version": "1.7.0"
}
</code></pre>
<p>Remote configuration:</p>
<pre><code class="text syntaxhl" data-language="text">[root@pulp32:~] pulp rpm remote show --name c8s-baseos-remote
{
"pulp_href": "/pulp/api/v3/remotes/rpm/rpm/dc58e56e-4a00-47f1-af0d-a9b6786af51c/",
"pulp_created": "2021-05-05T02:44:02.345456Z",
"name": "c8s-baseos-remote",
"url": "http://mirror.centos.org/centos/8-stream/BaseOS/x86_64/os/",
"ca_cert": null,
"client_cert": null,
"tls_validation": true,
"proxy_url": null,
"pulp_labels": {},
"pulp_last_updated": "2021-05-05T02:44:02.345474Z",
"download_concurrency": 10,
"policy": "immediate",
"total_timeout": null,
"connect_timeout": null,
"sock_connect_timeout": null,
"sock_read_timeout": null,
"headers": null,
"rate_limit": null,
"sles_auth_token": null
}
</code></pre>
<p>How to reproduce:</p>
<pre><code class="text syntaxhl" data-language="text">[root@pulp32:~] pulp rpm repository sync --name c8s-baseos --remote c8s-baseos-remote
Started background task /pulp/api/v3/tasks/e50f608f-9bf7-48a9-ad6a-03efb34014d6/
.Error: Task /pulp/api/v3/tasks/e50f608f-9bf7-48a9-ad6a-03efb34014d6/ failed: ''utf-8' codec can't decode byte 0xfd in position 0: invalid start byte'
</code></pre>
<p>Please let me know if you need any more details. Thank you in advance!</p> Debian Support - Task #8682 (CLOSED - CURRENTRELEASE): Migrate to new Distribution model for pulp...https://pulp.plan.io/issues/86822021-05-03T12:21:35Zquba42Debian Support - Issue #8671 (CLOSED - CURRENTRELEASE): Sync with Main Debian Repo fails on a tra...https://pulp.plan.io/issues/86712021-04-30T08:43:16Ztp-hlawatsch
<p>Hello everybody,</p>
<p>we plan to use pulp as our main tool for repos . We testing at the moment Debian mirroring
Our pulp testinstance is runing with the latest offical docker image</p>
<p>This is the configured remote repo</p>
<p>{
"name": "debian",
"url": "<a href="http://ftp.de.debian.org/debian/" class="external">http://ftp.de.debian.org/debian/</a>",
"ca_cert": null,
"client_cert": null,
"client_key": null,
"tls_validation": false,
"proxy_url": null,
"proxy_username": null,
"proxy_password": null,
"username": null,
"password": null,
"pulp_labels": {},
"download_concurrency": "10",
"policy": "immediate",
"total_timeout": null,
"connect_timeout": null,
"sock_connect_timeout": null,
"sock_read_timeout": null,
"rate_limit": null,
"distributions": "buster",
"components": "main",
"architectures": "amd64",
"sync_sources": false,
"sync_udebs": false,
"sync_installer": false,
"gpgkey": null,
"ignore_missing_package_indices": false
}</p>
<p>the sync run in to the following error</p>
<p>{
"pulp_href": "/pulp/api/v3/tasks/a7bd851b-f001-4f5f-89eb-0e0de529a556/",
"pulp_created": "2021-04-30T07:53:11.537345Z",
"state": "failed",
"name": "pulp_deb.app.tasks.synchronizing.synchronize",
"logging_cid": "00de2125667f4c7e8b8e01554c401eeb",
"started_at": "2021-04-30T07:53:11.648803Z",
"finished_at": "2021-04-30T07:53:19.135647Z",
"error": {
"traceback": " File "/usr/local/lib/python3.6/site-packages/rq/worker.py", line 1008, in perform_job\n rv = job.perform()\n File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 706, in perform\n self._result = self._execute()\n File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 729, in _execute\n result = self.func(*self.args, **self.kwargs)\n File "/usr/local/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py", line 122, in synchronize\n DebDeclarativeVersion(first_stage, repository, mirror=mirror).create()\n File "/usr/local/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 149, in create\n loop.run_until_complete(pipeline)\n File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete\n return future.result()\n File "/usr/local/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline\n await asyncio.gather(*futures)\n File "/usr/local/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in <strong>call</strong>\n await self.run()\n File "/usr/local/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 266, in run\n RemoteArtifact.objects.bulk_get_or_create(self._needed_remote_artifacts(batch))\n File "/usr/local/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 324, in _needed_remote_artifacts\n msg.format(rp=content_artifact.relative_path, c=d_content.content)\n",
"description": "No declared artifact with relative path "dists/buster/main/i18n/Translation-ml.bz2" for content """
},</p>
<p>We do not see a misconfiguration on our side so I raised this issue here</p>
<p>Regards
Stephan Hlawatsch</p> Debian Support - Task #8388 (CLOSED - CURRENTRELEASE): Handle ALLOWED_CONTENT_CHECKSUMS settinghttps://pulp.plan.io/issues/83882021-03-11T14:59:24Zquba42
<p>Right now the pulp_deb plugin is dependend on the following hashs being present on artifacts: md5, sha1, sha256, sha512.</p>
<p>Starting with pulpcore 3.11 will exclude md5 and sha1 from ALLOWED_CONTENT_CHECKSUMS by default.</p>
<p>Pulp deb will need to handle this gracefully. There is still some debate on whether pulp_deb makes sense without md5 feature wise. As a result, the solution may need to include demanding or encouraging a different configuration from pulp_deb users than the pulpcore default configuration.</p>
<p>See the following mailing list threads:</p>
<ul>
<li><a href="https://listman.redhat.com/archives/pulp-dev/2021-February/msg00017.html" class="external">https://listman.redhat.com/archives/pulp-dev/2021-February/msg00017.html</a></li>
<li><a href="https://listman.redhat.com/archives/pulp-dev/2021-March/msg00013.html" class="external">https://listman.redhat.com/archives/pulp-dev/2021-March/msg00013.html</a></li>
</ul> CertGuard - Issue #7182 (CLOSED - CURRENTRELEASE): The docs aren't buildinghttps://pulp.plan.io/issues/71822020-07-21T20:25:23Zbmbouterbmbouter@redhat.com
<a name="The-docs-site-on-RTD-is-no-longer-building"></a>
<h2 >The docs site on RTD is no longer building<a href="#The-docs-site-on-RTD-is-no-longer-building" class="wiki-anchor">¶</a></h2>
<p>You can see the 1.0.0 and 1.0.1 changelogs are not present: <a href="https://pulp-certguard.readthedocs.io/en/latest/changes.html" class="external">https://pulp-certguard.readthedocs.io/en/latest/changes.html</a></p>
<p>The last successful build was Completed June 30, 2020. 6:54 p.m from (6d951eb3203a19f627d1f3e0e729f8c3a3107c0b)</p>
<p>The RTD builder fails with:</p>
<pre><code>DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting PyOpenSSL
Downloading pyOpenSSL-19.1.0-py2.py3-none-any.whl (53 kB)
ERROR: Could not find a version that satisfies the requirement pulpcore<3.6,>=3.3 (from -r requirements.txt (line 2)) (from versions: 3.0.0a1.dev3, 3.0.0a1.dev4, 3.0.0a1.dev5, 3.0.0a1, 3.0.0a2, 3.0.0a3, 3.0.0a4, 3.0.0a5, 3.0.0a6, 3.0.0a7, 3.0.0a8, 3.0.0a9, 3.0.0a10, 3.0.0a11, 3.0.0a12, 3.0.0a13, 3.0.0a14, 3.0.0a15, 3.0.0a16, 3.0.0a17, 3.0.0a18, 3.0.0a19, 3.0.0a20, 3.0.0a21, 3.0.0a22, 3.0.0a23, 3.0.0a24, 3.0.0a25, 3.0.0a26, 3.0.0a27, 3.0.0a28, 3.0.0a29, 3.0.0b1, 3.0.0b2, 3.0.0b3, 3.0.0b4, 3.0.0b5, 3.0.0b6, 3.0.0b7)
ERROR: No matching distribution found for pulpcore<3.6,>=3.3 (from -r requirements.txt (line 2))
</code></pre> Pulp - Issue #4136 (CLOSED - WONTFIX): Installation docs for Fedora incorrectly use Fedora reposi...https://pulp.plan.io/issues/41362018-11-08T13:13:30Zamacdona@redhat.comaustin@redhat.com
<p>"Beginning with Fedora 24, Pulp is included in the Fedora project and no special repositories are needed to install there."</p>
<p>The fedora repo does have Pulp, but it's 2.13. We should remove Pulp from Fedora and fix the documentation.</p>
<p><a href="https://docs.pulpproject.org/user-guide/installation/fedora.html" class="external">https://docs.pulpproject.org/user-guide/installation/fedora.html</a></p> Python Support - Refactor #4132 (CLOSED - NOTABUG): Metadata is not downloaded in parallelhttps://pulp.plan.io/issues/41322018-11-07T17:01:10Zamacdona@redhat.comaustin@redhat.com
<p>Metadata for the python plugin is all retrieved in the PythonFirstStage[0]. Project level metadata is downloaded in a for loop [1]. Since we are using asyncio, the `await` does cede control while it is downloading a metadata file, so after the first metadata file is downloaded and processed, the PythonFirstStage continues to download 1 metadata file at a time while all later stages from pulpcore continue.</p>
<p>My assumption is that downloading the project level metadata in parallel would be a relatively small performance improvement when each project (corresponding to 1 metadata file) has many Python distributions associated with it. However, for lazy sync or cases where each project has a small number of Python distributions, downloading the metadata in parallel could be a very large performance increase.</p>
<a name="Background"></a>
<h3 >Background<a href="#Background" class="wiki-anchor">¶</a></h3>
<p>Unless a Stage implements parallel calls with asyncio, each Stage only operates 1 at a time (or 1 batch at a time). The only pulpcore Stage that runs multiple calls in parallel is the ArtifactDownloader[1], which uses the ArtifactDownloaderRunner to ensure_futures[2], which is what enables many calls to happen simultaneously.[2]</p>
<a name="Design-Options"></a>
<h3 >Design Options<a href="#Design-Options" class="wiki-anchor">¶</a></h3>
<p>We can download the metadata in parallel in at least 2 ways.</p>
<ol>
<li>Split up the PythonFirstStage. Even though the Python plugin doesn't have a "Project" Content unit, we can still create DeclarativeContent objects that can flow to later stages.
<ol>
<li>ProjectListStage - Rather than downloading metadata in the first stage[3], we could create a DeclarativeContent object for each project, and out_q.put(dc).</li>
<li>ArtifactDownloader (from core, unchanged) - This would download (in parallel) each of the project metadata files.</li>
<li>ProcessProjectMetadata -This Stage would open and read the project dc.d_artifact file, and create dcs for PythonPackageContent, and out_q.put(python_package_content_dc). It would <strong>not</strong> continue to pass project dcs down the pipeline. This would roughly correspond to part of the first stage[4], but would also need to include package filtering[5].</li>
<li>Suggested Pipeline: ProjectListStage -> ArtifactDownloader -> ProcessProjectMetadata -> ArtifactDownloader -> save artifacts, save content,etc</li>
</ol>
</li>
<li>It's probably also possible to implement parallel downloads directly into a monolithic first stage by using asyncio.ensure_future, but IMO this should not be idiomatic use of the Stages API. It would be complex and would require knowledge of asyncio that is not really expected of plugin writers. Besides, the tricky part of this is already implemented by the ArtifactDownloader.</li>
</ol>
<p>[0]: <a href="https://github.com/pulp/pulp_python/blob/master/pulp_python/app/tasks/sync.py#L57" class="external">https://github.com/pulp/pulp_python/blob/master/pulp_python/app/tasks/sync.py#L57</a><br>
[1]: <a href="https://github.com/pulp/pulp/blob/master/plugin/pulpcore/plugin/stages/artifact_stages.py#L215" class="external">https://github.com/pulp/pulp/blob/master/plugin/pulpcore/plugin/stages/artifact_stages.py#L215</a><br>
[2]: <a href="https://github.com/pulp/pulp/blob/master/plugin/pulpcore/plugin/stages/artifact_stages.py#L121-L132" class="external">https://github.com/pulp/pulp/blob/master/plugin/pulpcore/plugin/stages/artifact_stages.py#L121-L132</a><br>
[3]: <a href="https://github.com/pulp/pulp_python/blob/master/pulp_python/app/tasks/sync.py#L96" class="external">https://github.com/pulp/pulp_python/blob/master/pulp_python/app/tasks/sync.py#L96</a><br>
[4]: <a href="https://github.com/pulp/pulp_python/blob/master/pulp_python/app/tasks/sync.py#L121-L129" class="external">https://github.com/pulp/pulp_python/blob/master/pulp_python/app/tasks/sync.py#L121-L129</a><br>
[5]: <a href="https://github.com/pulp/pulp_python/blob/master/pulp_python/app/tasks/sync.py#L151" class="external">https://github.com/pulp/pulp_python/blob/master/pulp_python/app/tasks/sync.py#L151</a></p> Pulp - Story #3778 (CLOSED - CURRENTRELEASE): [Epic] As a user, I can run Pulp 3 in a FIPS-enable...https://pulp.plan.io/issues/37782018-06-21T16:13:49ZdaviddavisNectar - Issue #2235 (CLOSED - DUPLICATE): Download progress won't stream to clientshttps://pulp.plan.io/issues/22352016-09-07T14:20:15Zbehi_1370behnam.loghmani@gmail.com
<p>I cloned nectar master branch repository, install and replace it with nectar 1.5.2-1 and test download files from clients.<br>
The problem that keeps proxy is solved but now another problem occurred. Packages won’t stream while downloading is in progress. For example when I want to download some packages, every packages completely download in server first and after that it serve packages to client so I can’t see the progress bar of downloading packages and also if I want to download a large file, it will response a timeout error on proxyset</p> Pulp - Issue #2079 (CLOSED - DUPLICATE): Fails to sync EPEL 7.x repository with Pulp 2.9.0https://pulp.plan.io/issues/20792016-07-14T14:01:27ZAnonymous
<p>I was trying to synchronize my EPEL 6.x/7.x repositories using the workaround published here.</p>
<p><a href="http://www.pulpproject.org/2016/07/12/pulp-2-9-0-generally-available/" class="external">http://www.pulpproject.org/2016/07/12/pulp-2-9-0-generally-available/</a></p>
<p>The migration took a little while but after applying the work around i get the following error ONLY on EPEL 7.x repository.</p>
<pre><code>pulp-admin rpm repo sync run --repo-id epel-7-x86_64
+----------------------------------------------------------------------+
Synchronizing Repository [epel-7-x86_64]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[/]
... completed
Downloading repository content...
[|]
[============= ] 27%
RPMs: 2/3 items
Delta RPMs: 0/0 items
... completed
Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed
Importing package groups/categories...
[\]
... completed
Cleaning duplicate packages...
[\]
... completed
Task Succeeded
Initializing repo metadata
[-]
... completed
Publishing Distribution files
[-]
... completed
Publishing RPMs
[==================================== ] 72%
7495 of 10349 items
Task Failed
Invalid block tag: 'compress'
</code></pre>
<p>This does not happen on EPEL 6.x repository and I try the following in order to fix the problem with no luck.</p>
<p>1. Delete the EPEL 7.x repository<br>
2. Create a new EPEL repository<br>
3. Update the repo using the workaround --skip...<br>
4. Synchronize again</p> Pulp - Issue #1956 (CLOSED - WONTFIX): "Cleaning duplicate packages" fails after upgrading from 2...https://pulp.plan.io/issues/19562016-05-31T10:37:26Zakegata@gmail.comakegata@gmail.com
<p>After applying the patch in <a href="https://pulp.plan.io/issues/1952" class="external">https://pulp.plan.io/issues/1952</a>, the actual syncing of packages work, but the whole sync fails on the "Cleaning duplicate packages" step:</p>
<pre><code class="c syntaxhl" data-language="c"><span class="nl">Operations:</span> <span class="n">sync</span>
<span class="n">Resources</span><span class="o">:</span> <span class="n">RHEL7_test_x86_64</span> <span class="p">(</span><span class="n">repository</span><span class="p">)</span>
<span class="n">State</span><span class="o">:</span> <span class="n">Failed</span>
<span class="n">Start</span> <span class="n">Time</span><span class="o">:</span> <span class="mi">2016</span><span class="o">-</span><span class="mo">05</span><span class="o">-</span><span class="mi">31</span><span class="n">T10</span><span class="o">:</span><span class="mi">17</span><span class="o">:</span><span class="mo">00</span><span class="n">Z</span>
<span class="n">Finish</span> <span class="n">Time</span><span class="o">:</span> <span class="mi">2016</span><span class="o">-</span><span class="mo">05</span><span class="o">-</span><span class="mi">31</span><span class="n">T10</span><span class="o">:</span><span class="mi">31</span><span class="o">:</span><span class="mi">20</span><span class="n">Z</span>
<span class="n">Result</span><span class="o">:</span> <span class="n">N</span><span class="o">/</span><span class="n">A</span>
<span class="n">Task</span> <span class="n">Id</span><span class="o">:</span> <span class="n">d3b8443b</span><span class="o">-</span><span class="mf">0e61</span><span class="o">-</span><span class="mi">4027</span><span class="o">-</span><span class="mi">8</span><span class="n">ded</span><span class="o">-</span><span class="n">b419b38f13f6</span>
<span class="n">Progress</span> <span class="n">Report</span><span class="o">:</span>
<span class="n">Yum</span> <span class="n">Importer</span><span class="o">:</span>
<span class="n">Comps</span><span class="o">:</span>
<span class="n">State</span><span class="o">:</span> <span class="n">FINISHED</span>
<span class="n">Content</span><span class="o">:</span>
<span class="n">Details</span><span class="o">:</span>
<span class="n">Drpm</span> <span class="n">Done</span><span class="o">:</span> <span class="mi">0</span>
<span class="n">Drpm</span> <span class="n">Total</span><span class="o">:</span> <span class="mi">0</span>
<span class="n">Rpm</span> <span class="n">Done</span><span class="o">:</span> <span class="mi">1</span>
<span class="n">Rpm</span> <span class="n">Total</span><span class="o">:</span> <span class="mi">1</span>
<span class="n">Error</span> <span class="n">Details</span><span class="o">:</span>
<span class="n">Items</span> <span class="n">Left</span><span class="o">:</span> <span class="mi">0</span>
<span class="n">Items</span> <span class="n">Total</span><span class="o">:</span> <span class="mi">1</span>
<span class="n">Size</span> <span class="n">Left</span><span class="o">:</span> <span class="mi">0</span>
<span class="n">Size</span> <span class="n">Total</span><span class="o">:</span> <span class="mi">74037668</span>
<span class="n">State</span><span class="o">:</span> <span class="n">FINISHED</span>
<span class="n">Distribution</span><span class="o">:</span>
<span class="n">Error</span> <span class="n">Details</span><span class="o">:</span>
<span class="n">Items</span> <span class="n">Left</span><span class="o">:</span> <span class="mi">0</span>
<span class="n">Items</span> <span class="n">Total</span><span class="o">:</span> <span class="mi">0</span>
<span class="n">State</span><span class="o">:</span> <span class="n">FINISHED</span>
<span class="n">Errata</span><span class="o">:</span>
<span class="n">State</span><span class="o">:</span> <span class="n">FINISHED</span>
<span class="n">Metadata</span><span class="o">:</span>
<span class="n">State</span><span class="o">:</span> <span class="n">FINISHED</span>
<span class="n">Purge</span> <span class="n">Duplicates</span><span class="o">:</span>
<span class="n">Error</span><span class="o">:</span> <span class="n">command</span> <span class="nf">SON</span><span class="p">([(</span><span class="err">'</span><span class="n">mapreduce</span><span class="err">'</span><span class="p">,</span> <span class="n">u</span><span class="err">'</span><span class="n">units_rpm</span><span class="err">'</span><span class="p">),</span> <span class="p">(</span><span class="err">'</span><span class="n">map</span><span class="err">'</span><span class="p">,</span> <span class="n">Code</span><span class="p">(</span><span class="s">"</span><span class="se">\n</span><span class="s"> </span><span class="err">
</span><span class="s"> function ()</span> <span class="p">{</span><span class="err">\</span><span class="n">n</span> <span class="n">var</span> <span class="n">key_fields</span> <span class="o">=</span> <span class="p">[</span><span class="n">this</span><span class="p">.</span><span class="n">name</span><span class="p">,</span> <span class="n">this</span><span class="p">.</span><span class="n">epoch</span><span class="p">,</span>
<span class="n">this</span><span class="p">.</span><span class="n">version</span><span class="p">,</span> <span class="n">this</span><span class="p">.</span><span class="n">release</span><span class="p">,</span> <span class="n">this</span><span class="p">.</span><span class="n">arch</span><span class="p">]</span><span class="err">\</span><span class="n">n</span>
<span class="n">emit</span><span class="p">(</span><span class="n">key_fields</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="sc">'-'</span><span class="p">),</span> <span class="p">{</span><span class="n">ids</span><span class="o">:</span> <span class="p">[</span><span class="n">this</span><span class="p">.</span><span class="n">_id</span><span class="p">]});</span><span class="err">\</span><span class="n">n</span> <span class="p">}</span><span class="err">\</span><span class="n">n</span> <span class="s">", {})),</span><span class="err">
</span><span class="s"> ('reduce', Code("</span><span class="err">\</span><span class="n">n</span> <span class="nf">function</span> <span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">values</span><span class="p">)</span> <span class="p">{</span><span class="err">\</span><span class="n">n</span> <span class="c1">// collect</span>
<span class="n">mapped</span> <span class="n">values</span> <span class="n">into</span> <span class="n">the</span> <span class="n">first</span> <span class="n">value</span> <span class="n">to</span> <span class="n">build</span> <span class="n">the</span> <span class="n">list</span> <span class="n">of</span> <span class="n">ids</span> <span class="k">for</span>
<span class="n">this</span> <span class="n">key</span><span class="o">/</span><span class="n">nevra</span><span class="err">\</span><span class="n">n</span> <span class="n">var</span> <span class="n">collector</span> <span class="o">=</span> <span class="n">values</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="err">\</span><span class="n">n</span> <span class="c1">// since</span>
<span class="n">collector</span> <span class="n">is</span> <span class="n">values</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="n">start</span> <span class="n">this</span> <span class="n">loop</span> <span class="n">at</span> <span class="n">index</span> <span class="mi">1</span><span class="err">\</span><span class="n">n</span> <span class="c1">// reduce</span>
<span class="n">isn</span><span class="err">'</span><span class="n">t</span> <span class="n">called</span> <span class="k">if</span> <span class="n">map</span> <span class="n">only</span> <span class="n">emits</span> <span class="n">one</span> <span class="n">result</span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span><span class="err">\</span><span class="n">n</span> <span class="c1">// so</span>
<span class="n">there</span> <span class="n">is</span> <span class="n">at</span> <span class="n">least</span> <span class="n">one</span> <span class="n">value</span> <span class="n">to</span> <span class="n">collect</span><span class="err">\</span><span class="n">n</span> <span class="k">for</span> <span class="p">(</span><span class="n">var</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">1</span><span class="p">;</span> <span class="n">i</span> <span class="o"><</span>
<span class="n">values</span><span class="p">.</span><span class="n">length</span><span class="p">;</span> <span class="n">i</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span><span class="err">\</span><span class="n">n</span> <span class="n">collector</span><span class="p">.</span><span class="n">ids</span> <span class="o">=</span>
<span class="n">collector</span><span class="p">.</span><span class="n">ids</span><span class="p">.</span><span class="n">concat</span><span class="p">(</span><span class="n">values</span><span class="p">[</span><span class="n">i</span><span class="p">].</span><span class="n">ids</span><span class="p">)</span><span class="err">\</span><span class="n">n</span> <span class="p">}</span><span class="err">\</span><span class="n">n</span> <span class="k">return</span>
<span class="n">collector</span><span class="err">\</span><span class="n">n</span> <span class="p">}</span><span class="err">\</span><span class="n">n</span> <span class="s">", {})), ('out', {'inline': 1}), ('query',</span><span class="err">
</span><span class="s"> {}), ('finalize', Code("</span><span class="err">\</span><span class="n">n</span> <span class="nf">function</span> <span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">reduced</span><span class="p">)</span> <span class="p">{</span><span class="err">\</span><span class="n">n</span> <span class="k">if</span>
<span class="p">(</span><span class="n">reduced</span><span class="p">.</span><span class="n">ids</span><span class="p">.</span><span class="n">length</span> <span class="o">></span> <span class="mi">1</span><span class="p">)</span> <span class="p">{</span><span class="err">\</span><span class="n">n</span> <span class="k">return</span> <span class="n">reduced</span><span class="p">;</span><span class="err">\</span><span class="n">n</span>
<span class="p">}</span><span class="err">\</span><span class="n">n</span> <span class="c1">// if there's only one value after reduction, this key</span>
<span class="n">is</span> <span class="n">useless</span><span class="err">\</span><span class="n">n</span> <span class="c1">// undefined is implicitly returned here, which</span>
<span class="n">saves</span> <span class="n">space</span><span class="err">\</span><span class="n">n</span> <span class="p">}</span><span class="err">\</span><span class="n">n</span> <span class="s">", {}))]) on namespace pulp_database.$cmd</span><span class="err">
</span><span class="s"> failed: exception: BSONObj size: 17449063 (0x67400A01) is invalid.</span><span class="err">
</span><span class="s"> Size must be between 0 and 16793600(16MB) First element: 0: { _id:</span><span class="err">
</span><span class="s"> "</span><span class="mi">2048</span><span class="o">-</span><span class="n">cli</span><span class="o">-</span><span class="mi">0</span><span class="o">-</span><span class="mi">0</span><span class="p">.</span><span class="mi">9</span><span class="o">-</span><span class="mi">4</span><span class="p">.</span><span class="n">git20141214</span><span class="p">.</span><span class="mi">723738</span><span class="n">c</span><span class="p">.</span><span class="n">el5</span><span class="o">-</span><span class="n">x86_64</span><span class="s">", value: null }</span><span class="err">
</span><span class="s"> State: FAILED</span><span class="err">
</span><span class="s">Traceback: Traceback (most recent call last): File</span><span class="err">
</span><span class="s"> "</span><span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">python2</span><span class="p">.</span><span class="mi">6</span><span class="o">/</span><span class="n">site</span><span class="o">-</span><span class="n">packages</span><span class="o">/</span><span class="n">celery</span><span class="o">/</span><span class="n">app</span><span class="o">/</span><span class="n">trace</span><span class="p">.</span><span class="n">py</span><span class="s">", line</span><span class="err">
</span><span class="s"> 240, in trace_task R = retval = fun(*args, **kwargs) </span><span class="err">
</span><span class="s"> File</span><span class="err">
</span><span class="s"> "</span><span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">python2</span><span class="p">.</span><span class="mi">6</span><span class="o">/</span><span class="n">site</span><span class="o">-</span><span class="n">packages</span><span class="o">/</span><span class="n">pulp</span><span class="o">/</span><span class="n">server</span><span class="o">/</span><span class="n">async</span><span class="o">/</span><span class="n">tasks</span><span class="p">.</span><span class="n">py</span><span class="s">",</span><span class="err">
</span><span class="s"> line 473, in __call__ return super(Task,</span><span class="err">
</span><span class="s"> self).__call__(*args, **kwargs) File</span><span class="err">
</span><span class="s"> "</span><span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">python2</span><span class="p">.</span><span class="mi">6</span><span class="o">/</span><span class="n">site</span><span class="o">-</span><span class="n">packages</span><span class="o">/</span><span class="n">pulp</span><span class="o">/</span><span class="n">server</span><span class="o">/</span><span class="n">async</span><span class="o">/</span><span class="n">tasks</span><span class="p">.</span><span class="n">py</span><span class="s">",</span><span class="err">
</span><span class="s"> line 103, in __call__ return super(PulpTask,</span><span class="err">
</span><span class="s"> self).__call__(*args, **kwargs) File</span><span class="err">
</span><span class="s"> "</span><span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">python2</span><span class="p">.</span><span class="mi">6</span><span class="o">/</span><span class="n">site</span><span class="o">-</span><span class="n">packages</span><span class="o">/</span><span class="n">celery</span><span class="o">/</span><span class="n">app</span><span class="o">/</span><span class="n">trace</span><span class="p">.</span><span class="n">py</span><span class="s">", line</span><span class="err">
</span><span class="s"> 437, in __protected_call__ return self.run(*args,</span><span class="err">
</span><span class="s"> **kwargs) File</span><span class="err">
</span><span class="s"> "</span><span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">python2</span><span class="p">.</span><span class="mi">6</span><span class="o">/</span><span class="n">site</span><span class="o">-</span><span class="n">packages</span><span class="o">/</span><span class="n">pulp</span><span class="o">/</span><span class="n">server</span><span class="o">/</span><span class="n">controllers</span><span class="o">/</span><span class="n">repo</span>
<span class="n">sitory</span><span class="p">.</span><span class="n">py</span><span class="s">", line 810, in sync raise</span><span class="err">
</span><span class="s"> pulp_exceptions.PulpExecutionException(_('Importer indicated a</span><span class="err">
</span><span class="s"> failed response')) PulpExecutionException: Importer indicated</span><span class="err">
</span><span class="s"> a failed response </span><span class="err">
</span></code></pre> Pulp - Refactor #1066 (CLOSED - WONTFIX): pulp.server.db.model should be moved to pulp.server.db....https://pulp.plan.io/issues/10662015-06-18T16:14:16Zamacdona@redhat.comaustin@redhat.com
<p>This should happen for two reasons:</p>
<p>1. All of our models live there, it should be a plural module name.<br>
2. pulp.server.db.model is currently a package acting as a module. By importing any of the modules in this package, we are forced to import everything that is in model/__init__.py. This creates a precarious situation for circular imports.</p>
<p>The second issue could also be fixed by moving the modules out of the model package and turning model into a real module. Either way, this should happen very soon because we will be updating all the plugins to use the models directly, and we want to make this change before it will break the plugins.</p> Pulp - Refactor #869 (CLOSED - WONTFIX): convert RepoUnitAssociationManager.unassociate_by_criter...https://pulp.plan.io/issues/8692015-04-10T16:02:55Zbcourtbcourt@redhat.com
<p>in RepoUnitAssociationManager.unassociate_by_criteria Add a switch that checks if all of the types on the repo have been converted to models then call branch to a new call the uses the the MongoEngine repo model to perform the delete and then calls the importer with the repo_content_unit objects.</p>
<p>Deliverables:</p>
<ul>
<li>a RepoUnitAssociationManager.unassociate_by_criteria that will use MongoEngine models if all the types in the given repo have been converted</li>
<li>This can use pulp_docker for testing</li>
</ul> Pulp - Refactor #868 (CLOSED - WONTFIX): Update RepoUnitAssociationManager.associate_from_repo to...https://pulp.plan.io/issues/8682015-04-10T15:59:48Zbcourtbcourt@redhat.com
<p>In RepoUnitAssociationManager.associate_from_repo Add a switch that checks if all of the types on the source and target repo have been converted to models then call branch to a new call the uses the MongoEngine repository models to perform the associate and then calls the importer with the repo_content_unit objects</p>
<p>Deliverables:</p>
<ul>
<li>A working version of associate_from_repo that will use either AssociatedUnits or MongoEngine models if all units within a repo support ME models</li>
<li>This can be tested with pulp_docker</li>
</ul>