Debian Support

Thanks quba42 for your updatre

My preferred approach is to integrate all possible fields (except ones not present for the concerned package in upstream repository) in order to be able to publish an identical Packages content for the concerned packages. Most of the fields are described here : https://www.debian.org/doc/debian-policy/ch-controlfields.html However, Multi-Arch field is not present in this specification. More details abour this field here : https://wiki.ubuntu.com/MultiarchSpec

I understand that Debian documentation is inconsitent about content of Packages files. I can try to open a case to them if you want

Hint i you'd like to check by yourself if requrired fields are missing :

1. Setup a repo, sync it with remote, create distribution and publication
2. Connect your apt client to this newly created publication and patch your system using it
3. You will see that same packages apply forever
4. Play around with files in /var/lib/apt/lists/_Packages by adding content you find on the upstream repo Packages file
5. You will see that when you run apt-get upgrade, the packages for which you modified attributes in /var/lib/apt/lists/.... based on upstream repository content will not be proposed anymore for upgrade as apt will be able to identify this package is already up-to-date

Let me know if you'd like to have more details :)

Thanks a lot for your support

I think it could be interesting to add to your test fixtures the following scenario :

• After publishing a new distribution, patch your debian or ubuntu client
• After patching, apt client must say there are no more applicable updates

Hi, I applied the fix on my Pulp server. Now the repo sync fails with the following error :

 "description": "value too long for type character varying(255)\n",
                "traceback": "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 883, in perform_job\n    rv = job.perform()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 657, in perform\n    self._result = self._execute()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 663, in _execute\n    return self.func(*self.args, **self.kwargs)\n  File \"/var/lib/pulp/pulp_deb/pulp_deb/app/tasks/synchronizing.py\", line 106, in synchronize\n    DebDeclarativeVersion(first_stage, repository, mirror=mirror).create()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py\", line 148, in create\n    loop.run_until_complete(pipeline)\n  File \"/usr/lib64/python3.6/asyncio/base_events.py\", line 484, in run_until_complete\n    return future.result()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/api.py\", line 225, in create_pipeline\n    await asyncio.gather(*futures)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/api.py\", line 43, in __call__\n    await self.run()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/content_stages.py\", line 102, in run\n    d_content.content.save()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/base.py\", line 110, in save\n    return super().save(*args, **kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py\", line 741, in save\n    force_update=force_update, update_fields=update_fields)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py\", line 779, in save_base\n    force_update, using, update_fields,\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py\", line 870, in _save_table\n    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py\", line 908, in _do_insert\n    using=using, raw=raw)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/manager.py\", line 82, in manager_method\n    return getattr(self.get_queryset(), name)(*args, **kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py\", line 1186, in _insert\n    return query.get_compiler(using=using).execute_sql(return_id)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py\", line 1375, in execute_sql\n    cursor.execute(sql, params)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 67, in execute\n    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 76, in _execute_with_wrappers\n    return executor(sql, params, many, context)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 84, in _execute\n    return self.cursor.execute(sql, params)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/utils.py\", line 89, in __exit__\n    raise dj_exc_value.with_traceback(traceback) from exc_value\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 84, in _execute\n    return self.cursor.execute(sql, params)\n"

Details about the remote used for syncing :

        {
            "architectures": "amd64",
            "ca_cert": null,
            "client_cert": null,
            "client_key": null,
            "components": "main contrib",
            "distributions": "buster bullseye buster-updates bullseye-updates",
            "download_concurrency": 20,
            "gpgkey": null,
            "name": "debian",
            "password": null,
            "policy": "on_demand",
            "proxy_url": "http://10.2.5.69:3128",
            "pulp_created": "2020-07-24T09:23:25.241744Z",
            "pulp_href": "/pulp/api/v3/remotes/deb/apt/a3036ec8-8c0c-408d-a493-007668ee4ad9/",
            "pulp_last_updated": "2020-07-24T09:25:29.131085Z",
            "sync_installer": false,
            "sync_sources": false,
            "sync_udebs": false,
            "tls_validation": true,
            "url": "http://ftp.ch.debian.org/debian/",
            "username": null
        }

@swisscom We had the same issue, and fixed it under a separate issue: https://pulp.plan.io/issues/7257

I expect both fixes will be merged into master today.

However, there is one more related/similar issue (https://pulp.plan.io/issues/7236). The remaining problem is that the pulp_deb plugin is currently declaring some fields as mandatory even though Debian says they are only "recommended". This causes failed syncs when there are packages that do not provide those fields.

Once all three of these issues are fixed, it is my hope that we will have fixed all (or at least the vast majority) of "There is some package control field we are not handling correctly" type issues...