Project

Profile

Help

Issue #8840

duplicate key value violates unique constraint "core_artifact_sha256_key"

Added by knzivid about 2 months ago. Updated about 2 months ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version - Debian:
Platform Release:
Target Release - Debian:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

The sync REST call returns successfully

completed: 
[3 Downloading Artifacts] 
[1 Update ReleaseFile units] 
[1 Update PackageIndex units] 
[0 Associating Content] 
[0 Un-Associating Content]

In the database server log, I see

ERROR:  duplicate key value violates unique constraint "core_artifact_sha256_key"
DETAIL:  Key (sha256)=(addfd22c48ca7f6e67197a63d6ba78dc706678e4b198c901de1bfd99060629cc) already exists.
STATEMENT:  INSERT INTO "core_artifact" ("pulp_id", "pulp_created", "pulp_last_updated", "file", "size", "md5", "sha1", "sha224", "sha256", "sha384", "sha512") VALUES ('76c5f18d-0597-4781-bde3-82cf9455b546'::uuid, '2021-05-28T15:27:04.541372+00:00'::timestamptz, '2021-05-28T15:27:04.541385+00:00'::timestamptz, 'artifact/ad/dfd22c48ca7f6e67197a63d6ba78dc706678e4b198c901de1bfd99060629cc', 4645, NULL, NULL, '3c9ad31f31074b93fb2ca51d8618f851ff9edd67ae509da53eb9a6e2', 'addfd22c48ca7f6e67197a63d6ba78dc706678e4b198c901de1bfd99060629cc', '682032a010aacda85558a346cff70840ff3f0882942b6f22ae5651faca9171f4751574341c285fbc9c4201e693f92c0c', 'b504ff76b0ab65b4e894949a9ce65ced4dc820e4cca961c5926744c453fca0d24ec82bc1e2134c765d2916a9f0d173380d5d7d67cb64da4c5bc22f55b2940dd9'), ('65508db8-5726-474b-8ea4-9c96d1d477f5'::uuid, '2021-05-28T15:27:04.541972+00:00'::timestamptz, '2021-05-28T15:27:04.541981+00:00'::timestamptz, 'artifact/88/89e025a6e18c2a0e31edd2f430ee2a7be8a88c48f156706ec672cc87cb832d', 5527, NULL, NULL, 'd58236d07ed20cf31c96eba8148b733e891967f71b6c418759c3b7b2', '8889e025a6e18c2a0e31edd2f430ee2a7be8a88c48f156706ec672cc87cb832d', 'f026b266558924947a1bc796296f9ee6b15033df275d05a576a09505bb66a3870bc61ce166c405887864ae940f08a9a4', 'b8eb88c5b37e13e53fa549831758dbb7564197323f7fd3d2a19b3ba8a7a9f10718939cb8d94db47987b7e461b4f29012ee2479a7c2bc3d3a73bd1fb8746e338c'), ('c0394932-c14a-4c4c-9d0a-774f28f60be3'::uuid, '2021-05-28T15:27:04.542179+00:00'::timestamptz, '2021-05-28T15:27:04.542187+00:00'::timestamptz, 'artifact/b7/4740e506eec14a271e74ee7096b0ce6194a4810f568727da9557127f8db4b9', 833, NULL, NULL, '3d42a1ff9d8d9caaa5a9526be54d786af61a3d16d00f45f0a740639a', 'b74740e506eec14a271e74ee7096b0ce6194a4810f568727da9557127f8db4b9', '2d3b9145ff8fef94ea470404c3b2327a76347b40e1b23ee301d16146bbf139d6bef83e8a04b49af5d65d4f6a6c73be72', '9008fd1c21bc112726f9687536e2dd8c2a5c5d3133338eff8605b4268bf0dc5fb7a974ea4509c5040e9cf8e2676ff7da29a5da100187f9d0fd8f565626594036') RETURNING "core_artifact"."pulp_id"
ERROR:  duplicate key value violates unique constraint "core_artifact_sha256_key"
DETAIL:  Key (sha256)=(addfd22c48ca7f6e67197a63d6ba78dc706678e4b198c901de1bfd99060629cc) already exists.

This does not happen on the first time sync but only from the second time onwards.

Remote:

{
    "pulp_href": "/pulp/api/v3/remotes/deb/apt/f1d18e33-e8d7-4efc-9168-99c17c91d4aa/",
    "pulp_created": "2021-05-28T15:25:58.211906Z",
    "name": "bionic.apt.llvm.org",
    "url": "http://apt.llvm.org/bionic",
    "pulp_last_updated": "2021-05-28T15:25:58.211916Z",
    "download_concurrency": 2,
    "policy": "immediate",
    "distributions": "llvm-toolchain-bionic-11",
    "components": null,
    "architectures": "amd64",
    "sync_sources": false,
    "sync_udebs": false,
    "sync_installer": false,
    "gpgkey": null,
    "ignore_missing_package_indices": false
}

Pulp versions:

    "versions": [
        {
            "component": "core",
            "version": "3.13.0"
        },
        {
            "component": "file",
            "version": "1.7.0"
        },
        {
            "component": "deb",
            "version": "2.13.0"
        }
    ]

I don't notice anything wrong when I am fetching the packages but I am not sure if I should be worried about this error.

History

#1 Updated by knzivid about 2 months ago

Interestingly

When I run pulp/pulp official image with these package versions, I do not see this error from postgres.

pulp-ansible                    0.7.3
pulp-certguard                  1.3.0
pulp-container                  2.6.0
pulp-deb                        2.12.0
pulp-file                       1.7.0
pulp-maven                      0.2.0
pulp-python                     3.2.0
pulp-rpm                        3.12.0
pulpcore                        3.12.2

When I run the same in https://pulpproject.org/related-tooling/#docker-pulp-pulp-docker-images with these versions, I see this error in the log.

pulp-deb                        2.12.0
pulp-file                       1.7.0
pulpcore                        3.12.2

Then I looked at the logs in /var/lib/pgsql/data/log in the pulp/pulp docker image and I found similar logs!

2021-05-28 16:16:52.436 UTC [696] ERROR:  duplicate key value violates unique constraint "core_artifact_sha256_key"
2021-05-28 16:16:52.436 UTC [696] DETAIL:  Key (sha256)=(addfd22c48ca7f6e67197a63d6ba78dc706678e4b198c901de1bfd99060629cc) already exists.
2021-05-28 16:16:52.436 UTC [696] STATEMENT:  INSERT INTO "core_artifact" ("pulp_id", "pulp_created", "pulp_last_updated", "file", "size", "md5", "sha1", "sha224", "sha256", "sha384", "sha512") VALUES ('d437ca8a-0d13-4c7a-ba23-911316364809'::uuid, '2021-05-28T16:16:52.435103+00:00'::timestamptz, '2021-05-28T16:16:52.435116+00:00'::timestamptz, 'artifact/ad/dfd22c48ca7f6e67197a63d6ba78dc706678e4b198c901de1bfd99060629cc', 4645, NULL, NULL, '3c9ad31f31074b93fb2ca51d8618f851ff9edd67ae509da53eb9a6e2', 'addfd22c48ca7f6e67197a63d6ba78dc706678e4b198c901de1bfd99060629cc', '682032a010aacda85558a346cff70840ff3f0882942b6f22ae5651faca9171f4751574341c285fbc9c4201e693f92c0c', 'b504ff76b0ab65b4e894949a9ce65ced4dc820e4cca961c5926744c453fca0d24ec82bc1e2134c765d2916a9f0d173380d5d7d67cb64da4c5bc22f55b2940dd9'), ('18b3ee57-eb6b-404c-8e81-a7763c165c29'::uuid, '2021-05-28T16:16:52.435669+00:00'::timestamptz, '2021-05-28T16:16:52.435678+00:00'::timestamptz, 'artifact/88/89e025a6e18c2a0e31edd2f430ee2a7be8a88c48f156706ec672cc87cb832d', 5527, NULL, NULL, 'd58236d07ed20cf31c96eba8148b733e891967f71b6c418759c3b7b2', '8889e025a6e18c2a0e31edd2f430ee2a7be8a88c48f156706ec672cc87cb832d', 'f026b266558924947a1bc796296f9ee6b15033df275d05a576a09505bb66a3870bc61ce166c405887864ae940f08a9a4', 'b8eb88c5b37e13e53fa549831758dbb7564197323f7fd3d2a19b3ba8a7a9f10718939cb8d94db47987b7e461b4f29012ee2479a7c2bc3d3a73bd1fb8746e338c'), ('6fb3b30f-7d58-43db-8a48-2482e160c7b3'::uuid, '2021-05-28T16:16:52.435871+00:00'::timestamptz, '2021-05-28T16:16:52.435880+00:00'::timestamptz, 'artifact/b7/4740e506eec14a271e74ee7096b0ce6194a4810f568727da9557127f8db4b9', 833, NULL, NULL, '3d42a1ff9d8d9caaa5a9526be54d786af61a3d16d00f45f0a740639a', 'b74740e506eec14a271e74ee7096b0ce6194a4810f568727da9557127f8db4b9', '2d3b9145ff8fef94ea470404c3b2327a76347b40e1b23ee301d16146bbf139d6bef83e8a04b49af5d65d4f6a6c73be72', '9008fd1c21bc112726f9687536e2dd8c2a5c5d3133338eff8605b4268bf0dc5fb7a974ea4509c5040e9cf8e2676ff7da29a5da100187f9d0fd8f565626594036') RETURNING "core_artifact"."pulp_id"

Is this harmless?

#2 Updated by quba42 about 2 months ago

Not sure if this is harmless. I will try to reproduce this when I get to it.

If it is possible to install packages from the synced repository in question, it may well be harmless...

Please register to edit this issue

Also available in: Atom PDF