Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #1894

closed

unassociate rpms API request using unit_ids removes all rpm from repo

Added by martin@hatchlane.com over 7 years ago. Updated over 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
jortel@redhat.com
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
2.8.2
Platform Release:
2.9.2
OS:
RHEL 7
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 5
Quarter:

Description

Hi Folks,

In an attempt to work around #1863 I decided to just copy all, search for units to remove, and unassociate them by their _id. In doing this I seem to have found another bug.

Initially I thought it was related to https://pulp.plan.io/issues/379 but I am using type_ids

Using a local clone of: http://mirror.centos.org/centos/7/updates/x86_64/

[centos@pulp ~]$ pulp-admin rpm repo content rpm --repo-id centos7-development-upstream-updates --fields filename | grep Filename | wc -l
3101

I do a search for:

  "criteria": {
    "fields": {
      "unit": [
        "filename",
      ]
    },
    "filters": {
      "unit": {
        '$and': [
          { "name": { "$regex": '^(ipa.*)(?<!fonts)$' } },
          { "version": { "$not": '4.1.0' } }
        ]
      }
    },
    "type_ids": [
      "rpm"
    ],
  }

which returns the expected units i wish to remove

If i then POST the following to: https://localhost/pulp/api/v2/repositories/centos7-development-upstream-updates/actions/unassociate/

{
  "criteria": {
    "type_ids": [
      "rpm"
    ], 
    "filters": {
      "unit": {
        "_id": {
          "$in": [
            "00a9afaf-0980-43b4-a298-165b709e7148", 
            "03910212-e10e-49f4-afe5-6de38eef21e3", 
            "03e87aff-b14c-4cb6-9947-64c3bc3402f1", 
            "10103d7e-1d6b-408a-8183-2e259239e797", 
            "11240ebe-9b80-4e75-8a2f-617994a1d8c4", 
            "1343d135-f31a-4e12-a5c9-0d3bdf74acaf", 
            "31972fa4-fbd0-4320-abe5-fe8f0ff39bb3", 
            "3d58d7f9-06a9-4c28-aab2-a5e072971ef4", 
            "3dc343e9-5637-4c6b-88d6-d6c9822b53f5", 
            "3dd763dc-a3aa-453b-a561-1915fff8479f", 
            "63f84135-8a26-444f-b275-a871f7c27aa6", 
            "7118c6f1-ccff-4180-acd3-bf87fa8d29e3", 
            "749dcac7-a17a-4869-8677-679be94c6b18", 
            "8a7ef2c9-c701-4234-b010-e406effc56a3", 
            "9238f8ba-e081-4ac4-8545-fa86e67cc765", 
            "925289d7-5a69-473e-839e-316c458dfe67", 
            "9835962f-9fdc-4d03-9899-277a6cac8d7c", 
            "a6d349b2-b0ba-4504-8db1-6d87ed6dd46f", 
            "aa2b6dda-8e37-4f65-9adf-058a06c7c88f", 
            "c4a1d7f1-f235-4977-8c0e-7e2988ac3430", 
            "c849e49a-ccc8-4bf5-b9ec-5c539b5f346c", 
            "d6b26c1d-cfa0-49bf-9fff-9f916d5dcb99", 
            "d77848fc-9406-4ebd-8b81-bd0358482548", 
            "dc4f5a3d-62c8-42e1-95d2-634f0a6119e4"
          ]
        }
      }
    }
  }
}

The task is accepted:

{"spawned_tasks": [{"_href": "/pulp/api/v2/tasks/12e0c9ec-ec33-42ea-8409-6003e5fb291c/", "task_id": "12e0c9ec-ec33-42ea-8409-6003e5fb291c"}], "result": null, "error": null}

And the response looks to have the correct rpms removed:

{
  "exception": null, 
  "task_type": "pulp.server.managers.repo.unit_association.unassociate_by_criteria", 
  "_href": "/pulp/api/v2/tasks/12e0c9ec-ec33-42ea-8409-6003e5fb291c/", 
  "task_id": "12e0c9ec-ec33-42ea-8409-6003e5fb291c", 
  "tags": [
    "pulp:repository:centos7-development-upstream-updates", 
    "pulp:action:unassociate"
  ], 
  "finish_time": "2016-05-04T14:40:09Z", 
  "_ns": "task_status", 
  "start_time": "2016-05-04T14:39:26Z", 
  "traceback": null, 
  "spawned_tasks": [], 
  "progress_report": {}, 
  "queue": "reserved_resource_worker-1@pulp.central.byhiras.com.dq", 
  "state": "finished", 
  "worker_name": "reserved_resource_worker-1@pulp.central.byhiras.com", 
  "result": {
    "units_successful": [
      {
        "unit_key": {
          "name": "ipa-server-dns", 
          "checksum": "61a9457d063f3134a1b12a82d908ea92772a6f8c29a2bdf79c14b104faeb1569", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.el7.centos.3", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-admintools", 
          "checksum": "be195e50fcbd3c190e90ecbe5690a664e01bf953ae29be709a4a6aba662736bb", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.0.1.el7.centos.6", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-server-trust-ad", 
          "checksum": "b66ed2777a51f83dabff3dcc54fb1404c4f449b58f3bed8d8867b6c3cfd123e5", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.el7_2.6", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-python", 
          "checksum": "c64d59f138beb4fb8a8a3a200c7696213eebf5c07ac50bb43a7faaadd1b5b9c0", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.0.1.el7.centos.6", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-server-trust-ad", 
          "checksum": "07db046a97a586d9a82fa375ee6ea2e303fdb3bfbe0286f3b22670b56b9c01e1", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.el7.centos.3", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-server", 
          "checksum": "f2cc2a982821847d506eb2fad3626b3681ec93062ae495ee047cd952a2bf183f", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.el7.centos.3", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-server-dns", 
          "checksum": "a712e3b6e56bef910b9003d2b20220f0e29f340821285481b4f27e0ae4993eeb", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.el7_2.6", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-server-dns", 
          "checksum": "e8ddad4b3b9fb4ebff1cd2ac0f45305524f6b3d14428871752d29a1fee41bc63", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.0.1.el7.centos.6.1", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-server-trust-ad", 
          "checksum": "5d86bbc72ab86713c6df94031ed2391452222f1f7b36b65adb234b6d7712cec6", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.0.1.el7.centos.6.1", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-admintools", 
          "checksum": "ad7c0dec8d5ee592d1b324b0d8403e806bd7d6b2838c5c653ab41999451a11d0", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.el7.centos.3", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-server", 
          "checksum": "3c5a1f51b4cb3f7e18ccc51670fa46929c07c3e80275d1960fcd6a6d7eb38891", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.el7_2.6", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-python", 
          "checksum": "94828b591a73270e106610ac9c7faef93325748a92b27b416fda989014c35c3f", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.0.1.el7.centos.6.1", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-client", 
          "checksum": "46df2769ffc4e7439ddd6a8a140b0afcb4f45204aa717dbe4fc027dd4c5dda71", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.0.1.el7.centos.6", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-server-dns", 
          "checksum": "9accd7c6001f0f0c02b37eae8538e3045eaaf0f04008d5202fd4007368ba0a64", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.0.1.el7.centos.6", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-admintools", 
          "checksum": "8e45a123f026388b7d4892612740c1b483c4928c401c4d321e8ceb99c04b5f8d", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.el7_2.6", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-python", 
          "checksum": "9ef7db6a1dd41f84dc9f630cf2f824e45e60f18e525bc7df68320a64fb0e1429", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.el7_2.6", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-client", 
          "checksum": "f4455af98527f9a2df19cc9f746bd25e8995e9f30789a52b78e46125d0aadc43", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.0.1.el7.centos.6.1", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-server", 
          "checksum": "c20a31a1e4ac386e50c55839ef25b4ad1b3c261d60981da35d7c0d6a7d773ee9", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.0.1.el7.centos.6", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-client", 
          "checksum": "c74edfa45b31a666a4acbd399aa965c104e02b2bc1ff9187febdfa8a1d632119", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.el7_2.6", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-python", 
          "checksum": "5680100a1d32978f3df4d37208b84c10514b1743d90f5c1c869c32994e1a968a", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.el7.centos.3", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-server-trust-ad", 
          "checksum": "5a5a91ff922ba863eef85723f589d3c940d2e8e529683ca811016688f7bcc95a", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.0.1.el7.centos.6", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-client", 
          "checksum": "85ddb3435f112a139eb7d008f73132f2a823f48ccac0831875deea097b8a77af", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.el7.centos.3", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-server", 
          "checksum": "c5ac9c031a02a29b36a0edd3df3e3fc681b348c3f037137ecd170ffba78d185e", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.0.1.el7.centos.6.1", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }, 
      {
        "unit_key": {
          "name": "ipa-admintools", 
          "checksum": "d03419892bfbe6ede192042c5772327b23a87d7b13262422a31c0910766f8c36", 
          "epoch": "0", 
          "version": "4.2.0", 
          "release": "15.0.1.el7.centos.6.1", 
          "arch": "x86_64", 
          "checksumtype": "sha256"
        }, 
        "type_id": "rpm"
      }
    ]
  }, 
  "error": null, 
  "_id": {
    "$oid": "572a099d3b678d167b397faa"
  }, 
  "id": "572a099d3b678d167b397faa"
}

However, the repo is now empty:

[centos@pulp ~]$ pulp-admin rpm repo content rpm --repo-id centos7-development-upstream-updates --fields filename | grep Filename | wc -l
0

If i use a list of filenames, instead if ids it works correctly. and removes the 24 listed RPMS only

Actions
Copy link
 #1

Updated by martin@hatchlane.com over 7 years ago

Using unit_id with an associate filter rather than _id on a unit filter does work correctly.

Actions
Copy link
 #2

Updated by dkliban@redhat.com over 7 years ago

  • Priority changed from Normal to High
  • Severity changed from 2. Medium to 3. High
  • Triaged changed from No to Yes
Actions
Copy link
 #3

Updated by dkliban@redhat.com over 7 years ago

  • Platform Release set to 2.8.4
Actions
Copy link
 #4

Updated by semyers over 7 years ago

  • Platform Release changed from 2.8.4 to 2.8.5
Actions
Copy link
 #5

Updated by Anonymous over 7 years ago

  • Sprint/Milestone set to 22
Actions
Copy link
 #6

Updated by jortel@redhat.com over 7 years ago

wrote:

Using unit_id with an associate filter rather than _id on a unit filter does work correctly.

Does using unit_id instead of _id in the unassociate work correctly as well?

Actions
Copy link
 #7

Updated by martin@hatchlane.com over 7 years ago

Yes, That is how I'm currently working around it in my use case

  • associate
  • search for items that would have been excluded by the filter
  • unassociate those items
  • publish

As for the unassociate:

unassociate = {
  "criteria": {
    "filters": {
      "association": {
        "unit_id": {'$in': units_to_remove }
      }
    },
    "type_ids": [
      "rpm"
    ],
  }
} 
r = self.request('repositories/%s/actions/unassociate/' % dest_repo, data=unassociate)

Thanks,
Martin

Actions
Copy link
 #8

Updated by jortel@redhat.com over 7 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to jortel@redhat.com
Actions
Copy link
 #9

Updated by semyers over 7 years ago

  • Platform Release changed from 2.8.5 to 2.8.6
Actions
Copy link
 #10

Updated by semyers over 7 years ago

  • Platform Release deleted (2.8.6)
Actions
Copy link
 #11

Updated by mhrivnak about 7 years ago

  • Sprint/Milestone changed from 22 to 23
Actions
Copy link
 #12

Updated by jortel@redhat.com about 7 years ago

Reproducing with:

pulp-admin rpm repo create --repo-id=zoo --feed=https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/

curl -X POST -k -H "Content-Type: application/json"  https://admin:admin@localhost/pulp/api/v2/repositories/zoo/actions/unassociate/  -d "{\"criteria\":{\"type_ids\":[\"rpm\"],\"filters\":{\"unit\":{\"_id\":{\"\$in\":[\"123\"]}}}}}"
Actions
Copy link
 #13

Updated by jortel@redhat.com about 7 years ago

  • Status changed from ASSIGNED to POST

Added by jortel@redhat.com about 7 years ago

Revision ceb05fa1

Fix unit query by criteria when unit filter contains ids. closes #1894

Added by jortel@redhat.com about 7 years ago

Revision ceb05fa1

Fix unit query by criteria when unit filter contains ids. closes #1894

Actions
Copy link
 #14

Updated by jortel@redhat.com about 7 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions
Copy link
 #15

Updated by ttereshc about 7 years ago

  • Platform Release set to 2.9.2
Actions
Copy link
 #16

Updated by semyers about 7 years ago

  • Status changed from MODIFIED to 5
Actions
Copy link
 #17

Updated by pthomas@redhat.com about 7 years ago

  • Status changed from 5 to 6

verified
[root@tigger ~]# rpm -qa pulp-server
pulp-server-2.9.2-0.2.beta.el7.noarch
[root@tigger ~]#

[root@tigger ~]# pulp-admin rpm repo content rpm --repo-id zoo  --str-eq="_id=fe60419a-4d55-4ba4-8d00-aca230878f32"
Arch:         noarch
Buildhost:    smqe-ws15
Checksum:     9b16374232c3a256968abe1a7e2056dfd84210c675adcbdfbcd636adbc4d6a11
Checksumtype: sha256
Description:  A dummy package of tiger
Epoch:        0
Filename:     tiger-1.0-4.noarch.rpm
License:      GPLv2
Name:         tiger
Provides:     tiger = 1.0-4-0
Release:      4
Requires:     
Version:      1.0

[root@tigger ~]# 
[root@tigger ~]# 
[root@tigger ~]# 
[root@tigger ~]# curl -X POST -k -H "Content-Type: application/json"  https://admin:admin@localhost/pulp/api/v2/repositories/zoo/actions/unassociate/  -d "{\"criteria\":{\"type_ids\":[\"rpm\"],\"filters\":{\"unit\":{\"_id\":{\"\$in\":[\"fe60419a-4d55-4ba4-8d00-aca230878f32\"]}}}}}" 
{"spawned_tasks": [{"_href": "/pulp/api/v2/tasks/fd7de6cc-0bee-49c1-a673-7ebc4c37186d/", "task_id": "fd7de6cc-0bee-49c1-a673-7ebc4c37186d"}], "result": null, "error": null}[root@tigger ~]# 
[root@tigger ~]# 
[root@tigger ~]# pulp-admin rpm repo content rpm --repo-id zoo  --str-eq="_id=fe60419a-4d55-4ba4-8d00-aca230878f32"
[root@tigger ~]#
Actions
Copy link
 #18

Updated by semyers about 7 years ago

  • Status changed from 6 to CLOSED - CURRENTRELEASE
Actions
Copy link
 #20

Updated by bmbouter over 5 years ago

  • Sprint set to Sprint 5
Actions
Copy link
 #21

Updated by bmbouter over 5 years ago

  • Sprint/Milestone deleted (23)
Actions
Copy link
 #22

Updated by bmbouter over 4 years ago

  • Tags Pulp 2 added
Actions
Send by e-mail Copy link

Also available in: Atom PDF