Project

Profile

Help

Issue #6049

closed

On the pulp2 repositories api, distribution_hrefs and publication_hrefs are not serialized properly

Added by jsherril@redhat.com almost 5 years ago. Updated over 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 65
Quarter:

Description

This may be a bindings issue, but after migrating a pulp2 repository, the 'pulp3_distribution_hrefs' and the 'pulp3_publication_href' does not seem to be serialized properly. They both appear as a serialized json array string:

"[\"/pulp/api/v3/distributions/file/file/e1fbefc4-bd25-42cb-a160-9b3b7a72eff4/\"]"

when i'd expect it to be de-serialized:

["/pulp/api/v3/distributions/file/file/e1fbefc4-bd25-42cb-a160-9b3b7a72eff4/"]

also, is pulp3_publication_href supposed to be an array? its showing as an array, but the attribute name is singular:

"[\"/pulp/api/v3/publications/file/file/ecfc2157-2a3f-47d6-9835-6e89ba6c2f2b/\"]"

Actions #1

Updated by jsherril@redhat.com almost 5 years ago

I confirmed that the api is returning correctly:

{
  "count": 1,
  "next": null,
  "previous": null,
  "results": [
    {
      "pulp_href": "/pulp/api/v3/pulp2repositories/ab6dad7f-b745-4573-8c3e-f2a6701d8de8/",
      "pulp_created": "2020-01-23T20:28:14.485711Z",
      "pulp2_object_id": "5e29fb21b02e5354b9243ad3",
      "pulp2_repo_id": "5b08c8a6-d3bb-41af-9cd4-cadf8fec2a55",
      "is_migrated": true,
      "not_in_plan": false,
      "pulp3_repository_version": "/pulp/api/v3/repositories/file/file/b993a103-7835-46b6-b542-5521817c6613/versions/1/",
      "pulp3_remote_href": "/pulp/api/v3/remotes/file/file/ee50e8be-f57b-42cd-9ac5-2c1b2bdda431/",
      "pulp3_publication_href": [
        "/pulp/api/v3/publications/file/file/7c929ad0-ada9-48c7-afcb-6e2a11f879c3/"
      ],
      "pulp3_distribution_hrefs": [
        "/pulp/api/v3/distributions/file/file/ded48855-1cf6-4123-908f-6cd14406927a/"
      ]
    }
  ]
}

so its purely a bindings issue

Actions #2

Updated by dalley almost 5 years ago

The Python bindings seem to be OK for me, but I haven't tried the Ruby bindings. There's been some PRs about those recently: https://github.com/pulp/ansible-pulp/commit/c16114d92aea710df8e62d157f732f728c98a89b

Justin, could you try this code (and/or the Ruby equivalent) on your system and make sure you're getting what I get?

from pulpcore.client.pulp_2to3_migration import Pulp2repositoriesApi
from pulpcore.client.pulp_2to3_migration import ApiClient as MigrationApiClient
from pulpcore.client.pulpcore import Configuration, ApiClient as CoreApiClient
import socket                                                                                                                                                                                                                 
configuration = Configuration() 
configuration.username = 'admin' 
configuration.password = 'password' 
configuration.host = 'http://{}:24817'.format(socket.gethostname()) 
configuration.safe_chars_for_path_param = '/'

core_client = CoreApiClient(configuration)
migration_client = MigrationApiClient(configuration)

pulp2_repositories_api = Pulp2repositoriesApi(migration_client)
pulp2_repositories_api.list()

This gives me the results

In [14]: pulp2_repositories_api.list()                                                                                                                                                                                                        
Out[14]: 
{'count': 4,
 'next': None,
 'previous': None,
 'results': [{'is_migrated': True,
              'not_in_plan': False,
              'pulp2_object_id': '5e2f74146e50c91efcf36236',
              'pulp2_repo_id': 'file2',
              'pulp3_distribution_hrefs': "['/pulp/api/v3/distributions/file/file/cc079605-8c84-4db4-9820-b8b96425457d/']",
              'pulp3_publication_href': "['/pulp/api/v3/publications/file/file/82977a4b-e2cc-4a39-a99d-f8c6a3fcffdf/']",
              'pulp3_remote_href': '/pulp/api/v3/remotes/file/file/188e1661-7e0f-4302-9354-8c9e5fb46bea/',
              'pulp3_repository_version': '/pulp/api/v3/repositories/file/file/d4d2289a-b8d9-41f1-a608-c4709024275e/versions/1/',
              'pulp_created': datetime.datetime(2020, 1, 28, 22, 12, 21, 941340, tzinfo=tzlocal()),
              'pulp_href': '/pulp/api/v3/pulp2repositories/87079689-0d8d-436e-a084-24aa717fc21e/'},
             {'is_migrated': True,
              'not_in_plan': False,
              'pulp2_object_id': '5e2f74146e50c91efcf36239',
              'pulp2_repo_id': 'file-many',
              'pulp3_distribution_hrefs': "['/pulp/api/v3/distributions/file/file/54b3589a-29d3-43b6-8fcb-a81262d7da08/']",
              'pulp3_publication_href': "['/pulp/api/v3/publications/file/file/3c0dcfd0-8415-4ed2-b506-13691d04567e/']",
              'pulp3_remote_href': '/pulp/api/v3/remotes/file/file/db8b9c3b-7ce1-4510-a4b7-7770371a30fa/',
              'pulp3_repository_version': '/pulp/api/v3/repositories/file/file/4ba562c3-d318-4f09-85ca-45854b072c97/versions/1/',
              'pulp_created': datetime.datetime(2020, 1, 28, 22, 12, 21, 955990, tzinfo=tzlocal()),
              'pulp_href': '/pulp/api/v3/pulp2repositories/4e916797-c9f4-4c65-a713-5dfef01c6a3d/'},
             {'is_migrated': True,
              'not_in_plan': False,
              'pulp2_object_id': '5e2f74156e50c91efb7ffbd6',
              'pulp2_repo_id': 'file-large',
              'pulp3_distribution_hrefs': "['/pulp/api/v3/distributions/file/file/d9573e9b-c3ca-4970-b755-40f700044148/']",
              'pulp3_publication_href': "['/pulp/api/v3/publications/file/file/1e988c70-5ef0-4053-a78e-27a1c6ba902a/']",
              'pulp3_remote_href': '/pulp/api/v3/remotes/file/file/8176f10a-82a5-4cf8-9587-21c916b998f2/',
              'pulp3_repository_version': '/pulp/api/v3/repositories/file/file/43e0af6a-a3b0-4407-8954-cb4cae2083fd/versions/1/',
              'pulp_created': datetime.datetime(2020, 1, 28, 22, 12, 22, 3877, tzinfo=tzlocal()),
              'pulp_href': '/pulp/api/v3/pulp2repositories/f1f2c124-cde6-4134-96d5-2576dede6f58/'},
             {'is_migrated': True,
              'not_in_plan': False,
              'pulp2_object_id': '5e2f74146e50c91efd260fc5',
              'pulp2_repo_id': 'file',
              'pulp3_distribution_hrefs': "['/pulp/api/v3/distributions/file/file/c4203daa-5d1a-4f40-817a-2239390b2714/']",
              'pulp3_publication_href': "['/pulp/api/v3/publications/file/file/41f13c11-5056-4781-9fc7-11cd880b9702/']",
              'pulp3_remote_href': '/pulp/api/v3/remotes/file/file/53c75c41-4449-4e3e-8c19-a8fc2dcf4d99/',
              'pulp3_repository_version': '/pulp/api/v3/repositories/file/file/63643150-8731-4953-9e8d-bc016aba89dd/versions/1/',
              'pulp_created': datetime.datetime(2020, 1, 28, 22, 11, 18, 935566, tzinfo=tzlocal()),
              'pulp_href': '/pulp/api/v3/pulp2repositories/ed6f05bc-fc81-4edd-96a6-b10196fa522a/'}]}

In [15]:                                                                                                  
Actions #3

Updated by dalley almost 5 years ago

It's kinda weird that pulp3_publication_href is singular when it's a list. Maybe we should make that plural like pulp3_distribution_hrefs

Actions #4

Updated by dalley almost 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley
Actions #5

Updated by ipanova@redhat.com almost 5 years ago

i am not sure, maybe it makes sense to call publication_set.first() instead of publication_set.all(), [0] we are migrating remotes, publications and distributions on every run, plus we re-use publication for the repo_version if created so we should have 1 publication per repo version https://github.com/pulp/pulp-2to3-migration/blob/master/pulp_2to3_migration/app/plugin/iso/repository.py#L58

obviously this depends on the implementation of each plugin. Is there in plan to create different publications for same repo_version in rpm plugin?

[0] https://github.com/pulp/pulp-2to3-migration/blob/master/pulp_2to3_migration/app/serializers.py#L184

Actions #6

Updated by dalley almost 5 years ago

  • Status changed from ASSIGNED to POST

Added by dalley almost 5 years ago

Revision 140395fc | View on GitHub

Fix the bindings on the pulp2repositories API

closes #6049 https://pulp.plan.io/issues/6049#note-5

Added by dalley almost 5 years ago

Revision 140395fc | View on GitHub

Fix the bindings on the pulp2repositories API

closes #6049 https://pulp.plan.io/issues/6049#note-5

Added by dalley almost 5 years ago

Revision 140395fc | View on GitHub

Fix the bindings on the pulp2repositories API

closes #6049 https://pulp.plan.io/issues/6049#note-5

Actions #7

Updated by dalley almost 5 years ago

  • Status changed from POST to MODIFIED
Actions #8

Updated by ipanova@redhat.com almost 5 years ago

  • Description updated (diff)
  • Triaged changed from No to Yes
Actions #9

Updated by ipanova@redhat.com almost 5 years ago

  • Sprint set to Sprint 65
Actions #12

Updated by ttereshc over 4 years ago

  • Sprint/Milestone set to 0.1.0
Actions #13

Updated by ttereshc over 4 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions #14

Updated by ggainey over 4 years ago

  • Tags Katello added
  • Tags deleted (Katello-P2)

Also available in: Atom PDF