Project

Profile

Help

Issue #5976

Migrate only content from the specified repositories

Added by ipanova@redhat.com 9 months ago. Updated 6 months ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
(Total: 0:00 h)
Severity:
2. Medium
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Problem:
All immediate content get migrated, regardless the specified repositories

$ pulp-admin repo list
+----------------------------------------------------------------------+
                              Repositories
+----------------------------------------------------------------------+

Id:                  file
Display Name:        None
Description:         None
Content Unit Counts: 
  Iso: 3

Id:                  file2
Display Name:        None
Description:         None
Content Unit Counts: 
  Iso: 3

Id:                  file-many
Display Name:        None
Description:         None
Content Unit Counts: 
  Iso: 250

Id:                  file-large
Display Name:        None
Description:         None
Content Unit Counts: 
  Iso: 10

$ http POST :24817/pulp/api/v3/migration-plans/ plan='{"plugins": [{ "type": "iso", "repositories" :  [{"name": "file","pulp2_importer_repository_id": "file", "repository_versions": [{"pulp2_repository_id": "file"}]}]}]}'
HTTP/1.1 201 Created
Allow: GET, POST, HEAD, OPTIONS
Connection: close
Content-Length: 289
Content-Type: application/json
Date: Mon, 13 Jan 2020 18:49:40 GMT
Location: /pulp/api/v3/migration-plans/ae1e5041-3cb5-4ca7-aa52-33aea081f0a8/
Server: gunicorn/20.0.4
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN

{
    "plan": {
        "plugins": [
            {
                "repositories": [
                    {
                        "name": "file",
                        "pulp2_importer_repository_id": "file",
                        "repository_versions": [
                            {
                                "pulp2_repository_id": "file"
                            }
                        ]
                    }
                ],
                "type": "iso"
            }
        ]
    },
    "pulp_created": "2020-01-13T18:49:40.893261Z",
    "pulp_href": "/pulp/api/v3/migration-plans/ae1e5041-3cb5-4ca7-aa52-33aea081f0a8/"
}

]$ http GET :24817/pulp/api/v3/pulp2repositories/
HTTP/1.1 200 OK
Allow: GET, HEAD, OPTIONS
Connection: close
Content-Length: 552
Content-Type: application/json
Date: Mon, 13 Jan 2020 18:52:55 GMT
Server: gunicorn/20.0.4
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN

{
    "count": 1,
    "next": null,
    "previous": null,
    "results": [
        {
            "is_migrated": true,
            "not_in_pulp2": false,
            "pulp2_object_id": "5e1c932dc998ac367a9b79d0",
            "pulp2_repo_id": "file",
            "pulp3_distribution_hrefs": [],
            "pulp3_publication_href": [],
            "pulp3_remote_href": "/pulp/api/v3/remotes/file/file/3730ab1a-e0e3-458e-b479-35456b607ad7/",
            "pulp3_repository_version": "/pulp/api/v3/repositories/file/file/4788f61a-815a-42fb-9331-ec6787d08281/versions/1/",
            "pulp_created": "2020-01-13T18:50:03.589032Z",
            "pulp_href": "/pulp/api/v3/pulp2repositories/070e0fe0-fc41-477a-900e-be641e348634/"
        }
    ]
}

$ http GET :24817/pulp/api/v3/repositories/file/file/4788f61a-815a-42fb-9331-ec6787d08281/versions/1/
HTTP/1.1 200 OK
Allow: GET, DELETE, HEAD, OPTIONS
Connection: close
Content-Length: 577
Content-Type: application/json
Date: Mon, 13 Jan 2020 19:00:14 GMT
Server: gunicorn/20.0.4
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN

{
    "base_version": null,
    "content_summary": {
        "added": {
            "file.file": {
                "count": 3,
                "href": "/pulp/api/v3/content/file/files/?repository_version_added=/pulp/api/v3/repositories/file/file/4788f61a-815a-42fb-9331-ec6787d08281/versions/1/"
            }
        },
        "present": {
            "file.file": {
                "count": 3,
                "href": "/pulp/api/v3/content/file/files/?repository_version=/pulp/api/v3/repositories/file/file/4788f61a-815a-42fb-9331-ec6787d08281/versions/1/"
            }
        },
        "removed": {}
    },
    "number": 1,
    "pulp_created": "2020-01-13T18:50:08.125597Z",
    "pulp_href": "/pulp/api/v3/repositories/file/file/4788f61a-815a-42fb-9331-ec6787d08281/versions/1/"
}

(pulp) [vagrant@pulp2-nightly-pulp3-source-centos7 pulp-2to3-migration]$ django-admin shell_plus
# Shell Plus Model Imports
from django.contrib.admin.models import LogEntry
from django.contrib.auth.models import Group, Permission, User
from django.contrib.contenttypes.models import ContentType
from django.contrib.sessions.models import Session
from pulp_2to3_migration.app.models.base import MigrationPlan
from pulp_2to3_migration.app.models.content import Pulp2Content, Pulp2LazyCatalog
from pulp_2to3_migration.app.models.repository import Pulp2Distributor, Pulp2Importer, Pulp2RepoContent, Pulp2Repository
from pulp_2to3_migration.app.plugin.docker.pulp_2to3_models import Pulp2Blob, Pulp2Manifest, Pulp2ManifestList, Pulp2Tag
from pulp_2to3_migration.app.plugin.iso.pulp_2to3_models import Pulp2ISO
from pulp_container.app.models import Blob, BlobManifest, ContainerDistribution, ContainerRemote, ContainerRepository, Manifest, ManifestListManifest, Tag
from pulp_file.app.models import FileContent, FileDistribution, FileFileSystemExporter, FilePublication, FileRemote, FileRepository
from pulp_rpm.app.models.advisory import UpdateCollection, UpdateCollectionPackage, UpdateRecord, UpdateReference
from pulp_rpm.app.models.comps import PackageCategory, PackageEnvironment, PackageGroup, PackageLangpacks
from pulp_rpm.app.models.custom_metadata import RepoMetadataFile
from pulp_rpm.app.models.distribution import Addon, Checksum, DistributionTree, Image, Variant
from pulp_rpm.app.models.modulemd import Modulemd, ModulemdDefaults
from pulp_rpm.app.models.package import Package
from pulp_rpm.app.models.repository import RpmDistribution, RpmPublication, RpmRemote, RpmRepository
from pulpcore.app.models.content import Artifact, Content, ContentArtifact, RemoteArtifact
from pulpcore.app.models.exporter import FileSystemExporter
from pulpcore.app.models.progress import ProgressReport
from pulpcore.app.models.publication import BaseDistribution, ContentGuard, Publication, PublishedArtifact, PublishedMetadata
from pulpcore.app.models.repository import Remote, Repository, RepositoryContent, RepositoryVersion, RepositoryVersionContentDetails
from pulpcore.app.models.status import ContentAppStatus
from pulpcore.app.models.task import CreatedResource, ReservedResource, ReservedResourceRecord, Task, TaskReservedResource, TaskReservedResourceRecord, Worker
from pulpcore.app.models.upload import Upload, UploadChunk
# Shell Plus Django Imports
from django.core.cache import cache
from django.conf import settings
from django.contrib.auth import get_user_model
from django.db import transaction
from django.db.models import Avg, Case, Count, F, Max, Min, Prefetch, Q, Sum, When, Exists, OuterRef, Subquery
from django.utils import timezone
from django.urls import reverse
Python 3.6.8 (default, Aug  7 2019, 17:28:10) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.11.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: FileContent.objects.count()                                                                                                                                                                                
Out[1]: 13

In [2]: FileRepository.objects.count()                                                                                                                                                                             
Out[2]: 1

As you can see there have been migrated 13 isos, 10 from 'file-large' and 3 from 'file'. Rest got skipped because file2 and file-many have on_demand policy


Subtasks

Issue #5977: Migrated Content counters are wrongASSIGNEDdalleyActions

History

#1 Updated by ipanova@redhat.com 9 months ago

  • Description updated (diff)

#2 Updated by ttereshc 9 months ago

  • Project changed from Pulp to Migration Plugin

#3 Updated by dalley 8 months ago

edit: this comment was incorrect, disregard.

#4 Updated by ttereshc 6 months ago

I think it was intentional - to migrate all content for a plugin regardless. It allows us to perform bulk operations without granular search.

The assumption is that user wants to migrate all or almost all content. If the intention is to migrate very few repositories, maybe it's not worth a hassle to use a migration plugin and write a MP, it's just easier to create those repos in Pulp 3 from scratch.

#5 Updated by ttereshc 6 months ago

  • Triaged changed from No to Yes

Please register to edit this issue

Also available in: Atom PDF