Issue #2710: syncing same ostree repo multiple times causes duplicate branches - OSTree Support - Pulp

Actions

Send by e-mail Copy link

Issue #2710

closed

syncing same ostree repo multiple times causes duplicate branches

Added by paji@redhat.com over 7 years ago. Updated over 5 years ago.

Status:

CLOSED - NOTABUG

Priority:

Normal

Assignee:

Sprint/Milestone:

Start date:

Due date:

Estimated time:

Severity:

2. Medium

Version - OSTree:

Platform Release:

Target Release - OSTree:

OS:

Triaged:

Groomed:

Sprint Candidate:

Tags:

Pulp 2

Sprint:

Quarter:

Description

I have an ostree repo created with the following setup

$ pulp ostree repo create --repo-id=john --feed=https://partha.fedorapeople.org/test-repos/ostree-zoo/
$ pulp ostree repo sync run --repo-id=john

$ pulp ostree repo list --detail --repo-id=john
+----------------------------------------------------------------------+
                          OSTree Repositories
+----------------------------------------------------------------------+

Id:                   john
Display Name:         None
Description:          None
Content Unit Counts:  
  Ostree: 2
Notes:                
Scratchpad:           
  Remote: 
    Summary: 
      Commit:   b954defc0c21b23705bf5776a78ca434b2569260e64e19fb61bdbd5526b3bf0f
      Metadata: 
        Version: 23.3
      Name:     fedora-atomic/f21/x86_64/updates-testing/docker-host
      Commit:   b2179ef2c16343758ebebaa1dfdf40cca8a76d5418ad1780d19b708ff2202dcf
      Metadata: 
        Version: 23.3
      Name:     fedora-atomic/f21/x86_64/updates/docker-host
Importers:            
  Config:               
    Feed: https://partha.fedorapeople.org/test-repos/ostree-zoo/
  Id:                   ostree_web_importer
  Importer Type Id:     ostree_web_importer
  Last Override Config: 
  Last Sync:            2017-04-13T21:15:29Z
  Last Updated:         2017-04-13T21:14:00Z
  Repo Id:              john
  Scratchpad:           None
Distributors:         
  Auto Publish:         True
  Config:               
    Relative Path: /test-repos/ostree-zoo/
  Distributor Type Id:  ostree_web_distributor
  Id:                   ostree_web_distributor_name_cli
  Last Override Config: 
  Last Publish:         2017-04-13T21:15:29Z
  Last Updated:         2017-04-13T21:14:00Z
  Repo Id:              john
  Scratchpad:

important number here is the number of branches

Each time I re reun the sync operation the number of branches keep incrementing even though the upstream feed was not updated

$ pulp ostree repo sync run --repo-id=john
.....
Task Succeeded
pulp ostree repo list  --repo-id=john
+----------------------------------------------------------------------+
                          OSTree Repositories
+----------------------------------------------------------------------+

Id:                   john
Display Name:         None
Description:          None
Content Unit Counts:  
  Ostree: 4

Other Info

$ rpm -qa |grep ostree
pulp-ostree-plugins-1.2.0-1.el7.noarch
python-pulp-ostree-common-1.2.0-1.el7.noarch
ostree-2017.1-3.atomic.el7.x86_64
pulp-ostree-admin-extensions-1.2.0-1.el7.noarch

$ rpm -qa |grep pulp|grep -v ostree
pulp-admin-client-2.12.1-1.el7.noarch
pulp-puppet-admin-extensions-2.12.1-1.el7.noarch
python-pulp-oid_validation-2.12.1-1.el7.noarch
pulp-server-2.12.1-1.el7.noarch
rubygem-smart_proxy_pulp-1.3.0-1.el7.noarch
python-pulp-bindings-2.12.1-1.el7.noarch
python-pulp-client-lib-2.12.1-1.el7.noarch
pulp-rpm-admin-extensions-2.12.1-1.el7.noarch
pulp-docker-admin-extensions-2.3.0-1.el7.noarch
python-pulp-repoauth-2.12.1-1.el7.noarch
pulp-rpm-plugins-2.12.1-1.el7.noarch
pulp-docker-plugins-2.3.0-1.el7.noarch
python-pulp-streamer-2.12.1-1.el7.noarch
pulp-puppet-plugins-2.12.1-1.el7.noarch
pulp-client-1.0-1.noarch
pulp-katello-1.0.2-1.el7.noarch
python-pulp-common-2.12.1-1.el7.noarch
python-kombu-3.0.33-6.pulp.el7.noarch
pulp-selinux-2.12.1-1.el7.noarch
python-pulp-docker-common-2.3.0-1.el7.noarch
python-pulp-puppet-common-2.12.1-1.el7.noarch
python-isodate-0.5.0-4.pulp.el7.noarch
python-pulp-rpm-common-2.12.1-1.el7.noarch

Actions

Copy link

Updated by ttereshc over 7 years ago

@jortel is investigating it, we will triage it next time with more info available

Actions

Copy link

Updated by jortel@redhat.com over 7 years ago

Looks like the unique index for the unit key is missing:

> db.units_ostree.getIndexes()
[
    {
        "v" : 1,
        "key" : {
            "_id" : 1
        },
        "name" : "_id_",
        "ns" : "pulp_database.units_ostree"
    }
]

The index should be created by pulp-manage-db.

Actions

Copy link

Updated by bmbouter over 7 years ago

Has OSTree been converted to use mongoengine models? If so then I expect that index to get created by pulp-manage-db in this code.

Actions

Copy link

Updated by jortel@redhat.com over 7 years ago

bmbouter wrote:

Has OSTree been converted to use mongoengine models? If so then I expect that index to get created by pulp-manage-db in this code.

Yes.

Actions

Copy link

Updated by jortel@redhat.com over 7 years ago

Removed all ostree units and repo association to the repository.
Ran pulp-manage-db
Verified the index got created.
Ran a few syncs.
Only created the expected units.

Actions

Copy link

Updated by bmbouter over 7 years ago

This was discovered to be an environmental issue where OSTree was installed but pulp-manage-db was not run. @jortel should we close this as NOTABUG?

Also I removed the release blocker field since the root cause is not a defect.

Actions

Copy link

#10