Project

Profile

Help

Task #2454

Stub File Plugin

Added by amacdona@redhat.com almost 3 years ago. Updated 6 months ago.

Status:
MODIFIED
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

0%

Platform Release:
Blocks Release:
Backwards Incompatible:
No
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 17

Description

This task is to create the beginnings of a file plugin.


Checklist


Related issues

Blocks Pulp - Task #2249: Document platform Django app layout MODIFIED Actions

Associated revisions

Revision e0f228e2 View on GitHub
Added by semyers over 2 years ago

Update plugin API and base classes for plugins

re #2454
https://pulp.plan.io/issues/2454

Revision e0f228e2 View on GitHub
Added by semyers over 2 years ago

Update plugin API and base classes for plugins

re #2454
https://pulp.plan.io/issues/2454

Revision e0f228e2 View on GitHub
Added by semyers over 2 years ago

Update plugin API and base classes for plugins

re #2454
https://pulp.plan.io/issues/2454

Revision 0518201f View on GitHub
Added by semyers over 2 years ago

Initial commit of pulp_file plugin stub

closes #2454
https://pulp.plan.io/issues/2454

History

#1 Updated by mhrivnak almost 3 years ago

  • Checklist item changed from Create a file plugin in the pulp organization on github to Create a file plugin repository in the pulp organization on github

#2 Updated by jortel@redhat.com almost 3 years ago

  • Groomed changed from No to Yes
  • Tags Pulp 3 added

#3 Updated by mhrivnak almost 3 years ago

  • Sprint/Milestone set to 30

#4 Updated by mhrivnak over 2 years ago

  • Sprint/Milestone changed from 30 to 36

#5 Updated by semyers over 2 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to semyers

I'm not entirely sure how to proceed, but am likely going to create the repository in the pulp org as the last step. I'll do the work in a "pulp_file" repo in my personal github space, and once everything's stubbed out POST that entire repo for review. I also plan to get pulp-dev involved, due to the unique review process that I plan to employ for creating an entire new repo.

#6 Updated by semyers over 2 years ago

I added the "document ... stubbing out a plugin" checklist item, since that will likely be useful for future plugin developers. I'm not planning to do anything fancier for that than leave notes on this ticket about what I did.

#7 Updated by semyers over 2 years ago

POSTed stuff:

New repo stubbing out the plugin:
https://github.com/seandst/pulp_file/

platform PR that adds necessary bits to the plugin API for writing a plugin, as well as entry point support:
https://github.com/pulp/pulp/pull/2975

The pulp_file plugin will not work unless #2975 is applied.

Here's the docs of what I did to stub out the plugin:

# create the pulp_file repo with a pulp_file django app inside it.
git init pulp_file
cd pulp_file
steal the .gitignore file from platform
# create the plugin src dir, then create the django app inside it
mkdir pulp_file
cd pulp_file
django-admin startapp app
cd app

# set up the modules we do and don't want in the pulp_file app dir
# in the 'app' dir: (keep models, remove other stuff, add serializers and viewsets)
touch serializers.py viewsets.py
rm admin.py tests.py views.py

# create django app
in pulp_file/app/__init__.py:
- created PulpFilePluginAppConfig, subclass of pulp.plugin.PulpPluginAppConfig
- define app name and label, 'pulp_file.app' and 'pulp_file_app' respectively

# link pulp_file plugin to its app
in pulp_file/__init__.py:
- set 'default_app_config' to 'pulp_file.app.PulpFilePluginAppConfig'

# create models
in pulp_file/app/models.py:
- created FileContent model with 'name' field, TYPE = 'file', subclass of pulp.plugin.models.Content
- created FileImporter, TYPE = 'file', subclass of pulp.plugin.models.Importer
- created FilePublisher, TYPE = 'file', subclass of pulp.plugin.models.Publisher
- Make migrations: django-admin makemigrations pulp_file_app

# create serializers
in pulp_file/app/serializers.py:
- created FileContentSerializer with 'name' field, subclass of pulp.plugin.serializers.ContentSerializer
- created FileImporterSerializer, subclass of pulp.plugin.serializers.ImporterSerializer
- created FilePublisherSerializer, subclass of pulp.plugin.serializers.PublisherSerializer

# create viewsets
in pulp_file/app/viewsets.py
- created FileContentViewSet with FileContentFilterSet exposing 'name' as filterable,
  subclass of pulp.plugin.viewsets.ContentViewSet
- created FileImporterViewSet, subclass of pulp.plugin.viewsets.ImporterViewSet
- created FilePublisherViewSet, subclass of pulp.plugin.viewsets.PublisherViewSet

# create setup.py
- add normal setup.py fields (name, description, authorship, url, version, etc)
- add 'pulp.plugin' entry point
- install (python setup.py develop)
- should depend on 'pulp_app', but not sure if that's something we can do
  (or want to do) until the platform components are available on pypi

# notes
- The FileContent 'name' field is arbitrary. In Pulp 2, we uniquely identified files listed in ISO manifests
  by their name, checksum, and size. In Pulp 3, any content unit can have zero or more artifacts associated with
  it, and each artifact already stores these attributes, so I've opted to use the "name" field here as a bit of
  a placeholder while we figure out exactly how we plan to model FileContent (or someone finds the docs where we
  already did this) and implement that plan.
- Django freaks out if the platform migrations don't exist when trying to create the pulp_file migrations, since
  it requires pulp_app's migrations to exist. Telling django to only create the pulp_app migrations first doesn't
  seem to fix this, so platform migrations should be run *before* installing the pulp_file plugin. At some point,
  we should start checking the migrations into version control; I'm not sure what that point is.
- I created and added support for the 'pulp.plugin' entry point. That's technically out of scope for this issue,
  so I captured that work in separate commits if we want to separate that out into its own issue.

edit: fixed commit refs to new repo to be actual working links :)

#8 Updated by semyers over 2 years ago

  • Status changed from ASSIGNED to POST

#9 Updated by semyers over 2 years ago

Unfortunately, in the current state of the development environment and how we're handling migrations, this plugin is uninstallable. I debated filing a separate bug about this, but it seems reasonable to fix it here rather than creating a new plugin repo that we know can't be installed into the dev env.

I've hacked up my vagrant box to get it working, so tomorrow I'll take those hacks and open up a devel PR that is pulp_file ready.

While it won't outright fail, the dev env will still not yet be pulp_file aware, that one's definitely a followup story, which I've added to this story's checklist to ensure we don't drop the ball on our next steps with pulp_file.

Edit: This is the issue I mentioned in my notes in comment 7. In trying to write docs for plugin writers to use, it became clear that the need to hack up the platform settings file just to get your development plugin in a workable state was unreasonable, and a better option was required.

#10 Updated by semyers over 2 years ago

semyers wrote:

I've hacked up my vagrant box to get it working, so tomorrow I'll take those hacks and open up a devel PR that is pulp_file ready.

pclean calls out to the "db-reset" script in the pulp repo, so no changed to devel were needed. I've updated my pulp PR to have working migrations whether or not you have plugins installed.

#11 Updated by semyers over 2 years ago

  • Blocks Task #2249: Document platform Django app layout added

#12 Updated by semyers over 2 years ago

Austin had an idea for opening up the new repository with a PR, which is posted for review:
https://github.com/pulp/pulp_file/pull/1

#13 Updated by semyers over 2 years ago

  • Status changed from POST to MODIFIED

We don't yet have automation for the pulp_file repository, so this went MODIFIED with the merge of https://github.com/pulp/pulp_file/commit/fcad0ab90c2e0201575b8ad5dcd7b21da676a5c9

#14 Updated by bmbouter over 1 year ago

  • Sprint set to Sprint 17

#15 Updated by bmbouter over 1 year ago

  • Sprint/Milestone deleted (36)

#16 Updated by daviddavis 6 months ago

  • Sprint/Milestone set to 3.0

#17 Updated by bmbouter 6 months ago

  • Tags deleted (Pulp 3)

Please register to edit this issue

Also available in: Atom PDF