Project

Profile

Help

Task #2454

closed

Stub File Plugin

Added by amacdona@redhat.com over 5 years ago. Updated over 2 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Sprint:
Sprint 17
Quarter:

Description

This task is to create the beginnings of a file plugin.


Related issues

Blocks Pulp - Task #2249: Document platform Django app layoutCLOSED - CURRENTRELEASEsemyers

Actions
Actions #1

Updated by mhrivnak over 5 years ago

Actions #2

Updated by jortel@redhat.com over 5 years ago

  • Groomed changed from No to Yes
  • Tags Pulp 3 added
Actions #3

Updated by mhrivnak over 5 years ago

  • Sprint/Milestone set to 30
Actions #4

Updated by mhrivnak over 5 years ago

  • Sprint/Milestone changed from 30 to 36
Actions #5

Updated by semyers over 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to semyers

I'm not entirely sure how to proceed, but am likely going to create the repository in the pulp org as the last step. I'll do the work in a "pulp_file" repo in my personal github space, and once everything's stubbed out POST that entire repo for review. I also plan to get pulp-dev involved, due to the unique review process that I plan to employ for creating an entire new repo.

Actions #6

Updated by semyers over 5 years ago

I added the "document ... stubbing out a plugin" checklist item, since that will likely be useful for future plugin developers. I'm not planning to do anything fancier for that than leave notes on this ticket about what I did.

Actions #7

Updated by semyers over 5 years ago

POSTed stuff:

New repo stubbing out the plugin:
https://github.com/seandst/pulp_file/

platform PR that adds necessary bits to the plugin API for writing a plugin, as well as entry point support:
https://github.com/pulp/pulp/pull/2975

The pulp_file plugin will not work unless #2975 is applied.

Here's the docs of what I did to stub out the plugin:

# create the pulp_file repo with a pulp_file django app inside it.
git init pulp_file
cd pulp_file
steal the .gitignore file from platform
# create the plugin src dir, then create the django app inside it
mkdir pulp_file
cd pulp_file
django-admin startapp app
cd app

# set up the modules we do and don't want in the pulp_file app dir
# in the 'app' dir: (keep models, remove other stuff, add serializers and viewsets)
touch serializers.py viewsets.py
rm admin.py tests.py views.py

# create django app
in pulp_file/app/__init__.py:
- created PulpFilePluginAppConfig, subclass of pulp.plugin.PulpPluginAppConfig
- define app name and label, 'pulp_file.app' and 'pulp_file_app' respectively

# link pulp_file plugin to its app
in pulp_file/__init__.py:
- set 'default_app_config' to 'pulp_file.app.PulpFilePluginAppConfig'

# create models
in pulp_file/app/models.py:
- created FileContent model with 'name' field, TYPE = 'file', subclass of pulp.plugin.models.Content
- created FileImporter, TYPE = 'file', subclass of pulp.plugin.models.Importer
- created FilePublisher, TYPE = 'file', subclass of pulp.plugin.models.Publisher
- Make migrations: django-admin makemigrations pulp_file_app

# create serializers
in pulp_file/app/serializers.py:
- created FileContentSerializer with 'name' field, subclass of pulp.plugin.serializers.ContentSerializer
- created FileImporterSerializer, subclass of pulp.plugin.serializers.ImporterSerializer
- created FilePublisherSerializer, subclass of pulp.plugin.serializers.PublisherSerializer

# create viewsets
in pulp_file/app/viewsets.py
- created FileContentViewSet with FileContentFilterSet exposing 'name' as filterable,
  subclass of pulp.plugin.viewsets.ContentViewSet
- created FileImporterViewSet, subclass of pulp.plugin.viewsets.ImporterViewSet
- created FilePublisherViewSet, subclass of pulp.plugin.viewsets.PublisherViewSet

# create setup.py
- add normal setup.py fields (name, description, authorship, url, version, etc)
- add 'pulp.plugin' entry point
- install (python setup.py develop)
- should depend on 'pulp_app', but not sure if that's something we can do
  (or want to do) until the platform components are available on pypi

# notes
- The FileContent 'name' field is arbitrary. In Pulp 2, we uniquely identified files listed in ISO manifests
  by their name, checksum, and size. In Pulp 3, any content unit can have zero or more artifacts associated with
  it, and each artifact already stores these attributes, so I've opted to use the "name" field here as a bit of
  a placeholder while we figure out exactly how we plan to model FileContent (or someone finds the docs where we
  already did this) and implement that plan.
- Django freaks out if the platform migrations don't exist when trying to create the pulp_file migrations, since
  it requires pulp_app's migrations to exist. Telling django to only create the pulp_app migrations first doesn't
  seem to fix this, so platform migrations should be run *before* installing the pulp_file plugin. At some point,
  we should start checking the migrations into version control; I'm not sure what that point is.
- I created and added support for the 'pulp.plugin' entry point. That's technically out of scope for this issue,
  so I captured that work in separate commits if we want to separate that out into its own issue.

edit: fixed commit refs to new repo to be actual working links :)

Actions #8

Updated by semyers over 5 years ago

  • Status changed from ASSIGNED to POST
Actions #9

Updated by semyers over 5 years ago

Unfortunately, in the current state of the development environment and how we're handling migrations, this plugin is uninstallable. I debated filing a separate bug about this, but it seems reasonable to fix it here rather than creating a new plugin repo that we know can't be installed into the dev env.

I've hacked up my vagrant box to get it working, so tomorrow I'll take those hacks and open up a devel PR that is pulp_file ready.

While it won't outright fail, the dev env will still not yet be pulp_file aware, that one's definitely a followup story, which I've added to this story's checklist to ensure we don't drop the ball on our next steps with pulp_file.

Edit: This is the issue I mentioned in my notes in comment 7. In trying to write docs for plugin writers to use, it became clear that the need to hack up the platform settings file just to get your development plugin in a workable state was unreasonable, and a better option was required.

Actions #10

Updated by semyers over 5 years ago

semyers wrote:

I've hacked up my vagrant box to get it working, so tomorrow I'll take those hacks and open up a devel PR that is pulp_file ready.

pclean calls out to the "db-reset" script in the pulp repo, so no changed to devel were needed. I've updated my pulp PR to have working migrations whether or not you have plugins installed.

Actions #11

Updated by semyers over 5 years ago

  • Blocks Task #2249: Document platform Django app layout added

Added by semyers over 5 years ago

Revision e0f228e2

Update plugin API and base classes for plugins

re #2454 https://pulp.plan.io/issues/2454

Added by semyers over 5 years ago

Revision e0f228e2

Update plugin API and base classes for plugins

re #2454 https://pulp.plan.io/issues/2454

Actions #12

Updated by semyers over 5 years ago

Austin had an idea for opening up the new repository with a PR, which is posted for review:
https://github.com/pulp/pulp_file/pull/1

Added by semyers over 5 years ago

Revision 0518201f

Initial commit of pulp_file plugin stub

closes #2454 https://pulp.plan.io/issues/2454

Actions #13

Updated by semyers over 5 years ago

  • Status changed from POST to MODIFIED

We don't yet have automation for the pulp_file repository, so this went MODIFIED with the merge of https://github.com/pulp/pulp_file/commit/fcad0ab90c2e0201575b8ad5dcd7b21da676a5c9

Actions #14

Updated by bmbouter over 4 years ago

  • Sprint set to Sprint 17
Actions #15

Updated by bmbouter over 4 years ago

  • Sprint/Milestone deleted (36)
Actions #16

Updated by daviddavis over 3 years ago

  • Sprint/Milestone set to 3.0.0
Actions #17

Updated by bmbouter over 3 years ago

  • Tags deleted (Pulp 3)
Actions #18

Updated by bmbouter over 2 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF