Pulp 3 Developer Notes¶

This wiki page is intended for use during early development of Pulp 3. Over time, as our development practices become standard, the contents of this page should be moved into the Pulp Contributing Guide

Before reporting issues with the development environment, please ensure that you are using the latest version.

Migrations¶

In both platform and plugins, the data model is not complete. As a result, committing migrations to the 3.0-dev branch will result in merge/migration conflicts from pull request to pull request. The simplest solution for now is not to commit migrations to the repository.

Because User model depends on Django's auth app having been migrated, this means that you currently need to run python manage.py migrate auth before running a general python manage.py migrate to set up the pulp database.

Making migrations during development¶

Tests require migrations to run, so while we should not commit migrations to the repositories just yet, we do still need to make them. This can be done with the python manage.py makemigrations command. Apps that depend on the platform migrations existing (such as plugins) may cause errors when making migrations. To avoid these errors, platform migrations should be made prior to installing any plugins.

Once the initial migrations are created, and model changes made thereafter will require python manage.py makemigrations to be run again, following by @python manage.py migrate" so Django can apply the model changes to the database.

If you are using the vagrant environment this is done during provisioning. The pclean alias takes care of migrations after resting the db.

Debugging migrations¶

Migrate pulp_file
<script>python manage.py makemigrations pulp_file</script>

Migrate pulp_app first because (sometimes) makemigrations does not always create the initial migrations if they are not already there
<script>python manage.py makemigrations pulp_app</script>

Starting a Web Server¶

The Django development server can be started with python manage.py runserver. This will run a basic WSGI app that exposes the URLs routed in urls.py, allowing you to access the REST API.

If you're using the vagrant hostmanager plugin, you can easily access the API from the host machine by explicitly binding the web server to all interfaces, e.g. python manage.py runserver 0.0.0.0:8000. This should make the API browseable at http://dev.example.com:8000/api/v3/

Authentication¶

We currently enable Basic HTTP Authentication on the REST API. This can be temporarily disabled by commenting out the 'DEFAULT_PERMISSION_CLASSES': ('rest_framework.permissions.IsAuthenticated',) line in the REST_FRAMEWORK section in app/pulp/app/settings.py. Note that this doesn't disable authentication, it just authorizes unauthenticated users to take any action. Basic Authentication should still work.

Tasks¶

Starting Services¶

If using the vagrant environment, you can start the services with the alias pstart. Afterwards, use prestart.

To start the processes manually, if you are using a virtual environment, be sure that the path to celery matches the virtual environment that includes pulp, any plugins, and pulp dependencies.

sudo -u apache /usr/bin/celery beat --app=pulpcore.tasking.celery_app:celery --scheduler=pulpcore.tasking.services.scheduler.Scheduler -l=INFO --pidfile=/var/run/pulp/scheduler.pid

sudo -u apache /usr/bin/celery worker -A pulpcore.tasking.celery_app:celery -n resource_manager@%%h\
            -Q resource_manager -c 1 --events --umask 18 --pidfile=/var/run/pulp/resource_manager.pid\
            --heartbeat-interval=5 -l=INFO

sudo -u apache /usr/bin/celery worker -n reserved_resource_worker-123s@h\
          -A pulpcore.tasking.celery_app:celery -c 1 --events --umask 18\
          --pidfile=/var/run/pulp/reserved_resource_worker-123s.pid\
          --heartbeat-interval=5 -l=INFO

Deploy tasks from shell¶

apply_async and apply_async_with_reservation tasks can be tested from a django shell python manage.py shell_plus I usually do the following to test tasks

from pulpcore.app.tasks import repository
from pulpcore.app.models import Repository
import uuid
repo_uuid=str(uuid.uuid4())
repo=Repository(name=repo_uuid)
repo.save()

repository.delete.apply_async(kwargs={'repo_id':repo_uuid})
repository.delete.apply_async_with_reservation("foo","bar",kwargs={'repo_id':repo_uuid})

Deploy sync task from REST API¶

1. Make sure that you have the file plugin installed with a developer setup.
2. Define a sync method on the FileImporter model.
3. Create a repository on the browseable api:
http://dev.example.com:8000/api/v3/repositories/
4. Create an importer. Note, you must specify the repository and the plugin in the URL.
http://dev.example.com:8000/api/v3/repositories/<repository_name>/importers/file/
5. Sync at the importer's url http://dev.example.com:8000/api/v3/repositories/<repository_name>importers/file/<importer_name>/sync

PyPI¶

Publishing to PyPI¶

python setup.py sdist bdist_wheel --python-tag py3
twine upload -s dist/{package}*

Installing from PyPI¶

pip3 install pulpcore
export DJANGO_SETTINGS_MODULE=pulpcore.app.settings
set up database and migrations
django-admin runserver

Known Issues¶

• Creating a repo with notes/scratchpad and updating a repo does not work from the Crispy Forms UI because the UI does not send scratchpad and notes in a json object like the serializer expects. Calling the create/update from httpie or curl does work. http --auth admin:admin --json PUT http://127.0.0.1:3000/api/v3/repositories/test/ name=test description=123456 scratchpad:='{"test": "asgfdds"}'