Pulp

When considering how to implement this in Pulp 3.0 it is worth looking at what Spacewalk does to calculate errata/package applicability. Spacewalk is only concerned with two content types: RPM packages and Errata. Errata are composed of packages. So we can really just discuss packages. Spacewalk takes a list of packages (profile) for a particular consumer and determines which packages in all repositories the consumer is subscribed to have a version greater than the version of the same package in the consumer profile. This calculation is performed using a stored procedure[0] that is able to take advantage of a custom type used to represent NVRA[1]. The custom NVRA type defines how NVRAs should be compared. The stored procedure computes which errata/packages are actually applicable.

Plugin writers should be able to define custom types for PostreSQL as well as stored procedures that can be used to perform applicability calculations.

Pulp's Plugin API should provide
- A mechanism for adding new types and stored procedures to the database.
- A basic Profiler interface that plugin writers can implement. The implementation would most likely run a stored procedure.

Pulp should provide REST APIs for:
- Uploading unit profiles that consist of two lists: installed units, associated repositories.
- Calculating applicability for a particular unit profile.
- Calculating applicability for all unit profiles associated with a repository.
- Retreiving applicability for a particular unit profile and content type.

[0] https://github.com/spacewalkproject/spacewalk/blob/SPACEWALK-2.5/schema/spacewalk/postgres/packages/rhn_server.pkb#L662-L696
[1] https://github.com/spacewalkproject/spacewalk/blob/SPACEWALK-2.5/schema/spacewalk/postgres/class/evr_t.sql

Generally all of this sounds good. +1 to using a stored procedure and moving the applicability calculation into the database.

One thing I contemplate is if we will have to "manage" stored procedures or accept stored procedures written by plugin writers. I was hoping we could write the one or two stored procedures which could be reused for all content types. Somehow we need the plugin author to give us everything that stored procedure would need on the model definition.

If we did have a stored procedure per content type, how would Django support that such that they could be installed with the Django migrations system?

The plugin writer will create a data migration[0] that uses a RunSQL[1] operation to add the stored procedure to the database. The migration would then be committed to source control (just like schema migrations that are generated with help of Django).

Django's connection and cursor objects implement most of Python's DB-API[2] described in PEP 249. The cursor provides a callproc() method[3] that allows calling a stored procedure with a list of parameters expected by the stored procedure.

The stored procedure in Spacewalk inserts the results of applicability calculation into the database. I suspect we would want to do the same thing.

[0] https://docs.djangoproject.com/en/1.8/topics/migrations/#data-migrations
[1] https://docs.djangoproject.com/en/1.8/ref/migration-operations/#django.db.migrations.operations.RunSQL
[2] https://docs.djangoproject.com/en/1.8/topics/db/sql/#connections-and-cursors
[3] http://legacy.python.org/dev/peps/pep-0249/#callproc