Index by title

2.11.2 Release Status

This release is currently released.

2.11.2 can be released before 2.12.0, with the earliest beta of 2.11.2 on Monday, Jan 23, 2017.

This release is currently in testing. If tests do not pass, a 2.11.2 will probably not happen due to the aforementioned overlap with the 2.12.0 release timeline.

Issues Addressed

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.

If all goes well, and no blocking bugs delay further release stages, this release should become Generally Available Jan 30 2017. This page will be updated as progress is made.


2.12.0 Release Status

This release is currently Generally Available.

https://repos.fedorapeople.org/repos/pulp/pulp/stable/2.12/

Issues & Stories Addressed

This build was delayed for one day while we discussed how to handle potential blocking issues found while testing the pulp_python plugin. We decided not to release pulp_python 2.0 with pulp 2.12.0.

Checklist

The first item on the list not struck through is in progress.

If no blocking issues filed before the end of the RC waiting period, the Generally Available release will be built and released.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the 2.12.0 release progresses. See the Release Schedule for more details.

If all goes well, and no blocking bugs delay further release stages, this release should become Generally Available Jan 31 2017. This page will be updated as progress is made.


2.12.1 Release Status

This release is currently generally available.

2.12.1 can be released after 2.12.0, with the earliest beta of 2.12.1 roughly one week after 2.12.0 becomes Generally Available.

Issues Addressed

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.


2.12.2 Release Status

This release is currently released.

Issues Addressed

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.


2.13.0 Release Status

This release is currently released.

The release date for 2.13.0 has not yet been determined. The most likely release date, in following with the current 12-week beta-to-beta timeline, is April 12, 2017.

Issues + Stories Addressed

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the 2.13.0 release progresses. See the Release Schedule for more details.

If all goes well, and no blocking bugs delay further release stages, this release should become Generally Available April 25, 2017. This page will be updated as progress is made.


2.13.1 Release Status

This release is currently generally available.

"Issues Addressed": http://bit.ly/2pPKNCC

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.


2.13.2 Release Status

2.13.2 can be released after 2.13.1, with the earliest beta of 2.13.2 roughly one week after 2.13.1 becomes Generally Available.

Checklist

The first item on the list not struck through is in progress.

2.13.2 Released 6-20-2017

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.


2.13.3 Release Status

2.13.3 can be released after 2.13.2, with the earliest beta of 2.13.3 roughly one week after 2.13.2 becomes Generally Available.

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.


2.13.4 Release Status

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.


2.14.0 Release Status

The current tentative release date for 2.14.0 GA is August 03, 2017.

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the 2.14.0 release progresses. See the Release Schedule for more details.

If all goes well, and no blocking bugs delay further release stages, this release should become Generally Available August 03, 2017. This page will be updated as progress is made.


2.14.1 Release Status

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.


2142 Release Status

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.


2.14.3 Release Status

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.


2.15.0 Release Status

The current tentative release date for 2.15.0 GA is December 11, 2017.

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the 2.15.0 release progresses. See the Release Schedule for more details.

If all goes well, and no blocking bugs delay further release stages, this release should become Generally Available December 11, 2017. This page will be updated as progress is made.


2.15.1 Release Status

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.


2.15.2 Release Status

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.


2.15.3 Release Status

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.


2.16.0 Release Status

The current tentative release date for 2.16.0 is April 3, 2018

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the 2.16.0 release progresses. See the Release Schedule for more details.

If all goes well, and no blocking bugs delay further release stages, this release should become Generally Available April 3, 2018. This page will be updated as progress is made.


2.16.1 Release Status

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.


2.16.2 Release Status

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.


2.16.4 Release Status

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.


2.17.0 Release Schedule

The current tentative release date for 2.17.0 is August 30, 2018

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the 2.17.0 release progresses. See the Release Schedule for more details.

If all goes well, and no blocking bugs delay further release stages, this release should become Generally Available August 30, 2018. This page will be updated as progress is made.

Release Contact: ipanova on freenode (#pulp and#pulp-dev)


2.17.1 Release Schedule

The current tentative release date for 2.17.1 is October 11, 2018

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.

Release Contact: dkliban or ttereshc on freenode (#pulp and#pulp-dev)


2.18.0 Release Schedule

The release date for 2.18.0 is December 05, 2018

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the 2.18.0 release progresses. See the Release Schedule for more details.

If all goes well, and no blocking bugs delay further release stages, this release should become Generally Available December 05, 2018. This page will be updated as progress is made.

Release Contact: ttereshc on freenode (#pulp and#pulp-dev)


2.18.1 Release Schedule

The current tentative release date for 2.18.1 is February 4, 2019

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.

Release Contact: jortel on freenode (#pulp and#pulp-dev)

QE Coverage


2182 Release Schedule

This was an early draft for planning purposes. Determined we do not need this, we can just do a 2.19.0 build.


2.19.0 Release Schedule

The current tentative release date for 2.19.0 is April 2, 2019

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the release cycle; this page will be updated as the 2.19.0 release progresses. See the Release Schedule for more details.

If all goes well, and no blocking bugs delay further release stages, this release should become Generally Available April 2, 2019. This page will be updated as progress is made.

Release Contact: ttereshc on freenode (#pulp and#pulp-dev)

QE Coverage


2.19.1 Release Schedule

The current tentative release date for 2.19.1 is May 29, 2019

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.

Release Contact: dkliban on freenode (#pulp and#pulp-dev)

QE Coverage


2.20.0 Release Schedule

The release date for 2.20.0 is July 10, 2019

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the release cycle; this page will be updated as the 2.20.0 release progresses. See the Release Schedule for more details.

If all goes well, and no blocking bugs delay further release stages, this release should become Generally Available July 10, 2019. This page will be updated as progress is made.

Release Contact: ttereshc on freenode (#pulp and#pulp-dev)

QE Coverage


2.21.0 Release Schedule

The tentative release date for 2.21.0 is September 25, 2019

Checklist

The first item on the list not struck through is in progress.

* Dev Freeze date
* September 11, 2019
* RC1 Build date
* September 19, 2019
* Original date September 18, 2019
* Targeting all content here: https://pulp.plan.io/issues?query_id=61
* Prerequisites to building:
* Issues blocking release
* No procedural issues blocking this release (broken CI env, and similar
* waiting period (at least 1 week after RC1 release date) before the GA build
* In case of any issues found during this period, a new RC will be built and a new waiting period will be defined by the release contact person.
* GA Date: September 25, 2019

The rest of the checklist is conditional on whether or not blockers are found in the release cycle; this page will be updated as the 2.21.0 release progresses. See the Release Schedule for more details.

If all goes well, and no blocking bugs delay further release stages, this release should become Generally Available September 25, 2019. This page will be updated as progress is made.

Release Contact: dalley on freenode (#pulp and#pulp-dev)


2.21.1 Release Schedule

The current tentative release date for 2.21.1 is March 4, 2020

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the release progresses. See the Release Schedule for more details.

Release Contact: ipanova on freenode (#pulp and#pulp-dev)


30-Development

.. warning::
All documents within the 3.0-Development section should be considered temporary, and will be
removed or relocated prior to the release of Pulp 3.0.

3.0 Development
===============

The goal of this section is to create a place for living documents related to the development of
Pulp 3.0. Some parts may grow and replace current documentation (plugin API) and others may be
temporary guides to making changes (translating from Mongo to Postgres).

.. toctree::
:maxdepth: 3

app-layout
data-modeling
db-translation-guide
rest-api

Pulp 3 and Python 3
-------------------

Pulp 3 will only support Python 3.5+. When introducing new dependencies to Pulp,
ensure they support Python 3 (see http://fedora.portingdb.xyz/). If they do not,
please file an issue in Redmine (related to https://pulp.plan.io/issues/2247) to
track the conversion of the dependency to support Python 3 and begin working with
upstream to convert the package.

Docstrings
----------

`PUP-2 <https://github.com/pulp/pups/blob/master/pup-0002.md&gt;\`_ adopted Google style for
:ref:`google-docstrings`. When porting code from Pulp 2 to Pulp 3, convert all the docstrings to the
new style. vim-style regexes can be used to speed up the process. Together, these will convert all
of the parameters to Google Style::

  1. Typed params
    %s/\(\s*\):param\s\+\(.*\):\s\+\(.*\)\n\s*:type\s\+.\+:\s\+\(.*\)/\1 \2 (\4): \3
  1. Untyped params
    %s/\(\s*\):param\s\+\(.*\):\s\+\(.*\)/\1 \2: \3

Data Modeling
=============

Introduction
^^^^^^^^^^

The Pulp 3 data modeling effort is not just a translation or porting effort but instead
an effort to build Pulp 3 using Pulp 2. Remodeling will include changes to leverage the
relational capabilities of postgres but also to improve upon the Pulp 2 model.

When creating each Pulp 3 model object, consider the following:

- Understand what the Pulp 2 model (collection) stores and how the information
is used by pulp.

- Name the model/table appropriately. A table is implicitly plural. Naming a
table `repository` is more appropriate than `repositories`.

- Name the model class appropriately. Each model object is a row in the table
an not a collection. Class names should be singular. For example, `Repository`
is a more appropriate class name than `Repositories`.

- Do not use multiple inheritance (mixins) unless there is no other choice. Although
they are convenient, they greatly diminish the clarity of the model and as with
multiple inheritance in general, can result in untended behavior. With a little extra
effort, a proper class hierarchy is almost always possible and will be better in the end.

- Question the existence, type, and naming of all fields. Field names beginning with `_`
should not exist in Pulp 3. The primary key field name is `id` and already defined in the
`Model` base class. All `display_name` fields need to be renamed to `name`. Also, avoid
redundant scope by prefixing field names (or any attribute) with the name of the class.
For example: `Task.task_type`.

- Question all methods (including @property). We only want those methods that encapsulate
significant complexity and are widely used. Most of the methods on Pulp 2 models will likely
not be necessary or wanted in the Pulp 3 models. Let's keep the model interface as clean
as possible.

- Most string fields will be `TextField`. When the field is required and indexed,
use: `TextField(db_index=True)`. When required but not indexed,
use: `TextField(blank=False, default=None)`. This ensures integrity at the model and
database layer(s) and supports validation at the REST layer.

- All fields containing ISO-8601 strings must be converted to `DateTimeField`.

- Think about natural keys. For example, each repository has a unique name (instead of repo_id
like in Pulp 2) that is known to users and is not the primary key (`id` is). The `name` is
the natural key. Don't forget the index. See: https://en.wikipedia.org/wiki/Natural_key

- Define a `natural_key()` method in each model. This both documents the natural key and
will be used by the django serializer.
See: https://docs.djangoproject.com/en/1.8/topics/serialization/#serialization-of-natural-keys

Development Process
^^^^^^^^^^^^^^^^^

- The refactor tasks (in Redmine) group related collections.

- The Pulp 3 models are grouped into modules within the `models` package.

- Foreach model class defined, A refactor task needs to be created in Redmine for
migrating the data. See existing examples.

- Foreach model class defined, add a section to the Notes section below.

- Add unit tests for models. Only methods encapsulating complexity need tests.
In addition to this, add a set of howto tests to demonstrate common use cases for the
model. They are intended to provide an example to developers and sanity testing only.
100% coverage is not expected. Also, we don't need to test django itself.

Notes About Models
^^^^^^^^^^^^^^^^

.. note:: This article is a stub. You can help by expanding it.

This section captures notes about each model. Developers should explain differences
between the Pulp 2 and Pulp 3. This is a good place to give examples of how models are intended
to be used or extended by plugins. Except for how to extend a class, refer to the howto unit
tests instead of including code blocks here.

Consumer
^^^^^^

The consumer models are much like that in Pulp 2 except the fields related to the removed
nodes and agent functionality are gone. The `bind` has be replaced with a simple relation
that is managed by django because no addition fields on the join table are needed.

The applicability models/tables have not been included because applicability is not a generic
platform concept. Errata and RPM applicability is owned by the RPM plugin. That said,
the `ConsumerContent` model provides a base class for plugins to model content that is installed
on a consumer.

Repository
^^^^^^^^

First, `Distributor` has been renamed to `Publisher` because it seems more appropriate.

The repository models include importers and publishers. The main difference being the
consolidation of the importer and publisher and its configuration. In the model, a
`ContentAdaptor` is the base for plugin contributed models that can be associated to a repository.
On importer and publisher base models, the standard configuration settings that were
separate documents in Pulp 2 are attributes of the importer and publishers itself. These models
follow the master-detail pattern. Adaptors needing additional configuration, need to extend the
base model (master) and add the extra fields on a new (detail) model.

The concept of a repository group distributor has been discarded. This concept and associated flows
were flawed in Pulp 2.

Examples:

.. code-block:: python

Class MyImporter(Importer):
field_1 = models.TextField()
field_2 = models.TextField()

Database Field Translation Guide
================================

Introduction
------------

Converting from mongo to postgres should be recognized from the outset as a monumental task.
In choosing Django as our ORM, this task has hopefully been made a little bit easier by bringing in
such a mature and well-supported framework. Great care has been taken to make full use of Django's
offerings when coming up with techniques and guidelines for converting our non-relational mongo
database over to postgres.

Django
------

Django has many layers, including the data model layer, the view layer, etc. This document focuses
on the model layer, and uses Django's terminology where applicable. Furthermore, every effort should
be made to adhere to existing Django functionality so that the full benefits of adopting this
framework can be realized.

https://docs.djangoproject.com/en/1.8/topics/db/models/

Tutorial
^^^^^^

If you aren't already familiar with some of the features that Django provides, part 1 of the Django
tutorial provides an excellent introduction. It covers things like starting a project for the first
time, starting the development web server, and accessing models. The tutorial pages that come after
part 1 are largely irrelevant for Pulp 3, but are a good exercise nonetheless for someone looking to
get to know Django a little bit better.

https://docs.djangoproject.com/en/1.8/intro/tutorial01/

Django Concepts
---------------

There are specific Django concepts that deserve special attention before getting into some of the
finer details of how the relation data model should be structured, which are outlined in subsections
below.

Model Inheritance
^^^^^^^^^^^^^^^

https://docs.djangoproject.com/en/1.8/topics/db/models/#model-inheritance

Django provides three different mechanisms for supporting object-oriented inheritance in its
Model classes. Each method provides specific benefits to Pulp that are outlined here.

Additionally, each of these model inheritance mechanisms can be combined with the other mechanisms
as-needed to create a sound object-oriented design that also generates a reasonable database schema.

Abstract Base classes
***************

https://docs.djangoproject.com/en/1.8/topics/db/models/#abstract-base-classes

Many of our ContentUnit classes have common fields and/or behavior. Abstract base classes make it
trivial for us to put those common bits of code in a single place, to be inherited by Model classes in
completely standard and pythonic ways, such as with subclassing or mixins.

The many different RPM-like ContentUnit subclasses use this extensively, combining in the
"detail" classes to make the complete unit model for a given type.

Multi-table Inheritance
*****************

https://docs.djangoproject.com/en/1.8/topics/db/models/#multi-table-inheritance

This is probably the most important Model inheritance mechanism, as it is used to implement the
ContentUnit "master-detail" relationship, where the "master" content units contain all fields
common to all (or most) ContentUnits, and the "detail" content units contain all of the type-specific
fields for that unit type.

On the database level, the information representing a ContentUnit resides in at least two database
tables (the master ContentUnit table and the detail table), and is seamlessly joined by Django on
instances of the detail ContentUnit model (e.g. RPM, a puppet Module, etc).

More information on this is documented later in the section describing ContentUnit changes.

.. note::
If you go by what wikipedia has to say about master-detail relationships, this isn't quite that.
However, it makes it easy to refer to both sides of the master/detail relationship with terms that
are easy to understand in the context of ContentUnits, so it's worth appropriating those
terms for Pulp's purposes here.

https://en.wikipedia.org/wiki/Master%E2%80%93detail_interface#Data_model

Proxy Models
******

https://docs.djangoproject.com/en/1.8/topics/db/models/#proxy-models
https://docs.djangoproject.com/en/1.8/topics/db/queries/#backwards-related-objects

Not currently used in Pulp, but mentioned here for docs completeness.

Generic Relations
^^^^^^^^^^^^^^^

https://docs.djangoproject.com/en/1.8/ref/contrib/contenttypes/#generic-relations

Django's Generic Relations give us the ability to associate many models to one model in a more
flexible was than a normal ForeignKey. Normally used for things like object tagging,
the Generic Relations that come with Django's contenttypes framework can be used by Pulp to easily
associate a generic Django Model with any number of other models that can benefit from storing the
information captured by the generic model.

A good example of this are the various Generic Key/Value stores that can be associated with any other
Model, "Notes" and "Config".

MongoEngine to Django Field Conversions
---------------------------------------

These are the MongoEngine field types currently used by Pulp, and guidelines on converting them to
Postgres. Since MongoEngine started out to get mongodb working as a Django backend, most fields have
direct counterparts in Django. The following subsections are the MongoEngine fields currently used in
Pulp, with applicable postgres datatypes and Django field alternatives listed inside.

In each MongoEngine field section there will be a link to that MongoEngine field's documentation,
brief information about the corresponding postgres data type, subsections detailing the Django
field or fields that should be used when translating a given MongoEngine field, and a list of
files in which that MongoEngine field is being used.

StringField
^^^^^^^^^

http://docs.mongoengine.org/apireference.html#mongoengine.fields.StringField

This field can be represented by one of two Django fields, depending on which Postgres column is a
better fit for the data being stored in it.

Postgres datatype reference:
https://www.postgresql.org/docs/current/static/datatype-character.html

For our purposes, only varchar and text are interesting, the character type will be ignored. While
some database engines have differences in performance between the varchar and text data types,
this tip from the linked postgres docs is good to keep in mind:

.. note::
"There are no performance differences between these three types, apart from the increased storage size
when using the blank-padded type. While character(n) has performance advantages in some other database
systems, it has no such advantages in PostgreSQL. In most situations text or character varying should
be used instead."

The "blank-padded" type mentioned in that quote is the character type, so for our purposes there is no
difference in performance between varchar and text.

Used in:
- `pulp_rpm/plugins/pulp_rpm/plugins/db/models.py`
- `pulp_rpm/plugins/pulp_rpm/plugins/db/fields.py`
- `pulp_ostree/plugins/pulp_ostree/plugins/db/model.py`
- `pulp_docker/plugins/pulp_docker/plugins/models.py`
- `pulp_puppet/pulp_puppet_plugins/pulp_puppet/plugins/db/models.py`
- `pulp/server/pulp/server/db/model/__init__.py`
- `pulp/server/pulp/server/db/fields.py`
- `pulp_python/plugins/pulp_python/plugins/models.py`

CharField
***

https://docs.djangoproject.com/en/1.8/ref/models/fields/#charfield

Represented by a varchar field in postgres, the max_length argument is required.

When the maximum length of a string is known, such as when storing hash values of a known type (or
types), this is the field to use. String length validation is done at the database level.

TextField
***

https://docs.djangoproject.com/en/1.8/ref/models/fields/#textfield

Represented by a text field in postgres.

When the maximum length of a string is unknown, such as when storing large chunks of text like errata
descriptions/summaries, this is the field to use.

IntField
^^^^^^

http://docs.mongoengine.org/apireference.html#mongoengine.fields.IntField

There are more numeric types supported by postgres + Django than are offered by MongoEngine,
so converting from one of these MongoEngine fields to a postgres field should take
the available Django field types into account to ensure that the most appropriate
postgres data type is being used.

https://www.postgresql.org/docs/current/static/datatype-numeric.html

The only known MongoEngine FloatField in Pulp is a timestamp field on the Distribution document,
which could reasonably be converted to a DateTimeField.

Used in:
- `pulp_rpm/plugins/pulp_rpm/plugins/db/models.py`
- `pulp_docker/plugins/pulp_docker/plugins/models.py`
- `pulp/server/pulp/server/db/model/__init__.py`

IntegerField, SmallIntegerField, BigIntegerField
******************************************

https://docs.djangoproject.com/en/1.8/ref/models/fields/#integerfield
https://docs.djangoproject.com/en/1.8/ref/models/fields/#smallintegerfield
https://docs.djangoproject.com/en/1.8/ref/models/fields/#bigintegerfield

2-byte, 4-byte, and 8-byte (respectively) storage for signed integers.

PositiveIntegerField, PositiveSmallIntegerField
*****************************************

https://docs.djangoproject.com/en/1.8/ref/models/fields/#positiveintegerfield
https://docs.djangoproject.com/en/1.8/ref/models/fields/#positivesmallintegerfield

Positive-only variants of SmallIntegerField and IntegerField. These use the
same postgres data types as their non-"Positive" counterparts, but use database
validation to enforce values >= 0.

FloatField
^^^^^^^^

http://docs.mongoengine.org/apireference.html#mongoengine.fields.FloatField

Also numeric types, just like IntField and LongField, but there are some python representation options
when it comes to floats that are available in django fields.

https://www.postgresql.org/docs/current/static/datatype-numeric.html

Used in:
- `pulp_rpm/plugins/pulp_rpm/plugins/db/models.py`

FloatField
****

https://docs.djangoproject.com/en/1.8/ref/models/fields/#floatfield

Stored as the "double precision" data type, using 8 bytes of storage. Represents the python "float"
type.

DecimalField
******

https://docs.djangoproject.com/en/1.8/ref/models/fields/#decimalfield

Stored as the "numeric" data type, storage size varies based on the field precision declared when the
field is created. Very similar to FloatField, but values are represented by the python
"decimal.Decimal" type. Use this field instead of FloatField in cases where the "decimal.Decimal"
type is more appropriate.

For reference: https://docs.python.org/3/library/decimal.html

The postgres docs state that "The actual storage requirement is two bytes for each group of four
decimal digits, plus three to eight bytes overhead," so there's no obvious storage efficiency benefit
the be gained by using this field.

BooleanField
^^^^^^^^^^

http://docs.mongoengine.org/apireference.html#mongoengine.fields.BooleanField

A normal BooleanField, represented a True/False value in python.

https://www.postgresql.org/docs/current/static/datatype-boolean.html

Used in:
- `pulp_rpm/plugins/pulp_rpm/plugins/db/models.py`
- `pulp/server/pulp/server/db/model/__init__.py`

BooleanField, NullBooleanField
************************

Represented by the "boolean" data type in postgres. "BooleanField" stores only True or False,
and cannot be null/None, so a default must be specified. The "NullBooleanField" alternative
additionally allows for null/None values, useful in cases where a boolean value might be
unknown, or not required.

https://docs.djangoproject.com/en/1.8/ref/models/fields/#booleanfield
https://docs.djangoproject.com/en/1.8/ref/models/fields/#nullbooleanfield

DateTimeField, UTCDateTimeField, ISO8601StringField
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

http://docs.mongoengine.org/apireference.html#mongoengine.fields.DateTimeField

All mongoengine DateTimeFields should, at this point, be storing UTC datetime
stamps, represented in python as "datetime.datetime" instances. UTCDateTimeField and
ISO8601StringField are custom fields with special behavior for storage, but
all datetimes should be stored in postgres as postgres's native data type, so the only
Django field type we should be using for all of these mongo fields is DateTimeField.
Custom serialization/deserialization of datetime data should be done at the API layer.

https://www.postgresql.org/docs/current/static/datatype-datetime.html

Used in:
- `pulp_ostree/plugins/pulp_ostree/plugins/db/model.py`
- `pulp/server/pulp/server/db/model/__init__.py`
- `pulp/server/pulp/server/db/fields.py`

DateTimeField
*******

https://docs.djangoproject.com/en/1.8/ref/models/fields/#datetimefield

Represented in postgres as the "timestamp with time zone" data type. Django is configured
to use the UTC timezone, so tz-aware datetime objects will be properly converted to
UTC timestamps when stored, our custom UTCDateTimeField is not required with Django.

DateField, TimeField
^^^^^^^^^^^^^^^^^^

MongoEngine does not provide equivalents for these field types, but they're worth mentioning
in the event that only a date or time component of a datetime object needs to be stored.

https://docs.djangoproject.com/en/1.8/ref/models/fields/#datefield

DateField represents the postgres "date" data type, and is the "datetime.date" type in python.

https://docs.djangoproject.com/en/1.8/ref/models/fields/#timefield

TimeField represents the postgres "time" data type, and is the "datetime.time" type in python.
Unlike DateTimeField, TimeField appears to be unaware of time zones; the column type is
"time with

UUIDField
^^^^^^^

http://docs.mongoengine.org/apireference.html#mongoengine.fields.UUIDField

UUIDs, represented by instances of the "uuid.UUID" data type.

Used in:
- `pulp/server/pulp/server/db/model/__init__.py`

UUIDField
***

https://docs.djangoproject.com/en/1.8/ref/models/fields/#uuidfield

Postgres has native support for UUIDs with the "uuid" data type, storing the value
as the UUID's 128-bit/16-byte value, rather than the UUID string representation.

All models in Pulp 3 also use a UUIDField as their Primary Key by default.

ListField
^^^^^^^

http://docs.mongoengine.org/apireference.html#mongoengine.fields.ListField

In general, elements of ListField arrays should be turned into their own
Django Model, with a ForeignKey relationship back to the Model that originally
contained the ListField.

A sort of case-study regarding converting ListFields to models can be found in the
"ListField Conversion Example" section of this document.

Used in:
- `pulp_rpm/plugins/pulp_rpm/plugins/db/models.py`
- `pulp_docker/plugins/pulp_docker/plugins/models.py`
- `pulp_puppet/pulp_puppet_plugins/pulp_puppet/plugins/db/models.py`
- `pulp/server/pulp/server/db/model/__init__.py`

DictField
^^^^^^^

http://docs.mongoengine.org/apireference.html#mongoengine.fields.DictField

There are many and varied instances of DictFields in Pulp. DictFields can usually
either be reduced to key/value stores, or should (like with ListField) be turned
into Django Models that ForeignKey back to the Model that originally contained the
DictField. For the case of key/value stores, see the "Arbitrary User Data" section
for details on how to handle that case.

Used in:
- `pulp_rpm/plugins/pulp_rpm/plugins/db/models.py`
- `pulp_ostree/plugins/pulp_ostree/plugins/db/model.py`
- `pulp/server/pulp/server/db/model/__init__.py`

UUID Primary Keys
-----------------

Postgres has native support for the UUID datatype, as does Django, making a UUID a viable option
for primary keys. UUIDs are already being used at the de-facto Primary Key of the MongoEngine
ContentUnit. Keeping these UUIDs when migrating to Postgres makes it so that users integrating with
Pulp will be able to keep any references they may have in their own data stores to Pulp ContentUnit
by their existing UUID PK.

Master and Detail ContentUnit Types
-----------------------------------

The "master" ContentUnit model (ContentUnit itself) has some special behaviors added to accomodate
the master-detail inheritance implementation. ContentUnit instance have a `cast` method that will
return a "detail" instance of a ContentUnit type, e.g. the RPM instance for that ContentUnit. Calling
`cast` on a detail instance will return that instance, making `cast` idempotent.

Similarly, all ContentUnits have a `content_unit` property that, when accessed, will always be the
master ContentUnit instance. It functions similarly to `cast`, in that it is idempotent. This is a
property, not a method, because all detail ContentUnit instances are already ContentUnits in an
object-oriented sense, whereas `cast`-ing ContentUnits will most likely result in a database JOIN
operation.

ListField Conversion Example (Errata)
-------------------------------------

In Pulp 2, the Errata model has many ListFields associated with it:
- references, a list of items to which this Errata refers, such as BZ bugs and CVEs
- pkglist, a list of package collections (themselves a list) referred to by this errata

As a result, both "references" and "pkglist" should become their own Model with a corresponding table
in the database with a ForeignKey relationship back to Errata. Furthermore, because the "pkglist"
element in updateinfo.xml can contains package collections, another Model is needed to represent
those package collections, which then has a ForeignKey relationship back to the pkglist that contains
it.

To sum up, the single Pulp 2 Errata model, with its two ListFields, becomes four Django Models:
- Errata

- ErrataReference - Exposed on Errata instances at the "references" attribute
- ErrataCollection - Exposed on Errata instances as the "pkglist" attribute
- ErrataPackage - Exposed on ErrataCollection instances as the "packages" attribute

These models (probably!) meet the requirements for errata:
- Pulp can store all data found in errata updateinfo XML files when syncing repos.
- Pulp can generate equivalent updateinfo XML files when publishing repos.


Core 3.1+ Ideas (post MVP)

Authentication

As an API user, I can have documentation to generate a JSON Web Token (JWT) without the server being online.

As an administrator, I can disable JWT token expiration. This configuration is in the settings file and is system-wide.

As an administrator, I can configure the JWT tokens to expire after a configurable amount of time. This configuration is in the settings file and is system-wide.

The JWT shall have a username identifier

As an API user, I can authenticate any API call with a valid JWT

As a JWT authenticated user, I can refresh my JWT token if Pulp is configured with JWT_ALLOW_REFRESH set to True (default is False)

As an API user, I can invalidate all existing JWT tokens for a given user.

As an authenticated user, when deleting a user 'foo', all of user 'foo's existing JWTs are invalidated.

As an un-authenticated user, I can obtain a JWT token by using a username and password.

Metalink 4.0 support

As a plugin writer I can hand metalink URLs without digest to HttpDownloaders and have them Do The Right Thing™ [3415]

Authorization

<put ideas here>

Versioned Repository

As an authenticated user, I can create a repository version from any repository version

As an authenticated user, I can get a list of a repo's versions.

As an authenticated user, I can specify how many versions to keep at a time.

As an authenticated user, I can get a reference to a new repo version from any task that adds or removes content.

As an authenticated user, I can publish a repo and have it default to the latest version.

As an authenticated user, I can run a publisher with a repository version.

Content Manipulation

As an authenticated user, I can remove one or more units from one or more repositories

As an authenticated user I can specify a filter to identify content to be added to a repo

Content Deletion

As an authenticated user, artifacts are deleted if they were exclusively used by the content unit

As an authenticated user, I can delete multiple content units with filtering

Content Filtering

As a user, I can search all content for a specific content unit regardless of type

As a user, I can find out all the repos in which a piece of content appears

Publications

As an authenticated user, I have filters on the Publication list:

Upload

Repositories 3.1+
filter by content type(ex. repository_contains_type: rpm)
last_content_added(content_added_since)
last_content_removed(content_removed_since)
"partial" repo name search (name: substring)
"tagged" repo names (name: substring)

Importer 3.1+

Importers

Publishers

Task Management

Allow filtering of tasks on 'completed' or 'started'. These 'meta' states are not states directly, but they represent a group of states. For instance 'completed' would be represent 'skipped', 'completed', 'failed', and 'canceled'.

Additional filtering support:

As an authenticated user I can DELETE a task

Data Exports

As an authenticated user, I can export a group of published repositories to a single media

As an authenticated user, I can export an incremental publish of a group of repositories to a single media

For both use cases ^, the layout needs some more discussion.

Also there are two main options in this area.

1. One is a publish bundler that bundles up all the units published to disk. Then this media (e.g. an iso) is mounted and brought into another Pulp installation using a sync. This will only work for content types that don't require 'live APIs'

2. Another option is to export database model data and disk content units from one Pulp to media and then import by directly adding those units to another Pulp. This could be done through the API possibly. This would allow things like Docker to be exported and imported, but it may not work for OSTree??

Also there was discussion about OSTree possibly never supporting and incremental export/import due to how OSTree stores content.

Server Plugins (which content types are available and importers and publishers)

Orphans

As an authenticated user, I can force delete content units even when associated with repositories.
As an authenticated user, I can cleanup orphaned content units for a specific "type" without specifying the units specifically.
As an authenticated user, I can filter orphan cleanup to only remove orphaned content units and artifacts created before a specified datetime.
As an authenticated user, I list all orphaned content units and orphaned artifacts that are not in any repositories

Plugin API

Incremental publishing support

As a plugin writer, I expect units that are missing from the remote repository to not be created in Pulp when using the immediate download policy.

As a plugin writer, I expect units that are missing from the remote repository to be created in Pulp when using background or on_demand download policies.

As a plugin writer, the plugin API provides tooling whereby I can provide the content to be added and removed from the repository. This tooling supports both immediate and deferred downloading.

Consumer Profile Applicability

Using Consumer Profiles and repository bindings I can compute applicability with 2.y parity

Glossary term:
Consumer Profile - A set of installed units on a specific machine. In Pulp3 this machine is not a "consumer" in the same sense as Pulp2. Pulp is not "managing" the machine anymore; Pulp3 only collects Profile information.

Status API

Status API return status of Squid (aka Proxy), web server, streamer

API to view an overall health attribute that returns a message when something is not operating properly or True.

I can view information about unapplied migrations

I can view a verbose Status API which returns a Pulp version for each component along with a list of all plugins and their versions.

Alternate content source support

Deployment

As a user, I can deploy the Pulp content serving view without all of Pulp's requirements.

Plugin Feature Set 3.1+ Ideas (post MVP)

Python

Event Listener Notifier

I can receive serialized task info via AMQP on each task save

Can this be restated in more pedantic terms? Does this mean that an arbitrary host can attach itself to Pulp's AMQP message bus and get updates on the progress of tasks?


Crane 3.2.0 Release Status

The current tentative release date for 3.2.0 is May 30, 2018

Checklist

The first item on the list not struck through is in progress.

The rest of the checklist is conditional on whether or not blockers are found in the beta cycle; this page will be updated as the 3.2.0 release progresses. See the Release Schedule for more details.

If all goes well, and no blocking bugs delay further release stages, this release should become Generally Available May 30, 2018. This page will be updated as progress is made.


Air Ticket Cancellation Policy For Singapore Airlines

There are some circumstances around which one has to cancel the flights. For that situation, one must have knowledge about the cancellation policy of that particular airline. In the case of Singapore airlines, the cancellations of flight without any charges are allowed within 24 hours of booking. If the ticket of Singapore airline id canceled with the 24 hours of purchase then no cancellation fee will be charged on the passenger by Singapore airlines.

In addition to this, the passenger can cancel their flight if the flight is delay by three or four hours. In that case, Singapore airlines reservations will allow you to cancel the flight without making any changes from the passengers. The candidates who have booked their tickets from the official website of Singapore airlines are eligible for online cancellation. If the passenger booked the tickets from any traveling agents or any other online reservation website then they have to contact them for their refunds.

Cancellation Charges for Singapore Airlines Flights

If the passenger wants to cancel the ticket, then they need to know that the cancellation fees will be based on the class of ticket. The passenger will have to pay 75 dollars if the category of the ticket is saver fares. The charges for advantage fare are 50 dollars. As the promo tickets are so cheap, they are non-refundable. The compensation given the airlines will depend on the mode of payment. Mostly the passenger gets the refund back within one week to three weeks.

How to cancel the Singapore Airlines ticket?

After confirmation, you will receive an email from Singapore airlines about your flight cancellation confirmation. For more queries related to Singapore airlines booking, the passenger can call on helpline numbers from the official websites. These numbers are available for 24 hours. The helpline executives will surely help you by justifying your queries related to any type of booking or cancellation of Singapore Airlines flights.


American Airlines Reservations
Got to see amazing content at your site! I must say that you provided the much needed information that I was looking for so long. Well hi! I work in American Airlines, and if you’re planning a trip somewhere, feel free to contact us at
American Airlines Reservations Number. We have great offers and deals going on lately. Must check them out!


NOTE: Current a draft. See examples from other projects here: community.redhat.com/brandguide/

Key Message

Pulp helps you fetch, upload, organize, and distribute software packages.

Visual Style

Logo

With words:

SVG version

Without words:

SVG version:

Black and white:

SVG black on white version
SVG white on black version

Colors

Logo colors: PMS 368 C, PMS 660 C, PMS 144 C

Elevator Pitches

25 Words

Pulp fetches, uploads, organizes, and distributes software packages. For example a Python package. Mirror and mix software repositories, test them, and then go into production.

50 Words

Pulp fetches, uploads, organizes, and distributes software packages. For example a Python or RPM package. Pulp downloads remote packages either immediately, as-needed, or upload your own packages. Then mirror, mix, or organize them. Use Pulp to publish the repository of packages for use by pip, yum/dnf, etc.

100 Words

Pulp fetches, uploads, organizes, and distributes software packages. For example a Python or RPM package. Pulp downloads remote packages either immediately, as-needed, or upload your own packages. Then mirror, mix, or organize them. Use Pulp to publish the repository of packages for use by pip, yum/dnf, etc.

For system administrators, being able to sync packages to test machines before rolling all updates out to consumers increases stability and confidence. Also storing older versions and rolling back changes is easier.

For developers, Pulp can support a continuous integration pipeline that test and versions software against ever-changing dependency stacks.

Print Materials

A Pulp-themed bookmark template is available that can be customized and printed. It is attached to this page.


Bug Trends Report 2016-02-10

Report from 02/04-02/10

*Summary *

Total Issues Filed : 8

Master

Total Issues Filed: 0

Pulp 2.12

Total Issues: 3

Pulp 2.11

Total Issues Filed : 0

Pulp 2.10

Total Issues Filed : 0

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 3

Pulp 3.0

Total Issues Filed : 0

Other areas/plugins breakup

Documentation :2

Summary

3 issues filed this week were on 2.11 . 3 issue were against 2.8 and 2 documentation issues

The following smash issue has been filed.
https://github.com/PulpQE/pulp-smash/issues/550
https://github.com/PulpQE/pulp-smash/issues/551
https://github.com/PulpQE/pulp-smash/issues/552
https://github.com/PulpQE/pulp-smash/issues/553


Bug Trends Report 2016-08-26

Friday Aug 26 EDT 2016

Item Summary

Pulp 2.10

Total Issues Filed : 9

Break down as per feature area

Installation fails on Fedora 24 when installing the OSTree packages

rsync distributor (new feature) :4 Blocker 1
rsync doc improvement : 1

Related to syncing uploaded content

Units test failure on el6

Pulp 2.9.z

Pulp 2.8.z

Total Issues Filed: 2

Summary

A total of 11 issues were filed during the week of 8/22-8/26. 9 were against 2.10 beta. The new feature rsync introduced with 2.10 has the 5 issues filed against it and 1 of them (Issue#2199) is currently blocking 2.10. Installation is being blocked on Fedora24 due to Issue#2203.

2 bugs were filed against the 2.8 stream. 1 is identified as a qpid related issue and a second one on missing release notes to 2 previous 2.8 releases. (2.8.6 & 2.8.7)

An issue was also filed against pulpproject.org usability


Bug Trends Report 2016-09-02

Friday September 2 EDT 2016

Item Summary

Pulp 2.10

Total Issues Filed : 4

Break down as per feature area

*Documentation issue

Pulp 2.9.z

Pulp 2.8.z

Total Issues Filed: 1

Pulp 2.11

Summary

A total of 6 issues were filed during the week of 8/27-9/2. 4 were against 2.10 beta. The new feature rsync introduced with 2.10 has the 1 issues filed against it. This issue has been automated in pulp smash. Proxy issues have always been recurring in pulp releases. We will have to prioritize and automate proxy issues to avoid future breakage of pulp plugins with proxy.

1 bug was filed against the 2.8 stream. Looks like this is a sync schedule issue that was originally filed against 2.4 . https://bugzilla.redhat.com/show_bug.cgi?id=1218785

1 bug is filed against master (2.11) on failure to sync from mirrorlist . Pulp smash issue will be filed to automate this in the future. Also will be added to the manual test until automation is completed on this issue.


Bug Trends Report 2016-09-09

Friday September 9 EDT 2016

Item Summary

Pulp 2.10

Total Issues Filed : 3

Break down as per feature area

Docker -2
OSTree-1

Pulp 2.9.2

Upgrade RPM - 2

Pulp 2.8.z

None

Master

Nectar :1

Summary

A total of 6 issues were filed during the week of 9/4 - 9/10. 2 issues were against Docker and 1 OSTree. OSTree issues are trickling in with some basic functionalities. So we would need to take a look at the OSTree and Docker plugin automated tests to see if we can improve coverage.

2 bugs were filed against the 2.9.2 upgrade. Both issues are cases that are not covered in our automation upgrade script. These cases will need to be included in the upgrade automation as well to avoid future incidents.


Bug Trends Report 2016-09-16

Friday, September 16 EDT 2016

Item Summary

Pulp 2.10

Total Issues Filed : 5

Break down as per feature area

rpm -3

Publish Issue -1
Unit Association -1
Package Signature-1

OSTree-2 Documentation

Pulp 2.9.2

None

Pulp 2.8.z

Total Issues Filed : 2

Issue with metadata copy -1
How pulp handles broken qpid -1

Summary

A total of 7 issues were filed during the week of 9/11 - 9/16. 3 rpm plugin related issues were filed for 2.10. 2 OSTree issues documentation. These were filed during an effort improve automation coverage for OSTree plugin.

2 Issues were filed in 2.8.z.

The rpm plugin issues are related to repo publish, unit association and copy. We need to spend some time to look at automation tests to make sure that these cases are included in the tests.


Bug Trends Report 2016-09-23

*Summary
*
Total Issues Filed : 7

Pulp 2.10

Total Issues Filed : 2

Breakdown as per feature area

Rpm plugin

drpm -1
Sync -1

Pulp 2.9.2

Total Issues Filed : 1

Iso - 1

Pulp 2.8.z

Total Issues Filed : 2

Upgrade from 2.7 to 2.8
Kicskart with on demand with --feed file

Master

Total Issues Filed : 2

Selinux -1
Lazy Sync -1

Summary

A total of 7 issues were confirmed during the week of 9/17 - 9/23. 2 issues were against master (2.10.1). And they were filed due to pulp-smash test failures.

2 issues were against 2.10

One of the issues is repo sync failing for a Suse repo. This is the kind case that was referred here

https://github.com/PulpQE/pulp-smash/issues/33#issuecomment-154501820

1 Bug was filed against Iso upload in 2.9

2 bugs were filed against the 2.8 . 1 related to upgrade from 2.7 to 2.8. A second issue is related to kickstart with repo synced using --feed file://and on_demand syncs. We currently do not have smash tests with --feed file// . pulp-smash issues to keep track of these cases will be filed.


Bug Trends Report 2016-09-30

*Summary
*
Total Issues Filed : 4

Pulp 2.10

Total Issues Filed : 3

Breakdown as per feature area

upgrade from 2.7 ->2.10 1
Installation 1

Pulp 2.9.2

Total Issues Filed 1

Upgrade 2.7 to 2.9 1

Pulp 2.8.z

Total Issues Filed : 1

Upgrade from 2.7 to 2.8 1

Master

None

Summary

The issue trend this week shows 3 centos installtion/upgrade issues. We will have to explore ways to add centos to the test matrix on Jenkins to catch similar issues in the future.


Bug Trends Report 2016-10-07

*Summary
*
Total Issues Filed : 4

Pulp 2.11 - Master

Total Issues Filed :3
Breakdown as per area
Mirrorlist-1
publish fail due to selinux issue -1
docker-1

Pulp 2.10

Total Issues Filed : 0

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 0

Pulp 3.0

Total Issues Filed : 1

Summary

A total of 4 issues were confirmed during the week of 10/1 - 10/7 . 3 issues were against master (2.11). And 2 were filed due to pulp-smash test failures. The mirrlorlist automation that is one of the most recent automation is already identified an urgent issue.

Also selinux issues have been identified int he recent weeks. This has been raised to the development team

1 issue was against 3.0


Bug Trends Report 2016-10-14

*Summary
*
Total Issues Filed : 2

Pulp 2.11 - Master

Total Issues Filed : 0

Pulp 2.10

Total Issues Filed : 0

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 2

Pulp 3.0

Total Issues Filed : 0

Summary

There were only 2 issues filed during this week. Both were against 2.8.7 and affecting satellite. One is about pulp error handling and 2nd one high priority although reported against 2.8.7 is affecting 2.10 and master. This is regarding sync reporting updated when there are no changes. This issue has already been automated with the PR https://github.com/PulpQE/pulp-smash/pull/413


Bug Trends Report 2016-10-28

Combined Report from 10/21 & 10/28

*Summary
*
Total Issues Filed : 13

Master

Total Issues Filed: 0

Pulp 2.11

Total Issues Filed : 4

Pulp 2.10

Total Issues Filed : 2

Pulp 2.9.2

Total Issues Filed : 2

Pulp 2.8.z

Total Issues Filed : 5

Pulp 3.0

Total Issues Filed : 0

Summary

The issues filed the last 2 weeks include 5 on 2.8 affecting satellite. 2 of which are lazy sync related 2. 1 on search , 1 on applicability and 1 on rpm upload. 2 issues filed on 2.9 are on repoview feature. On 2.10 we have 1 on importer update and 1 on forcefull sync issues . The 4 issues on 2.11 were, 1 httpd failing to start if the optional AMQP bus for consumers is unavailable, 1 the status reporting of wrong AMQP message bus, and 1 on python egg upload and 1 on sync silently failing without error.

Lazy sync issues affecting satellite are already part of the automation backlog. The others need to be looked at automation issues need to be added. We need to revisit the repo upload tests to include the missing scenarios. Also the sync error handling scenarios need to be added.


Bug Trends Report 2016-11-04

*Summary
*
Total Issues Filed : 4

Master

Total Issues Filed: 0

Pulp 2.11

Total Issues Filed : 0

Pulp 2.10

Total Issues Filed : 4

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 0

Pulp 3.0

Total Issues Filed : 0

Summary

All the issues filed this week were on 2.10. One was a migration issue that will be fixed in 2.11. All the issues were rpm plugin issues.

We will be looking at the automation tests to include these scenarios in the rpm automation suites. (skip cases, repo update etc)


Bug Trends Report 2016-11-11

*Summary
*
Total Issues Filed : 4

Master

Total Issues Filed: 0

Pulp 2.11

Total Issues Filed : 0

Pulp 2.10

Total Issues Filed : 3

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 0

Pulp 3.0

Total Issues Filed : 0

Summary

All the issues filed this week were on 2.10. 1 was a python plugin related issue. 1 with incremental publish, And 1 --skip requiring export_distributor.

We will be looking at the automation tests to include these scenarios in the rpm automation suites. (skip cases, repo update etc). --skip issue is the 2nd of its kind getting reported in the last 2 weeks.


Bug Trends Report 2016-11-18

*Summary
*
Total Issues Filed : 4

Master

Total Issues Filed: 0

Pulp 2.11

Total Issues Filed : 0

Pulp 2.10

Total Issues Filed : 3

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 0

Pulp 3.0

Total Issues Filed : 1

Summary

3 issues filed this week were on 2.10. 1 with upgrade from 2.7, 1 documentation and 1 SELinux related to RHEL 7.3


Bug Trends Report 2016-11-28

*Summary
*
Total Issues Filed : 4

Master

Total Issues Filed: 0

Pulp 2.11

Total Issues Filed : 0

Pulp 2.10

Total Issues Filed : 2

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 2

Pulp 3.0

Total Issues Filed : 1

Summary

2 issues filed this week were on 2.10.2 and SELinux related. https://github.com/PulpQE/pulp-smash/issues/442 has been filed to automate.

1 of the issue on 2.8 is on documentation and the 2nd one on re-uploading the same file multiple times resulting in discrepancy. This case will need to be added to pulp-smash.


Bug Trends Report 2016-12-09

Combined report for 12/3/16 and 12/09/16

*Summary
*
Total Issues Filed : 7

Master

Total Issues Filed: 0

Pulp 2.11

Total Issues Filed : 0

Pulp 2.10

Total Issues Filed : 4

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 2

Pulp 3.0

Total Issues Filed : 2

Summary

2 issues filed this week were on 2.10.2 and SELinux related. Multiple SELinux related issues have been added including https://github.com/PulpQE/pulp-smash/issues/442. Also a test has been merged to check selinux issue. There is also plans to add more detailed SELinux tests after 2.11 GA. 1 issue for spec file and 1 for error reporting with lazy sync has been filed as well.

2 issues were on 2.8. 1 is docker unassociate. Pulp-smash issue has been added for that. The 2nd issue is for reassociating already associated units. Will add that to the smash issue.


Bug Trends Report 2016-12-22

*Summary
*
Total Issues Filed : 6

Master

Total Issues Filed: 0

Pulp 2.11

Total Issues Filed : 0

Pulp 2.10

Total Issues Filed : 5

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 1

Pulp 3.0

Total Issues Filed : 0

Summary

5 issues filed this week were on 2.10.2 . 1 related to service showing wrong status, 1 on memory leak . 1 applicability issue was filed.1 is migrations issue on upgrading from 2.7. And 1 issue related to package group id discrepancy in 2.10

1 issue was filed on 2.8 with error message.

We will have to go through each of these issues to determine if it can be automated and issues will be filed accordingly.


Bug Trends Report 2017-01-13

*Summary *

Total Issues Filed : 7

Master

Total Issues Filed: 2

Pulp 2.11

Total Issues Filed : 3

Pulp 2.10

Total Issues Filed : 1

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 1

Pulp 3.0

Total Issues Filed : 0

Summary

3 issues filed this week were on 2.11 . 2 issue were against master which is about error message getting swallowed.

The following smash issue has been filed to handle 1 of the issues. QE will file more smash issues for issues that can be automated.

https://github.com/PulpQE/pulp-smash/issues/477


Bug Trends Report 2017-02-03

Report from 01/14-02/03/

*Summary *

Total Issues Filed : 18

Master

Total Issues Filed: 3

Pulp 2.11

Total Issues Filed : 4

Pulp 2.10

Total Issues Filed : 2

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 2

Pulp 3.0

Total Issues Filed : 0

Other areas/plugins breakup

Documentation :2
Pulp-admin help :1
python plugin- 2
OSTree- 2

Summary

3 issues filed this week were on 2.11 . 2 issue were against master which is about error message getting swallowed.

The following smash issue has been filed.

https://github.com/PulpQE/pulp-smash/issues/526
https://github.com/PulpQE/pulp-smash/issues/522
https://github.com/PulpQE/pulp-smash/issues/539
https://github.com/PulpQE/pulp-smash/issues/540
https://github.com/PulpQE/pulp-smash/issues/542
https://github.com/PulpQE/pulp-smash/issues/543
https://github.com/PulpQE/pulp-smash/issues/544


Bug Trends Report 2017-02-10

Report from 02/04-02/10

*Summary *

Total Issues Filed : 8

Master

Total Issues Filed: 0

Pulp 2.12

Total Issues: 3

Pulp 2.11

Total Issues Filed : 0

Pulp 2.10

Total Issues Filed : 0

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 3

Pulp 3.0

Total Issues Filed : 0

Other areas/plugins breakup

Documentation :2

Summary

3 issues filed this week were on 2.11 . 3 issue were against 2.8 and 2 documentation issues

The following smash issue has been filed.
https://github.com/PulpQE/pulp-smash/issues/550
https://github.com/PulpQE/pulp-smash/issues/551
https://github.com/PulpQE/pulp-smash/issues/552
https://github.com/PulpQE/pulp-smash/issues/553


Bug Trends Report 2017-02-17

*Summary *

Total Issues Filed : 6

Master

Total Issues Filed: 2

Pulp 2.12.1

Total Issues: 1
lazy sync

Pulp 2.12

Total Issues: 0

Pulp 2.11

Total Issues Filed : 0

Pulp 2.10

Total Issues Filed : 1

Pulp 2.9

Total Issues Filed : 1

Pulp 2.8.z

Total Issues Filed : 0

Pulp 3.0

Total Issues Filed : 0

Other areas/plugins breakup

Documentation: 1

Summary

2 issues filed this week were on 2.11 lazy sync related( 2.10 & 2.12). 2 issues were on master (2.13). There is one issue on Pulp workers/beat/resource_manager go missing. This needs to investigated and to file a smash issue

The following smash issue has been filed.
https://github.com/PulpQE/pulp-smash/issues/560
https://github.com/PulpQE/pulp-smash/issues/559


Bug Trends Report 2017-02-24

*Summary *

Total Issues Filed : 4

Master

Total Issues Filed: 0

Pulp 2.12

Total Issues Filed : 3

Pulp 2.11

Total Issues Filed : 0

Pulp 2.10

Total Issues Filed : 1

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 0

Pulp 3.0

Total Issues Filed : 0

Summary

3 issues filed this week were on 2.12. 1 was a release engineering issue which is already fixed.

The following smash issues have been filed

https://github.com/PulpQE/pulp-smash/issues/566
https://github.com/PulpQE/pulp-smash/issues/567
https://github.com/PulpQE/pulp-smash/issues/568


Bug Trends Report 2017-03-03

*Summary *

Total Issues Filed : 5

Master

Total Issues Filed: 2

Pulp 2.12

Total Issues Filed : 2

Pulp 2.11

Total Issues Filed : 0

Pulp 2.10

Total Issues Filed : 1

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 0

Pulp 3.0

Total Issues Filed : 0

Summary

2 issues filed this week were on 2.12 which are edge cases for features that are already implemented. 2 issues are on master relating to Celery4+Kombu4. This does not affect pulp now but will in the future (F26) or when the environment is manually installed. 1 issue on 2.10.

The following smash issues have been filed/updated

https://github.com/PulpQE/pulp-smash/issues/572
https://github.com/PulpQE/pulp-smash/issues/573
https://github.com/PulpQE/pulp-smash/issues/521 -updated

2 issues already have automation in place

python3 -m unittest pulp_smash.tests.rpm.api_v2.test_schedule_publish.ScheduledPublishTestCase.test_total_run_count
python3 -m unittest pulp_smash.tests.rpm.api_v2.test_broker.BrokerTestCase.test_broker_reconnect


Bug Trends Report 2017-03-10

*Summary *

Total Issues Filed : 7

Master

Total Issues Filed: 0

Pulp 2.12

Total Issues Filed : 7

Pulp 2.11

Total Issues Filed : 0

Pulp 2.10

Total Issues Filed : 0

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 0

Pulp 3.0

Total Issues Filed : 0

Summary

The issues filed were related to the following areas in 2.12

search regression
sync fails with non-ASCII
Errata publish performance degradation
Puppet sync error message
Max-speed parameter not working
Drpm upload with --checksum-type option

The following smash issues have been filed/updated

https://github.com/PulpQE/pulp-smash/issues/581
https://github.com/PulpQE/pulp-smash/issues/583
https://github.com/PulpQE/pulp-smash/issues/584
https://github.com/PulpQE/pulp-smash/issues/585


Bug Trends Report 2017-03-24

*Summary *

Total Issues Filed :7

Master

Total Issues Filed: 0

Pulp 2.12

Total Issues Filed : 7

Pulp 2.11

Total Issues Filed : 0

Pulp 2.10

Total Issues Filed : 0

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 0

Pulp 3.0

Total Issues Filed : 0

Summary

The issues filed were related to the following areas in 2.12

Docker
iso rsync distributor
Iso unit association
repo create with auth
plugin upgrade affection 2.12& 2.13 in fedora
rpm repo

The following smash issues have been filed/updated
https://github.com/PulpQE/pulp-smash/issues/601
https://github.com/PulpQE/pulp-smash/issues/602
https://github.com/PulpQE/pulp-smash/issues/603
https://github.com/PulpQE/pulp-smash/issues/604
https://github.com/PulpQE/pulp-smash/issues/605


Bug Trends Report 2017-04-11

*Summary *

Total Issues Filed :14

Master

Total Issues Filed: 0

Pulp 2.12

Total Issues Filed : 14

Pulp 2.11

Total Issues Filed : 0

Pulp 2.10

Total Issues Filed : 0

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 0

Pulp 3.0

Total Issues Filed : 0

Summary

The issues filed were related to the following areas in 2.12

Crane
Python
rsync distributor
Streamer
Documentation
rpm repo
Task cancelling

The following smash issues have been filed/updated
https://github.com/PulpQE/pulp-smash/issues/550
https://github.com/PulpQE/pulp-smash/issues/613
https://github.com/PulpQE/pulp-smash/issues/611
https://github.com/PulpQE/pulp-smash/issues/615


Bug Trends Report between 2017-04012 and 2017-05-06

*Summary *

Total Issues Filed :25

Master

Total Issues Filed: 0

Pulp 2.12

Total Issues Filed : 22

Pulp 2.11

Total Issues Filed : 0

Pulp 2.10

Total Issues Filed : 0

Pulp 2.9.2

Total Issues Filed : 0

Pulp 2.8.z

Total Issues Filed : 2

Pulp 3.0

Total Issues Filed : 1

Summary

The issues filed were related to the following areas
platform -6
Crane -2
Docker -1
rsync distributor -1
iso -2
Documentation -3
rpm repo -7
streamer-3

https://github.com/PulpQE/pulp-smash/issues/638
https://github.com/PulpQE/pulp-smash/issues/637
https://github.com/PulpQE/pulp-smash/issues/636
https://github.com/PulpQE/pulp-smash/issues/635
https://github.com/PulpQE/pulp-smash/issues/634
https://github.com/PulpQE/pulp-smash/issues/633
https://github.com/PulpQE/pulp-smash/issues/632


Bug Trends Report 2017-05-31

*Summary *

Total Issues Filed :20

Pulp 3

Total Issues :3

Master

Total Issues Filed: 0

Pulp 2.13

Total Issues :5

Pulp 2.12

Total Issues Filed: 3

Pulp 2.11

Total Issues Filed: 1

Pulp 2.10

Total Issues Filed : 0

Pulp 2.9.2

Total Issues Filed : 1

Pulp 2.8.z

Total Issues Filed : 1

Summary

The issues filed were related to the following areas
platform -4
Crane -1
iso -2
Documentation -1
rpm repo -2
RCM -2

Pulp-smash issues

https://github.com/PulpQE/pulp-smash/issues/662
https://github.com/PulpQE/pulp-smash/issues/663
https://github.com/PulpQE/pulp-smash/issues/664
https://github.com/PulpQE/pulp-smash/issues/667


Bug Trends Report 2017-06-13

Total Bugs filed: 17

Pulp 3.0: 6

Pulp 2.14: 1

Pulp 2.13: 7

Pulp 2.12.1 :1

pulp-manage-db: 1
rpm :3
rcm :2
Doc 4
applicability: 1

Pulp-Smash issues filed

https://github.com/PulpQE/pulp-smash/issues/679
https://github.com/PulpQE/pulp-smash/issues/680
https://github.com/PulpQE/pulp-smash/issues/681
https://github.com/PulpQE/pulp-smash/issues/682
https://github.com/PulpQE/pulp-smash/issues/683
https://github.com/PulpQE/pulp-smash/issues/684


Bug Trends Report 2017-06-23

*Total Issues: 7
*
Pulp 3 : 2

Pulp 2.13: 3

Base auth issue :1
Documentation 1
Iso sync issue for RHEL6.7

Pulp 2.11 :1

Suse repo sync

Pulp 2.8 : 1

*pulp-smash issues
*
https://github.com/PulpQE/pulp-smash/issues/688


Bug Trends Report 2017-06-30

*Summary *

Total Issues Filed :6

Pulp 3

Total Issues :0

Master

Total Issues Filed: 0

Pulp 2.13

Total Issues : 6

Pulp 2.12

Total Issues Filed:

Pulp 2.11

Total Issues Filed:

Pulp 2.10

Total Issues Filed : 0

Pulp 2.9.2

Total Issues Filed :

Pulp 2.8.z

Total Issues Filed :

Summary

The issues filed were related to the following areas
platform -5
Docker-1

Pulp-smash issues
https://github.com/PulpQE/pulp-smash/issues/701
https://github.com/PulpQE/pulp-smash/issues/695
https://github.com/PulpQE/pulp-smash/issues/694


Bug Trends Report 2017-07-14

*Summary *

Total Issues Filed :7

Pulp 3

Total Issues :1

Master

Total Issues Filed: 0

Pulp 2.13

Total Issues : 1

Pulp 2.12

Total Issues Filed:

Pulp 2.11

Total Issues Filed:

Pulp 2.10

Total Issues Filed : 0

Pulp 2.9.2

Total Issues Filed :

Pulp 2.8.z

Total Issues Filed : 2

Summary

The issues filed were related to the following areas
platform -5
Documentation:3
Consumer/agent :1
Applicability : 1

https://github.com/PulpQE/pulp-smash/issues/714
https://github.com/PulpQE/pulp-smash/issues/715


Bug Trends Report 2017-07-24

*Summary *

Total Issues Filed :7

Pulp 3

Total Issues :4

2.14

Total Issues Filed: 2

Pulp 2.13

Total Issues : 0

Pulp 2.12

Total Issues Filed:

Pulp 2.11

Total Issues Filed:

Pulp 2.10

Total Issues Filed : 0

Pulp 2.9.2

Total Issues Filed :

Pulp 2.8.z

Total Issues Filed : 0

Summary

The issues filed were related to the following areas

Crane: 1
Docker:1
Packaging: 1
Pulp3: 4

The 2 docker/crane issues have been added the docker pulp-smash issue
https://github.com/PulpQE/pulp-smash/issues/713


Bug Trends Report 2017-07-28

*Summary *

Total Issues Filed :3

Pulp 3

Total Issues :2

2.14

Total Issues Filed: 0

Pulp 2.13

Total Issues : 1

Pulp 2.12

Total Issues Filed:

Pulp 2.11

Total Issues Filed:

Pulp 2.10

Total Issues Filed : 0

Pulp 2.9.2

Total Issues Filed :

Pulp 2.8.z

Total Issues Filed : 0

Summary

The issues filed were related to the following areas

Tasking: 1


Bug Trends Report 2017-08-04

*Summary *

Total Issues Filed :4

Pulp 3

Total Issues:1

2.14

Total Issues Filed: 0

Pulp 2.13

Total Issues : 3

Pulp 2.12

Total Issues Filed:

Pulp 2.11

Total Issues Filed:

Pulp 2.10

Total Issues Filed : 0

Pulp 2.9.2

Total Issues Filed :

Pulp 2.8.z

Total Issues Filed : 0

Summary

The issues filed were related to the following areas

User creation:1
Celery :1
Node Sync :1

*Smash Issues *
https://github.com/PulpQE/pulp-smash/issues/695
https://github.com/PulpQE/pulp-smash/issues/746


Bug Trends Report 2017-08-11

*Summary *

Total Issues Filed :8

Pulp 3

Total Issues:3

2.14

Total Issues Filed: 3

Pulp 2.13

Total Issues :

Pulp 2.12

Total Issues Filed:

Pulp 2.11

Total Issues Filed:

Pulp 2.10

Total Issues Filed : 0

Pulp 2.9.2

Total Issues Filed :

Pulp 2.8.z

Total Issues Filed : 2

Summary

The issues filed were related to the following areas

Tasking:1
Unicode error :1
Fedora 26 install:1
Debian plugin:1
python plugin:1

No pulp-smash issues were filed for these issues as they are not automation candidates.


Bug Trends Report 2017-08-18

*Summary *

Total Issues Filed :4

Pulp 3

Total Issues:2

2.14

Total Issues Filed: 2

Pulp 2.13

Total Issues :0

Pulp 2.12

Total Issues Filed:

Pulp 2.11

Total Issues Filed:

Pulp 2.10

Total Issues Filed : 0

Pulp 2.9.2

Total Issues Filed :

Pulp 2.8.z

Total Issues Filed : 2

Summary

The issues filed were related to the following areas

Replica Set: 1
Docker plugin:1

No pulp-smash issues were filed for these issues as they are not automation candidates.


Bug Trends Report 2017-08-25

*Summary *

Total Issues Filed :2

Pulp 3

Total Issues:1

2.14

Total Issues Filed: 1

Pulp 2.13

Total Issues :0

Pulp 2.12

Total Issues Filed:

Pulp 2.11

Total Issues Filed:

Pulp 2.10

Total Issues Filed : 0

Pulp 2.9.2

Total Issues Filed :

Pulp 2.8.z

Total Issues Filed : 0

Summary

The issues filed were related to the following areas

Tasking & Celery 4: 1

No pulp-smash issues were filed for these issues as they are not automation candidates.


Bug Trends Report 2017-09-01

*Summary *

Total Issues Filed :2

Pulp 2.13

Total Issues :1

Pulp 2.8.z

Total Issues Filed : 1

Summary

The issues filed were related to the following areas

qpid related:1

errata update:1

No new pulp-smash issues were filed based on bugs reported this week.


Bug Trends Report 2017-09-08

Total Bugs : 3

Pulp3 : 1

Pulp 2.14: 1

Release Engineering:1

Summary

The issues filed were related to the following areas

Task Cancel:1

No new pulp-smash issues were filed based on bugs reported this week.


Bug Trends Report 2017-09-15

Total Bugs: 4

Pulp3

Total : 2

Pulp 2.13

Total: 2

Summary

There were a total of 4 issues filed this week. 2 were in Pulp3 code base. 1 issue in 2.13 was due to docker sync failure related to changes in 2.14 and 2nd issue was filed on recursive copy of errata.

1 github issue was filed

https://github.com/PulpQE/pulp-smash/issues/769


Bug Trends Report 2017-09-22

Total Bugs: 13

Pulp 3.0

Total: 10

Pulp 2.13

Total :1

Sync related issue for a specific feed (Cent OS

2.10

Total:1

Warnings in mongo logs after moving to Mongo 3.4

2.8

Total:1

Issue related to how Pulp defines uniqueness among different "distribution" units. This is a data model change which is identified as a pulp3 candidate.

Summary

Mojority of the issues were filed in Pulp3 space. No new pulp-smash issues were filed based on these issues.


Bug Trends Report 2017-10-22 - 2017-10-09

Total Issues: 14

Pulp3

Total: 6

Pulp 2:14

Total: 5

Proxy related: 1
Iso repo issue:1
scheduled tasks:1
Uninstalling:1
F26:1
Pulp 2.13

Total: 1

Pulp 2.8+

Total :2

Orphan remove issues:2

https://github.com/PulpQE/pulp-smash/issues/784
https://github.com/PulpQE/pulp-smash/issues/783


Bug Trends Report 2017-10-13

Total Bugs: 3

Pulp3

Total : 2

Documentation: 1

Summary

There were a total of 3 issues filed this week. 2 were in Pulp3 code base. 1 issue was documentation related.

No github issues were filed.


Bug Trends Report 2017-10-20

Total Bugs: 3

Pulp3

Total : 2

Pulp 2.8+

Total: 2

Summary

There were a total of 3 issues filed this week. 2 were in Pulp3 code base. 1 issue was in 2.8+
No new github issues were filed.


Bug Trends Report 2017-10-27

Total Bugs: 7

Pulp3

Total : 3

Pulp 2.14

Total:4

Docker:1
Upload:1
Iso:
Consumer/Tasks:1

Summary

There were a total of 7 issues filed this week. were in Pulp3 code base. 4 issues were in 2.14. No github issues were filed. https://pulp.plan.io/issues/3090 was a regression and has an automation issue associated with it.

https://pulp.plan.io/issues/3090


Bug Trends Report 2017-11-03

Total Issues: 2

Pulp 2.14: 2

Documentation :1
Crane :1

There are no new pulp-smash issues filed on these issues. These issues were not customer facing.

https://pulp.plan.io/issues/3115 - Documention
https://pulp.plan.io/issues/3111 - Crane issue


Bug Trends Report 2017-11-09

Total Bugs Reported : 6

Pulp3 : 2

Pulp Master/ 2.15: 1

Docker: https://pulp.plan.io/issues/3122

Pup2.14: 3

Documentation: https://pulp.plan.io/issues/3115
Parallel Sync with smart proxy : https://pulp.plan.io/issues/3120 Satellite issue already fixed
Docker repo create issue: https://pulp.plan.io/issues/3118
Satellite Issue

2.15/ Master issue is a docker issue in the master build (3.1) and is related to skopeo.

2.14 1 issue is documentation. 1 is a satellite issue related to syncing and 1 issue is on docker.


Bug Trends Report 2017-11-21

Total Bugs: 10

Pulp3: 3
https://pulp.plan.io/issues/3138: Broken Sync
https://pulp.plan.io/issues/3134: qpid install
https://pulp.plan.io/issues/3145: strange error when feed_url is not a valid manifest

Pulp 2-Master:1
https://pulp.plan.io/issues/3127: Docker

Pulp2.14: 2

https://pulp.plan.io/issues/3139: Sync from crane broken
https://pulp.plan.io/issues/3140: pulp_streamer doesn't set HTTP response headers

Pulp 2.13: 3

https://pulp.plan.io/issues/3135: Workers missing
https://pulp.plan.io/issues/3130: Upgrade from Pulp 2.13.2- to Pulp 2.13.3+ can result in "duplicate key error index

Pulp 2.8+: 2

https://pulp.plan.io/issues/3129- http segfault
https://pulp.plan.io/issues/3133: applicability

Summary

There were a total of 10 issues filed for this period. 3 were in Pulp3 code base. 2 issues in 2.13 2, 2 in 2.14 and 2 in 2.8+ They are edge cases and need to be investigated further to see if they can be automated.


Bug Trends Report 2017-12-08

Total Bugs: 6

Pulp3

Total : 2

Pulp 2.14+
https://pulp.plan.io/issues/3161 - CentOs issue
https://pulp.plan.io/issues/3162 - Logging related/Error handling
https://pulp.plan.io/issues/3165 - Docker - https://pulp.plan.io/issues/3165

Total:3

Pulp 2.8+

https://pulp.plan.io/issues/3172 - Celery worker consumes large number of memory

Total: 1

Summary

2 issues on pulp3 were filed. 3 on Pulp 2.14 and 1 on 2.18+. 1 issue was filed for automation.

1 github issue was filed.

https://github.com/PulpQE/pulp-smash/issues/822


Bug Trends Report 2017-12-15

Total Bugs: 1

Pulp 2.14+

Pulp-docker :1

https://pulp.plan.io/issues/3208- Error uploading v1 docker image to repository.

There was only 1 issue filed during this week. 1 github issue has been filed for automation.

https://github.com/PulpQE/pulp-smash/issues/828


Bug Trends Report 2018-01-05

Week 2017-12-15 to 2018-01-05

Total Bugs: 15

Pulp3

Total: 5

Pulp 2.13+

Total: 7

Importer
importer config update fails with traceback - 3210
Docker:5
copy tag issue not preserve user_metadata - 3242
docker_distributor_web not supporting the unit types docker_blob, docker_manifest, docker_tag - 3241
return unit_key data for the manifest that was uploaded in order to facilitate further processing - 3232
REST API should let one upload layers/blobs individually, and then a manifest.json file to tie everything together. 3231
Rewrite redirect URL
3227

ISO
ISO sync fills up /var/cache/pulp with isos - 3229

Documentation: 3

Summary
Total of 15 bugs were filed during this period. 1 was satellite related. 5 are related to the Docker feature in 2.15. 5 issues are for pulp3 and 3 for documentation. Pulp-smash issues were filed for automation.

Pulp-smash Issues

https://github.com/PulpQE/pulp-smash/issues/837
https://github.com/PulpQE/pulp-smash/issues/838
https://github.com/PulpQE/pulp-smash/issues/836
https://github.com/PulpQE/pulp-smash/issues/839


Bug Trends Report 2018-01-26

2018-01-06 - 2018-01-25

Total Bugs: 20

Pulp3

Total: 7

Pulp 2.13+

Docker: 2
Manifest upload:1 3286
Sync logic assuming v1 as default - 3258
Pulp: 4
Celery status: 1 3281
Certificate related:1 3319
upload :1 3307
rsync :1 3317
Rpm: 4
rsync seLinux - 3313
mirrorlist sync -3310
entitlement cerkey - 3257
sslclient cert naming issue
3256

Total: 10

Documentation: 3

Summary


Bug Trends Report 2018-02-16

2018-01-25  - 2018-02-16

Total Bugs: 17

Pulp3

Total: 7

Pulp 2.13+

Docker: 2
docker_distributor: 3357
Sync logic assuming v1 as default - 3258
Pulp: 4
Celery 4.0 task status -3378
applicability: 3368
Worker issue: 3358, 3356, 3349
Repo copy : 3344
RHEL 5 issue : 3342
rsync distributor syncs: 3338
RPM:
Suse repo sync : 3377
Ubuntu repo sync : 3370

Summary

Total no of issues filed was 19. 7 issues were file against pulp3 and 10 for pulp2. There are some worked related issues on 2.15 that we will make sure to test once they are fixed. 1 issue was filed for pulp smash

https://github.com/PulpQE/pulp-smash/issues/869


Bug Trends Report 2018-02-28

2018-02-17 - 2018-02-28

Total Bugs: 8

Pulp3

Total:4

Pulp 2.13+

Docker:

docker_rsync_distributor:3397

Rpm
Debian repo sync: 3388

Pulp
Celery 4.0: 3383

Summary

The total number of issues filed was 8. 4 issues were file against pulp3 and 2 for pulp2. No new issues were filed in github.


Bug Trends Report 2018-03-13

Total Bugs: 15

Pulp3

Total: 11

Pulp 2.13+

Docker: 1
removed tags showing incorrectly - 3462

rpm
srpm packages not included when generating errata -3461
Not able to skip SRPMs during sync - 3440

Total: 10

Documentation: 1

Summary

Total number of bugs filed are 15. 11 issues are filed for pulp3. 1 pulp-smash issue was filed to automate issue 3440


Bug Trends Report 2018-03-31

Total Bugs: 12

Pulp3

Total: 7

Pulp 2.13+

Override config - 3521
gpg_cmd - 3498, 3474
OL7 repo sync issue: 3525

Total: 4

Documentation: 1

Summary

Total number of bugs filed are 12. 7 issues are filed for pulp3. 1 pulp-smash issue was filed to automate issue 3498/3474

Total untriaged bugs: 13

https://github.com/PulpQE/pulp-smash/issues/912


Bug Trends Report 2018-04-15

Total Bugs: 9

Pulp3

Total: 6

Pulp 2.13+

Epel Sync issue: 3564
Repodata issue: 3551

Total: 2

Documentation: 1 (pulp3)

Summary

Total number of bugs filed are 9. 6 issues are filed for pulp3. 1 pulp-smash issue was filed to automate issue 3564

https://github.com/PulpQE/pulp-smash/issues/957


Bug Trends Report 2018-04-27

Total Bugs: 22

Pulp3

Total: 16

Pulp 2.13+

Docker google registry not synching : 3573
Sync/Publish : 3585
Checksum error after copy repo: 3574
whl file upload fail with python plugin: 3602
Total: 6

Documentation: 1 (pulp3)

Summary

The total number of bugs filed are 22. 16 issues are filed for pulp3. No pulp-smash issues were filed.


Community Plugins

Support

This page is an index for community supported plugins to be posted. These are not supported by the core dev team. Please contact the plugin maintainer or developer for support.

Discussion

Plugin development discussion is appropriate by anyone on the pulp-dev mailing list. Plugin usage discussion is appropriate in the pulp-list mailing list.

Plugin List

Name Description URL
pulp_win Pulp plugin for Windows (.msi, .msm) https://github.com/mibanescu/pulp_win
pulp-snapshot Pulp plugin for snapshotting a repository https://github.com/sassoftware/pulp-snapshot
pulp_deb Pulp plugin for Debian packages https://github.com/sassoftware/pulp_deb

Continuous Delivery of Pulp 3

The plan for quality involves the continuous delivery of Pulp 3 where both unit and integration are run with each PR and prior to each release to PyPI. This will affect both pulpcore and pulp_file. Beta builds of pulpcore and pulp_file will be published to PyPI once a week without human intervention when pulp-smash and unit tests are passing.

Pull Requests (100% Travis)

The following tests are used to check pulpcore pull requests:

For pulp_file, the following jobs are used:

Releases (100% Travis)

The following tests are used to check pulpcore releases:

For pulp_file, we use the following tests:

Plugins

Other than pulp_file, plugins will be responsible for providing their own set of continuous integration jobs.

[0] http://github.com/pulp/pulp/
[1] http://github.com/pulp/pulp_file/
[2] https://github.com/PulpQE/pulp-smash


The helpline number is being operated by our travel experts. They are professionally qualified to be a travel expert. After choosing, we provide them training so that they can resolve any query within least time and provide complete satisfaction. All you need to do is make a call at Delta Airlines Phone Number. Whenever you call at our helpline number, you instantly get connected with our travel experts. By knowing your issue, they provide solution instantly. Visit: https://delta.airlines-phone-number.com/


Docs for Youtube Channel Managers

Creating Demo Checklist

0. Goto the 'Event' page: https://www.youtube.com/my_live_events
1. Click 'New Live Event'
2. Set Title: 'Community Demo Month XX, 2018'
3. Set Time then Date of the future recording
4. Check 'Also share on Twitter'
5. Add a Twitter message which will become the Twitter post. I usually use "Watch the Pulp Community Demo to see what's new!" as a start. Closer to the event I try to tailor this message to the content a bit more, but this generic one will do at creation time.
6. Go into 'Advanced Settings' tab and...
7. Set the 'Recording Date'
8. Set 'Promote on my channel page' to '1 week prior to scheduled start time'
9. Select 'Optimize for interaction (low latency)'
10. Tidy up the Community Demo Agenda etherpad https://etherpad.net/p/Pulp_Community_Demo_Agenda

11. Update the calendar invite description which is sent to the core team and Satellite. It's on the "Satellite share" calendar. The video URL is the main thing. It has this format:

Watch here: https://www.youtube.com/watch?v=XXkHrqutYlU
Agenda is at: https://etherpad.net/p/Pulp_Community_Demo_Agenda

12. Update the date on the Get Involved page of pulpproject.org

Finalizing Demo Checklist

0. Goto the 'edit' page for the Community Demo
1. Remove the Google default 'Tags'
2. Add the demo to the 'Community Demos' playlist
3. Upload and set the custom thumbnail with the agenda
4. Create the description from the template below and set it as the description 'Description'.

0:45 State of Pulp (mhrivnak)
4:58 New Pulp logo (bmbouter)

Helpful tools

bmbouter published a script to help generate the different text outputs (email, blog, description comments).

https://github.com/bmbouter/pulp_community_tools

Coordinating a Demo

0. Ensure the Youtube Event is created and correctly configured.
1. Update the invite with the "watch page" URL. Talk to @bmbouter for access to this invite.
2. 10 days before the demo send out a call for presenters to pulp-list. Here is an example.
3. 3 days before (the Monday before) send out an advertisement to pulp-list and satellite6-list. Here is an example.
4. Post a post-demo Blog post which embeds and includes links to the agenda. Here is an example. I make these links with http://www.youtubetime.com/. Also be sure to test your urls.
5. Post a followup email to pulp-list and satellite6-list with a link to the blog post and the text agenda with links.


Demo Presenter Notes

Presentations are 5 min or less video of something accomplished that affects Pulp, a pulp Plugin, or the Pulp community. Presenters can be anyone from the community.

Deciding to present

There is a call for presenters sent to the pulp-dev@redhat.com email address whenever a demo is being organized. It has instructions on how to signup which is putting your name, nick, and title onto an etherpad.

All presentations are pre-recorded

Demo length

All videos must be under 5 minutes. The ideal demo length being about 3 - 4 minutes based on the Wistia report.

Recording a Video

Typically a tool like simplescreenrecorder or recordmydesktop is used to record both audio and video at the same time. OpenBroadcasterSoftware is also a good choice, but more heavyweight and fully featured than those two. Whatever you use, make sure that you:

1. Increase the font! Whatever you think is readable, make it a lot bigger.
2. Check your audio level. We can't fix audio that is recorded too soft or too loud when we playback the video.

Testing your Video

All video playback happens in Firefox. Ensure that a normal Firefox browser can natively play your video.

Sending your video:

1. Come up with a title that is 50 chars or less.
2. Come up with a description. It can be one or more paragraphs.
3. Send the recorded video as an attachment, the title, and the description to dawalker@redhat.com. Alternatively, you can send a link to download the video file from. By submitting a video you are acknowledging you are the copyright owner of that video content and you agree to grant Pulp the right to broadcast and playback that content.


Vagrant Options

Full configuration of Vagrant is outside the scope of our documentation, but there are some common options that might be useful to multiple developers. These tips are documented here, but are not "supported". This guide assumes familiarity with our official dev setup guide: http://docs.pulpproject.org/en/3.0/nightly/contributing/dev-setup/index.html

Keep a Vagrant box for Pulp 2 and Pulp 3

To quickly switch bewteen Pulp 2 and Pulp 3, the simplest solution is to keep a completely separate vm for each, which means that you need a full set of git repositories checked out to the correct branches for each .

1. Create a second set of repositories. Create a copy of the directory that contains all of your Pulp git repositories. The names is not important, but I will reference this example:

cp -r ~/devel ~/3dev
mv ~/devel ~/2dev

2. Align the git checkouts. You can use `checkout.py` or manually align the git branches.

3. Update your Vagrantfile.
"~/2dev/devel/Vagrantfile" picks up the changes in https://github.com/pulp/devel/blob/master/Vagrantfile.example
"~/3dev/devel/Vagrantfile" picks up the changes in https://github.com/pulp/devel/blob/3.0-dev/Vagrantfile.example

4. Playbooks tracked by git have been updated. If you use additional playbooks, you may need to update the `hosts` line.

5. Delete your old libvirt or docker machine using virt-manager

6. `vagrant destroy` and `vagrant up`

Vagrant providers:

There are two Vagrant providers available for use: ``libvirt`` (using a virtual machine) and``docker`` (using a `docker <https://www.docker.com/&gt;\`_ container).

libvirt:

Reasons to prefer libvirt:

Requirements:

docker:

Requirements:

You will need to enable and start the docker service:

$ sudo systemctl enable docker
$ sudo systemctl start docker

Cache RPM Packages

You can configure your Vagrant enviroment to cache RPM packages you download with dnf. To do
this, uncomment the line ``'.dnf-cache' => '/var/cache/dnf'``, which syncs the ``.dnf-cache``
directory (relative to the Vagrantfile) to ``/var/cache/dnf``. You will need to create the
``.dnf-cache`` directory manually with ``mkdir .dnf-cache``.

Vagrant without sudo

When using Vagrant, you probably have noticed that you are frequently prompted for passwords to
manage libvirt. You can configure your system policy to allow your user to manage libvirt without
needing root privileges. Create ``/etc/polkit-1/localauthority/50-local.d/libvirt.pkla`` with the
following contents, substituting with your user id::

[Allow your_user_id_here libvirt management permissions]
Identity=unix-user:your_user_id_here
Action=org.libvirt.unix.manage
ResultAny=yes
ResultInactive=yes
ResultActive=yes

Speed improvements:

KVM unsafe cache

You can configure your Vagrant environment to use
`kvm's unsafe cache mode <http://libvirt.org/formatdomain.html#elementsDisks&gt;\`_. If you do this,
you will trade data integrity on your development environment's filesystem for a noticeable speed
boost. In your Vagrantfile, there is a commented line ``domain.volume_cache = "unsafe"``. To use
the unsafe cache mode, simply uncomment this line.

You can also configure Vagrant to use the unsafe cache for all Vagrant guests on your system by
creating ``~/.vagrant.d/Vagrantfile`` with the following contents::

  1. * mode: ruby *
  2. vi: set ft=ruby :
    Vagrant.configure(2) do |config|
    config.vm.provider :libvirt do |domain|
    # Configure the unsafe cache mode in which the host will ignore fsync requests from the
    # guest, speeding up disk I/O. Since our development environment is ephemeral, this is
    # OK. You can read about libvirt's cache modes here:
    # http://libvirt.org/formatdomain.html#elementsDisks
    domain.volume_cache = "unsafe"
    end
    end
    .. warning::
    This is dangerous! However, the development environment is intended to be "throw away", so
    if you end up with a corrupted environment you will need to destroy and recreate it.
    Fortunately, the code you are working on will be shared from your host via NFS so your work
    should have data safety.

Vagrant with PyCharm

PyCharm 5.0.1 is mostly usable with Vagrant.

Remote Debugging

To use a remote debugger provided by PyCharm, ensure the PyCharm debug egg is installed in the
Vagrant environment. This can be done in the Vagrant environment using ``sudo pip``
so it is available in all virtualenv environments the Vagrantfile sets up.

When SSHing to Vagrant, use a reverse SSH tunnel to allow the Vagrant environment to connect
back to your host system where the PyCharm remote debugger is listening. ``vagrant ssh`` allows
you to specify arbitrary SSH commands using the ``--`` syntax. Assuming a PyCharm remote debugger
is listening on port 12345, connect to Vagrant with a reverse tunnel using::

$ vagrant ssh -- -R 12345:localhost:12345

You'll also need to configure local to remote path mappings to allow PyCharm to treat your host
code checkout corresponds with the remote Vagrant code. To do this, edit the PyCharm remote
debugger instance and add the following path mapping configuration::

/home/&lt;your_username&gt;/devel=/home/vagrant/devel

Resolving References

With Vagrant, Pulp is not installed on your host system preventing PyCharm from knowing an object
through static analysis. Practically speaking, this causes all Pulp objects to be shown as an
unresolved reference and prevents jumping to the declaration (Ctrl + B).

To resolve this, configure your project with a Vagrant-aware, remote interpreter. In settings,
find the 'Project Interpreter' area and add a Remote Interpreter. Select 'Vagrant'
and give it the path to your vagrant file. In my case this is ``/home/<username>/devel/pulp``.

.. note:: The remote interpreter copies the indexed remote code locally into PyCharm's cache.
          Be aware, when you jump to a declaration (Ctrl + B), you are being shown PyCharm's
          cached version. For reading code this is fine, but when applying changes, be sure
          you know if you are editing the actual code or a cached copy.

Use Vagrant's Inventory

You may find it convenient to have more manual control
over the running of the playbook by running ``ansible-playbook`` directly against the
Vagrant inventory. Vagrant stores its inventory in a ``.vagrant`` dir in the same place
as the ``Vagrantfile`` after running ``vagrant up``. Vagrant stores the inventory in
``.vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory``, so pass that to
``ansible-playbook`` as the invetory when running the dev playbook, remember to also
enable the vagrant role::

ansible-playbook -e ansible_python_interpreter=/usr/bin/python3 -e use_vagrant_role=true \
    -i .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory \
    ansible/dev-playbook.yml

Downloader Performance Comparison

Overview

The Pulp3 Plugin API has two sets of "downloaders" currently. There are the asyncio downloaders and the futures downloaders

Goal

Run some basic performance comparisons between the asyncio and futures downloaders.

The tests

The tests all use the file-example fixture data hosted here

There are 5 different manifests in that repo, each which has the following characteristics

num_files, size (MB), url
100, 407.679, https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/file-example/PULP_MANIFEST_100
200, 819.088, https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/file-example/PULP_MANIFEST_200
300, 1206.11, https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/file-example/PULP_MANIFEST_300
400, 1565.02, https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/file-example/PULP_MANIFEST_400
500, 1945.95, https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/file-example/PULP_MANIFEST_500

Methodology

Compare the runtime average and std deviation of the asyncio and futures downloaders across 5 different repos. The methodology needs to control for a variety of things:

Differences in how the downloaders are operated

pulp_example contains an importer that works with the "asyncio" downloader and one that works with the "futures" downloader. That implementation is very similar and differs meaningfully only with its use of the downloader. Neither of them use the changesets; both importers performing testing use the respective downloader directly.

Code changes between tests

All tests use the same commit with pulp at ( 1c8040150e11e9808d0cf84fe067a3e1e9da48c3 ) and pulp_example at ( 75087c7b1d5ca4c3678f4f121b55196f979e05d5 ). These commits are used as is with 0 line changes applied onto them throughout testing.

Network conditions

Each test is run 10 times with the final runtime average and standard deviation computed from the 10 runs. For any given repo (100, 200, etc) the asyncio and futures downloaders are run immediately back-to-back which should subject them to mostly similar network conditions. The different repo tests (100, 200, etc) are not expected to have run in similar network conditions so we should only compare a single test type, e.g. asyncio-100 vs futures-100 is ok to compare but asyncio-100 versus asyncio-200 is not safe to compare.

Local data or state

Three things are done before each test to reset the state: a)the Vagrant environment has its database fully resets by running pclean run on it. b) All downloaded data from previous runs is removed from the artifact directory. c) all services are restarted.

Test Environment Setup

1. Start a Pulp3 Vagrant VM with pulp at ( 1c8040150e11e9808d0cf84fe067a3e1e9da48c3 ) and pulp_example at ( 75087c7b1d5ca4c3678f4f121b55196f979e05d5 ).
2. Ensure you have the testing scripts checked out (3 files).
3. Install jq `sudo dnf install jq`.
4. Run the tests from the pulp virtualenv. You can do this by running `workon pulp`.

Test Plan

1. Edit the perftest.sh to configure your test. Set `asyncio=1` to test asyncio, set `asyncio=0` to test futures. Set the `num` to the test type you want, e.g. 200.
2. Run the test with `./manytests.sh`
3. Read the runtimes (in seconds) in the data file perf_data.txt

Data Collected

https://docs.google.com/spreadsheets/d/1E4sRA_xKMq1kNjOvWLz6BkGOvzKNPuTozm7xUzN1Snc/edit?usp=sharing

Data Summary

Mean Download Time

Std. Deviation of Download Time

Conclusions

From this testing both, the asyncio downloaders download roughly 19.5% faster on average than the futures downloaders. This is an average of the percentage change across all 5 tests.

Note that both downloaders could be tuned using their respective with the concurrency settings. For the futures downloads, this is the number of threads used, and for the asyncio downloaders, it's the number of co-routines being scheduled. Anecdotally, increasing the asyncio concurrency does result in significantly faster downloading, but increasing the thread count for futures strangely results in slower downloading.


Downloading

In pulp3, there are two competing technologies and designs being considered. For the purposes of the discussion we'll name them Jupiter and Saturn. The Jupiter solution is based on concurrent.futures and the Saturn solution is based on asyncio. In addition to the underlying technology difference, the solutions meet the requirements in different ways. The Jupiter solution includes more classes, provides more abstraction and supports customization through delegation and object composition. The Saturn solution meets the requirements with the fewest classes possible and minimum abstraction. Customization is supported though subclassing.

The three actors for our use cases is the Importer, Streamer and Plugin Writer. The ChangeSet shares a subset of the Streamer requirements but not included in this discussion.

Design Goals & Constraints

The requirements define the minimum criteria to be satisfied by both solutions. The design constrains and goals define how the requirements are met.

juniper:

saturn:

Use Cases

Importer


As an importer, I need to download single files.

jupiter:

download = HttpDownload(
    url=url,
    writer=FileWriter(path),
    timeout=Timeout(connect=10, read=15),
    user=User(name='elmer', password='...'),
    ssl=SSL(ca_certificate='path-to-certificate',
            client_certificate='path-to-certificate',
            client_key='path-to-key',
            validation=True),
    proxy_url='http://user:password@gateway.org')

try:
    download()
except DownloadError:
    # An error occurred.
else:
   # Go read the downloaded file \o/

saturn:

ssl_context = aiohttpSSLContext()
ssl_context.load_cert_chain('path-to-CA_certificate')
ssl_context.load_cert_chain('path-to-CLIENT_certificate')
ssl_context.load_cert_chain('path-to-CLIENT_key')

connector=aiohttp.TCPConnector(verify_ssl=True, ssl_context=ssl_context)

session = aiohttp.ClientSession(
    connector=connector,
    read_timeout=15,
    auth=aiohttp.BasicAuth('elmer', password='...', encoding='utf-8'))

downloader_obj = HttpDownloader(
    session,
    url,
    proxy='http://gateway.org',
    proxy_auth=aiohttp.BasicAuth('elmer', password='...', encoding='utf-8')

downloader_coroutine = downloader_obj.run()
loop = asyncio._get_running_loop()
done, not_done = loop.run_until_complete(asyncio.wait([downloader_coroutine]))
for task in done:
    try:
        result = task.result()  # This is a DownloadResult
    except aiohttp.ClientError:
        # An error occurred.

question: How can the connect timeout be set in aiohttp?


As an importer, I can leverage all settings supported by underlying protocol specific client lib.

jupiter:

Commonly used settings supported by abstraction. Additional settings could be supported by subclassing.


class SpecialDownload(HttpDownload):

    def _settings(self):
        settings = super()._settings()
        settings['special'] = <special value>
        return settings

saturn:

The underlying client lib arguments directly exposed.


As an importer, I can create an Artifact with a downloaded file using the size and digests calculated during the download.

jupiter:

Using the optional DownloadMonitor to collect statistics such as size and calculate digests.


download = HttpDownload(..)
monitor = DownloadMonitor(download)
...  # perform download.
artifact = Artifact(**monitor.facts())
artifact.save()

saturn:

The size and all digests always calculated.


downloader_obj = HttpDownloader(...)
...  # perform download.
result = task.result()
artifact = Artifact(**result.artifact_attributes)
artifact.save()

As an importer, I need to download files concurrently.

jupiter:

Using the Batch to run the downloads concurrently. Only 3 downloads in memory at once.


downloads = (HttpDownload(...) for _ in range(10))

with Batch(downloads, backlog=3) as batch:
    for plan in batch():
        try:
            plan.result()
        except DownloadError:
            # An error occurred.
        else:
            # Use the downloaded file \o/

saturn:

Using the asyncio run loop. This example does not restrict the number of downloads in memory at once.


downloaders = (HttpDownloader...) for _ in range(10))

loop = asyncio._get_running_loop()
done, not_done = loop.run_until_complete(asyncio.wait([d.run() for d in downloaders]))
for task in done:
    try:
        result = task.result()  # This is a DownloadResult
    except aiohttp.ClientError:
        # An error occurred.

As an importer, I want to validate downloaded files.

jupiter:

Supported by adding provided or custom validations to the download. A validation error raises ValidationError which IsA DownloadError.


download = HttpDownload(...)
download.validation.append(DigestValidation('sha256', '0x1234'))

try:
    download()
except DownloadError:
    # An error occurred.

saturn:

Supported by passing the expected_digests dictionary and catching DigestValidationError.


downloader_obj = HttpDownloader(..., expected_digests={'sha256': '0x1234'})

downloader_coroutine = downloader_obj.run()
loop = asyncio._get_running_loop()
done, not_done = loop.run_until_complete(asyncio.wait([downloader_coroutine]))
for task in done:
    try:
        result = task.result()  # This is a DownloadResult
    except (aiohttp.ClientError, DigestValidationError):
        # An error occurred.

As an importer, I am not required to keep all content (units) and artifacts in memory to support concurrent downloading.

jupiter:

Using the Batch to run the downloads concurrently. The input to the batch can be a generator and the number of downloads in
memory is limited by the backlog argument.


downloads = (HttpDownload(...) for _ in range(10))

with Batch(downloads, backlog=3) as batch:
    for plan in batch():
        try:
            plan.result()
        except DownloadError:
            # An error occurred.
        else:
            # Use the downloaded file \o/

saturn:

@bmbouters: please describe and provide examples.


As an importer, I need a way to link a downloaded file to an artifact without keeping all content units and artifacts in memory.

jupiter:

Using the Batch to run the downloads concurrently and specifying the backlog to limit the number of downloads in memory. See other examples.

The Download.attachment provides linkage to objects like Artifacts that are related to the download.


download = HttpDownload(...)
download.attachment = Artifact(..)

saturn:

@bmbouters: please describe and provide examples.


As an importer, I can perform concurrent downloading using a synchronous pattern.

jupiter:

Using the Batch. See other examples.

saturn:

Using the asyncio loop directly. See other examples.


As an importer, concurrent downloads must share resources such as sessions,connection pools and auth tokens across individual downloads.

jupiter:

The Download.context is designed to support this. The shared context can be used to safely share anything This includes python-requests sessions (using a Cache), auth tokens and resolved mirror lists. The sharing is done through collaboration. When it's appropriate for individual downloads to share things, an external actor like the Batch or the Streamer will ensure that all of the download
objects have the same context.

saturn:

Each downloader could define a class attribute. This global can be used share anything. This includes python-requests sessions (using a Cache), auth tokens and resolved mirror lists. The sharing is done through collaboration. Sharing is global and unconditional.

Question: how will thread safety be provided? The streamer will have multiple twisted threads using these downloaders.


As an importer I can customize how downloading is performed. For example, to support mirror lists

jupiter:

All download objects can be customized in one of two ways. First, by delegation using events. And, second by subclassing.

Delegation example.


class MirrorDelegate:
    # Any download can delegate mirror list resolution
    # and hunting to this object.

    def __init__(self):
        self.mirrors = iter([])

    def attach(self, download):
        download.register(Event.PREPARED, self.on_prepare)
        download.register(Event.ERROR, self.on_error)

    def on_prepare(self, event):
        # Resolve the mirror list URL
        # May already be stored in the context or need to be downloaded and parsed.
        with event.download.context as context:
            try:
                mirrors = context.mirrors
            except AttributeError:
                download = event.download.clone()
                download.writer = BufferWriter()
                download()
                _list = download.writer.content()
                mirrors = [u.strip() for u in _list.split('\n') if u.strip()]
                context.mirrors = mirrors
        # Align retries with # of mirrors.
        event.download.retries = len(mirrors)
        self.mirrors = iter(mirrors)
        # Start
        event.download.url = next(self.mirrors)

    def on_error(self, event):
        try:
            event.download.url = next(self.mirrors)
        except StopIteration:
            # no more mirrors
            pass
        else:
            event.repaired = True

# importer
def get_download(...):
    download = Factory.build(...)
    delegate = MirrorDelegate()
    delegate.attach(download)

Subclass example.


class MirrorDownload(HttpDownload):
    # Support HTTP/HTTPS mirror list downloading.

    def _prepare(self):
        super()._prepare()
        # Resolve the mirror list URL
        # May already be stored in the context or need to be downloaded and parsed.
        with self.context as context:
            try:
                mirrors = context.mirrors
            except AttributeError:
                download = self.clone()
                download.writer = BufferWriter()
                download()
                _list = download.writer.content()
                mirrors = [u.strip() for u in _list.split('\n') if u.strip()]
                context.mirrors = mirrors
        # Align retries with # of mirrors.
        self.retries = len(mirrors)
        self.mirrors = iter(mirrors)
        # Start
        self.url = next(self.mirrors)

    def _on_error(self, event):
        super()._on_error(event)
        try:
            self.url = next(self.mirrors)
        except StopIteration:
            # no more mirrors
            return False
        else:
            return True

# importer
def get_download(...):
    # Factory needs to support custom class.

saturn:


As an importer, concurrent downloading must limit the number of simultaneous connections. Downloading 5k artifacts cannot open 5k connections.

jupiter:

This is supported by sharing connection pools and limiting the total number of downloads in progress concurrently. See resource sharing and concurrency limiting use cases.

saturn:

This is supported by sharing connection pools and limiting the total number of downloads in progress concurrently. See resource sharing and concurrency limiting use cases.


As an importer, I can terminate concurrent downlading at any point and not leak resources.

jupiter:

The loop using the iterator returned by Batch can be safely exited at any point and all resources are then free to be garbage collected.

saturn:

The loop using the asyncio loop can be safely exited at any point and all resources are then free to be garbage collected.


As an importer, I can download using any protocol. Starting with HTTP/HTTPS and eventually FTP.

jupiter:

Classes extending Download may implement any protocol. HTTP/HTTPS is supported by HttpDownload. See other use case examples.

saturn:

HTTP/HTTPS is supported by HttpDownloader. See other use case examples.


Streamer


As the streamer, I need to download files related to published artifacts and metadata but delegate the implementation (protocol, settings, credentials) to the importer. The implementation must be a black-box.

jupiter:

The Download is a callable.


download = importer.get_downloader(...)
download()

saturn:

@bmbouters: please describe and provide examples.


downloader = importer.get_downloader(...)
self.not_done.append(downloader.run())

As the streamer, I want to validate downloaded files.

jupiter:

The Download may be configured by the importer with a list of Validation objects. Validation is performed on the downloaded bit stream.


download = importer.get_downloader(...)
download()

saturn:

The HttpDownloader may be configured by the importer with expected size and expected digests. Validation is performed on the downloaded bit stream.


downloader = importer.get_downloader(...)
self.not_done.append(downloader.run())

As the streamer, concurrent downloads must share resources such as sessions,connection pools and auth tokens across individual downloads without having knowledge of such things.

jupiter:

Each download may be configured with a shared context. The download objects collaborate to share resources using the context. The streamer updates each Download provided by the importer to use the same (shared) context.


download = importer.get_downloader(...)
download.context = self.context  # This is a Context.
download()

saturn:

Each downloader has a class attribute used to globally share resources.


downloader = importer.get_downloader(...)
self.not_done.append(downloader.run())

As the streamer, I need to support complex downloading such as mirror lists. This complexity must be delegated to the importer.

jupiter:

The downloader object provided by the importer will handle the mirror list.

saturn:


downloader = importer.get_downloader(...)
self.not_done.append(downloader.run())

As the streamer, I need to bridge the downloaded bit stream to the Twisted response. The file is not written to disk.

jupiter:

The Download.writer can be reassigned to the base Writer.


class TwistedWriter(Writer):

    def __init__(self, request):
        self.request = request

    def append(self, buffer):
        reactor.callFromThread(self.request.write, buffer)

    def close(self):
        reactor.callFromThread(self.request.finish)


download = importer.get_downloader(...)
download.writer = TwistedWriter(request)
download()

saturn:

@bmbouters: please describe and add example.


As the streamer, I need to forward HTTP headers from the download response to the twisted response.

jupiter:

The headers can be forwarded using a delegate.


class HeaderDelegate:

    def __init__(self, request):
        self.request = request

    def attach(download):
        download.register(Event.REPLIED, self.on_reply)

    def on_reply(event):
        for header, value in event.download.reply.headers:
            # do filtering here
            self.request.setHeader(header, value)

download = importer.get_downloader(...)
delegate = HeaderDelegate(request)
delegate.attach(download)
download()

saturn:

@bmbouters: please describe and add example.



File Plugin

Terms:

Repository: A directory containing a Manifest and file tree.

Manifest: A metadata file containing references to files within the directory tree to be included in the repository.
Format: <relative path>, <sha256 digest>, optional <size>.

Use Cases

As a user I can CRUD file importer. [done]

As a user I can CRUD file publisher. [done]

As as user, I want to synchronize file repositories by specifying a feed url that points to a manifest.

As a user, I want to publish file repositories.

As a user, I can upload file content to a repository. [done]

As a user, I can distribute a file publication so that it can be synced/downloaded. [done]

Future Roadmap

As a user, I want to support signed files.


Flatpak

Manage flatpak content in Pulp 3.

Uniqueness would be defined by "Identifier triples:"http://docs.flatpak.org/en/latest/introduction.html#naming :

content unit = flatpak

client tool = flatpak

Discovery
How to discover content type discovered? Not sure yet.

Other notes:
Need to look at OSTree for versioning?


Helm Support

Manage Charts, the content type for the Helm package manager.

The Content Unit Anatomy

A chart is a named, compressed tarball that, besides the actual content, contains a mandatory chart file which describes the chart. charts may optionally nest further chart dependencies.

The Content Unit Unique Constraints

Two charts can be considered identical if their name, version and digest attributes are equal. A charts repository index tracks these attributes of each published chart.

Additional Mandatory Content Unit Attributes

There are multiple optional attributes a chart can have, that the helm inspect command will display. The helm search command utilises matching based on the tags and description attributes in addition to the name attribute. It therefore would be reasonable to track these attributes in Pulp too.

Content Discovery

The helm client supports the notion of a chart repository ; multiple repositories can be tracked by the client. A chart can be discovered by searching thru a repository index that contains all the mandatory attributes of all the charts of the repository. In addition, both the index and the chart file include a list of urls that a chart can be downloaded from. Both these lists should probably contain a link back to the Pulp repository itself. The helm content plug-in should therefore rebuild the index upon a repository publish.

Summary

Relevant chart content type attributes:


Current Infrastructure & Hosting

Websites

pulpproject.org - The main website is a Jekyll based website. The content comes from the pulp/pulpproject.org repo. It is currently built by the OSCI maintained Jekyll builder and hosted on OSCI infrastructure. The OSCI also manages redirects, TLS cert renewal, and DNS for the site.

docs.pulpproject.org - The docs website is built via Sphinx by Jenkins nightly using this script. That builds a specific branch of the docs from pulp/pulp and then pushes them to the hosting environment hosted by OSCI infrastructure. The OSCI also manages redirects, TLS cert renewal, and DNS for the site.

Testing Environments.

Jenkins Master - Jenkins master is hosted by Red Hat central CI. This environment provides lots of functions: including PR test running, PR docs testing, nightly docs building, nightly rpm production, upstream/downstream automation, and other functions.

Jenkins Slaves - All Jenkins work is coordinated by the master but performed by slaves. The Jenkins slaves run in an environment called ci-rhos which is a Red Hat openstack instance which provides slave resources.

Travis - Travis is the test runner for Pulp3 specifically. Those tests are not run by Jenkins.

Distribution of Bits

PyPI - Pulp3 is being distributed via PyPI.

repos.fedorapeople.org - The official Pulp rpms are available via https://repos.fedorapeople.org/pulp/pulp/stable/ We rsync the bits to that environment whenever releases occur.

Issue tracking

pulp.plan.io - Hosted redmine instance for issues and features to be filed for Pulp and some plugins. All plugins maintained by the core team are expected to be there. Community plugins are also welcome to root their issue trackers there.

Github Issues - Some parts of Pulp have their issues tracked via Github Issues. Specifically, pulp-smash and the repo tracking defects on pulpproject.org both use Github issues.

Bots

pulpbot is hosted in an OSCI environment. It is managed using an Ansbile playbook hosted on GitHub.

Infrastructure Needs

Code Performance Testing

We've wanted to understand the performance impact of code changes for a long time. We want to run tests like these. Overall this idea is inspired by reports like these for the Lucene project.

We likely need one-bare metal machine that we can load fresh everyday and test on. We also need some way to freshly load that machine each day with a tool like oVirt or Foreman. It would be great if we could:


Issue filing template

Description of problem:

Version-Release number of selected component (if applicable):

Hint:
Run rpm -qa | grep pulp to find versions of pulp packages.
Run cat /etc/redhat-release to find out what OS and version you are on

How reproducible:

Steps to Reproduce:
1.
2.
3.

Actual results:

Expected results:

Additional info:

Hint:
Attach the logs from your activity by outputting them to a file. You can do this by:
journalctl --follow > logs.txt


Manage my booking with KLM airlines

KLM airlines provide an easy interface for the passenger where passengers can easily do any modifications in their scheduling of the KLM flights. All the modifications will be done to manage my booking tab. So it becomes very easy for a passenger to change any of the fields related to their booking in few minutes by logging in into the official website of KLM airlines.

Options related to the FLIGHTS in KLM manage my booking tab

• Seat selection option

There are so many options related to the flights to manage my booking tab. The first and foremost option is that the passenger can choose their seat during the KLM reservations. It is the most important factor in your journey. Sitting on a seat of your own choice will make you feel comfortable and make your journey joyful.

• Cancel or change flight option

Manage my booking tab will also allow the passenger to do any of the changes in their scheduling. This means that the passenger can change or cancel the flight if the passenger is unable to travel. The flight change fares will be charged from the passengers according to the guidelines of the KLM airline reservation official website.

• Ticket rebooking option

In case, the passenger cannot travel and will have to cancel the flight, the manage my booking tab gives an option to the passenger to rebook the ticket again by charges some fares according to the official website’s guidelines.

• Selection of in-flight meal

Under the Manage my booking tab, the passenger has the option to select or order on flight meal. Moreover, the passenger can view the inflight entertainment offered by the KLM airline flights.

• Baggage information

The passenger can see that how much baggage you can take with you on your KLM flights and the passenger can also report about the delayed baggage after arrival with the help of managing my booking tab.

Additional Services of KLM Airlines

The passengers can do tax-free shopping with the help of Shop@KLM. The passenger can also book a carrycot and sports equipment for babies. Moreover, passengers can have the benefits of frequent flyers.

These are the options that the passenger can easily modify in manage my booking tab. If the passenger finds any issues related to the KLM manage my booking, then the passenger can call on helpline number by sign in to the official website of KLM airlines to talk to the airline executives. The executives will surely help you out and justify your queries regarding to travelling with KLM flights.


Maven Plugin

I think this should actually be a maven plugin, since it needs to download both jar files and pom files which are maven artifacts. With each, there is a sha and md5 signature.

uniqueness : group_id + artifact_id + version (maybe, the hash???)
types per unit : 1 pom, many jars. 2 sigs per item
attriubtes : updated date (datetime)

clients: maven
maybe https://pypi.python.org/pypi/jip

Content

Discovery

Notes from https://pulp.plan.io/issues/854

Create an initial framework for supporting syncing & publishing Maven artifacts from within Pulp. This issue is an ok place for planning and gathering comments and information, but that should be broken up into smaller stories if/when implemented.

Deliverables

Questions

archetype-catalog.xml

Some maven repositories appear to have an index of content named archetype-catalog.xml, and some do not. What are the expectations around that file? When is/should it be present?

When Pulp does a sync, if that catalog is not present, we will need some other way for a user to specify what content to get. Should they provide a pom.xml file to the importer? Or perhaps csv/json that enumerates the content to get?

When Pulp does a publish, is it always safe for it to create an archetype-catalog.xml file? It is handy to have so that other Pulp deployments can sync from it and duplicate the exact same content.

SNAPSHOT

Should Pulp support "snapshot" repositories?

https://blog.packagecloud.io/eng/2017/03/09/how-does-a-maven-repository-work/#snapshot-repositories

The challenge is that the content with a given unique identifier that includes "-SNAPSHOT" can change from one sync to the next, and there's no way to know if it did. So it would need to be downloaded every time. We would also have to accept that if the content had been promoted into an additional repository, it could get updated in-place without another promote operation.

Upload

Is there a use case for upload? What does that look like? In what format would data get uploaded?

References

Repo in the sky

It may be worth looking at http://artificer.jboss.org/ just to see if there's any opportunity to integrate with that.


NuGet Packages

This would manage .nupkg files, which are renamed ZIP files with a special <id>.nuspec file in the root.

The .nuspec is documented with an xsd on GitHub: https://github.com/NuGet/NuGet.Client/blob/dev/src/NuGet.Core/NuGet.Packaging/compiler/resources/nuspec.xsd
Uniqueness is determined as a combination of id and version from the spec, with added digest.
Core attributes are id and version. They are both strings and are required to discover NuGet packages.

Additional simple optional attributes that could be added are;
- authors : string
- copyright : string
- description : string
- language : string : default en-US
- owners : string
- summary : string
- tags : string
- title : string

Clients that access NuGet packages are; nuget (Both by itself as well as a package-manager plugin for Visual Studio), the dotnet CLI tool, the Chocolatey tool, and just plain curl works too.

For content discovery, a core index JSON lists available services on the registry (https://registry.example/v3/index.json)
The Catalog/3.0.0 service provides json blobs for full content listing, split into several "pages" of JSON. (https://docs.microsoft.com/en-us/nuget/guides/api/query-for-all-published-packages)
Additionally, the PackageBaseAddress/3.0.0 service provides the base address for generating a download URL from {@id}/{LOWER_ID}/{LOWER_VERSION}/{LOWER_ID}.{LOWER_VERSION}.nupkg (https://docs.microsoft.com/en-us/nuget/api/package-base-address-resource)

The JSON blobs contain identical metadata to the nuspec XML, just in JSON format.

To make the client happy about finding packages in large registries, the SearchQueryService would probably have to be implemented as well - as a live API. (https://docs.microsoft.com/en-us/nuget/api/search-query-service-resource)


OSTree Plugin Roadmap

Terms

Ref - A named commit. Think of it as a branch:

Commit Metadata - A set of key/value pairs.

Use Cases

As a user, I want to CRUD an ostree importer.
Additional importer attributes:

As a user, I want to CRUD an ostree publisher.
Additional publisher attributes:

As a user, I want to synchronize with a remote repository as defined by the feed URL.
Additional options:

As a user, I want each content (unit) to be a Ref.

As a user, I want to list all available Refs in the remote repository (by repository). This should be refreshed on each sync.

As a user, I want to publish a repository version.

As a user, I want to distribute OSTree repository via HTTPS.

After-MVP ideas

As a user, I want to synchronize with a remote repository as defined by the feed URL.

As a user, I want to export my ostree repository to a filesystem as an ISO.

As a user, I want to distribute OSTree repository via HTTP.


Performance Test Plan

Overview

Overall this idea is inspired by reports like these for the Lucene project.

Idea

Add performance tests to pulp-smash in a specific Python module. They will not be run by default. They will only if the specific performance module is run. The runtime measurement will be built into these tests and will be reported in seconds.

By default these tests will write their runtime output to stdout as a table

If given an environment variable (or option?), write the contents to a json file. The file will be keyed on the test name and its value is the runtime.

Create a Jenkins job which installs the version of Pulp under test, and runs the performance tests with the environment variable set. The install happens on one machine and all tests run on that one machine. If all tests do not complete the Jenkins job takes no further action and e-mail and reports in IRC that the performance test did not run.

Longitudinal raw data will be stored outside of Jenkins in a sqlite database. Assuming the tests completed, after the runtime json file is produced, the Jenkins job will download the sqlite file via scp. It will then add that run's data into the sqlite file. The scp will then be scp delivered back to its original location overwritting the file it downloaded.

Using the updated sqlite database, one summarized timeseries chart will be produced using matplotlib for each test in the sqlite database. These will do a summarization (see statistics below) to consolidate several points into one. These charts will be SCP'd to a web accessible filesystem to be viewed by the community.

The sqlite database will have a table for Annotations which will allow matplotlib to annotate data points with information about changes in performance or release information.

Controlling for External Factors over Time

Add things here...

The Tests

Each test will live on its own page and at the top will have a simple description of the test does. The graph will be below the description, and the annotations are at the bottom. Here is an example (minus the description)

RPM Tests

Fresh sync only of Fedora 21 w/ lazy off from http://archive.linux.duke.edu/fedora/pub/fedora/linux/releases/21/Everything/x86_64/os/

Re-sync only w/ lazy off http://archive.linux.duke.edu/fedora/pub/fedora/linux/releases/21/Everything/x86_64/os/

Fresh sync only of Fedora 21 w/ lazy on from http://archive.linux.duke.edu/fedora/pub/fedora/linux/releases/21/Everything/x86_64/os/

Re-sync only w/ lazy on http://archive.linux.duke.edu/fedora/pub/fedora/linux/releases/21/Everything/x86_64/os/

Uploading of 100 RPMs with Pulp's httpd config defaults

Copy of all rpms from Fedora 21 repo ^ one repo to an empty repo without depsolve

Copy of a limited set of rpms from Fedora 21 repo ^ to an empty repo with depsolve

Fetch all rpm ids of a large repo. Katello tries to fetch all ids and if that doesn't respond within 2 minutes we fetch in 500 chunks

Related test would be to take the same large repository and create 100-200 copies of that repository (thus creating a lot of related links among the content in Mongo) and do the same fetches

Fetch all errata ids of a repo (May need to sync RHEL or EPEL for this), same logic as rpm ids

After fetching all the errata ids of a repo, fetch all the errata in chunks of 500 by chunks of 500 ids

After fetching all the rpms ids of a repo, fetch all the errata in chunks of 500 by chunks of 500ids

Search performance using a simple Criteria filter

Fresh publish runtime of a Fedora 21 repo

Incremental (second) publish runtime of the Fedora 21 repo with no changes

Incremental (second) publish runtime of the Fedora 21 repo after uploading 20 new rpms.


Plugin Planning

Maven Plugin

NuGet Packages

Helm Support

Flatpak

Requested Plugins (Unplanned)

Windows Chocolatey: https://chocolatey.org/
Mac homebrew: https://brew.sh/
Anaconda: https://www.anaconda.com/
JavaScript NPM: https://www.npmjs.com/
Solaris SmartOS: https://wiki.smartos.org/display/DOC/Home
R CRAN: https://cran.r-project.org/
Puppet: https://forge.puppet.com/
Git: https://git-scm.com/
Go: https://golang.org
Rust Cargo: https://crates.io/


Plugin Wishlist

rpm
puppet
ostree
python
file_plugin
docker


Provisioning Test Machine from Cloud

In this example it will be used https://www.digitalocean.com/ as cloud provider. Feel free to use any cloud provider that you want to.

Few steps to have a machine up and running.

  1. Create an account in a cloud provider, such as : https://www.digitalocean.com/
  2. Login, then click create droplet.
  3. Select a distro, fedora25_x64.

  1. Choose a size. For test purposes the basic is good enough.
  2. Choose a datacenter region. Default is ok.
  3. Change the name of droplet, droplet is just the name of your machine.
  1. Click Create! And you will receive an email with root password and IP.

There are few options to access the previous created machine


Pulp 2 10 3 Test Result Summary

Summary

Started On: 21 November 2016
Finished On: 1 December 2016
Version: 2.10.3
Result : Success
Build URL: N/A
Summary

This release included bug fixes. The QE tasks included in the release process included manual verification some of the issues included in the release, making sure automation jobs pass and upgrade automation jobs passing.

The upgrade automation failure on RHEL6 was identified as a problem not with Pulp, but with how the upgrade is performed. Pulp 2.8 is installed using ansible. Then the upgrade is performed using a bash script that Jenkins runs. As a result, when the the services are started after the upgrade the pid files started in the java_t context. When pulp-smash, or anything else, tries to stop the services the following denial is recorded in the audit.log:

type=AVC msg=audit(1480428276.495:3071): avc: denied { signal } for pid=23019 comm="celery" scontext=unconfined_u:system_r:celery_t:s0 tcontext=unconfined_u:system_r:unconfined_java_t:s0-s0:c0.c1023 tclass=process

The above log says that a signal being sent by command 'celery' that has the unconfined_u:system_r:celery_t:s0 context is being denied. The signal is being sent to a process running in the unconfined_u:system_r:unconfined_java_t:s0-s0:c0.c1023 context.

The solution is to convert the upgrade job from a bash script to an ansible playbook.

Due to this failure upgrade on RHEL 6 was done manually and tests passed.

Beta History
Beta 1: 21 November 2016

*Bugs Included
*

2354 RPM Support Incorrect URL on lazy catalog entries created for existing RPM content units
2362 RPM Support applicability calculation wastes time scanning a list
2374 RPM Support Can't skip distribution content during sync
2436 Pulp pulp-selinux RPM fails to run restorecon statements post install
2195 Pulp can't sync unit that was previously uploaded
2372 RPM Support updating importer ssl_client_cert to null fails with error
2424 Pulp restorecon runs unecessarily for all 2.10+ upgrades

*New Issues Filed
*
https://pulp.plan.io/issues/245

*Action Items

https://github.com/PulpQE/pulp-smash/issues/442


Pulp 2 11 1 Test Result Summary

Summary

Started On: 09 January 2017
Finished On: 16 January 2017
Version: 2.11.1
Result : Success
Build URL: N/A
Summary

This release included bug fixes. The QE tasks included in the release process included identifying issues that needed to be added to automation backlog, creating github issues, manual verification some of the issues, making sure automation jobs pass and upgrade automation jobs passing.

Pulp QE approve the release of pulp 2.11.1
Beta History
Beta 1: 09 January 2016

*Bugs Included *

Pulp
      2439    No data available in publish step (OSError: [Errno 61] No data available)
      2483    pulp_rpm migration 0031 fails: cursor not found
      2491    When stopping pulp_workers, pulp_celerybeat, and pulp_resource_manager gracefully, the status API still shows them as running
      2464    pulp-admin broken in vagrant environment on 2.y
      1287    Repo sync failing with KeyError
      2006    iso importer fails without useful error message
      2136    publish step error handling incorrectly assumes open file
      2476    pulp publish fail when selinux is disabled
      2510    django logs warning when responding with 404
Puppet Support
      2500    ValueError exception is raised when uploading module with invalid name
      1981    unnecessary pulp error on puppet repo sync
      1853    pulp-puppet-module-builder should not overwrite existing module files
RPM Support
      2514    Creating a group no longer updates the existing group
      2365    rpm upload fails to populate "files" field on model
      2457    When syncing do not associate units that are already associated to the repo
      2096    Additional updateinfo.xml after second publish
      2466    Remove unnecessary `deepcopy` calls for sync

View this list in Redmine:
http://bit.ly/2iWc1I7

Action Items

Make sure that all the issues with associated bugzilla are either automated or manually verified.


Pulp 2 11 2 Test Result Summary

Summary

Started On: 23 January 2017
Finished On: 30 January 2017
Version: 2.11.2
Result : Success
Build URL: N/A
Summary

This release included bug fixes. The QE tasks included in the release process included identifying issues that needed to be added to automation backlog, creating github issues, making sure automation jobs pass and upgrade automation jobs passing.

Pulp QE approve the release of pulp 2.11.2
Beta History

|ssues Addressed
================

Pulp
1847    last_unit_added is not added in mongo repo collection records
2520    credentials in feed URL are not url-unquoted
RPM Support
1086    pulp_distribution.xml sync should ignore repodata/*

View this list in Redmine:
http://bit.ly/2k9Whiy


Pulp 2.11 Test Result Summary

Summary

Status: [ ] Not Started [ ] In Progress [ ] Paused [ ] Blocked [X ] Finished
Started On: October 26
Finished On: December 14
Beta/RC URL: https://repos.fedorapeople.org/pulp/pulp/beta/2.11/
Redmine URL: http://https://pulp.plan.io/issues?query_id=61

The number of new features/stories were limited. Upgrade testing needed to be done manually and automation needed to be updated due to some issues found. Automated tests have increased from 432 in 2.10 to 472 in 2.11 . The jobs were run nightly on EL6, EL7 , Fedora 23 and Fedora 24. Jenkins Jobs were set to run on master before beta builds. As a result, the beta produced were stable. Manual testing was limited to the scenarios that do not yet have pulp smash tests. All the new features have been added to pulp-smash backlog to automate or already automated.

Pulp QE team approves the release of the 2.11 RC to GA.

Beta History
Beta 1: October 26, 2016
Beta 2: November 3, 2016
Beta 3: November 23, 2016
RC 1: November 30, 2016
RC 2: December 9, 2016

Automation

New tests were added for the following

https://github.com/PulpQE/pulp-smash/issues/444
https://github.com/PulpQE/pulp-smash/issues/427
https://github.com/PulpQE/pulp-smash/issues/425

[[Status: [X] PASSED [ ] FAILED
System: x86_64
RHEL 6.8 RHEL 7.3
Installation PASS PASS
Smoke PASS PASS
API PASS PASS
CLI PASS PASS
Upgrade PASS PASS]]

NOTES:

2.11 Features and Bugs
https://pulp.plan.io/issues?query_id=61

New Features

============

Here are the specific stories done for 2.11:

Docker Support
      2114    As a user, I can perform incremental publish with Docker rsync distributor
OSTree Support
      2205    As a user, I want to pull older versions of trees available to atomic hosts.
Pulp
      1983    As a user, an importer config change or content removal will cause the next sync to
be full
      2172    Memory Improvements with Process Recycling
      2186    As a user, pulp-manage-db refuses to run if other pulp processes are running locally
RPM Support
      1806    As a user, I can use pulp-admin to upload drpms
      2113    As a user, I can perform an incremental publish using ISO rsync distributor
      2132    As a consumer of Pulp content I want ISO & ISO_rsync distributor to support
relative_url for consistency with yum
      2211    As a user, I can sync a distribution unit with the existing yum_importer without
requiring yum metadata
      2212    As a user I can see in the task report of copy operation list of units which did not
pass signature filter 

View this list in redmine:
http://bit.ly/2eQz5Ui

Issues Addressed

================

Here are the bug fixes specific to 2.11:

Pulp
      2377    0025_importer_schema_change hits issue with last_sync being None
      1977    basic auth in URL fails when using authenticated proxy
      1069    extra Task ID in docs example to bind a repo
      2133    Permissions docs should include some info on /v2/
      752     Suggestion to add -v remains even if you use -v
RPM Support
      2224    Cannot sync from mirrorlists
      2321    Pulp can't handle mirrorlists with invalid entries
      2363    Pulp acts like all is well when a sync completely fails
      2071    Creating yum distributor without https, http keys in config crashes

Action Items

The following process improvements will be made as a result of this release
Automation: Continue to Improve automation coverage. And add more matrix to automated upgrade testing.
Manual Testing: Organize manual testing efforts better. Identify areas for upgrade testing and feature testing for manual testing that are not covered in the automation.

New Pulp-smash issues added

https://github.com/PulpQE/pulp-smash/issues/454
https://github.com/PulpQE/pulp-smash/issues/453
https://github.com/PulpQE/pulp-smash/issues/431
https://github.com/PulpQE/pulp-smash/issues/421
https://github.com/PulpQE/pulp-smash/issues/420


Pulp 2 12 1 Test Result Summary

Summary

Started On: 16 February 2017
Finished On: 23 February 2017
Version: 2.12.1
Result : Success
Build URL: N/A
Summary

This release included bug fixes. The QE tasks in the release process included identifying issues that needed to be added to automation backlog, creating github issues, making sure automation jobs pass and upgrade automation jobs passing. 2 Issues were identified and fixed during the cycle. Both have been fixed and tested

https://pulp.plan.io/issues/2587
https://pulp.plan.io/issues/2600

Pulp QE approve the release of pulp 2.12.1

Beta History

|ssues Addressed

================

Pulp     
      1594    deprecation warnings when running migrations on EL7
      1781    Files ending in .gz are delivered with incorrect content headers
      2004    Pulp logs many DeprecationWarnings
      2458    Lazy download repo content task fails silently.
      2503    remove_missing does not seem to remove on_demand catalog entries
      2518    rpm - confusion around pulp_streamer files
      2542    Streamer needs to try all available catalog entries.
      2550    Publishing via rsync does not correctly look at publish records
      2563    MongoDB deployment link broken in docs
      2576    Update build system to properly implement el6 support policy
      2577    last_unit_added is not updated when units are copied over from another repo
      2587    on_demand downloads aren't being cached by Squid
Pulp RPM
      2263    Fast-forward yum publish skips units if previous publish was cancelled
      2502    cursor timeout in _duplicate_key_id_generator_aggregation() during sync
      2568    DocumentTooLarge error during merge of errata pkglists     

View this list in Redmine:
http://bit.ly/2k9Whiy

The issues added to the backlog for automation

https://github.com/PulpQE/pulp-smash/milestone/14


Pulp 2 12 2 Test Result Summary

Summary

Started On: 10 March 2017
Finished On: 11 April 2017
Version: 2.12.2
Result : Success
Build URL: N/A
Summary

This release included bug fixes. The QE tasks in the release process included identifying issues that needed to be added to automation backlog, creating github issues, making sure automation jobs pass and upgrade automation jobs passing. 2 Issues were identified later in the cycle and fixed. Both have been fixed and tested

https://pulp.plan.io/issues/2688
https://pulp.plan.io/issues/2657

Pulp QE approve the release of pulp 2.12.2

Beta History
Beta1 3/10/2017
Beta 2 4/5/2017

Issues Addressed
================

Pulp
723 RPMs with large number of files can exceed mongo document size limit
1389 Disallow re-uploading the same package twice
2240 Can't export rpm repo
2524 Vagrant systemd and upstart scripts are not symlinked to source
2527 Ensure Pulp 2.y works with both python-celery 3.1.z and 4.0.z
2551 Pulp task error messages should be more informative
2584 Mongo cursor times out during task pulp.server.managers.content.orphan.delete_all_orphans
2585 Bugs fixed in version link is not correct for plugins
2615 Scheduled calls support broken for Celery4+Kombu4
2630 disassociate_units updates last_unit_removed timestamp even if no units are removed
2680 Scheduled RPM installation and uninstallation does not work at all
2688 last_unit_added is not updated when units are copied over from another repo

RPM Support
1414 pulp rpm profiler INFO-level logging is very verbose
1903 RPM import traceback (non-utf-8 metadata slipping through)
2274 Uploading duplicate content results in ambiguous error message
2505 Getting an incorrect exception when trying to sync a local repo with bad permissions
2559 Empty pkglist for the erratum may be generated which may cause yum not to find applicable packages
2560 Non-unique collection names in erratum pkglists causes yum not to show all packages
2599 Deprecation warning in the logs during errata migration
2620 All RPM repo searches are broken
2621 Syncing an immediate repo with 'on_demand' overridden no longer populates the catalog
2622 Sync fails when non-ASCII characters are present in primary.xml
2623 Errata publish performace degradation
2627 Uploading drpm by pulp-admin with --checksum-type fails
2655 units_successful is empty for successful ISO assocaiting task
2657 Publishing iso_rsync_distributor fails when iso_distributor is published with serve_https==False

OSTree Support
2544 copy always matches every unit in the repo
2552 updating ostree rpm gives error on sync: LibError: GLib.Error('No such file or directory', 'g-io-error-quark', 1)

Puppet Support
2606 Fix the ability to forge importer to mirror upstream repository on sync

View this list in Redmine:
http://bit.ly/2mnYfxD

The issues added to the backlog for automation
https://github.com/PulpQE/pulp-smash/milestone/16
3- closed
10-Open


Pulp 2 12 Test Result Summary

Summary

Status: [ ] Not Started [ ] In Progress [ ] Paused [ ] Blocked [X ] Finished
Started On: January 12, 2017
Finished On: January 30. 2017
Beta/RC URL: https://repos.fedorapeople.org/pulp/pulp/beta/2.12/
Redmine URL: http://https://pulp.plan.io/issues?query_id=61

The number of new features/stories were limited.Automated tests have increased from 472 in 2.11 to 481 in 2.12 . The jobs were run nightly on EL6, EL7 , Fedora 23 and Fedora 24. Upgrade jobs were run on RC builds. The results were reported and failures were discussed. https://pulp.plan.io/issues/2545 was found during RC1. RC2 was built with the fix. Manual testing was limited to the scenarios that do not yet have pulp smash tests. All the new features have been added to pulp-smash backlog to automate or already automated.

Pulp QE team approves the release of the 2.13 RC to GA.

Beta History
Beta 1: January 12, 2017
RC 1: January 24, 2017
RC 2: January 25, 2017

Automation

[[https://github.com/PulpQE/pulp-smash/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Coverage+for+2.12%22]]
[[Status: [X] PASSED [ ] FAILED
System: x86_64
RHEL 6.8 RHEL 7.3
Installation PASS PASS
Smoke PASS PASS
API PASS PASS
CLI PASS PASS
Upgrade PASS PASS]]

NOTES:

2.12 Features and Bugs
https://pulp.plan.io/issues?query_id=61

New Features
============

Here are the specific stories done for 2.12:

Pulp
      2525    Failover events should be explicitly logged as such when they occur
      2509    Pulp process failure detection and any failover should occur within 30 seconds
      2186    As a user, pulp-manage-db refuses to run if other pulp processes are running locally
      1939    As a user, I would like to be able to profile Pulp tasks
      1268    As a user, orphan content delete reports how many units were deleted
Python Support
      1882    Rebuild model to support all package types
      140     As a user, I can sync Python packages from another Pulp server
      136     As a user, I can upload wheel packages to Pulp
      135     As a user, I can synchronize wheels from PyPI
RPM Support
      1976    As user, I can have packages sorted in "Packages/$x" directories when published
Docker Support
      2189    As a user, I can update docker_tag units without using sync

View this list in redmine:
http://bit.ly/2iMdYpK

Issues Addressed
================

Here are the bug fixes specific to 2.12:

Pulp
      2264    better error reporting during import/association
      1488    Deprecate nodes
      1321    Commit message requirements for Pulp contributions are out of date and could be better
      103     Document the commit message keywords that will interact with Redmine
RPM Support
      1823    RPMs partially downloaded

All bug fixes from Pulp 2.11.1 and earlier are also included in Pulp 2.12.

View this list in redmine:
http://bit.ly/2iMgzjJ

Action Items
The following process improvements will be made as a result of this release

Automation: Continue to Improve automation coverage. And add more matrix to automated upgrade testing.
Manual Testing: Organize manual testing efforts better. Identify areas for upgrade testing and feature testing for manual testing that are not covered in the automation.


Pulp 2 13 1 Test Result Summary

Summary

Started On: 16 May 2017
Finished On: 24 May 2017
Version: 2.13.1
Result : Success
Build URL: N/A
Summary

This release included bug fixes. The QE tasks in the release process included identifying issues that needed to be added to automation backlog, creating github issues, making sure automation jobs pass and upgrade automation jobs passing. Upgrade automation failed to run the pulp-smash on the upgraded pulp. So manual upgrade was performed and ran pulp-smash on the pulp-server.

Pulp QE approve the release of pulp 2.13.1

Beta History
Beta1 5/16/2017
Beta 2 5/19/2017

IIssues Addressed
================

Pulp
2689 Don't use ssh connection sharing in rsync distributor
2722 Local variable 'published_after_predistributor' referenced before assignment"
2733 Pulp's test certs are bad and do not conform to candlepin's entitlement cert format
2746 Add documentation about Transparent Huge Pages
2726 Fix pulp_resource_mananger typo

RPM Support
2704 Catalog entries for existing content units with old-style storage path get created with new-style storage path.
2720 Can't sync file repo that uses basic auth
2721 Race condition on errata model save during sync of multiple similar repos
2666 Rsync publish for RPM repo fails to run in fast forward mode
2605 Ovirt, Openstack | Malformed repository: metadata is missing for some packages in filelists.xml and in other.xml

Puppet Support
1846 pulp does not sync puppetforge correctly

nectar
2315 Better logging at INFO level
2696 Pulp streamer not 'streaming' content to squid

View this list in Redmine:
http://bit.ly/2pPKNCC

The issues added to the backlog for automation
https://github.com/PulpQE/pulp-smash/milestone/18
2 - Closed
5 - Open


Pulp 2 13 2 Test Result Summary

Summary

Started On: June 13, 2017
Finished On: June 20, 2017
Version: 2.13.2
Result : Success
Build URL: N/A
Summary

This release included bug fixes. The QE tasks in the release process included identifying issues that needed to be added to automation backlog, creating github issues, making sure automation jobs pass and upgrade automation jobs passing. Upgrade automation failed to run the pulp-smash on the upgraded pulp.

Pulp QE approve the release of pulp 2.13.2

Beta History
Beta1 : June 13, 2017

Issues Addressed
================

Pulp
       2776    pulp-manage-db requires pulp services to be shut down even in dry run mode
       2770    Tasks stuck in waiting state if received while qpidd is down
       2741    Pulp is not compatible with Django 1.10
       2728    Check and fix Pulp compatibility with python-mongoengine >= 0.11.0 in Fedora 26/Rawhide
       2639    Upgrade fails on Fedora

Puppet Support
2750 pulp_puppet is not compatible with django 1.10
1237 Puppet Install Distributor does not raise exception when non-optional install_path is missing

RPM Support
2797 Single consumer aplicability generation does not work unless profile already has applicability generated

View this list in Redmine:
http://bit.ly/2sxXeIz

Fedora Uprade
=============
This release resolves the issue (#2639) of upgrading from the pulp packages in Fedora's repositories to the upstream pulp packages.

Pulp-Smash issues

https://github.com/PulpQE/pulp-smash/milestone/20

Open

https://github.com/PulpQE/pulp-smash/issues/686
https://github.com/PulpQE/pulp-smash/issues/650

Closed

https://github.com/PulpQE/pulp-smash/issues/679
https://github.com/PulpQE/pulp-smash/issues/676


Pulp 2 13 3 Test Result Summary

Summary

Started On: July 12 , 2017
Finished On: July 19, 2017
Version: 2.13.3
Result : Success
Build URL: N/A
Summary

This release included bug fixes. The QE tasks in the release process included identifying issues that needed to be added to automation backlog, creating github issues, making sure automation jobs pass and upgrade automation jobs passing.

Pulp QE approve the release of pulp 2.13.3

Beta History
Beta1 : July 12, 2017
Beta 2 July 13, 2017

Issues Addressed ================

Issues Addressed
================
Pulp
2532 rsync distributor without force_full incorrectly skips publishing some units
2677 Occasional HTTP 403 errors when downloading rpms from on_demand / lazy sync repositories
2791 Rsync publish includes units which have not yet been published via the predistributor
2898 Django 1.10 breaks content app

RPM Support
      2798    content can't be fetched for background or on-demand repositories
      2773    ISO repo does not handle updates to files on manifest during re-sync correctly
      2844    Rsync publish for RPM repo fails to run in fast forward mode
      2681    Store errata package lists in a separate collection

View this list in Redmine:
http://bit.ly/2tfgXsY

Pulp-Smash issues

https://github.com/PulpQE/pulp-smash/milestone/21
Open

https://github.com/PulpQE/pulp-smash/issues/682
https://github.com/PulpQE/pulp-smash/issues/656
https://github.com/PulpQE/pulp-smash/issues/664

Closed

https://github.com/PulpQE/pulp-smash/issues/615
https://github.com/PulpQE/pulp-smash/issues/526


Pulp 2 13 4 Test Result Summary

Summary

Started On: September 15 , 2017
Finished On: September 25, 2017
Version: 2.13.4
Result : Success
Build URL: N/A
Summary

This release included bug fixes. The QE tasks in the release process include making sure automation jobs pass and upgrade automation jobs pass. There were no new GitHub issues filed. The bugs included were part of 2.14 and have already been filed.

Pulp QE approve the release of pulp 2.13.4

Beta History
Beta1 : September 15, 2017

Issues Addressed

Pulp
     2874    Race condition during applicability regeneration for consumers with same profile
     2903    Process Recycling feature causes HTTP event notifier to be unreliable
     2734    task group never quite finishes if pulp is restarted in the middle of task run
     2927    Messages in resource manager queue are not persisted after restarting Qpid and cause tasks never start
Docker Support
     3010    Can't sync Docker images

The github issues as the following.

https://github.com/PulpQE/pulp-smash/issues/714
https://github.com/PulpQE/pulp-smash/issues/715
https://github.com/PulpQE/pulp-smash/issues/768

QE was not able to do an upgrade of pulp to pulp 2.13.4 on a satellite6.3 due to a migration issue.

Migrations in downstream to be different a bit is because in 6.2 a lot of cherry picks were required and some of them contained migrations. So in Satellite order of migrations is different and for that reason, we had to re-number them. Which means in order to upgrade we would have to apply the patches and then do an upgrade.

We will continue to pursue this task after the upstream release and report any failures.


Pulp 2 13 Test Result Summary

Summary

Status: [ ] Not Started [ ] In Progress [ ] Paused [ ] Blocked [X ] Finished
Started On: April 12, 2017
Finished On: April 26. 2017
Beta/RC URL: https://repos.fedorapeople.org/pulp/pulp/beta/2.13/
Redmine URL: http://https://pulp.plan.io/issues?query_id=61

Automated tests have increased from 481 in 2.12 to 521 in 2.13 . The jobs were run nightly on EL7 , Fedora 24 and Fedora 25. Upgrade jobs were run on RC builds. Upgrade test jobs passed for EL7. Fedora still experiences the issue https://pulp.plan.io/issues/2639. Manual testing was limited to the scenarios that do not yet have pulp smash tests. Docker tests have been improved to use crane. This provides better test coverage for Docker plugin tests. All the new features have been added to pulp-smash backlog to automate or
are already automated.

Pulp QE team approves the release of the 2.13 RC to GA.

Beta History
Beta 1: April 12, 2017
RC 1: April 24, 2017

Automation

[[ https://github.com/PulpQE/pulp-smash/milestone/15]]
[[Status: [X] PASSED [ ] FAILED
System: x86_64
RHEL 7.3
Installation PASS PASS
Smoke PASS PASS
API PASS PASS
CLI PASS PASS
Upgrade PASS PASS]]

NOTES:

2.13 Features and Bugs
https://pulp.plan.io/issues?query_id=61


New Features
============

Here are the specific stories done for 2.13:

  Crane
        2336    Update crane to serve manifest schema 2
  Docker Support
        2099    As a user, I can sync v2 schema manifests
        2368    As a user, I want to provide synced images with multiple "/" in them.
  Pulp
        1988    As a new developer, I am able to get started from the documentation
        2324    As a user, I can see the first 8 characters of a task id in every log statement
emitted from a running task
        2519    Enable workers to record their own heartbeat records to the database
        2548    As an administrator, I can configure how pulp logs
  Puppet Support
        2108    Add option to puppet_install distributor to install to 'modules' directory and
cleanup base directory
  Python Support
        135     As a user, I can synchronize wheels from PyPI
        136     As a user, I can upload wheel packages to Pulp
        137     As a user, I can publish repositories that contain wheel packages
        140     As a user, I can sync Python packages from another Pulp server
        1882    Rebuild model to support all package types
        1883    As a user, I can sync and publish all package types
  RPM Support
        2575    As a user, I don't have to wait for migration 0020_nested_drpm_directory if
there are no DRPMs

View this list in redmine:
http://bit.ly/2plvHGd

Issues Addressed
================

Here are the bug fixes specific to 2.13:

  Crane
        2608    Update crane installation and configuration docs
  Docker Support
        2643    DKR1008: Could not find registry API at https://docker-registry.engineering.redhat.c
om
        2644    pulp fails to correctly process WWW-Authenticate headers
  Pulp
        1839    sn.dat is succeptable to race conditions
        2013    SSL certs are created at install time, but should be at setup runtime
        2613    Worker Heartbeats Broken After Broker Reconnect
        2496    Killing pulp_workers, pulp_celerybeat, and pulp_resource_manager causes the status
API still shows them as running
        2516    worker heartbeat log statement mishandles timezone
        2589    Pulp 2.13 nightly fails to start on RHEL 7.3
        2687    Rsync publish removes symlinks and replace them with directories
  Puppet Support
        2512    Puppet Importer swallows exception when one is raised during upload
        2581    Update puppet release notes through 2.13.0
  Python Support
        2561    pulp_python 2.0 new features are not documented
  RPM Support
        2543    RPM Importer swallows exception when one is raised during upload
        2101    'pulp-admin iso repo create' command does not set default --serve-http config

All bug fixes from Pulp 2.12.2 and earlier are also included in Pulp 2.13.

View this list in redmine:
http://bit.ly/2oA1q8q

Action Items

The action items post 2.13 release is to continue adding automation tests for 2.13 issues.


Pulp 2 14 1 Test Result Summary

Summary

Started On: September 27, 2017
Finished On: October 5, 2017
Version: 2.14.1
Result : Success
Build URL: N/A
Summary

This release included bug fixes. The QE tasks in the release process include making sure automation jobs pass and upgrade automation jobs pass. QE also spend time on verifying the high priority issues on this release. There were no new GitHub issues filed.

Pulp QE approve the release of pulp 2.14.1

Beta History
Beta1 : September 27, 2017

Issues Addressed

================

Pulp
2961 Pulp 2.14 broken on Fedora 26
2734 task group never quite finishes if pulp is restarted in the middle of task run
2954 Ensure that queued tasks are not lost by enabling task_reject_on_worker_lost for Celery 4
2959 Task results can be lost due to race condition
2758 Documentation on Pulp's storage logic
3018 mongo 3.4 warning over the use of aggregates

RPM Support
3004 recursive errata copy does not copy rpms

Docker Support
2956 As a user, I can sync from registries that use basic auth

View this list in Redmine:
http://bit.ly/2xAPLbV

The github issues are the following.

https://github.com/PulpQE/pulp-smash/issues/695
https://github.com/PulpQE/pulp-smash/issues/769


Pulp 2 14 2 Test Result Summary

Summary

Started On: October 17, 2017
Finished On: October 26, 2017
Version: 2.14.1
Result : Success
Build URL: N/A

Summary

This release included bug fixes. The QE tasks in the release process include making sure automation jobs pass and upgrade automation jobs pass. QE also spend time on verifying some issues on this release. There were 2 GitHub issues for automation.

During QE testing we found out that Issue#3047 was not fully fixed and hence it was removed from the release. A new related issue has also been filed.

Pulp QE approve the release of pulp 2.14.2

Beta History
Beta1 : October 17, 2017

Issues Addressed

================

Pulp
3036 Pulp streamer dumps core on F26
2979 Celery workers may deadlock when PULP_MAX_TASKS_PER_CHILD is used

Nectar
2960 UnicodeEncodeError in case of a non-ASCII character in comments provided with SSL cert/key/CA

Python Support
2964 Too easy for user to create empty python repo when syncing from feed

Docker Support
2847 Skip download of blobs with foreign mediatype.

New Issue

https://pulp.plan.io/issues/3100

Pulp-Smash issues

https://github.com/PulpQE/pulp-smash/issues/800
https://github.com/PulpQE/pulp-smash/issues/784


Pulp 2 14 3 Test Result Summary

Summary

Started On: November 14, 2017
Finished On: November 20 26, 2017
Version: 2.14.3
Result : Success
Build URL: N/A

Summary

This release included bug fixes. The QE tasks in the release process include making sure automation jobs pass and upgrade automation jobs pass. QE also spend time on verifying some issues on this release. There were 2 GitHub issues for automation.

Pulp QE approve the release of pulp 2.14.3

Beta History
Beta1 : November 14, 2017

Issues Addressed
================

Pulp
3039 NotFound exception if consumer is deleted when its queue is gone
3090 Uploading an invalid rpm produces an error message: "unexpected error occurred
importing uploaded file: 'primary'"

RPM Support
3120 Race condition on saving a distribution unit during "smart proxy" sync
3047 ISO repo doesn't correctly handle updates to files for content already in Pulp
3100 Removal of existing iso units doesn't work if there are multiple iso files
3115 One sample request in API doc for regenerate_applicability is wrong

Pulp-Smash issues

https://github.com/PulpQE/pulp-smash/issues/544
https://github.com/PulpQE/pulp-smash/issues/784


Pulp 2 14 Test Day Recap

Test Day August 8/8/2017
Build - 2.14 rc

Email announcements about the event were sent well in advance. We were able to get 3 participants from the community which is a big win from last year when we had 0 community participation

What went well:

We had 3 participants from the community (milkman, cixil & mxey). All of the participants had challenges with installing pulp on their test systems. This was in part due to pulp 2.14 not working on Fedora 26. We were not aware of the issue while writing the Test Day plan [1] and had announced it as one of the supported operating systems. Once identified the test day plan doc was updated. Pulp-QE was able to get everyone onboard quickly. So we able to engage the participants and continue testing. This was also a good exercise for Pulp-QE to get some testing done and verify fixed issues for Pulp 2.14

I also want to give a shout out to Elijah- the pulp QE intern for taking ownership of the test day and making it a successful event.

The following issues were verified

https://pulp.plan.io/issues/2769
https://pulp.plan.io/issues/2724
https://pulp.plan.io/issues/2783

The following issue was reproduced on 2.14 and updated

https://pulp.plan.io/issues/2346

Following new Issues were filed.

https://pulp.plan.io/issues/2964
https://pulp.plan.io/issues/2966
https://pulp.plan.io/issues/2967

What did not go well and suggestions for improvement:-

Explore ideas to attract more community participation.

Coming up with a better plan to get get test systems easily will help us to make use of the test day more effectively

Come up with a better plan to list what need to be tested

We had a broad list of how to participate
It may be more beneficial to come up with more detailed tasks and group them into sections.

[1] https://pulp.plan.io/projects/pulp/wiki/Test_Day_on_August_8_2017


Pulp 2 14 Test Result Summary

Summary

Status: [ ] Not Started [ ] In Progress [ ] Paused [ ] Blocked [X ] Finished
Started On: July 25, 2017
Finished On: August 8. 2017
Beta/RC URL: https://repos.fedorapeople.org/pulp/pulp/beta/2.14/
Redmine URL: http://https://pulp.plan.io/issues?query_id=61

Automated tests have increased from 521 in 2.13 to 528 in 2.14. The jobs were run nightly on EL7, Fedora 24 and Fedora 25. The install on Fedora 26 is broken [1]. Upgrade jobs were run on RC builds. Upgrade test jobs passed for EL7, F24 and F25. Manual testing was limited to the scenarios that do not yet have pulp smash tests. New docker tests were added to include the new feature [2]. All the issues have been added to pulp-smash backlog to automate or are already automated [3]. Pulp 2.14 is also the first release of pulp with the Debian plugin. Pulp-QE have not done any testing of the plugin other than installation and smoke testing.

A Test day was organized to get some community testing done on 2.14 RC, The report here [4]

Pulp QE team approves the release of the 2.14 RC to GA.

Beta History
Beta 1: July 25, 2017
Beta2: July 26, 2017
RC 1: August 2, 2017

Automation

[[ https://github.com/PulpQE/pulp-smash/milestone/15]]
[[Status: [X] PASSED [ ] FAILED
System: x86_64
RHEL 7.3/7.4
Installation PASS PASS
Smoke PASS PASS
API PASS PASS
CLI PASS PASS
Upgrade PASS PASS]]

NOTES:

2.14 Features and Bugs
Here are the bug fixes specific to 2.14:

Pulp
      2903    Process Recycling feature causes HTTP event notifier to be unreliable
      2819    set DJANGO_SETTINGS_MODULE env variable in docs/config.py
      2808    Clarify use of `pulp-consumer`
      2786    The content wsgi does not respect logging configuration
      2783    CopyDirectoryStep should copy mtime
      2874    Race condition during applicability regeneration for consumers with same profile
RPM Support
      2754    RPM uploads appear to be missing metadata information
Crane
      2920    Crane does not redirect correctly to schema 1 path
      2759    Set followlinks to True to visit directories pointed to by symlinks
      2723    As a user GET /crane/repositories/v2 shows v2 repositories
Debian Support
      878     Convert pulp_deb to use MongoEngine models
Docker Support
      2924    Accept headers are not correct during the request to the registry
      2848    remove tag and name field on the Manifest model
      2724    `pulp-admin docker repo create --help` states v1 content is synced by default
      2678    'relative_repo_path' in docs should be 'repo_relative_path'
OSTree Support
      2769    Relative paths are not checked for collisions
Python Support
      2900    Add http config change install documentation for Pulp Python
      2899    Pulp Python plugin spams logs when downloading wrong filetypes from pypi

All bug fixes from Pulp 2.13.3 and earlier are also included in Pulp 2.14.

View this list in redmine:

http://bit.ly/2gOEG22

*Action Items
*
The action items post 2.13 release is to continue adding automation tests for 2.13 issues.

[1] https://pulp.plan.io/issues/2961
[2] https://github.com/PulpQE/pulp-smash/issues/713
[3] https://github.com/PulpQE/pulp-smash/milestone/19
[4] https://pulp.plan.io/projects/pulp/wiki/Pulp_2_14_Test_Day_Recap


Pulp 2 15 1 Test Result Summary

Summary

Started On: January 23, 2018
Finished On: January 30, 2018
Version: 2.15.1
Result: Success
Build URL: N/A

Summary

This release included bug fixes. The QE tasks in the release process include making sure automation jobs pass and upgrade automation jobs pass. Since we still have outstanding SELinux issues for Fedora 27, Pulp 2.15.1 will not be released on F27.

Pulp QE approve the release of pulp 2.15.1

Beta History
Beta1 : January 23, 2018

Issues Addressed

ssues Addressed
================

Crane
3303 v2.py: Accept header handling is sketchy

Docker Support
3118 cannot create docker repository with repo-registry-id containing "--" or "__"
3126 As a user, I can verify blobs checksum during v2 image import
3232 As a user, I should be told what I have just uploaded
3241 docker_distributor_web does not advertise supporting docker_manifest, docker_tag, docker_blob (any v2 unit types)
3242 copying a tag does not preserve its pulp_user_metadata
3250 docker tag creation: inconsistent field names
3251 Uploading a docker tag with pulp_user_metadata specified
3290 Refactor PublishTagsStep to make clear where data comes from

Pulp
3159 Celery AVC Denials on F27
3133 CursorNotFound while queueing applicability tasks
3208 Clarify docs to import an uploaded v1 docker image to repository.
3210 Update Importer configuration fails.
3245 Broken link to protected-branches.py

RPM Support
3306 Unable to sync Lazy Atomic KS repos from CDN due to missing '/'


Pulp 2 15 2 Test Day Recap

Summary

Started On: February 20, 2018
Finished On: February 26, 2018
Version: 2.15.2
Result: Success
Build URL: N/A

Summary

This release included bug fixes. The QE tasks in the release process include making sure automation jobs pass and upgrade automation jobs pass. This release is tested and ready for release onF27.

Pulp QE approve the release of pulp 2.15.2

Beta History
Beta1 : February 20, 2018

Issues Addressed
================

Pulp
2835 Tasks stuck in waiting after restart of pulp services
3129 occasional httpd segfault
3317 rsync_distributors: 'rsync_extra_args' not used in all calls to rsync
3319 Certificates used in unit testing are expired
3349 pulp-admin -vvv returns incorrect information for "worker_name" when querying task results
3356 queue_name property on worker model returns invalid results when the worker name is None
3383 Worker model's "queue_name" property returns the wrong name on Celery 4.x

Puppet Support
3314 puppet install distributor broken on F27 due to SELinux denials

RPM Support
3342 Relative import breaks Python 2.4 (RHEL 5)


Pulp 2 15 2 Test Result Summary

Summary

Started On: February 20, 2018
Finished On: February 26, 2018
Version: 2.15.2
Result: Success
Build URL: N/A

Summary

This release included bug fixes. The QE tasks in the release process include making sure automation jobs pass and upgrade automation jobs pass. 2.15.2 was released on F27.

Pulp QE approve the release of pulp 2.15.2

Beta History
Beta1 : January 23, 2018

Issues Addressed
================

Pulp
2835 Tasks stuck in waiting after restart of pulp services
3129 occasional httpd segfault
3317 rsync_distributors: 'rsync_extra_args' not used in all calls to rsync
3319 Certificates used in unit testing are expired
3349 pulp-admin -vvv returns incorrect information for "worker_name" when querying task results
3356 queue_name property on worker model returns invalid results when the worker name is None
3383 Worker model's "queue_name" property returns the wrong name on Celery 4.x

Puppet Support
3314 puppet install distributor broken on F27 due to SELinux denials

RPM Support
3342 Relative import breaks Python 2.4 (RHEL 5)


Pulp 2 15 3 Test Result Summary

Summary

Started On: March 13, 2018
Finished On: March 19, 2018
Version: 2.15.3
Result: Success
Build URL: N/A

Summary

This release included bug fixes. The QE tasks in the release process include making sure automation jobs pass and upgrade automation jobs pass. 2.15.2 was released on F27.

Pulp QE approve the release of pulp 2.15.3

Beta History
Beta1 : March 13, 2018

Issues Addressed
================

Docker Support
3286 Sync logic should not assume on schema1 by default existence
3357 docker_distributor_web is racy, not atomic
3258 Update with new recipe for v2s1 manifest upload

Pulp
3246 Update branching.rst to match pup-0003
3347 Advise users to use `setsebool` to set pulp_manage_rsync
3386 PEP8 Pulp's docs/conf.py

RPM Support
3307 import_upload: improve data validation
3339 Missing docs on how to import RPM package which has rich dependencies in Requires
3353 Missing docs on importing RPM repository module metadata
3411 Document Implications that Pulp2 does not support metalink for rpm syncing

Summary:

Most of the issues are documentation-related. The candidates for automation are:
Issue #3286: Sync logic should not assume on schema1 by default existence
Issue #3357: docker_distributor_web is racy, not atomic
Issue #3307: import_upload: improve data validation
#3286 might be automatable. To automate this issue, we need a test that does the following:

#3357 has a reproducer that requires patching source code.

#3307 can be tested quite easily. We already have tests that upload files to Pulp. To automate this issue, we need a test that starts and upload, but provides some invalid parameters.


Pulp 2 15 Test Result Summary

Summary

Status: [ ] Not Started [ ] In Progress [ ] Paused [ ] Blocked [X ] Finished
Started On: November 28, 2017
Finished On: January 16. 2018
Beta/RC URL: https://repos.fedorapeople.org/pulp/pulp/beta/2.14/
Redmine URL: http://https://pulp.plan.io/issues?query_id=61

Automated tests have increased from 528 in 2.14 to 584 in 2.15. The jobs were run nightly on EL7, Fedora 25 and Fedora 26. Test run on Fedora 27 failed due to SELinux issues [1]. Upgrade jobs were run on RC builds. Upgrade test jobs passed for EL7, F25 and F26. Manual testing was limited to the scenarios that do not yet have pulp smash tests. All the issues have been added to pulp-smash backlog to automate or are already automated [2].

Pulp QE team approves the release of the 2.15 RC to GA.

Beta History
Beta 1: November 28, 2017
Beta2: December 5, 2017
Beta3: December 18, 2017
RC 1: January 10, 2017

Automation

[[Status: [X] PASSED [ ] FAILED
System: x86_64
RHEL 7.4
Installation PASS PASS
Smoke PASS PASS
API PASS PASS
CLI PASS PASS
Upgrade PASS PASS]]

NOTES:

New Features
============

Here are the specific stories done for 2.15:

Pulp
      3119    As a user, I can see in logs that force_full option for sync or publish is enabled
      1282    As an EC2 user, I would like to set up a RHUI as an alternate content source
Crane
      3110    As a user i can perform docker search of docker v2 content
Docker Support
      2993    As a user, I can upload a docker_manifest_list so that I can remove arches from a manifest list without performing a sync
      2810    As a user, I can upload a schema 2 manifest with blobs
RPM Support
      3055    As a user, I can publish a Yum repository that works with repo_gpgcheck=1
      2788    As a user i can configure removal of old published repodata

View this list in redmine:
http://bit.ly/2j1OeW9

Issues Addressed
================

Here are the bug fixes specific to 2.15:

Crane
    3111    Crane should continue to load rest of the metadata after corrupted data is encountered
Docker Support
    3122    config layer is served gzip-compressed to docker client
    3067    Document image manifest and manifest list peculiar behaviour during copy, remove and other actions
    3139    Sync from Crane fails with "Not Found"
RPM Support
    3130    Upgrade from Pulp 2.13.2- to Pulp 2.13.3+ can result in "duplicate key error index: pulp_database.erratum_pkglists.$errata_id_1_repo_id_1 dup key"
    3104    repomd.xml is empty

All bug fixes from Pulp 2.14.3 and earlier are also included in Pulp 2.15.

View this list in redmine:
http://bit.ly/2AieENo

*Action Items *

[1] https://pulp.plan.io/issues/3159
[2] https://github.com/PulpQE/pulp-smash/issues/766
[3] https://github.com/PulpQE/pulp-smash/issues/681
[4] https://github.com/PulpQE/pulp-smash/issues/829


Pulp 2 16.1 Test Result Summary

Summary

Status: [ ] Not Started [ ] In Progress [ ] Paused [ ] Blocked [X ] Finished
Started On:
Finished On: April 3, 2018
GA URL: https://repos.fedorapeople.org/repos/pulp/pulp/stable/2.16/
Redmine URL: https://pulp.plan.io/issues?query_id=59

The QE tasks included in the release process included manual verification some of the issues included in the release, making sure automation jobs pass and upgrade automation jobs passing.

Pulp QE team approves the release of the 2.16.1 RC to GA.

Beta History
Beta 1: April 24, 2018

RC 1: May 03, 2018

Automation

[[Status: [X] PASSED [ ] FAILED
System: x86_64
RHEL 7.5
Installation PASS PASS
Smoke PASS PASS
API PASS PASS
CLI PASS PASS
Upgrade PASS PASS]]

Issues Addressed
================

Docker Support
3462 Pulp produces incorrect crane json for removed tags

RPM Support
3564 2.16 update broke EPEL sync - Task Failed Importer indicated a failed response
3440 It is not possible to skip SRPMs during sync, only together with RPMs


Pulp 2 16.2 Test Result Summary

Summary

Status: [ ] Not Started [ ] In Progress [ ] Paused [ ] Blocked [X] Finished
Started On:
Finished On: June 26, 2018
GA URL: https://repos.fedorapeople.org/repos/pulp/pulp/stable/2.16/
Redmine URL: https://pulp.plan.io/issues?query_id=59

The QE tasks included in the release process included manual verification of some of the issues included in the release, making sure automation jobs pass and upgrade automation jobs passing.

Pulp QE team approves the release of the 2.16.2 RC to GA.

Beta History
Beta 1: June 19, 2018

RC 1: June 26, 2018

Automation

[[Status: [X] PASSED [ ] FAILED
System: x86_64
RHEL 7.5
Installation PASS PASS
Smoke PASS PASS
API PASS PASS
CLI PASS PASS
Upgrade (Not working, builds are broken but this issue is not blocking the release)]]

Issues Addressed
================

Docker Support 4
  3703    Docker Support    Update image upload to be compatible with skopeo Directory Transport 1.1    Actions
  3573    Docker Support    Not able to sync from google registry.    Actions
  3667    Docker Support    Merrory error when trying to upload a big tar    Actions
  3686    Docker Support    Misleading error when trying to sync private repo with wrong creds    Actions
Pulp 8
  3687    Pulp    Pulp 2 doesn't work on Fedora 27+    Actions
  3521    Pulp    last_override_config exposes sensitive info    Actions
  3745    Pulp    Applicability task fails if consumer is removed in the middle of the task    Actions
  3768    Pulp    FIPS related failures    Actions
  3642    Pulp    Get Pulp 2 working with mongodb in a FIPS-enabled environment    Actions
  3646    Pulp    Get scripts (e.g. pulp-gen-ca-certificate) working on FIPS enabled environment    Actions
  3663    Pulp    Investigate orphan cleanup performance    Actions
  3676    Pulp    Get pulp-admin login working in FIPS mode    Actions
Puppet Support 1
  3753    Puppet Support    Confirm that pulp_puppet works in FIPS mode    Actions
RPM Support 3
  3654    RPM Support    CursorNotFound during checksums_and_templates migration    Actions
  3172    RPM Support    Celery worker consumes large number of memory when regenerating applicability for a consumer that binds to many repositories with many errata.    Actions
  3535    RPM Support    Error syncing Oracle EPEL repository

Pulp 2.16.3 Test Result Summary

Status: Approved

Testing History

2018-06-29: RC released
2018-06-29: RC approved

This release was expedited, to fix an regression found in 2.16.2. For this release, QE verified that all existing automated tests pass, and that the regression is fixed. Upgrade tests are not working, but this is a known issue that is not specific to this particular release.

Issues Addressed

RPM support

https://pulp.plan.io/issues/3795 Errata are not shown as applicable if epoch info is absent in pkglist


Pulp 2 16 Test Result Summary

Summary

Status: [ ] Not Started [ ] In Progress [ ] Paused [ ] Blocked [X ] Finished
Started On:
Finished On: April 3, 2018
Beta/RC URL: https://repos.fedorapeople.org/pulp/pulp/beta/2.16/
Redmine URL: http://https://pulp.plan.io/issues?query_id=61

Automated tests have increased from 584 in 2.15 to 614 in 2.16. The jobs were run nightly on EL7, Fedora 25 and Fedora 26 and Fedora 27. Upgrade jobs were run on RC builds. Upgrade had some issues related to the upgrade to Celery 4.0, but they were addressed by the release engineering before RC. Manual testing was limited to the scenarios that do not yet have pulp smash tests.

Pulp QE team approves the release of the 2.16 RC to GA.

Beta History
Beta 1: March 20, 2018

RC 1: March 27, 2018

Automation

[[Status: [X] PASSED [ ] FAILED
System: x86_64
RHEL 7.4
Installation PASS PASS
Smoke PASS PASS
API PASS PASS
CLI PASS PASS
Upgrade PASS PASS]]

Issues Addressed
================

Debian Support
80 Design data model to support Debian repos
2938 Add pep8speaks
2764 Ability to publish more than one dist+component combination
2765 Ability to sync more than one dist/component

Packaging
3407 Upgrade the Celery stack that Pulp ships on EL7 to Celery + Kombu 4.x

Pulp
3135 As a user, I have a setting to mitigate when workers go missing under heavy loads
3352 Make config.get_boolean return given default value

RPM Support
3474 gpg_cmd configuration option should not be accepted in repo config or overrides
3091 As a user, I can create a manifest for the files in a directory
3377 Add support for SUSE Errata format
3444 I can sign packages ONLY with gpg, and only with one key

Issues automated:

https://pulp.plan.io/issues/3377

Issue added to the automation backlog

https://pulp.plan.io/issues/3474


Pulp 2 17.0 Test Result Summary

Summary

Status: [ ] Not Started [ ] In Progress [ ] Paused [ ] Blocked [x] Finished
Started On:
Finished On: Aug 31, 2018
GA URL: https://repos.fedorapeople.org/repos/pulp/pulp/stable/2.17/
Redmine URL: https://pulp.plan.io/versions/63

The QE tasks included in the release process included manual verification of some of the issues included in the release, making sure automation jobs pass and upgrade automation jobs passing.

Issues that had verification flag:

https://pulp.plan.io/issues/3847
https://pulp.plan.io/issues/3715

Pulp QE team approves the release of the 2.17.0 RC to GA.

Beta Release date:
August 14, 2018
Generally Available
Aug 31, 2018

Automation

[[Status: [X] PASSED [ ] FAILED
System: x86_64
RHEL 7.5
Installation PASS PASS
Smoke PASS PASS
API PASS PASS
CLI PASS PASS
Upgrade (Not working, builds are broken but this issue is not blocking the release)]]


Pulp 2 17.1 Test Result Summary

Summary

Status: [ ] Not Started [ ] In Progress [ ] Paused [ ] Blocked [x] Finished
Started On:
Finished On: Oct 12, 2018
GA URL: https://repos.fedorapeople.org/repos/pulp/pulp/stable/2.17/

The QE tasks included in the release process included manual verification of some of the issues included in the release, making sure automation jobs pass and upgrade automation jobs passing.

Pulp QE team approves the release of the 2.17.1 RC to GA.

Beta Release date:
Oct 04, 2018
Generally Available
Oct 12, 2018

Automation

[[Status: [X] PASSED [ ] FAILED
System: x86_64
RHEL 7.5
Installation PASS PASS
Smoke PASS PASS
API PASS PASS
CLI PASS PASS
Upgrade (Not working, builds are broken but this issue is not blocking the release)]]


Pulp 2 18 0 Test Result Summary

Summary
Status: [ ] Not Started [ ] In Progress [ ] Paused [ ] Blocked [x] Finished
Started On:
Finished On: Dec 05, 2018
GA URL: https://repos.fedorapeople.org/repos/pulp/pulp/stable/2.18/
The QE tasks included in the release process included manual verification of some of the issues included in the release, making sure automation jobs pass and upgrade process passes – manually verified.
Pulp QE team approves the release of the 2.18 RC to GA.
Beta Release date:
Nov 20, 2018
Generally Available
Dec 05, 2018
Automation
[[Status: [X] PASSED [ ] FAILED
System: x86_64
RHEL 7.6
Installation PASS
Smoke PASS
API PASS
CLI PASS
Upgrade PASS


Pulp 2 18 1 Test Result Summary

Summary

Status:

[] Not Started [] In Progress [ ] Paused [ ] Blocked [X] Finished

Tracking Ticket: https://pulp.plan.io/issues/4359

Status Meta

The QE tasks included in the release process included manual verification of some of the issues included in the release, making sure automation jobs pass and upgrade automation jobs passing.

Changes were made to FIPS and non-FIPS installation to prioritize pulp.repo installation over epel or other source repos. Without, a resulting failure, such as with amqp, can and did occur.

See the above Tracking Ticket for more information, linked to #4387

Started On: 2019-02-19
Finished On: 2019-02-20
GA URL: https://repos.fedorapeople.org/repos/pulp/pulp/stable/2.18/
Redmine URL: https://pulp.plan.io/projects/pulp/wiki/2181_Release_Schedule https://pulp.plan.io/issues?query_id=123
Development Release Information: https://pulp.plan.io/projects/pulp/wiki/2181_Release_Schedule

Automation

Status:

[] In Progress [X] PASSED [ ] FAILED

Info/Topics Covered

Issues Addressed - 6

RPM Support
    4152 Regression Pulp 2.17.1: recursive copy of RPMs does not copy partially resolvable dependencies
    4225 0029_applicability_schema_change.py fails for some users
    4252 modules.yaml file is generated on repository with no modularity information
    4253 modules.yaml reference in repomd.xml does not use selected checksum
    4309 Vendor field migration fails with 'NoneType' object has no attribute 'text'
OS Tree Support
    3999 Publishing incorrect branch head.

Pulp 2 19 0 Test Result Summary

Summary

Status:

[] Not Started [] In Progress [ ] Paused [ ] Blocked [X] Finished

Status Meta

The QE tasks included in the release process included manual verification of some of the issues included in the release, making sure automation jobs pass and upgrade automation jobs passing.

Started On: N/A
Finished On: N/A
GA URL: https://repos.fedorapeople.org/repos/pulp/pulp/stable/2.19/
Redmine URL: https://pulp.plan.io/projects/pulp/wiki/2190_Release_Schedule https://pulp.plan.io/issues?query_id=123
Development Release Information: https://pulp.plan.io/projects/pulp/wiki/2190_Release_Schedule

Automation

Status:

[] In Progress [X] PASSED [] FAILED

Info/Topics Covered

Issues Covered


Pulp 2 19 1 Test Result Summary

Summary

Status:

[ ] Not Started [ ] In Progress [ ] Paused [ ] Blocked [X] Finished

Status Meta

The QE tasks included in the release process included manual verification of some of the issues included in the release, making sure automation jobs pass and upgrade automation jobs passing.

Started On: May 22, 2019
Finished On: May 29, 2019
GA URL: https://repos.fedorapeople.org/repos/pulp/pulp/stable/2.19/
Redmine URL: https://pulp.plan.io/projects/pulp/wiki/2191_Release_Schedule
Development Release Information: https://pulp.plan.io/projects/pulp/wiki/2191_Release_Schedule

Automation

Status:

[ ] Not Started [] In Progress [X] PASSED [ ] FAILED

Info/Topics Covered

Issues Covered


Pulp 2 20 0 Test Result Summary

Summary

Status:

[ ] Not Started [ ] In Progress [ ] Paused [ ] Blocked [X] Finished

Status Meta

The QE tasks included in the release process included manual verification of some of the issues included in the release, making sure automation jobs pass and upgrade automation jobs passing.

Started On: 07/03/2019
Finished On: 07/10/2019
GA URL: https://repos.fedorapeople.org/repos/pulp/pulp/stable/2.20/
Redmine URL: https://pulp.plan.io/projects/pulp/wiki/2200_Release_Schedule
Development Release Information: https://pulp.plan.io/projects/pulp/wiki/2200_Release_Schedule

Automation

Status:

[ ] Not Started [] In Progress [X] PASSED [ ] FAILED

Info/Topics Covered

Issues Covered


Pulp 2 Release Planning

This serves as a step-by-step guide to coordinating a Pulp 2 release. This is mostly about facilitating the required communication to keep everyone on the same page.

Preparing a release

1. Identify that a release needs to happen via pulp-dev. This can be something that is requested by anyone who wants to release bits that have been merged.

2. Create a Release Planning Page specific for that release. For example here is a Z Release Status page and a Y Release Status page. At a minimum it should contain the following:

3. Link to the new page made in step (2) from the overall Release Schedule.

4. Communicate the dev feeze datetime to pulp-dev with a link to the new release schedule.

5. Make sure the version being planned has a 'Platform Release' entry in Redmine's custom field. You can edit this here: https://pulp.plan.io/custom_fields/4/edit

6. Update the relevant Redmine filter for the next bugfix or next feature release. Update both the name and the filter value. These queries are important as they show the set of issues for the upcoming release.

Dev Freeze

To coordinate the dev freeze you should send 2 emails to the pulp-dev list.

1. 24 hours (or earlier) prior to dev feeze it's good to send a reminder to pulp-dev. Here is an example
2. After the freeze is done you should send an email with a link to the Redmine query showing the list of fixes and features in that release. Here is an example This email serves also to notify release engineering that development is done for that release and those are the issues.

Besides sending email, after the dev freeze occurs, you need to update the Release Schedule:

1. strikethrough the dev freeze date since it occurred
2. Talk with @pcreech or @ehelms to update the page with a firm (not-tentative) beta date.
3. Add a link to the redmine query for issue to be included.

Before Beta/RC create/update release notes for every project which is a part of that release. It is good to submit them against master and include them into the cherrypicks.

Beta Announcing

To be ready to announce a Beta, you must first receive an ack that the beta is built and ready to be published from the build team. Prior to acking that it's ready, the built team works with engineering to review automated tests ensuring that the beta can be released. Make sure jenkins jobs are green and there are no tests failures. Once the build team says it's ready, do the following:

1. Move all of the issues from MODIFIED to ON_QA
2. Trigger the docs build to ensure the beta's docs are pushed to the right place
3. Create the beta announcements so you're ready to send them. This is a subset of the GA process. Specifically for Betas we only: (a) trigger the docs build, (b) email announce, (c) twitter announce. We do not blog post or update IRC for beta announcements.
4. Add the Beta to the list of Beta/RC on this page with a PR like this one You can merge to that without a review if you are handling the release.
5. Ask the build team to push the bits to the testing repos and wait for the to ack that they did
6. Publish and send out the announcements
7. Strike through the Beta on the Wiki Release page and ensure that the next date (GA) is firm and accurate not tentative

Release Candidate Announcing

To be ready to announce a RC, you must first receive an ack that the RC is built and ready to be published from the build team. Prior to acking that it's ready, the built team works with engineering to review automated tests ensuring that the RC can be released. Make sure jenkins jobs are green and there are no tests failures. Once the build team says it's ready, do the following:

1. Trigger the docs build to ensure the RC's docs are pushed to the right place
2. Create the RC announcements so you're ready to send them. This is a subset of the GA process. Specifically for RC we only: (a) trigger the docs build, (b) email announce, (c) twitter announce. We do not blog post or update IRC for beta announcements.
3. Add the RC to the list of Beta/RC on this page with a PR like this one You can merge to that without a review if you are handling the release.
4. Ask the build team to push the bits to the testing repos and wait for the to ack that they did
5. Publish and send out the announcements
6 Strike through the RC on the Wiki Release page and ensure that the next date (GA) is firm and accurate not tentative

GA Announcing

On GA release day, the build team will build the final assets and work with engineering to have them tested. Make sure jenkins jobs are green and there are no tests failures. Once they are ready a developer needs to trigger a final docs build and then can send the final announcements. There are several announcements: email, blog, twitter, irc, wiki. For the blog you'll need merge rights to the github.com/pulp/pulpproject.org/ repo. For twitter, you'll need the pulpproj twitter credentials, or know someone who can post. For irc updating you'll need to become an op in #pulp or know someone who can.

Update the docs configs in pulp/pulp 2-master branch.

1. In case of a new Y release create new config like this at the RC stage and update at later stages accordingly.
In case of a new Z release update that config accordingly at all stages.

2. In case of a new Y release, update the latest version of the docs and update the supported releases, so the warning that new docs are from unsupported version will go away.

Triggering docs build

0. Ensure the correct version/release is set in the sphinx config on the release branch, e.g. 2.19-release. If it's not, talk to the build team to figure out if you should still wait or if you should change it yourself with a PR.

Use custom travis build to trigger the build:

1. Trigger a build on this page: https://www.travis-ci.org/pulp/pulp/

2. Ensure it passes . You might need to manually trigger the publish of the docs. Ensure build is green 3. Load https://docs.pulpproject.org/ and ensure it shows the expected new version as the home page docs.

Move issues to CLOSED - CURRENTRELEASE

Use the query for all issues in the release to set all issues to CLOSED - CURRENTRELEASE

Email announce

Use the community_announce.py tool to produce the email. Send this email to pulp-list

Blog announce

The same tool above produces a blog post. Post it using these instructions:

1. Make a new .md file in the _posts directory that is dated and named appropriately. e.g. 2018-02-27-pulp-2.15.2-generally-available.md

2. Push a PR with that file and merge it yourself or ask someone in #pulp-dev to merge it. The blog posts do not require review. It should show up on pulpproject.org within a few minutes after you merge it.

Twitter Announce

Post on twitter. Feel free to personalize it, but the announce tool also produces a generic tweet for you.

1. Get the tweet content from the announce tool
2. Add the link to the blog post on the end
3. Post to twitter as @pulpproj

Update IRC

1. Become operator in #pulp with: /msg ChanServ op #pulp
2. Post your content by running something like: /topic http://pulpproject.org/ | Current Release: Pulp 2.15.2 | To report a bug: https://pulp.plan.io/projects/pulp/issues/new | Development chat: #pulp-dev | 2.15.2 Generally Available!
3. Take away your op priviledges with: /msg ChanServ op #pulp -username where username is your irc nick, e.g. bmbouter.

Update the Wiki Release Page

Strikethrough the release date since it's completed. A completed page should be fully struckthrough like this one


Pulp 2 16.4 Test Result Summary

Summary

Status: [ ] Not Started [ ] In Progress [ ] Paused [ ] Blocked [x] Finished
Started On:
Finished On: Aug 3, 2018
GA URL: https://repos.fedorapeople.org/repos/pulp/pulp/stable/2.16/
Redmine URL: https://pulp.plan.io/issues?query_id=59

The QE tasks included in the release process included manual verification of some of the issues included in the release, making sure automation jobs pass and upgrade automation jobs passing.

There are 2 issues with this build on Jenkins

1. tests.rpm.api_v2.test_updateinfo.OpenSuseErrataTestCase.test_01_create_sync_publish
Is failing by timeout to sync the repo https://download.opensuse.org/update/leap/42.3/oss/ this test case is PASSING locally but as it takes long time to run QE will investigate on ways to enhance this test case. ref: https://github.com/PulpQE/Pulp-2-Tests/issues/6

2. tests.puppet.api_v2.test_sync_publish.SyncInvalidFeedTestCase.test_task_error_traceback
One-off test failure, due to network or timeout issues, PASSING locally QE is also putting on to-do list to investigate.

Pulp QE team approves the release of the 2.16.4 RC to GA.

Beta Release date:
July 24, 2018
Generally Available
Aug 3, 2018

Automation

[[Status: [X] PASSED [ ] FAILED
System: x86_64
RHEL 7.5
Installation PASS PASS
Smoke PASS PASS
API PASS PASS
CLI PASS PASS
Upgrade (Not working, builds are broken but this issue is not blocking the release)]]

Issues Addressed (non of the issues required verification)
================
3370 Debian Support Unable to sync Ubuntu Trusty Universe
3498 Debian Support gpg_cmd configuration option should not be accepted in repo config or overrides
3828 Pulp pulp-gen-ca-certificate script fails when /proc/sys/crypto/fips_enabled doesn't exist
3578 Python Support pulp-python wrong path to pypi
3632 Python Support Pulp_python does not normalize project name before publish
3736 RPM Support When the checksum type of the upstream repository changed, syncing a "on_demand" repository will fail


Pulp 3 Developer Notes

This wiki page is intended for use during early development of Pulp 3. Over time, as our development practices become standard, the contents of this page should be moved into the Pulp Contributing Guide

Before reporting issues with the development environment, please ensure that you are using the latest version.

Migrations

In both platform and plugins, the data model is not complete. As a result, committing migrations to the 3.0-dev branch will result in merge/migration conflicts from pull request to pull request. The simplest solution for now is not to commit migrations to the repository.

Because User model depends on Django's auth app having been migrated, this means that you currently need to run python manage.py migrate auth before running a general python manage.py migrate to set up the pulp database.

Making migrations during development

Tests require migrations to run, so while we should not commit migrations to the repositories just yet, we do still need to make them. This can be done with the python manage.py makemigrations command. Apps that depend on the platform migrations existing (such as plugins) may cause errors when making migrations. To avoid these errors, platform migrations should be made prior to installing any plugins.

Once the initial migrations are created, and model changes made thereafter will require python manage.py makemigrations to be run again, following by @python manage.py migrate" so Django can apply the model changes to the database.

If you are using the vagrant environment this is done during provisioning. The pclean alias takes care of migrations after resting the db.

Debugging migrations

Migrate pulp_file
python manage.py makemigrations pulp_file

Migrate pulp_app first because (sometimes) makemigrations does not always create the initial migrations if they are not already there
python manage.py makemigrations pulp_app

Starting a Web Server

The Django development server can be started with python manage.py runserver. This will run a basic WSGI app that exposes the URLs routed in urls.py, allowing you to access the REST API.

If you're using the vagrant hostmanager plugin, you can easily access the API from the host machine by explicitly binding the web server to all interfaces, e.g. python manage.py runserver 0.0.0.0:8000. This should make the API browseable at http://dev.example.com:8000/api/v3/

Authentication

We currently enable Basic HTTP Authentication on the REST API. This can be temporarily disabled by commenting out the 'DEFAULT_PERMISSION_CLASSES': ('rest_framework.permissions.IsAuthenticated',) line in the REST_FRAMEWORK section in app/pulp/app/settings.py. Note that this doesn't disable authentication, it just authorizes unauthenticated users to take any action. Basic Authentication should still work.

Tasks

Starting Services

If using the vagrant environment, you can start the services with the alias pstart. Afterwards, use prestart.

To start the processes manually, if you are using a virtual environment, be sure that the path to celery matches the virtual environment that includes pulp, any plugins, and pulp dependencies.

sudo -u apache /usr/bin/celery beat --app=pulpcore.tasking.celery_app:celery --scheduler=pulpcore.tasking.services.scheduler.Scheduler -l=INFO --pidfile=/var/run/pulp/scheduler.pid

sudo -u apache /usr/bin/celery worker -A pulpcore.tasking.celery_app:celery -n resource_manager@%%h\
            -Q resource_manager -c 1 --events --umask 18 --pidfile=/var/run/pulp/resource_manager.pid\
            --heartbeat-interval=5 -l=INFO

sudo -u apache /usr/bin/celery worker -n reserved_resource_worker-123s@h\
          -A pulpcore.tasking.celery_app:celery -c 1 --events --umask 18\
          --pidfile=/var/run/pulp/reserved_resource_worker-123s.pid\
          --heartbeat-interval=5 -l=INFO

Deploy tasks from shell

apply_async and apply_async_with_reservation tasks can be tested from a django shell python manage.py shell_plus I usually do the following to test tasks

from pulpcore.app.tasks import repository
from pulpcore.app.models import Repository
import uuid
repo_uuid=str(uuid.uuid4())
repo=Repository(name=repo_uuid)
repo.save()

repository.delete.apply_async(kwargs={'repo_id':repo_uuid})
repository.delete.apply_async_with_reservation("foo","bar",kwargs={'repo_id':repo_uuid})

Deploy sync task from REST API

1. Make sure that you have the file plugin installed with a developer setup.
2. Define a sync method on the FileImporter model.
3. Create a repository on the browseable api:
http://dev.example.com:8000/api/v3/repositories/
4. Create an importer. Note, you must specify the repository and the plugin in the URL.
http://dev.example.com:8000/api/v3/repositories/<repository_name>/importers/file/
5. Sync at the importer's url http://dev.example.com:8000/api/v3/repositories/<repository_name>importers/file/<importer_name>/sync

PyPI

Publishing to PyPI

python setup.py sdist bdist_wheel --python-tag py3
twine upload -s dist/{package}*

Installing from PyPI

pip3 install pulpcore
export DJANGO_SETTINGS_MODULE=pulpcore.app.settings
set up database and migrations
django-admin runserver

Known Issues


Pulp 3.0.0 Minimum Viable Product (MVP)

Lines highlighted in red need more attention.

Overall Guarantees

Authentication

As an authenticated user I can manage user(s). [done]

As an API user, I can authenticate any API call with Basic auth [done]

As an authenticated user, I can filter users by: [3142]

Repositories

As an authenticated user, I can list all repos.

As an authenticated user I can use filters on Repositories list: [3079]

As an authenticated user, I can CRUD a repository

As an authenticated user, when viewing a repository, I can discover a URL to the latest version of a repository. [done][3235]

Remotes

note: Remote attributes will commonly be available on remotes, but aren't guaranteed to be used by all remotes.

As an authenticated user, I can CRUD an remote

As an authenticated user, I have filters on the Remote list: [3080]

As an authenticated user I can configure the following attributes on an Remote: [done]

Publishers

note: Publisher attributes will commonly be available on publishers, but aren't guaranteed to be used by all publishers.

As an authenticated user, I can CRUD a publisher

As an authenticated user, I have filters on the Publisher list: [3081]

As an authenticated user I can configure the following attributes on a Publisher:

Distributions

As an authenticated user, I can CRUD Distributions:

As a user, my distribution base paths don't conflict and my create/update is rejected identifying the conflicting distributions [2987]

As an authenticated user, I can create or update a distribution that is not associated with any publication (NULL)

As an authenticated user, I can create or update a distribution that is not associated with any publisher/repository (NULL)

As a user, I can update a Distribution to distribute a specific Publication
As a user, I want a newly created publication to be automatically served by the content as defined by distributions.

As a user, I can see the full urls my base path is served at
As an authenticated user, I have filters on the Distribution list: [3082]%

Publications

As an authenticated user, when creating a Publication, I can post a repo version href to be published. [3221]

As an authenticated user, I can publish a repository's latest version by posting a repository href to be published. [3223]

As an authenticated user, I can view which repository version was used to create a particular publication. [3237]

As an authenticated user, I can read Publication(s)

As an authenticated user, I can delete publications.
- asynchronously with a lock on the repository version.
- prevented if associated with a distribution.
- single object only.

As an authenticated user, I can list publications and I have enough information to choose which ones to delete.

As an authenticated user, I can list publications and i have enough information to select a publication to be associated with a distribution.

As an authenticated user, I can determine if and how a publication is distributed.

Exporters

As a plugin writer, I can contribute an exporter that is discovered by core
As a plugin writer, I have docs on how to create a discoverable exporter
As a plugin writer, I can contribute an exporter that uses a publication.

Artifacts

As an authenticated user, I can create an Artifact by uploading a file. [done]

As an authenticated user, I can optionally specify a size and/or digest to validate the uploaded file. [done]

Content Units

As an authenticated user, I can create a content unit.

As an authenticated user, I can read and list content units.
* As an authenticated user, I can filter content by repository version

As an authenticated user, I can delete a specific content unit if it is not in any repository version. (orphan)

Versioned Repositories

CRD

As an authenticated user, I can list versions for a particular repository. [done]

As an authenticated user, I can filter repository versions by: [3238]

As an authenticated user, I can delete any repository version. [3219]

As an authenticated user, I can view content that was added or removed in a particular repository version (as compared to the previous version). [done]

As an authenticated user, I can see the number of content unit types with counts for each [done][3059]

Repository Version Content

As a user, I know content sets for repository versions are immutable. [done]

As an authenticated user, I can list the content in a particular repository version [done]

As an authenticated user, I can create a new version by adding or removing content to the latest version. [3234]

Orphan Content Units and Artifacts

As an authenticated user, I can cause an action that cleans up both orphaned content units and orphaned artifacts.

Task Management

As an authenticated user, I can list all tasks

As an authenticated user, I can see a detail view for a specific task [done]

As an authenticated user, I can cancel a task [done]

As an authenticated user, I can delete tasks.

As an authenticated user, I can filter tasks by: [3144]

Status

As an unauthenticated user I can view the status of Pulp workers and resource managers. [done]

As an unauthenticated user I can view the status of the web server's connection to the database and message broker. [done]

As an unauthenticated user I can view the versions of core and each installed plugin.

Workers

As an authenticated user, I can filter workers by: [3143]

Plugin User Content Management stories

Simple Copy

As an authenticated user of a plugin, I can search (synchronous call) a repository version's content using filtering.

Complex Copy

As a plugin writer I can provide a rich search features with arbitrary viewsets. e.g. depsolving, versioning, etc

Examples of specific plugin use cases motivating the above general viewset

Plugin API

As a plugin writer, I have a plugin API that is semantically versioned at 0.x separate from the REST API [done]

As a plugin writer, my app will be discovered by Pulp's app via an entry point provided by the plugin writer [done]

Task

As a plugin writer, I can report progress with a message and state [done]

As a plugin writer, I can report progress with an optional suffix [done]

As a plugin writer, I can report progress with a total count of things to do an the current count of things done [done]

As a plugin writer, non-fatal exceptions on the Task and are included in the Task detail. non_fatal exceptions do not cause the Task to be marked as failed, but may be interpreted by the user as not fully successful. [done]

As a plugin writer, the working directory is set before Task work is done and cleaned up afterwards. I should not need to interact with the file system outside of the working dir. [done]

Remote

As a plugin writer, I can provide a subclassed Remote. I can add custom fields to the subclassed Remote.

As a plugin writer, I can provide a UserFacingTask to perform a sync operation using information stored in a Remote.

As a plugin writer, I can provide a ViewSet for the subclassed Remote that has a 'sync' endpoint that dispatches the sync task with a reservation for Repository and Remote.

Publisher

As a plugin writer, I can provide a subclassed Publisher. I can add custom fields to the subclassed Publisher.

As a plugin writer, I can provide a UserFacingTask to perform a publish operation using information from the subclassed Publisher.

As a plugin writer, I can provide a ViewSet for the subclassed Publisher that has a 'publish' endpoint that dispatches the publish task with a reservation for Repository and Publisher.

Content

As a plugin writer, I can provide a subclassed Content unit. I can add custom fields to the subclassed Content. [done]

As a plugin writer, I have documentation that shows how I can add filters to filter content responsibly.

As a plugin writer, I have documentation on how to write a filter for my Content that can use the RepositoryVersion manager.
* note: This will allow users to filter content by repository version

Content Management

As a plugin writer, I can interact with and create Artifacts [done]

As a plugin writer, I can query content units/artifacts associated with a repository. [done]

As a plugin writer, I can create a new repository version: [done]

Publishing

As a plugin writer, I can create a new publication: [done]

"live APIs"

As a plugin writer, I can register views and viewsets to arbitrary endpoints. [3360]
As a plugin writer, I have documentation on what URLs I should not use for my views and viewsets [3473]

Here are some concrete use cases driving the very Live API use cases above:

# Concrete user use cases:
    As an authenticated user, I can use the puppet client to fetch content from Pulp using the Forge API
    As an authenticated user I can use the docker client to fetch content from Pulp using the Docker v1 API
    As an authenticated user I can use the docker client to fetch content from Pulp using the Docker v2 API

# Concrete plugin writer use cases
    As a puppet plugin developer, I can provide a viewset which handles the server side of the puppet Forge v3 API
    As a docker plugin developer, I can provide a viewset which handles the server side of the docker v1 API
    As a docker plugin developer, I can provide a viewset which handles the server side of the docker v2 API

h4. Storage

As a plugin writer, the plugin API provides an API that returns a fully qualified path to a shared and namespaced storage location used to store content. ["3182"https://pulp.plan.io/issues/3182\]
******

Webserver Deployment

As a system administrator, I can deploy all Pulp web applications on one process
As a system administrator, I can deploy the Pulp REST API exclusively in one process
As a system administrator, I can deploy the Pulp content serving view exclusively in one process

As a system administrator, I can deploy all Pulp web applications inside a virtualenv.
As a system administrator, I can deploy all Pulp web applications without root permissions.

CLI

We will use coreapi-cli to generate a one to one mapping of cli commands to rest api schema #3068
We will have a wrapper for coreapi-cli. This wrapper will handle parallel progress reporting

Download API

As a plugin writer, I can download files via

As a plugin writer, I can configure a downloader with:

As a plugin writer, I can provide arbitrary behaviors for customized downloaders

As a plugin writer, I can have connection pooling/reuse

As a plugin writer, I have proxy settings

As a plugin writer, I can have great logs

As an authenticated user, I have documentation about how to use something for bandwidth limiting

As a plugin writer, I can configure the validation mechanisms used at download time

As a plugin writer I can manage the catalog by using ChangeSets

As a plugin writer, the plugin can participate in adding content for cases where the decision to add additional content is based on the content that has been downloaded.

As a plugin writer, I can fetch content myself (but I am not encouraged to do so) with code I write

As a plugin writer, I can CRUD content units need a convention to handle multiple content units - see https://pulp.plan.io/issues/3472

Migrations only involving Pulp 3

Users can run "pulp-manager migrate" to migrate the database and adjust state in other locations (filesystem, message broker, ...). [done]

Web Server Integration

As a user, I can have content efficiently served to me by Apache by Pulp using the X-SEND response headers. [done]
As a user, I can have content efficiently served to me by Nginx by Pulp using the X-Accel-Redirect response headers. [done]

Glossary

Add (Content Unit): An operation causing a new repository version to contain a content unit(s)

Applicability - A plugin defined term meaning when a package update available in a repository is applicable to a given consumer as determined by the Consumer Profile.

Artifact - A file associated with one content (unit). Artifacts are not shared between content (units). Create a content unit using an uploaded file ID as the source for its metadata. Create Artifacts associated with the content unit using an uploaded file ID for each; commit as a single transaction.

Content (unit) - A single piece of content manged by Pulp. Each file associated with a content (unit) is called an Artifact. Each content (unit) may have zero or many Artifacts.

Distribution: Where and how the content app serves a Publication. i.e. http vs https and base path component of the URL. A Distribution defines:

Exporter: An Exporter exports a Publication out of Pulp. e.g. rsync exporter exports content to a remote server

Live API: a viewset endpoint contributed by plugin. For examples see the associated MVP section

Orphan Artifact: An Artifact that is associated with 0 Content Units and 0 Publications

Orphan Content (unit): A content unit that is a member of 0 repository versions

Remove (content unit): An operation causing a new repository version to not contain a content unit(s)

Repository - A named collection of repository versions.

Repository Version - An immutable set of content which is versioned by a sequential number.


Pulp 3 Roadmap


Pulp 3.0 Smoke Test Plan

Verify that the following functionalities are working. This document assumes that the tests are performed by an authenticated user unless otherwise specified. We will use file plugin to test some of these core functionalities but will not be testing File Plugin itself.

Authentication

  1. Add a user

       Userid -Required
       Username - Required
       Password - Required
       Other info - Non Required
     
    
  2. View user(s)

        List single user
        List single user detail
        List multiple users
    
  3. Update any user details

        Update required user info
        Update optional user info
        List the user to verify
    
  4. Delete a user

       Delete a user by id
       Delete admin user
       Delete last user
    

Repository

  1. Using the API Create a repo

       Required info
       Optional info   
    
  2. Update all mutable repo fields

    Update required info
    Update optional info
    Add optional info
    Remove optional info
    
  3. Delete a repo

     Delete a repo by repo-id
     Delete multiple repos
     
    
  4. List all repos

       # All fields are included
       # Pagination is supported
    # List a repository's associated importers and publishers by type
       # All fields are included
       # Pagination is supported
     
    

Importer

  1. Using the API Create an importer

       Required info
       Optional info   
    
  1. Read an importer

       # All fields are included 
    # List a repository's associated importers and publishers by type
       # All fields are included
     
    
  2. Update all mutable importer fields

    Update required info
    Update optional info
    Add optional info
    Remove optional info
    
  3. Delete an importer (asynchronous)

 Delete an importer
 Delete multiple importers
 

Publisher

  1. Create a publisher

       Required info
       Optional info   
    
  1. Read a publisher
   # All fields are included 
# List a repository's associated importers and publishers by type
   # All fields are included
 
  1. Update all mutable publisher fields

    Update required info
    Update optional info
    Add optional info
    Remove optional info
    
  1. Delete a publisher (asynchronous)

     Delete a publisher
     Delete multiple publishers
    

Tasks

  1. List all tasks

      All tasks are listed
      Tasks can be filtered by ['state', 'id', 'group']
      See a detail view for a specific task 
    
  2. Cancel a task

    Cancel a task by task_id
    Cancel completed task
    Cancel multiple tasks
    

*Un Authenticated user

  1. View the status of Pulp workers, resource managers, and celerybeats.

    View pulp-worker status
    View resource manager status
    View celerybeat status
    
  2. View the status of httpd's connection to the database and message broker.


Pulp Automation Services

This is not an exhaustive list. If any services are missing, please add them as needed.

The "automation" VM

Some of our automated services live on the "automation" VM, which currently resides in the Openstack tenancy provided by Red Hat Central CI. Being provided by Central CI, this system is only accessible if you have the Red Hat corporate network, similar to our Jenkins instance itself. Access to this VM is freely provided as-needed to Red Hat Pulp team members.

nodepool

Nodepool is a service created by the Openstack project, and is responsible for creating Jenkins slave nodes for our Jenkins master. Nodepool periodically inspects the jenkins job queue, and spawns nodes matching the node label require by jobs in that queue. Once a job completes, nodepool destroys that slave node. In this way, nodepool is able to efficiently utilize the resource quotas in our Openstack tenancy by ensuring that only nodes actually needed by Jenkins are online at any given time.

Instance Types

Replace <dist> in the list below with an identifier that obviously represents a given OS distribution. For example, f24-np would be based off of a Fedora 24 Cloud image, and rhel7-vanilla-np would be based off of a RHEL 7 Cloud image.

Common node labels available to Jenkins jobs:

Holding Instances

It is occasionally useful, when debugging or reproducing issues, to hold a nodepool instance. Holding an instance prevents nodepool from destroying it, allowing use to get on the held instance and inspect/interact with that node.

Upon logging into the pulp automation VM, you can run the "nodepool" command. To list instances, run "nodepool list". To hold an instance, run "nodepool hold <instance id>". To log in to an instance, ssh to that instance by IP address from the automation VM. The automation VM's ssh client config will ensure that you are able log into the node with the correct user ID. Both the instance ID and IP Address are listed in the output of "nodepool list". Futhermore, the instance label and ID are both included in the instance's name, which can be seen in Jenkins itself when viewing the "Build Executor Status" block, found on the left side of Jenkins dashboard pages. the node name template is <label>-<provider>-<id>. Provider is always a single word, so a node named "rhel7-vanilla-np-cios-54482" means the node label is "rhel7-vanilla-np", and the node ID is 54482.

Rebuilding images

Nodepool rebuilds images nightly for each node label. Nodepool also keeps the previous image for a given node label, so under normal circumstances there will be two images for a label. Older images are deleted at the end of nodepool's nightly image rebuild.

Occasionally, nodepool's automated image build fails. If this happens, we normally do one of two things. We either delete the most recently created image for the affected label and fall back on the previous image, or we rebuild the image.

To delete the most recently created image, run "nodepool image-list" to get the list of images. Find the image(s) matching the affected label, and then delete the most recent image. You can see which is most recent by looking at the "Age" column and picking the one with a smaller duration, or looking at the "Version" column and deleting the one with a higher version.

To rebuild an image for a label, run "nodepool image-update <provider> <label>". The provider for an image can been seen in the output of "nodepool image-list".

Once a new image is available (either by deleting the most recent image or building a new one), any current instances running the failing image should be destroyed. It's not very easy to cross-reference images with the instances running them, so the simplest thing to do is delete all instances that appear in "nodepool list" in the "ready" state for the affected label.


Pulp Container Roadmap

This is a living document that is moving towards a long term plan to develop Container plugin for Pulp 3.0 and beyond.
With Pulp as a Container registry you can:

- Mirror container image repositories hosted on Docker-hub, Google Container Registry, Quay.io, etc
- Reduce disk use space by mirroring container image repositories using the "on demand" policy. An image is only downloaded once it has been requested by a client.
- Use local filesystem or an object storage store such as S3 to host all the container images TBD
- Curate container images by whitelisting what is mirrored from an external repository.
- Curate container images by creating repository versions with a specific set of images.
- Create versioned repositories that can be promoted or rolled back with a single operation.

Pulp also has:
- tasking system that can be used to perform a variety of specialized work such as analysis of content. e.g. integration with clair-scanner
- a large community of users
- a commuity of plugin developers

green

MVP

Supported Content Types

Pulp Container Plugin Use Cases

MVP

Sync

NOTE: dropping enable_v1, enable_v2, mask_id options

Publish

As a result the above section should enable clients to perform `docker/podman pull`

NOTE dropping protected option

Filtering
Addition of the content to the repo with deps
Removal of the content from the repo with deps

* As a user I can remove Manifest and all Blobs it references from the repo
* As a user I can remove Manifest List and all Manifests and Blobs it references from the repo
* As a user I can remove Tag and all its' associated content it references from the repo

NOTE units that are referenced by other units are not removed

Copy of the content form source repo to the dest repo with deps
Adding/removing Tag via pulp api

Post-MVP 4.1+ ( subject to extension)

Sync
Publish
Force removal of the content from the repo with deps
Export
Docker/podman push
Skopeo copy
Enable v2/catalog endpoint

What will be dropped in Pulp3


Pulp Licensing FAQ

What license does pulpcore and pulpcore-plugin use?

The pulpcore and pulpcore-plugin packages both extend their license to users a "GPLv2 or later".

https://github.com/pulp/pulpcore/blob/master/COPYRIGHT#L5
https://github.com/pulp/pulpcore-plugin/blob/master/COPYRIGHT#L5

So a plugin can choose either GPLv2 or GPLv3.

Why would a Pulp plugin be affected by the license of the pulpcore-plugin package?

The GPL FAQ states that subclassing as a mechanism is creating a derivative work. https://www.gnu.org/licenses/gpl-faq.en.html#OOPLang

Coupled with the fact that the plugin API uses subclassing as a mechanism ( https://docs.pulpproject.org/en/pulpcore-plugin/nightly/plugin-writer/concepts/index.html#subclassing ) plugin code must be a "GPLv2 or later", that is GPLv2 or GPLv3.

Why does the pulpcore package matter? Plugin code is only imported from pulpcore-plugin, not pulpcore?

Many of the objects in pulpcore-plugin are imported from the pulpcore package itself. There are enough of these cases that offering pulpcore and pulpcore-plugin as separate license types would be too confusing.

Can I combine GPLv2 or GPLv3 licensed code with other license types?

It depends on if the other license is "compatible" with GPLv2 or GPLv3. https://www.gnu.org/licenses/gpl-faq.html#WhatDoesCompatMean

Can I combine GPLv2 or GPLv3 licensed code with Apache 2.0 licensed code?

Yes, you can combine GPLv3 and Apache 2.0 licensed code together into a larger program as long as the larger work is GPLv3 licensed. You can do this because Apache v2 is compatible with GPLv3: https://www.gnu.org/licenses/license-list.html#apache2

You cannot combine GPLv3 and Apache 2.0 licensed code together into a larger program and have the larger work licensed as Apache v2. This is stated as prohibited by Apache Software Foundations interpretation of the Apache v2 license. See https://www.apache.org/licenses/GPL-compatibility.html for more details.

You cannot combine GPLv2 and Apache 2.0 licensed code also as stated here: https://www.gnu.org/licenses/license-list.html#apache2


Pulp Python Roadmap

Legend:
(Red, undecided)
(Yellow, decided- needs story)
(blue, decided with story)
(Black - Finished bc green is hard to read)

This is a living document that is moving towards a long term plan to develop pulp_python 3.0 and beyond.

Pulp Python 3.0 Use Cases

All open issues in this list have the Pulp Python 3.0 GA Sprint/Milestone.

Synced Package Index use case:

As a user, I can configure an importer to sync a list of projects [#2884]

Publish Use Case:

As a user, I can publish a repository (Published projects will include all releases and distributions)

Upload Use Case:

As a user, I can upload individual distribution packages (name, version, platform) [#3197]

As a user I have documentation that covers all above in this format:

Pulp Python 3.1+ Use Cases

Upload Use Case:

A twine user can publish a distribution package to Pulp [#2887]

Migrations:

As a user, I can migrate Pulp2 packages to Pulp3 in-place. [#2888]

Cache Use Case:

As a user, I can use Pulp as a PyPI cache.

Granular Sync

Blacklist: As a user, I can disinclude some content of a project

>> By specifying (release, distribution package) (Included in 3.0)

I can disinclude content by distribution type

> Whitelist (Curated Package index Use Case) As a user, I can sync packages that reproduce a specific environment (Included in 3.0)

From the output of pip freeze (loose use case)
With the name, exact versions, and distribution type (tight use case)

Glossary:

Project

A library, framework, script, plugin, application, or collection of data or other resources, or some combination thereof that is intended to be packaged into a Distribution.
[ https://packaging.python.org/glossary/#term-project ]

Package Index:

A repository of distributions with a web interface to automate package discovery and consumption. (this should align to a pulp repo). A Package Index is assumed to implement the PyPI APIs

Release

A snapshot of a Project at a particular point in time, denoted by a version identifier.
Making a release may entail the publishing of multiple Distributions. For example, if version 1.0 of a project was released, it could be available in both a source distribution format and a Windows installer distribution format.

Distribution Package

"Distribution" for short. If we mean linux, we will say "distro"
A versioned archive file that contains Python packages, modules, and other resource files that are used to distribute a Release. The archive file is what an end-user will download from the internet and install. A project may contain many releases, and releases may contain many distribution packages. Can be type sdist, bdist, etc. "Distribution package" is used instead of "package" to avoid confusion with "import packages" or linux "distributions".

Relevant PEPs:


Pulp RPM Roadmap

This is document is no longer updated. Please see Roadmap/Milestone for RPM support to check what is released in which released and what 

K2 - Katello gap
K3 - Katello gap, lower priority than K2

Supported Content Types

MVP

Post-MVP

Pulp RPM Plugin Use Cases

MVP

Sync
Publish

Post-MVP

Sync
Publish
Export
Upload
Copy

Not much yet discussed use cases

Content Protection
Applicability
Rsync Distributor

Pulp 2 features not present in Pulp 3



Content Unit Creator - Downloads and creates objects

open questions: should this associate each unit as its made?


Content Unit Downloader - Parallelized downloads, but it emits a content unit only when all files are downloaded


Concurrent Downloader - Downloads any number of artifacts in parallel


Synchronous Downloader - download a single file synchronously



Pulp3 Release Guide

This serves as a step-by-step guide to implement a Pulp 3 release.

Releasing Pulpcore

Releases always happen on the X.Y branch. The release itself is a tagged commit.

1. Build the changelog with the towncrier --version x.y.z --draft command. If it looks good rebuild with towncrier --version x.y.z. This will stage git changes for you.

2. Commit the changelog by itself. (first commit)

3. Increment the versions in setup.py and pulpcore/__init__.py to the release versions. e.g. 3.0.1

4. Commit the version bump (second commit). Note the hash of this commit.

5. Increment the versions in setup.py and pulpcore/__init__.py to the next, unreleased version for that branch. For a y-release of 3.1.0 on 'master' would be 3.2.0.dev. For a z-release 3.0.1 on the 3.0 branch, the next unreleased version would be 3.0.2.dev. If you're making a new y-release, also bump the value for stable_branch in template_config.yml.

6. Commit the version bump (third commit).

7. Make a PR and merge it.

8. Make a tag of the "second commit" from above. That one has the changelog and a correct version number. Tag it with e.g. git tag 3.0.1 9fceb02

9. Push the tag with git push origin 3.0.1.

10. Create the github release from the tag. After the PR is merged, go to https://github.com/pulp/pulpcore/releases/new to create a new release from the existing tag.

11. Creating a new tag starts a Travis job that pushes the package to PyPI. Go to https://travis-ci.com/pulp/pulpcore/builds and monitor the job. After the job is done, validate that the new package is on PyPI.

12. Create a pulpcore (Not Shared) Sprint/Milestone with the name of the release, e.g. 3.0.1. You can do this here Then do these Redmine steps below. To make this easier, I take the issue IDs and for a query like this: https://pulp.plan.io/issues?set_filter=1&status_id=*&issue_id=5707,5894,5896,5941,5955,5955,5833,5867,5870,5873,5706,5941

13. Announce to pulp-dev@redhat.com with a summary of all the changes.

14. If making a z-stream release, cherry pick the change log (second commit) to the 'master' branch such as git cherry-pick -x 420dd1ff2f326db454b216f33d848d5267489dfb and merge it to master via a PR

15. If making a y-stream release, create branch and push branch from the tag. See docs on this step in the section below.

Preparing a Y-release

If you are creating a y-stream release, e.g. 3.1.0, you'll need to perform branching. If making a z-stream release you don't need to do this section because you already have a branch to work on.:

1. Update the supported-releases.json file here to the 3.y version in a PR like this one. This drives a javascript banner on docs.pulpproject.org and if not set correctly the latest, supported Y-release will show a banner (incorrectly) that this version is unsupported.

2. Create the new X.Y branch from a clean checkout of the y-release tag. For example to make the 3.1 branch from 3.1.0 we ran:

git fetch pulp    # assumes the remote 'pulp' is https://github.com/pulp/pulpcore
git checkout 3.1.0    # This is the tag you're branching from
git reset --hard 3.1.0
git checkout -b 3.1
git push pulp 3.1

3. Increment the versions in setup.py and pulpcore/__init__.py to the next z-release. So if you are making 3.0.0, the 3.1 branch would become 3.0.1.dev.

4. Commit the version bump to the 3.y branch you created

5. Update the stable_branch variable in template_config.yml and set it to the branch you just created (eg "3.1"). This is used to determine the latest stable branch when for example the cherry pick processor needs to cherry pick commits.

Cherry Picking Process

For pulpcore, Travis has a cron job that runs and checks merged PRs for the 'Needs Cherry Pick' label. Any PR that has that label will be cherry picked and included on a PR which is then tested per the usual process. A human merged that final PR and the label is removed.

Other plugins can opt in to using the plugin_template "cherry_pick_automation" variable. See the plugin_template docs for more infomartion on opting in to this CI/CD feature.

Releasing a Plugin

Releasing a plugin has the same process as described above except it happens on the plugin repo instead of https://github.com/pulp/pulpcore


Pulp3 RPM MVP Use Cases

<your idea here>


Python Plugin Minimum Viable Product

As a user I can CRUD repositories.

As a user I can CRUD a `PythonImporter`

As a user I can CRUD a `PythonPublisher`

As a user I can upload a Python content unit.

As a user I have documentation that covers all above in this format:

For each above:


QE Test Summary

Pulp 2.19.0 Test Result Summary
Pulp 2.18.1 Test Result Summary
Pulp 2.18.0 Test Result Summary
Pulp 2.17.1 Test Result Summary
Pulp 2.17.0 Test Result Summary
Pulp 2.16.4 Test Result Summary
Pulp 2.16.3 Test Result Summary
Pulp 2 16 2 Test Result Summary
Pulp 2 16 1 Test Result Summary
Pulp 2 16 Test Result Summary
Pulp 2 15 3 Test Result Summary
Pulp 2 15 2 Test Result Summary
Pulp 2 15 1 Test Result Summary
Pulp 2 15 Test Result Summary
Pulp 2 14 2 Test Result Summary
Pulp 2 14 1 Test Result Summary
Pulp 2 13 4 Test Result Summary
Pulp 214 Test Result Summary
Pulp 2 13 3 Test Result Summary
Pulp 2 13 2 Test Result Summary
Pulp 2 13 1 Test Result Summary
Pulp 2 13 Test Result Summary
Pulp 2 12 2 Test Result Summary
Pulp 2 12 1 Test Result Summary
Pulp 2 12 Test Result Summary
Pulp 2 11 2 Test Result Summary
Pulp 2 11 1 Test Result Summary
Pulp 2 11 Test Result Sumary
Pulp 2 10 3 Test Result Sumary


Releases In Progress

Future Releases

Following the 2.11 release, Pulp development will have transitioned almost entirely to working on the upcoming Pulp 3 release. Any releases after Pulp 2.11 will be scheduled as-needed.

Release Stages

Beta - Dev believes the Z release is complete and ready for testing. Any bug fixes related to this release are still acceptable.
Release Candidate (RC) - Dev believes the Y release is complete and ready for testing. Any bug fixes related to this release are still acceptable.
Generally Available (GA) - Release is generally available in the stable repositories.

Release Stage Conditions

Releases spend a minimum time of one week in a given release stage. Subsequent releases in a given stage do not "reset the clock" on this waiting period. Care is taken to verify release-breaking bugs prior to advancing to the next release stage, which usually adds a few days to the release cycle, preventing "rapid-fire" releases.

Z Release Beta

There are no release candidates for Z releases. Z releases advance directly to GA after testing reveals no new blocking issues, and have passed acceptance testing by QE (including upgrade testing).

Y Release RC

There are no Beta for Y releases. Y releases advance to GA after testing reveals no new blocking issues, and have passed acceptance testing by QE. This release stage cannot advance to GA until the upgrade path from previous versions of pulp is confirmed to be working.
There can be as many release candidates as needed.

Y Release Cadence

Due to the unpredictable timeline of the release schedule once a Beta is released, the Y release cycle begins on the day of the previous Beta Y release. While unpredictable, the 12-week period between beta releases should be more than adequate to get that release Generally Available and still allow a few weeks before the next Beta is announced.

This release cadence was introduced to coincide with the release of the 2.11.0 Beta on 2016/10/25. If desired, the previous Y release cadence can be seen in the history of this wiki document.

2.y releases are no longer time based and therefore are not dependent on when previous y release beta was cut. The rest of the schedule is followed

Z Release Cadence

The Z release cycle starts with a Dev Freeze and then advances to a Beta. The Z release will spend a minimum of one week in Beta before being eligible to become Generally Available.

Z releases are released as often as possible based on this schedule, but are subject to upgrade testing which often extends the beta period beyond a week.

Version Numbering

To the best of our ability, our releases adhere to semantic versioning. The releases in this document correspond to releases that increment a given release component (Y or Z) as defined by semantic versioning:

http://semver.org/#spec-item-2

In short, a "Z Release" increments the "Z" release component (e.g. 1.0.0 -> 1.0.1), and indicates no new features; only bug and security fixes have taken place. A "Y Release" increments the "Y" release component (e.g. 1.0.0 -> 1.1.0), and indicates that new features have been introduced, but backward-incompatible API changes have not taken place.

"X" Releases are not covered by this document. At the moment, they are rare enough as to be handled case-by-case. The next X release, Pulp 3, is currently being actively discussed on the pulp-dev mailing list. If you're interested in participating, go here to subscribe:

https://www.redhat.com/mailman/listinfo/pulp-dev

Release Schedule and Past Releases

Below is a list of all Versions including upcoming planned releases and versions already Generally Available listed for historical reference. 2.11.2 was the first release to have a release status page in this wiki.


Repository Versions

For background on use cases and value, please see this thread: https://www.redhat.com/archives/pulp-dev/2017-May/msg00045.html

Publications and RepositoryVersions

As one addendum, it’s worth reminding ourselves how a managed Publication relates to a RepositoryVersion. In the time since the above email thread, we moved forward with the Publication and Distribution models being first-class.

A Publication is a set of artifacts and metadata that can be presented to a particular type of client for consumption. It is produced from the set of content that was present in a repository at the time of publication.

A RepositoryVersion is an immutable content set that represents the state of a repository’s content at a point in time. It can be used to create (or recreate) a Publication. It can be used to make two different types of publications from exactly the same content set. It can be used to answer the question: given a publication, what content was used to create it? What content is currently available to my production environment? What content was available to that environment last week when some event took place? Exactly what changed between two publishes, or since the most recent publish?

Neither takes the place of the other, and each provides the most value when both are in use.

Planning Details

Having updated the proof of concept to work with the latest Pulp 3 code base (as of late Nov 2017), and with all of the REST API improvements we have made, it became clear that the following changes would be necessary in the REST API and Plugin API to enable versioning of repositories.

The most problematic endpoint is /api/v3/repositorycontents/. It’s not entirely unreasonable to think that a client could add relationships by POSTing to this endpoint with a reference to a repository and another to a content unit, which would spawn a task that creates a new version along with the requested relationship. The side-effect of creating a version isn’t ideal, but it could be livable.

The real problem came with removing content from a repo via this endpoint. Calling DELETE on a relationship would not be allowed, because removal of a content unit from a repo requires creating a new repo version and updating the relationship to reference the new version as the one which removes the content. Allowing a client to directly create a new version, and then PUT/PATCH a specific relationship to reference that version, would be full of problems; the repo wouldn’t be locked, the content set in the version would be mutable (violating the value proposition of immutable content sets), and we’d be exposing the implementation details of how additions and removals are tracked (something that’s otherwise not necessary).

REST API

Changed Endpoints

/api/v3/repositories/{id}/publishers/file/{name}/publish/

/api/v3/tasks/{id}/

Moved Endpoints

Current:
/api/v3/repositories/{id}/content/
Becomes:
/api/v3/repositories/{id}/versions/{number}/content/

Moved Attributes

Current:
“content_summary” at /api/v3/repositories/{id}/
Becomes:
“content_summary” at /api/v3/repositories/{id}/versions/{number}/

New Endpoints

/api/v3/repositories/{id}/versions/

Removed Endpoints

/api/v3/repositorycontents/

Plugin API

Sync

The task in core should retrieve the current version, create a new version, and hand each of them to the importer’s sync method. The current version is used to determine what content is currently in the repository, which is necessary to know during sync. The new version is used as the API for adding a removing content:

new_version.add_content(rpm1)
new_version.remove_content(rpm2)

Publish

The task in core should determine the version to use and pass it to the Publisher’s publish method. It should be determined either based on a reference provided by a client, or default to the latest.

Models

The RepositoryContent model should be removed from the plugin API, because it would no longer be intended for direct use by a plugin writer.


Everything is quite easy from planning a trip to having flight reservations but what is the main issue passengers generally stuck on? One can have their airline reservations anyhow but the main problem one faces is the boarding process of the flights. Also if you are traveling alone and traveling for the very first time on the flights then you will surely waste your time in the flight boarding process without any knowledge. But if you have planned your journey with Alaska Airlines Flights then you will have the most simplifies boarding process as compared to other airlines.

This is one of the reasons that make Alaska a very satisfying and promising airline, as they look out for the passenger’s comfort. Alaska Airlines flights and booking process keeps updating and changing with time to offer the best services among all the airlines.

Group Boarding Of Passengers Introduced By Alaska:-

Alaska Airlines has introduced a new boarding process known as the group boarding process. Passengers who get to know about group boarding for the very first time they think it's for passengers who are traveling in a group, but it's a big No.!! Because this process has been introduced to make easy boarding of passengers by dividing passengers into different groups. The following are the groups that have been divided.

For Alaska, Flight Bookings Go to the URL https://www.airlinesreservationsflights.com/Reservations/Alaska-Airlines


Sprint Plans

Sprint 69

Dates: Friday March 20, 2020 - Thursday April 3, 2020 Sprint Goals/Focus:

Sprint 68

Dates: Friday March 6, 2020 - Thursday March 20, 2020

Sprint Goals/Focus:

Sprint 67

Dates: Friday February 21, 2020 - Thursday March 5, 2020

Sprint Goals/Focus:

Sprint 66

Dates: Friday February 7, 2020 - Thursday February 20, 2020
Sprint Goals/Focus:

Sprint 65

Dates: Friday January 24, 2020 - Thursday February 6, 2020
Sprint Goals/Focus:

Sprint 64

Dates: Friday January 10, 2020 - Thursday January 23, 2020
Sprint Goals/Focus:

Sprint 63

Dates: Friday December 6, 2019 - Thursday January 9, 2020
Sprint Goals/Focus:

Sprint 62

Dates: Thursday November 14, 2019 - Thursday December 5, 2019
Sprint Goals/Focus:

Sprint 61

Dates: Friday October 25, 2019 - Wednesday November 13, 2019
Sprint Goals/Focus:

Sprint 60

Dates: Friday September 27, 2019 - Thursday October 24, 2019
Sprint Goals/Focus:

Sprint 59

Dates: Friday September 13, 2019 - Thursday September 26, 2019
Sprint Goals/Focus:

Sprint 58

Dates: Friday August 24, 2019 - Thursday September 13, 2019
Sprint Goals/Focus:

* Pulp 2

***** 2.21.0

***** Module Dependency solving

Sprint 57

Dates: Friday August 2, 2019 - Thursday August 23, 2019
Sprint Goals/Focus:

* Pulp 2

***** 2.21.0

***** Module Dependency solving

***** Bug fixes

**** Docker performance

***** Prep for Pulp 3 migration - packaging to leave file system in right state before migration.

Sprint 56

Dates: Friday July 12, 2019 - Thursday August 1, 2019
Sprint Goals/Focus:

* Pulp 2

***** 2.21.0

***** Dependency solving

***** https://pulp.plan.io/issues/5108 - multi resource locking enhancement

***** Bugs

Sprint 55

Dates: Friday June 21, 2019 - Thursday July 11, 2019
Sprint Goals/Focus:

Sprint 54

Dates: Friday May 31, 2019 - Thursday June 20, 2019
Sprint Goals/Focus:

Sprint 53

Dates: Friday May 10, 2019 - Thursday May 30, 2019
Sprint Goals/Focus:

Sprint 52

Dates: Thursday April 18, 2019 - Thursday May 9, 2019
Sprint Goals/Focus:

Sprint 51

Dates: Friday Mar 29, 2019 - Wednesday 17-Apr-2019
Sprint Goals/Focus: Pulp 3 Plugins released working with Core RC
Notes

Sprint 50

Dates: Friday Mar 8, 2019 - Thursday 28-Mar-2019
Sprint Goals/Focus: Pulp 3 RC + Plugins to Core RC feature parity
Notes

Sprint 49

Dates: Friday Feb 15, 2019 - Thursday 7-Mar-2019
Sprint Goals/Focus: Pulp 3 RC + Plugins to Core RC feature parity
Notes

Sprint 48

Dates: Friday Jan 25, 2019 - Thursday 14-Feb-2019
Sprint Goals/Focus: Pulp3 RC
Notes
Sprint Goals/Focus: Pulp 2 + Pulp 3 RC + Pulp 3 Docker beta
Notes

Sprint 47

Dates: Friday Jan 04, 2019 - Thursday 24-Jan-2019
Sprint Goals/Focus: Pulp 2 + Pulp 3 RC + Pulp 3 Docker beta
Notes

Sprint 46

Dates: Wednesday Nov 29, 2018 - Wednesday 20-Dec-2018
Sprint Goals/Focus: Pulp 2 + Pulp 3 RC + Pulp 3 Docker beta
Notes

Sprint 45

Dates: Friday Nov 2, 2018 - Tuesday 28-Nov-2018
Sprint Goals/Focus: Pulp 2 + Pulp 3 RC
Notes

Sprint 44

Dates: Friday Oct 5, 2018 - Thursday Oct 26 01-Nov, 2018
Sprint Goals/Focus: Pulp 2 + Pulp 3 RC
Notes

Sprint 43

Dates: Friday Sept 14, 2018 - Thursday Oct 4, 2018
Sprint Goals/Focus: Pulp 2 + Pulp 3 RC
Notes

Sprint 42

Dates: Friday Aug 24, 2018 - Thursday Sept 13, 2018
Sprint Goals/Focus: Pulp 2 + Pulp 3 GA
Notes

Sprint 41

Dates: Friday Aug 3, 2018 - Thursday August 23, 2018
Sprint Goals/Focus: Pulp 2 + Pulp 3 GA
Notes

Sprint 40

Dates: Friday July 13, 2018 - Thursday August 2, 2018
Sprint Goals/Focus: Pulp 2 + Pulp 3 GA
Notes

Sprint 39

Dates: Friday June 22, 2018 - Thursday July 12, 2018
Sprint Goals/Focus: Pulp 2 + Pulp 3 GA
Notes

Sprint 38

Dates: Friday June 1, 2018 - Thursday June 21, 2018
Sprint Goals/Focus: Pulp 3 GA + Pulp 2
Notes

Sprint 37

Dates: Friday May 11, 2018- Thursday May 31,2018
Sprint Goals/Focus: Pulp 3 GA + Pulp 2
Notes

Completion

Sprint 36

Dates: Friday April 20, 2018- Thursday May 10,2018
Sprint Goals/Focus: Pulp 3 Core Beta
Notes

Sprint 35

Dates: Friday March 30, 2018- Thursday April 19,2018
Sprint Goals/Focus: Pulp 3 Core Beta
Notes

Sprint 34

Dates: Friday March 9, 2018- Thursday March 29,2018
Sprint Goals/Focus: Pulp 3 Core Beta Dev Freeze
Notes


Test Day for 2.14 and pulp deb plugin

Join us on August 8, 2017 to test the 2.14 release and the new pulp_deb plugin!

Contents

Can't make the date?

If you come to this page before or after the test day is completed, your testing is still valuable, and you can use the information on this page to test, file any bugs you find here on https://pulp.plan.io, and report your results as detailed in "How to report results" section below.

How to join

RSVP with this form to let us know you are coming and request help getting set up:
RSVP and request help

If you don't get around to RSVPing, just show up!

Join the #pulp channel on freenode
You can join via your favorite IRC client, or either of the two web clients below:

pulp channel on freenode webchat

Specify a nickname that you would like to go by, and keep it safe for work please!
Many people choose something close to their name, as in Elijah's nickname is "elijah_d".

== How to contribute ==

The August 8th Pulp Test Day will focus on the Pulp 2.14 beta release and the new pulp debian plugin

Things you can do:

== Who's available ==

The following cast of characters will be available testing, workarounds, bug fixes, and general discussion.

Quality Engineering

Specifically in the case of problem related to test day wiki, please reach out to Elijah DeLee at kdelee@redhat.com or on irc (nick: elijah_d)

== Prerequisite for Test Day ==

Ansible install on Fedora 26 is currently broken. Rumor is, it is manually installable. Unless you want to battle install issues today, it is best to go with Fedora 25

pulp-admin login -u admin -p admin
pulp-admin status

For a guide on some ways to spin up a VM on your machine. see Virtualization quickstart
For some information on how to provision a machine from a cloud provider such as digital ocean see Provisioning Test Machine from Cloud

== Resources (where to get RPMs, fedora images, etc) ==

The Pulp 2.14 beta repository is included in the pulp repo files:
2.14 beta repo file for fedora 24 & 25
2.14 beta repo file for RHEL7

You can also find the packages themselves here:
2.14 beta RPMs

Fedora images:
You can find fedora "live" images here to install with:
Fedora 24
Fedora 25
Fedora 26

== How to test? ==

== Run the tests ==

If you'd like to run the integration test suite, pulp-smash or any individual tests, you can run them either locally on the pulp_server or remotely from any machine that can SSH into the pulp_server.

For docs on pulp-smash, see http://pulp-smash.readthedocs.io/en/latest/ where you can find instructions on how to install and configure pulp-smash to run against your pulp install.

If you have trouble, email us or ping us on IRC and we can help you get it set up!

== Exploratory testing ==

You can help out by playing around with the tool in whatever ways you can think of: try out all the things you can find.
Get creative! Any problems you find please file a bug, or report to the IRC channel. See Filing new issues

Find the 2.14 User Guide here.
Find the 2.14 RPM Quick Start Guide here.

Look for other quick start guides among the plugin documentation. Note that the main docs.pulpproject.org may direct you to docs for 2.13, check the URL to make sure you are looking at the right version.

Think CRUD
As a user, you should be able to Create, Read, Update, and Delete repositories and the data within. Is this the case? Where do you run into trouble?

== Verifying existing issues ==

There are three different ways you can use existing issues.

1. Verify that user stories marked "closed" work

User stories closed in the 2.14 beta release

2. Verify that bugs fixed in the 2.14 release are indeed fixed!

Bugs fixed in 2.14

3. Recreate outstanding bugs

Validate existence of outstanding bugs in 2.14 beta.

If the bug is replicable and seems suitable for for pulp-smash tests, file an issue on pulp-smash github issues

== Filing new issues ==

If you have problems with any of the tests, first search for a issue related to the same bug and add a comment on the issue at https://pulp.plan.io.

Otherwise, report the bug to https://pulp.plan.io in the appropriate project section.

For example, you can file a new bug or find an existing issue for the debian plugin under the Debain issues

Again, if you are having trouble finding where to file your issue or search for existing ones, join us on IRC to get help!

== How to report results ==

New bugs

If you found a new bug, then you can file a new issue as described above. Also, refer to Issue filing template

Verified bugs

If you verified that a fix is correct or that a bug still exists, then comment on the issue with details about your system including all information pertinent to filing a new bug: Issue filing template

More thoughts on what make a good bug report


** Tips to travel with Spirit Airlines**

Low-cost is just a word and Spirit airline makes it real. As Spirit Airlines Flights is an epitome of offering the lowest flight fares. There are many travelers who mostly prefer budget-friendly trips and for this, they need to choose low-cost airlines to maintain their own budget. Many people make up their minds and keep high expectations with the airlines because they think airlines are offering the lowest fares and so they will provide service like some big airlines. But they forget why airlines are called low cost and why they are able to maintain their lowest fares and this is the biggest reason why Spirit Airlines gets so much curse from its passengers.

Though Spirit Airlines receives so much curse if you will compare completely with a big airline then you will be able to see good points of Spirit Airlines and will be able to think why its been called a good airline to travel with. Just to offer low fares spirit needs to make every in-flight service paid and this way they are able to offer you the best. After paying for everything to Spirit Airlines, its fares become lower than large airlines. Here are some tips for not losing your shit while traveling with Spirit Airlines Flights. .

Get The Best Seat:- Like every other airline, Spirit Airline also charges you to have a seat of your own choice. We all keep comparing and saying that about the airline's fees but if you seriously notice than the prices are so low. First, you need to decide how expensively you want to travel and then think about seat selection. If you want to travel then get the seat assigned on the same day by the airlines or else Spirit charges $5 to $38 as per the choice of your seat and what's the big deal you are already saving hundreds.

How To Save On Baggage Charges:- Almost every airline charges for checked-in baggage and some charge for carry-on also. It's a huge burden for passengers to pay for everything for traveling to someplace. So Spirit Airlines charges you for both carry-on and checked-in baggage. There are different ways to save on baggage and first is to have your Spirit reservations as early as possible. The later you book the more you pay. Spirit Airlines flights allow you to carry one personal item free of cost and if you can adjust your carry on stuff in that than you can save here also.

Bring Your Own Beverages:- This is because Spirit Airlines flights even charge you for water and if you want to have happy travel better carry your own eating and drinking stuff and that not a big thing as you are already paying for a carry-on baggage and this way you can save your other expense and this will help to travel happily with Spirit airlines and also in budget. Also why Spirit Airlines is best to travel at a low price is because they are available 24*7 to serve the passengers for any help. Every passing day Spirit Airlines makes their service better. You can also have your Spirit Airlines reservations through their reservation number and also they will help you to get the best deals.


United Airlines Change Flight Policy:-

United Airlines is the major airline of America and is the one that fulfills passengers desires to have a luxury journey. The airline offers all types of cabins and also outstanding in-flight services. United Airlines is always known because of its best customer service and comfortable feels. United Airlines is most picked by the people who always prefer to travel in high-class airlines. The airline offers sophisticated in-flight services with such professional and co-operative cabin crew. The words are less if we talk about the services and flight experience which United offers. United Airlines have a solution for every situation and if you have booked a United Airlines Flights and now you want to change because of certain reasons then you can do it. Just follow the instructions given below in the article and you will know how to change a flight with United airlines.

How To Change United Airlines Flights:- There are many passengers who travel with United Airlines flights on a regular basis but are not aware of the flight change policy of the airline till the time they aren't in that situation. So passengers may know how to change scheduled flights by the below listed ways as any sudden situation can arise where one needs to make changes in their preplans.

Flight Change Fee Of United Airlines:-

Also if you are new to United Airlines then you can directly contact the airlines for this process on their United Airlines Reservations Number and their executives will help you with any of your queries regarding your new United Airlines Booking or Existing Bookings.


Virtualization quickstart

This quickstart guide has been tested on Fedora 25.

Within code blocks, a prefix of $ indicates that a command should be executed as a regular user, and # indicates that a command should be performed as root.

Option 1: Using libvirt and Virtual Machine Manager

Install required packages:

# dnf install libvirt qemu-kvm kvm libguestfs-tools virt-install

Then, download an image for the OS you want to run in your VM. Fedora ISOs may be downloaded from here.

Launch the "Virtual Machine Manager" application and click:

1. File → New Virtual Machine
2. Local install media → Browse to downloaded ISO
3) Accept default settings

SETTING UP THE VM:

1. Start the VM with the "play button"
2. Choose to install fedora:
3. set hostname: give it a name other than localhost.
4. make it meaningful
5. set root password (weak one is OK)
6. if you make a user, make sure to give them sudo access

NOTE: If you make a user and forgot to give them sudo access, start the VM, open a terminal, and:

$ su
# gpasswd --add admin wheel

Then restart.

WARNING: localhost is a bad hostname. Each of your VMs should have a unique hostname. If you clone a VM, both hosts will have the same hostname. It is imperative that you change your a host's hostname before installing any pulp components, as Pulp uses its host's hostname, and Pulp can produce undefined behavior if its host's hostname is changed after the fact.

To see your host's hostname, execute hostname. To change your host's hostname, run:

# hostnamectl set-hostname <new-hostname>

SETTING UP ANSIBLE (on the VM):

For details on Ansible, see the Ansible documentation.

Install Ansible:
For Fedora, you can install ansible without enabling any extra repos. For RHEL7 you will have to set up epel.

For RHEL 7 only

$ wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
# rpm -i epel-release-latest-7.noarch.rpm
# dnf install ansible git

Download the pulp_packaging repo:

$ git clone https://github.com/pulp/pulp_packaging.git

Now we need to make a hosts file for Ansible. Use the results of the hostname command in the place of 'localhost':

$ cd pulp_packaging/ci/ansible
$ echo "$(hostname)" > hosts

Choosing what version of pulp to install

To choose the version of pulp that you will install:

Now you are ready to run the ansible playbook:
The ansible user needs to be root, if you are allready root, or have passwordless sudo, you can go without the ask_pass and ansible_user variables.

$ cd pulp_packaging/ci/ansible
$ ansible-playbook -i hosts \
   -e ansible_connection=local \
   pulp_server.yaml \
   -e ask_pass=True \
   -e ansible_user=root

For RHEL7 you will have to provide your RHN credentials as well,

$ cd pulp_packaging/ci/ansible
$ ansible-playbook -i hosts \
   -e ansible_connection=local \
   pulp_server.yaml \
   -e rhn_password=${RHN_PASS} \
   -e rhn_pool=${RHN_POOL} \
   -e rhn_username=${RHN_USER}  \
   -e ask_pass=True \
   -e ansible_user=root

Option 2: Using Vagrant with either libvirt or docker

See Pulp Developer Setup


Weekly Bug Trends

Click on the link to see bug trends per week
Bug Report 04/15/18
Bug Report 03/31/18
Bug Report 03/13/18
Bug Report 02/28/18
Bug Report 02/16/18
Bug Report 01/05/18
Bug Report 09/15/17
Bug Report 09/08/17
Bug Report 09/01/17
Bug Report 08/25/17
Bug Report 08/18/17
Bug Report 08/11/17
Bug Report 08/04/17
Bug Report 07/28/17
Bug Report 07/24/17
Bug Report 07/14/17
Bug Report 06/30/17
Bug Report 06/23/17
Bug Report 06/13/17
Bug Report 05/31/17
Bug Report 05/06/17
Bug Report 04/11/17
Bug Report 03/24/17
Bug Report 03/10/17
Bug Report 03/03/17
Bug Report 02/24/17
Bug Report 02/17/17
Bug Report 02/10/17
Bug Report 02/03/17
Bug Report 01/13/17
Bug Report 12/22/16
Bug Report 12/09/16
Bug Report 11/28/16
Bug Report 11/18/16
Bug Report 11/11/16
Bug Report 11/04/16
Bug Report 10/28/16
Bug Report 10/14/16
Bug Report 10/07/16
Bug Report 9/30/16
Bug Report 9/23/16
Bug Report 9/16/16
Bug Report 9/9/16
Bug Report 9/2/16
Bug Report 8/26/16


Welcome to the Pulp Wiki. Here are some useful links:

Plugins

Community Plugins
Core Plugins

Current Development

Release Schedule
Sprint Plans
Pulp 2 Release Planning

Quality Engineering

QE Test Summary
Weekly Bug Trend Reports
Smoke Test Plans for Pulp 3 0

QE Past Events

Test Day on August 8 2017
Pulp 2 14 Test Day Recap

Pulp 3.0 development

Pulp 3 Minimum Viable Product
Pulp 3 Developer Notes
Developer Install Options
Pulp3_Plugin_Brainstorming
Continuous_Delivery_of_Pulp_3
Pulp3_Release_Guide
Plugin Planning

Downloader Performance Comparison

Pulp 3.0 Plugins development

Python plugin

Pulp Python Roadmap

RPM plugin

Pulp RPM Roadmap

Puppet plugin
Container plugin

Container Roadmap

OSTree plugin

Roadmap

File plugin

Roadmap

Pulp 3.1+ development

Pulp 3.1+ Ideas

Demos / Videos

Demo Organizer Notes
Demo Presenter Notes

Brand

Brand Guide