Pulp

Against the most recent 3.0-dev along with a latest checkout of pulp_example (from dkliban's PR) I run the following commands to perform a basic sync using the asyncio downloaders:

http POST http://127.0.0.1:8000/api/v3/repositories/ name=example-repo notes:={} scratchpad:={}
http POST http://127.0.0.1:8000/api/v3/repositories/example-repo/importers/example-asyncio/ name=example-importer download_policy='immediate' repository='http://127.0.0.1:800/api/v3/repositories/example-repo/' feed_url='https://repos.fedorapeople.org/pulp/pulp/fixtures/file-mixed/PULP_MANIFEST' sync_mode='mirror'
http POST http://127.0.0.1:8000/api/v3/repositories/example-repo/importers/example-asyncio/example-importer/sync/

The repo above is expected to have 2 units which 404. If you run this repo over and over, most of the time you'll get the following snippet in your task status output:

<snip>
    "non_fatal_errors": [
        {
            "traceback": null,
            "description": "404, message='Not Found' for url https://repos.fedorapeople.org/pulp/pulp/fixtures/file-mixed/missing-2.iso",
            "code": null
        },
        {
            "traceback": null,
            "description": "404, message='Not Found' for url https://repos.fedorapeople.org/pulp/pulp/fixtures/file-mixed/missing-1.iso",
            "code": null
        }
    ],
</snip>

However occasionally you'll get only one 404 recorded, not two, which looks like this:

<snip>
    "non_fatal_errors": [
        {
            "traceback": null,
            "description": "404, message='Not Found' for url https://repos.fedorapeople.org/pulp/pulp/fixtures/file-mixed/missing-2.iso",
            "code": null
        }
    ],
</snip>

Note that the codepath taken by pulp_example proves that the downloaders are correctly emitting the 404 exception back to the plugin code. The thing that seems to not be happening correctly is the recording of the non-fatal exceptions on the Task in the database.

After some further experimentation this occurs when the task runtime is very short with runtimes like:

    "started_at": "2017-09-18T21:48:51.402544Z",
    "finished_at": "2017-09-18T21:48:51.886003Z",

The theory is that the task is ending and not flushing all writes to the database before the task is finished.