Project

Profile

Help

Task #3698

Story #3693: Lazy for Pulp3

Make the streamer download using a configured downloader, configured by the correct remote.

Added by dkliban@redhat.com over 1 year ago. Updated 6 months ago.

Status:
MODIFIED
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Platform Release:
Blocks Release:
Backwards Incompatible:
No
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 46

Description

The inbound request is mapped to a RemoteArtifact just like the Content app does. After that it gets the url and remote from there. It then can:

  • forward headers first, then stream data
  • Header injection requirements from squid (see old streamer)
  • asyncio not twisted

Related issues

Blocks Pulp - Task #3699: Make the streamer create an Artifact and update ContentArtifact with the foreign key MODIFIED Actions

Associated revisions

Revision 8219efd5 View on GitHub
Added by bmbouter 10 months ago

Adds the content app to pulpcore.content

The Settings can be a lot simpler now that the architecture no longer
required redirection. It does need to account for the content being
served on a different HOST. This PR handles that.

Since the settings are now single values, they are much easier to
override using Dynaconf, which is great.

The content app can be removed now that the streamer code is handling it
all.

This PR is a prototype and has no docs. Use Pulp as normal , but run the
streamer from github.com/bmbouter/pulp_streamer/ and run the streamer
with:

`gunicorn pulpcore.streamer:server --bind localhost:8080 --worker-class aiohttp.GunicornWebWorker -w 2`

This PR needs pulp-smash changes from pulp_file so it requires the PR
below:

Required PR: https://github.com/pulp/pulp_file/pull/140

https://pulp.plan.io/issues/4239
closes #4239

https://pulp.plan.io/issues/3698
closes #3698

https://pulp.plan.io/issues/3699
closes #3699

https://pulp.plan.io/issues/4181
closes #4181

https://pulp.plan.io/issues/4243
closes #4243

Revision 8219efd5 View on GitHub
Added by bmbouter 10 months ago

Adds the content app to pulpcore.content

The Settings can be a lot simpler now that the architecture no longer
required redirection. It does need to account for the content being
served on a different HOST. This PR handles that.

Since the settings are now single values, they are much easier to
override using Dynaconf, which is great.

The content app can be removed now that the streamer code is handling it
all.

This PR is a prototype and has no docs. Use Pulp as normal , but run the
streamer from github.com/bmbouter/pulp_streamer/ and run the streamer
with:

`gunicorn pulpcore.streamer:server --bind localhost:8080 --worker-class aiohttp.GunicornWebWorker -w 2`

This PR needs pulp-smash changes from pulp_file so it requires the PR
below:

Required PR: https://github.com/pulp/pulp_file/pull/140

https://pulp.plan.io/issues/4239
closes #4239

https://pulp.plan.io/issues/3698
closes #3698

https://pulp.plan.io/issues/3699
closes #3699

https://pulp.plan.io/issues/4181
closes #4181

https://pulp.plan.io/issues/4243
closes #4243

History

#1 Updated by bmbouter about 1 year ago

  • Description updated (diff)

#2 Updated by dkliban@redhat.com about 1 year ago

  • Groomed changed from No to Yes
  • Sprint Candidate changed from No to Yes

#3 Updated by dkliban@redhat.com about 1 year ago

  • Sprint set to Sprint 41

#4 Updated by dkliban@redhat.com about 1 year ago

  • Related to Task #3699: Make the streamer create an Artifact and update ContentArtifact with the foreign key added

#5 Updated by dkliban@redhat.com about 1 year ago

  • Related to deleted (Task #3699: Make the streamer create an Artifact and update ContentArtifact with the foreign key)

#6 Updated by dkliban@redhat.com about 1 year ago

  • Blocks Task #3699: Make the streamer create an Artifact and update ContentArtifact with the foreign key added

#7 Updated by CodeHeeler about 1 year ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to CodeHeeler

#8 Updated by rchan about 1 year ago

  • Sprint changed from Sprint 41 to Sprint 42

#9 Updated by rchan about 1 year ago

  • Sprint changed from Sprint 42 to Sprint 43

#10 Updated by rchan about 1 year ago

  • Sprint changed from Sprint 43 to Sprint 44

#11 Updated by CodeHeeler 11 months ago

  • Assignee changed from CodeHeeler to bmbouter

#12 Updated by rchan 11 months ago

  • Sprint changed from Sprint 44 to Sprint 45

#13 Updated by bmbouter 11 months ago

  • Status changed from ASSIGNED to POST

The github.com/bmbouter/pulp_streamer/ master branch currently does this so I'm moving to POST.

We can move to MODIFIED once the repo is moved to the Pulp organization pending a bigger review/test.

#14 Updated by rchan 11 months ago

  • Sprint changed from Sprint 45 to Sprint 46

#15 Updated by bmbouter 10 months ago

#16 Updated by bmbouter 10 months ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#17 Updated by daviddavis 6 months ago

  • Sprint/Milestone set to 3.0

#18 Updated by bmbouter 6 months ago

  • Tags deleted (Pulp 3)

Please register to edit this issue

Also available in: Atom PDF