Project

Profile

Help

Story #7965

Updated by bmbouter about 3 years ago

## Problem 

 Servers like galaxy.ansible.com and cloud.redhat.com have rate limits, and they are pretty low. This causes sync's to fail because they do not correctly respond with 429 when rate limiting is in effect. Instead during rate limiting they respond with 500+ errors which causes the sync to fail. 

 ## Why won't concurrency resolve this? 

 The Galaxy API causes many, very small requests to occur, so even a rate limit of 1 can cause so many requests to occur in a short amount of time to trigger the rate limit situation. 

 ## Solution 

 Add a `rate_limit` option to the CollectionRemote which will be disabled by default. It will be set in terms of an integer number of requests per second. minute. 

 This will include all requests including token fetches and actual data fetches, etc. 

 ## Implementation 

 #### The Model Field 

 rate_limit = models.IntegerField(null=True) 

 #### The Serializer 

 rate_limit = serializers.IntegerField(help_text=_("Limits total download rate in requests per second")) 

 

Back