Skip to content
This repository was archived by the owner on Sep 19, 2018. It is now read-only.

Rate Limiting Handling #12

Open
z64 wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Rate Limiting Handling #12

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c61375a
implement preemptive rate limiting mutex and 429 handling, closes #9
z64 Jul 26, 2017
862e828
fixup! implement preemptive rate limiting mutex and 429 handling, clo…
z64 Jul 26, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 45 additions & 2 deletions lib/rls/api.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

require 'rest-client'
require 'json'
require 'time'
require 'rls/objects/platform'
require 'rls/objects/player'
require 'rls/objects/season'
Expand Down Expand Up @@ -122,19 +123,61 @@ def seasons(renew = false)
end
end

# Performs a mutex-protected request to the API with rate limit handling.
# @param type [String, Symbol] HTTP verb
# @param endpoint [String, Symbol] The API endpoint
# @param attributes [Array<Hash>] Header and query parameters
# passed along with the request
# @return [Hash] The parsed JSON response
def request(type, endpoint, *attributes)
attributes << {} if attributes.empty?
response = raw_request(type, endpoint, attributes)
JSON.parse(response)
@mutex ||= Mutex.new

@mutex.synchronize do
sleep until_reset if will_be_rate_limited?

begin
@last_response = raw_request(type, endpoint, attributes)
rescue RestClient::TooManyRequests
sleep until_reset
retry
end

JSON.parse(@last_response)
end
end

private

# @return [String, nil] if it exists, the specific header from the last API request
def last_header(key)
@last_response&.headers&.dig(key)
end

# @return [Float] the amount of time until the rate limit resets
def until_reset
(last_header(:x_rate_limit_reset_remaining) || 0).to_i / 1000.0
end

# @return [Integer] number of requests until we're rate limited
def remaining_requests
last_header(:x_rate_limit_remaining)&.to_i || -1

@z64 z64 Jul 26, 2017 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify this defaulting to -1:

last_header(:x_rate_limit_remaining)&.to_i returns [Integer, nil] which would fail the subsequent #zero? check in #will_be_rate_limited?. I opted just to return -1 as it isn't zero (it's unknown until we make our first request).

In contrast, if we default to zero, then #will_be_rate_limited? may return true on the first request and sleep for 0 seconds. This isn't a big deal either, but if we implement something like rate limit logging later, it may look pretty confusing. 1 may also be an acceptable default for this method, but it isn't true (we didn't get that from the API) ; so I think this justifies -1 as a special case.

end

# @return [Time, nil] if it exists, when the rate limit will be reset
def rate_limit_reset
str = last_header(:x_rate_limit_reset)
Time.parse(str) if str
end

# @return [true, false] if the next request will be rate limited
def will_be_rate_limited?
return false unless @last_response
return false if Time.now > rate_limit_reset
remaining_requests.zero?
end

# Performs a request to the API. Not protected by a rate-limit mutex!
# @param type [String, Symbol] HTTP verb
# @param endpoint [String, Symbol] The API endpoint
# @param attributes [Array<Hash>] Header and query parameters
Expand Down