itmecho

« back to blog

Handling LaunchDarkly rate limits

At work we use LaunchDarkly for feature flags. As part of a migration process, we update a segment to remove a value on successful migration. This has been working fine but whilst scaling up the process, we started to hit the rate limit on the segment PATCH endpoint. As far as I can tell, the current limit is 5 requests within 10 seconds.

Note: I have simplified the real code in this post to limit it to the actual subject!

We can get two different rate limit errors from LaunchDarkly, IP based and global/route based. I started by capturing the error:

type RateLimitError struct {
  RetryAfterSeconds int
  ResetTime time.Time
}

func (e RateLimitError) Error() string {
  // ...
}

func getRateLimitError(resp *http.Response) (error, bool) {
  if resp.StatusCode != http.StatusTooManyRequests {
    return nil, false
  }
  var rlErr RateLimitError
  if v := resp.Header.Get("X-Ratelimit-Reset"); v != "" {
    millis, err := strconv.ParseInt(v, 10, 64) 
    if err != nil {
      return err, true
    }
    rlErr.ResetTime = time.UnixMillis(millis).UTC()
  }
  if v := resp.Header.Get("Retry-After"); v != "" {
    seconds, err := strconv.Atoi(v)
    if err != nil {
      return rlErr, true
    }
    rlErr.RetryAfterSeconds = seconds
  }
  return rlErr, true
}

Now we have an error, we can check and return it in the request code:

  // ...
  resp, err := client.Do(req)
  if err != nil {
    return err
  }
  if rlErr, ok := getRateLimitError(resp); ok {
    return rlErr
  }
  // ...

Finally, in the calling code, we can check for this error and retry the request after waiting for the delay!

func doWithRateLimitHandling(req *http.Request, maxAttempts int) error {
  var attempt int
  for {
    attempt += 1
    if attempt >= maxAttempts {
      return fmt.Errorf("failed to complete request in %d attempts", maxAttempts)
    }
    err := doRequest(req)
    if err == nil {
      return nil
    }
    var rlErr RateLimitError
    if errors.As(err, &rlErr) {
      time.Sleep(time.Until(rlErr.ResetTime))
      continue
    }
    return err
  }
}

I’ve only included the ResetTime handling in this post but in reality there’s a function on the error to return the delay based on whether the ResetTime or RetryAfterSeconds is set. There’s also some padding seconds added to the delay to account for a little clock drift.