Working with unreliable web APIs

To be clear, you should not compromise on API quality for mission critical things. We do not do this.

Sometimes you have to work with web APIs that have reliability issues and or inconsistent error responses – and changing APIs is not always an option.

Here are a five tips we’ve compiled at Instavest to help you deal with finicky web APIs:

1: Retry loops

This tip only applies to information requests (not changing data).

We had a third party data source that gives us 500 errors and undocumented error responses at high rates (90% of so of requests succeeded). Mathematically speaking, trying twice should give us an error rate of only 1%:

90% + 10% * 90% = 99%

Re-trying 2 times should lower it another order of magnitude:

90% + 10% * 90% + 1% * 90%= 99.9%

  success = False
  for x in xrange(3):
    try:
      response = requests.get('https://api.bigco.com/v1/entries/')
      if response.status == 200:
        success = True
        break
    except ValueError:
      logger.info(“Request failed {} time(s)”.format(x+1))

  if not success:
    logger.info(“Failed to complete a request for...”)
    # Handle error

2: Thorough logging

Each time you have an error for an API request, make it crystal clear what went on so that the API provider can speed up debugging. This includes:

Your code should log this stuff regardless of a success of failure – this is your audit trail.

3: Set up automatic emails

As you work with more third parties, error reporting can become a lower priority especially if you’ve built a robust application with solid error handling. This does not mean its OK – we have several third party APIs that are not critical to Instavest’s functionality but they help provide better user experiences and therefore are important (e.g. data APIs).

We found that automating emails to tech support teams with detailed error diagnostics (as described above) can be helpful. We also use casual language in the emails so that they don’t think the emails are automated.

request_params = {
        'url': 'https://api.bigco.com/entries/.../',
        'request_type': 'POST',
        'data': {},
        'headers': {},
        ...
}

success = False
for x in xrange(3):
  try:
    response = execute_request(request_params)
    if response.status == 200:
      success = True
      break
  except ValueError as e:
    logger.info(“API request {url} failed: {e}, details: {request_params}”.format(**request_params)

if not success:
  logger.info(“Failed to complete a request for...”)
  send_email_for_bigco_api_error(request_params)
  # Handle error

4: Use error reporting tools

We use Sentry to report unhandled exceptions. This is critical because you will never anticipate every error and log reporting tools may have latency (some people periodically grep their logs for exceptions and email alerts).

You should be using this regardless of whether you use third party web APIs or not! Sentry is just one option, there are several well regarded options

Integrating one of these tools usually requires minimal work (it fits in as middleware). So if an uncaught exception occurs, you’re notified (along with frequency and users affected).

5: Track response latency and warn yourself internally

Response times increasing can be a leading indicator of errors. It’s not too difficult to time this in your logic and add it to logs. The Python library, Requests, (has this built in)[http://docs.python-requests.org/en/latest/api/?highlight=elapsed#requests.Response.elapsed]

Start by logging every request to response time delta, and find the distribution curve of these data points.

For example, find the median time response and distribution. When a response time greater than the 95th percentile occurs, give yourself a warning or contact the party responsible for this to warn them. You might help them avoid heavy load errors!

Moreover, this data can help with periphery things like timing heavy data pulls. We run several periodic scripts at Instavest that heavily poll data APIs to calculate interesting statistics for our users. We saw some of these scripts running very slowly – but moving them to times where the third party APIs were more available drastically sped things up.

Understanding this data can help you answer import questions on error handling and user experience.

We hope these tips help you integrate and work better with finicky web APIs. Remember, just because an API isn’t as beautiful as Stripe, doesn’t mean you should ignore it – we’ve had great success working with a variety of APIs at Instavest. Comments? Reach us at hello@instavest.com - we’d love to hear from you.

 
43
Kudos
 
43
Kudos

Now read this

Infographic: Tesla’s Delay Game

Yesterday, Tesla [(NASDAQ:TSLA)]([https://www.google.com/finance?q=NASDAQ:TSLA) reported earnings for Q2 2015. Tesla beat revenue and earnings estimates of -$1.17 billion (non-GAAP loss per share of $0.60) by posting results non-GAAP... Continue →