The Complete Guide to Retrying Failed Requests with Axios

Before jumping into retries, let's first understand why Axios is useful. Axios is a promise-based HTTP client for making requests to APIs and servers in JavaScript. Whether it's fetching data from a REST endpoint or posting data to your backend - Axios makes the process seamless.

For example, here's a simple GET request with Axios:

import axios from 'axios';

axios.get('/api/users')
  .then(response => {
    console.log(response.data);
  })
  .catch(error => {
   console.log(error);
  });

Compared to native methods like XMLHttpRequest or the Fetch API, Axios stands out for its ease of use, browser and Node.js support, and handy response handling via promises. Over the years, it has become a staple in web projects.

Of course, even with Axios, stuff can go wrong. Requests can fail due to network errors, server issues or any number of reasons. Let's look at some common cases...

When Requests Fail

From my experience building web scrapers, some typical situations where requests fail include:

Network Issues: Connectivity blips or DNS failures can cause requests to hang and eventually time out. Last month, while scraping a site behind Cloudflare bot protection, two of my IPs lost DNS resolution mid-script!

Overloaded Servers: An influx of traffic can overwhelm servers, causing them to throw 500 or 503 errors under load. This happened recently when one of our monitored ecommerce APIs got flooded during a sale.

Rate Limiting: APIs often enforce thresholds to prevent abuse, throttling requests once limits are crossed. Not ramping up traffic gradually has tripped rate limits for me.

Bad Deploys: There's nothing like broken releases to ruin your day! I've seen CI pipelines directly deploy half-baked features. API responses suddenly error out after months of stability. Fun times!

The costs of failing requests can be huge, specially if critical transactions are involved. This is why manually retrying failed requests is such a common, if tedious, workaround.

But what if retries could happen automatically in code?

Enter Automated Retries

Here are some benefits of automating retries instead of relying on crude manual interventions:

Let's look at how we can bake automated retries into Axios using interceptors.

Building a Retry Mechanism

Axios interceptors allow handling requests before they are sent or after a response is received. They are ideal for cross-cutting tasks like logging, authentication or, you guessed it, retries!

Here is a simple interceptor that retries failed requests up to 3 times:

import axios from 'axios';

// Add a request interceptor
axios.interceptors.request.use(function (config) {

    // Do something before request is sent
    return config;

  }, function (error) {

    // Request error handling

    return Promise.reject(error);

});

// Add a response interceptor
axios.interceptors.response.use(function (response) {

    // Any status code that lie within the range of 2xx cause this function to trigger
    // Do something with response data
    return response;

  }, async function (error) {

    // Any status codes that falls outside the range of 2xx cause this function to trigger
    // Do something with response error

    if (error.response.status === 500) {

      const config = error.config;

      if (!config.retryCounter) {
        config.retryCounter = 1;
      } else {
        config.retryCounter++;
      }

      if (config.retryCounter > 3) {
        // Reject with error
        return Promise.reject(error);
      }

      // Try again
      return axios(config);

    }

    // Reject with error
    return Promise.reject(error);

});

There are a few interesting bits here:

This shows the basic mechanism. But there's a lot more we can configure for effective retries.

Configuring Retries

Some key controls we have for configuring retries include:

Number of Retries

Too few, and transient issues will still cause failures. Too many, and recovering requests may overload systems. There's no ideal value - start small with 2-3 attempts.

Delay Between Retries

Adding a delay between retries allows systems to recover before hammering them again. A one-size-fits-all delay doesn't work well. More on this soon.

Conditional Retries

Blindly retrying certain errors like 400 Bad Request won't help. Having a conditional check to retry only specific cases is important.

Let's enhance our mechanism to incorporate some of these capabilities:

// Import exponential backoff function
import { exponentialDelay } from './backoffUtils';

// Response interceptor
axios.interceptors.response.use(
  function (response) {
    // ...
  },

  async function (error) {

    // Conditional retry
    if (shouldNotRetry(error)) {
      return Promise.reject(error);
    }

    if (error.config.retryCount <= 3) {

      // Increase counter
      error.config.retryCount++;

      // Apply exponential backoff delay
      const delay = exponentialDelay(retryCount);

      // Wait before retrying
      await timeout(delay);

      // Return error config again
      return axios(error.config);

    }

    // Max retries reached
    return Promise.reject(error);

  }
);

// Check if error should not be retried
function shouldNotRetry(error) {
  // Add logic here
  return false;
}

Observe how we:

Besides fixed/exponential delays, jitter delays help stagger retry clusters hitting servers in unison. This post has a good Primer.

Strategies for Effective Retries

There are additionally some best practices worth highlighting around retrying:

Idempotent vs Non-idempotent Requests

Methods like GET and PUT are safe to retry since results will be identical. However, POST/PATCH calls modifying data may duplicate records if blindly retried on certain errors. Handling them correctly is vital.

Avoiding Retry Overloads

Too many clients furiously retrying failed requests can directly overwhelm recovering systems. Exponential backoffs and conditional retries based on error cause can help.

Caching/Queuing

Sometimes it helps to temporarily cache failed requests and replay from persistent storage once issues are mitigated. This takes pressure off overloaded systems.

There are advanced strategies like the Circuit Breaker pattern which "trip" after too many errors to prevent clobbering fragile backends. Netflix's Hystrix is a good reference implementation.

While the examples above directly use Axios interceptors for brevity, let's look at another option...

Using the Axios-Retry Plugin

For convenience, there are handy plugins like axios-retry that wrap retrying capabilities into simple packages:

import axios from 'axios';
import axiosRetry from 'axios-retry';

// Initialize
axiosRetry(axios, {
   retries: 3,
   retryCondition: isServerError // Condition method
});

// Use Axios as normal
axios.get('/items')
  .then(/* ... */)
  .catch(/* ... */)

The plugin automatically retries failed requests which match criteria like server errors or network issues.

Some benefits over a manual interceptor are:

For full documentation on usage and configuration parameters, check the axios-retry README.

Let's also briefly contrast some alternatives.

Comparison of Libraries

Besides axios-retry, there are couple of other shared libraries like retry-axios. They have a lot of overlap in capabilities but few differences exist around implementation:

For most use cases, axios-retry strikes a good balance without forcing much config coding.

However, for advanced needs like custom retry flows or specialized HTTP clients, using raw Axios interceptors directly allows finer-grained control.

There are also niche tools like axios-circuitbreaker focused solely on specific capabilities like circuit breakers which can complement these libraries.

Testing and Debugging Retries

Here are some quick tips around testing and troubleshooting retry logic:

Unit Testing

Mock error responses using libraries like Jest to simulate failures and assert retries occur expected times.

Logging

Trace retry attempts in interceptor handlers during development by logging metadata like retry counts.

Monitoring

Monitor metrics like request counts, retry frequency, errors behind retries etc to optimize configurations.

Handling Circular Retries

Beware infinite loops if reused errors re-trigger interceptors! Have fallback termination logic as shown earlier via max counters.

FAQs

Why use Axios vs Fetch API?

Axios provides automatic JSON parsing, support for older browsers, and richer error handling out-of-the-box compared to the Fetch API.

Does Axios require backend/server code?

No, Axios is a frontend HTTP client library that does not need corresponding server code. You make requests to external APIs and services.

Why is Axios widely used in React apps?

Axios integrates seamlessly with React since it is promise-based. It also avoids opt-in async configurations needed for Fetch.

Is Axios only meant for REST APIs?

No, Axios can call any HTTP backend - REST, RPC, GraphQL etc. It simply helps make HTTP calls and handle responses.

Should all requests be retried by default

No, blindly retrying non-idepotent requests for example can cause data corruption. Having smart conditional retry logic is important.

So in summary, automated retries form an integral safety net when building robust apps dealing with remote services. Leverage tools like Axios interceptors and plugins to transparently bake resilience against failures, without expensive manual intervention.

I hope walking through real-world examples and tips from my web scraping experience helps you be better equipped to handle crashes...because code never lies, but APIs always fail!