Resilience4j Circuit Breaker Not Working with Retry in Spring Boot 3

Integrating Resilience4j Circuit Breaker and Retry mechanisms in Spring Boot 3 is a popular approach to create robust, fault-tolerant applications. However, developers frequently encounter situations where the circuit breaker refuses to cooperate smoothly with retry policies. If you’re struggling with similar issues—relax, you’re not alone.

Understanding the Problem

You’ve implemented Resilience4j’s Circuit Breaker and Retry together in your Spring Boot 3 application. But despite careful configuration, it feels like these two are not working well together. For instance, retries may fire as expected, but the circuit breaker won’t trip at the right moment, or the retries don’t trigger at all.

This situation is pretty common. It often arises because these resilience patterns, though complementary, have specific interaction rules we sometimes overlook.

Let’s dive right into the solution and walk through the correct implementation.

Code Implementation of Circuit Breaker and Retry in Spring Boot

When combining Retry and Circuit Breaker, there are usually two strategies:

Circuit Breaker protecting against repeated failures.
Retry providing additional attempts before giving up.

To best use these two, you need clearly defined methods incorporating both annotations. Let’s illustrate with an example.

Function Executed with Retry and Circuit Breaker Combined

Here’s a simple Spring Boot service method configured to first retry failed calls, and then apply circuit breaker as a safeguard against constant system-level failures:

@Service
public class PaymentService {

    private final RestTemplate restTemplate;
    
    public PaymentService(RestTemplate restTemplate) {
        this.restTemplate = restTemplate;
    }

    @Retry(name = "paymentRetry")
    @CircuitBreaker(name = "paymentCircuitBreaker", fallbackMethod = "paymentFallback")
    public String makePayment(String orderId) {
        return restTemplate.getForObject("https://payment-service/orders/{id}", String.class, orderId);
    }

    public String paymentFallback(String orderId, Throwable t) {
        return "Fallback response: payment service unavailable.";
    }
}

Configure Properly Using YAML

Next, ensure you’ve correctly configured Resilience4j Circuit Breaker and Retry in your Spring Boot application’s application.yml:

resilience4j:
  circuitbreaker:
    instances:
      paymentCircuitBreaker:
        slidingWindowSize: 5
        permittedNumberOfCallsInHalfOpenState: 2
        failureRateThreshold: 50
        waitDurationInOpenState: 10000
  retry:
    instances:
      paymentRetry:
        maxAttempts: 3
        waitDuration: 2000
        retryExceptions:
          - org.springframework.web.client.HttpServerErrorException
          - java.io.IOException

Here’s what’s happening: Retry kicks in first, retrying certain transient exceptions like IOException or HttpServerErrorException. If all retries fail, the Circuit Breaker records it as a single failure event. When the number of failure events (like exhausted retries) passes the threshold, the Circuit Breaker trips.

Managing Spring Boot Dependencies

For Spring Boot 3 and Resilience4j, include these Gradle dependencies in your project’s build.gradle:

dependencies {
    implementation 'org.springframework.boot:spring-boot-starter-webflux'
    implementation 'org.springframework.cloud:spring-cloud-starter-circuitbreaker-reactor-resilience4j'
    implementation 'io.github.resilience4j:resilience4j-spring-boot3:2.1.0'
}

Ensure your dependencies match your Spring Boot version to avoid compatibility issues.

Testing Circuit Breaker and Retry Functionality

To verify that both behaviors work as expected, you’ll need clear testing scenarios:

Testing Retry Behavior

To test Retry, simulate transient server errors. For example, set up a test controller mocking intermittent failures:

@RestController
public class MockPaymentController {
    
    private AtomicInteger requestCount = new AtomicInteger(0);
    
    @GetMapping("/orders/{id}")
    public ResponseEntity mockPayment(@PathVariable String id) {
        if(requestCount.incrementAndGet() <= 2) {
            return ResponseEntity.status(500).body("Error, please retry.");
        }
        return ResponseEntity.ok("Payment successful for order: " + id);
    }
}

The above endpoint will fail twice before finally succeeding, exactly matching our Retry scenario of three attempts.

Testing Circuit Breaker Behavior

Similarly, validating Circuit Breaker functionality requires continuous failures past the configured sliding window threshold. Increment your test to always return 500 HTTP error for more than five consecutive requests to trigger the breaker properly.

Analyzing Behavior: Retry Vs Circuit Breaker

While testing, you'll notice:

Retry handles transient failures, improving resilience to temporary glitches such as network instability.
Circuit Breaker protects your system from repeatedly attempting calls if the dependent service is down or severely degraded.

In essence, Retry deals with transient issues, while Circuit Breaker handles long-running or systemic problems. Proper interplay between them ensures resilient and fault-tolerant applications.

Troubleshooting: Why the Circuit Breaker Might Not Work with Retry

If you've carefully set everything but still can't get them working correctly, consider these common problems:

Incorrect order of annotations: The sequence matters. Always use @Retry first, then @CircuitBreaker.
Misconfigured exceptions: Make sure you explicitly list correct exceptions for Retry. Circuit Breaker will record events after retries finish their attempts.
External Call Problem: Ensure your endpoint actually throws exceptions and not just returns successful HTTP statuses with failure content.
Logging Issues: Without detailed logs (Resilience4J logging configuration), it's harder to debug what's happening at runtime.

Clearing the Issue: Best Practices and Recommendations

After careful testing and troubleshooting, keep these best practices in mind:

Itemize clearly in your YAML config all exceptions suitable for retry and frequent failures.
Log failures clearly to avoid confusion about Retry successes versus real Circuit Breaker events.
Test separately first to ensure Retry works on its own; then add Circuit Breaker config afterwards.

Next Steps & Future Improvements

Here are some useful enhancements and next tasks to explore:

Implementing resilience tests in your Continuous Integration (CI) pipelines.
Combining frontend JavaScript strategies with backend resilience frameworks for holistic stability.
Exploring strategies integrating Bulkhead and Rate Limiter patterns alongside existing Retry and Circuit Breaker setups.

Acknowledgements

Appreciation goes out to extensive documentation by Spring Boot, Resilience4j maintainers, and the broader community at Stack Overflow. They've been instrumental for clarity and technical guidance.

References

Have you experienced other challenges integrating resilience patterns in your apps? Share your own experiences or doubts about Resilience4j Retry and Circuit Breakers in the comments!