Build for failure

The demand of distributed computing requires services to be written so that they are capable of handling failures. The failure of one service can lead to a cascading effect across all and can be the biggest bottleneck in the application. When invoking REST APIs, consider using time outs, as waiting for long periods will have an adverse effect on the response times. When a response doesn't meet the defined service level agreement (SLA), you could have a default response kept available for such cases.

If a service is timing out, then there's a good chance that it may be down or it may not be available for a longer period. Using a circuit breaker allows you to avoid calling the service that is timing out frequently. If a service is timing out beyond a defined number of times, then the circuit breaker would prevent any further calls. After a defined interval, the circuit-breaker would retry the target service, thereby allowing it time to recover.

These patterns can be written in plain Java code without relying on libraries to do it for you. As an example, you could create an interceptor for all external calls made, which keeps track of failures. Once the threshold of failures is reached, the interceptor can have logic that tells it to not make any further calls to the target service until a defined period. After the configured period is over, the interceptor can start allowing new calls to the service.