Proper client setup for multithreaded environment and lock contention on client initialization

[ptorkko]

We run a public api service on spring boot 2.6.14 and its platform bom which depends on jersey 2.35.

We havily utilize jersey clients to make calls to internal services to provide data to the end users, e.g.

• client request
  • collect data from apis
    • async api call 1
    • async api call 2
    • async api call 3
  • mangle responses to a reply to client

In our current setup, we create a WebTarget, with a client setup with apache pooling connector, jackson, etc etc.
Each WebTarget is created as the "root" of the service: client.target("https://internal.service/".
Each WebTarget has it's own client instance, they are not shared, the apache connector is shared.

We've recently had some issues during releases where the application reports itself healthy to the load balancer, but in reality, nothing related to jersey clients is initialized at this point. When a large amount of traffic suddenly hits the service, we have several points of lock contention, one of them being ClientConfig.state.runtime, which is lazy loaded.

It's possible to call e.g. ((JerseyWebTarget)webTarget).preInitialize() to load the lazy runtime before accepting traffic, but our services are built like

@RequiredArgsConstructor
class Service {
    private final WebTarget webTarget;
    public ResponseDTO api1(String someParam) {
        return webTarget.path("v1").path("fancy-api").path(someParam).get(ResponseDTO.class);
    }
    public OtherResponseDTO api2(String someParam) {
        return webTarget.path("v1").path("other-api").path(someParam).get(OtherResponseDTO.class);
    }

where only the "root" WebTarget is preinitialized as any calls to .path etc seem to invalidate the runtime for the new WebTargetfor the path.

As I failed to find any good resource on "how to actually set up jersey clients for high parallel use" i'd like to know how to properly set this up, it seems this has been "wrong" for some time (see this for example.

And if there's something to do to make the clients actually ready to deal with incoming requests.

[jansupol]

Each WebTarget contains its own copy of the client runtime. Anything that can modify the WebTarget creates a new instance of WebTarget (as per WebTarget javadoc) keeping the previous WebTarget unchanged. The new copy is created because any change can affect the behavior of the Connector, DI, or the JAX-RS components.

Instead of WebTargets, it is possible to store and reuse Invocation, or Invocation.Builder, that is already initialized. Would it help?

[ptorkko]

Don't think that's feasible as path and query parameters can be dynamic, and usually are.

Do new instances of WebTarget generated from calls to .path, .queryParam etc not benefit if the initial target was already initialized? They seem to retain the previous state via their ClientConfig but the runtime always requires initialization after being copied.

Could the initial ClientBuilder.newClient(config) be shared among all the different targets? I'm currently a bit unsure of how this should be set up to minimize impact on a cold instance of the service.

This happens to be on AWS ECS, and currently there is no 'slow start' support for gradual traffic increase, so the service has to be prepared to accept 1/nth of the requests when it reports itself healthy to the loadbalancer.

[ptorkko]

Ok I dug a bit deeper and i'm not sure where my original assumptions came from.

with the following:

var target = client.target("https://myapi.com/");
if (target instanceof JerseyWebTarget jwt) jwt.preinitialize();

var actualApiTarget = target.path("v1").path("fancyapi").path(someDynamicPathPAram).queryParam("query", "indeed");

actualApiTarget.request("application/json").....

It seems each new JerseyWebTarget created from .path or .queryParam etc calls actually just retain the initially inizialied client runtime as it's passed with the ClientConfig to each new instance.

As long as the targets are configured at this point and no changes are needed, we should be fine with a setup where we just share WebTargets for each service with the root uri and call the preinitialize method for each of them.