My application has two outgoing SOAP connections. For those, I want to implement TLS. Both are created using CXF.
The javax.xml.ws.Service.getPort() returns an individual bindingProvider (both connections use their own WSDL) but both use the same org.apache.cxf.bus.spring.SpringBus instance.
Before using the bindingProvider, I set the TLS client parameters on the Conduit:
Client client = ClientProxy.getClient(bindingProvider); // different
HTTPConduit httpConduit = (HTTPConduit) client.getConduit(); // same for both connections
TLSClientParameters tlsClientParameters = new TLSClientParameters();
tlsClientParameters.setTrustManagers(getTrustmanagers());
httpConduit.setTlsClientParameters(tlsClientParameters);
The issue is, the retrieved client is different for both connections, but the conduit is the same object. Thus, when I set the parameters for the second connection on the same object, I overwrite the priorly set settings.
The FAQ answer if CXF is threadsafe with "yes" and a number of exceptions. I think the second exception applies here. It says:
CXF answer: CXF proxies are thread safe for MANY use cases. The exceptions are:
[...]
Settings on the conduit - if you use code or configuration to directly manipulate the conduit (like to set TLS settings or similar), those are not thread safe. The conduit is per-instance and thus those settings would be shared. Also, if you use the FailoverFeature and LoadBalanceFeatures, the conduit is replaced on the fly. Thus, settings set on the conduit could get lost before being used on the setting thread.
[...]
For the conduit issues, you COULD install a new ConduitSelector that uses a thread local or similar. That's a bit complex though.
I'm not entirely sure if thread safety is my issue. I create the two connections each in their own component. Springs uses only one thread for initializing all the components, so both connections are initialized by the same thread. But afterwards, the connection is used threads from a pool. Overwriting the settings happens during intialization, so before sending actual SOAP messages using different threads.
When the Conduit is created in org.apache.cxf.endpoint.AbstractConduitSelector#getSelectedConduit, it is done using the SpringBus which is the same instance for both objects.
So, the FAQ tell me to use my own custom ConduitSelector.
I tried setting it before the initialization above:
Client client = ClientProxy.getClient(bindingProvider);
client.setConduitSelector(
new UpfrontConduitSelector(
new URLConnectionHTTPConduit(client.getBus(),
client.getEndpoint().getEndpointInfo())));
and I tried the same after the initialization. In both cases, after setting the conduit selector, when something uses the BindingProvider (which is a Proxy-object), it gets a NullPointerException although the object is not null.
My issue here is to either get the custom conduit selector running or to see that my issue can be solved completely differently or just to get some inspiration :)
Some guy on SO seems to have solved this here, but the answer to his question is not helping me.
I found a solution.
The issue indeed had nothing to do with multithreading, but with the way the SpringBus is wired into my objects and how the Conduit is created from it.
The solution was to give each service its own SpringBus.
So before I create each SOAP-service by calling its c'tor in javax.xml.ws.Service, I do
BusFactory bf = BusFactory.newInstance();
Bus b = bf.createBus();
BusFactory.setThreadDefaultBus(b);
which sets a new threadlocal default bus which is then used for the service created. Thus, my two services each have their own SpringBus and they both create their own Conduit.
This works because each service is a spring #Component and all spring components are created by the main thread. So there is only one thread and no way this code will not be executed sequentially.
Related
we have a larger multi service java spring app that declares about 100 exchanges and queues in RabbitMQ on startup. Some are declared explicitly via Beans, but most of them are declared implicitly via #RabbitListener Annotations.
#Component
#RabbitListener(
bindings = #QueueBinding(key = {"example.routingkey"},
exchange = #Exchange(value = "example.exchange", type = ExchangeTypes.TOPIC),
value = #Queue(name = "example_queue", autoDelete = "true", exclusive = "true")))
public class ExampleListener{
#RabbitHandler
public void handleRequest(final ExampleRequest request) {
System.out.println("got request!");
}
There are quite a lot of these listeners in the whole application.
The services of the application sometimes talk to each other via RabbitMq, so take a example Publisher that publishes a message to the Example Exchange that the above ExampleListener is bound to.
If that publish happens too early in the application lifecycle (but AFTER all the Spring Lifecycle Events are through, so after ApplicationReadyEvent, ContextStartedEvent), the binding of the Example Queue to the Example Exchange has not yet happend and the very first publish and reply chain will fail. In other words, the above Example Listener would not print "got request".
We "fixed" this problem by simply waiting 3 seconds before we start sending any RabbitMq messages to give it time to declare all queues,exchanges and bindings but this seems like a very suboptimal solution.
Does anyone else have some advice on how to fix this problem? It is quite hard to recreate as I would guess that it only occurs with a large amount of queues/exchanges/bindings that RabbitMq can not create fast enough. Forcing Spring to synchronize this creation process and wait for a confirmation by RabbitMq would probably fix this but as I see it, there is no built in way to do this.
Are you using multiple connection factories?
Or are you setting usePublisherConnection on the RabbitTemplate? (which is recommended, especially for a complex application like yours).
Normally, a single connection is used and all users of it will block until the admin has declared all the elements (it is run as a connection listener).
If the template is using a different connection factory, it will not block because a different connection is used.
If that is the case, and you are using the CachingConnectionFactory, you can call createConnection().close() on the consumer connection factory during initialization, before sending any messages. That call will block until all the declarations are done.
Since gRPC makes service call on new thread and gRPC context is Thread Local, how can I propagate this gRPC context? I found that Context.currentContextExecutor() and ContextPropagatingExecutorService can be used but I haven't found enough resources or example for these 2 options. Can someone help to implement these?
ClientInterceptors shouldn't change the context instance seen by the application. The Context behavior shouldn't really change whether using blocking, async, or future stubs and a blocking API would not be able to change the current context.
While an interceptor is free to modify a pre-existing (mutable) value in the Context, there's generally little need. It is normally easier to create a new interceptor instance each RPC and communicate with the interceptor directly, or communicate via a custom CallOption.
If you have just a single call site that needs access to response headers, then MetadataUtils.newCaptureMetadataInterceptor() is a convenient (although roundabout) way to get the Metadata. It was designed for testing, but is appropriate for small-scale use outside of testing situations.
AtomicReference<Metadata> headers = new AtomicReference<>();
AtomicReference<Metadata> trailers = new AtomicReference<>();
// Using blocking for simplicity, but applies equally to futures
stub.withInterceptors(MetadataUtils.newCaptureMetadataInterceptor(headers, trailers))
.someRpc();
Metadata headersSeen = headers.get();
If you need to access the same header from multiple callsites, it is better to create a custom interceptor that does what you need.
CustomInterceptor interceptor = new CustomInterceptor();
stub.withInterceptors(interceptor)
.someRpc();
... = interceptor.getWhateverValue();
This is demonstrating a "general" use case. Specific instances commonly can tweak their API further to be more convenient and natural.
Context
I have an instance of org.apache.hc.client5.http.impl.classic.CloseableHttpClient
and a singleton instance of org.apache.hc.client5.http.io.HttpClientConnectionManager to communicate with some remote API.
I have created a singleton pool of HTTP connections to create an instance of the HTTP client.
What is the right way to use HttpClientConnectionManager?
Should I really use try-with-resource (or old-style try-finally) to work with CloseableHttpClient?
If I close the HTTP client, a connection will be closed also from this pool. And then I can't use this pool to communicate with a remote API.
Of course, I've read the documentation https://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html
Maybe something I missed. Could anyone explain me?
You should also be using CloseableHttpClient as a singleton (on a per distinct service basis).
If you want to continue creating short lived instances of CloseableHttpClient with the same HttpClientConnectionManager please make sure to mark it as shared when creating CloseableHttpClient instances with HttpClientBuilder.
When I use a SimpleRequestFactory with my AsyncRestTemplate I can easily configure an HTTP proxy server.
I can either do (sample code in Kotlin):
#Bean
open fun asyncRestTemplate(): AsyncRestTemplate {
val proxy = Proxy(Proxy.Type.HTTP, InetSocketAddress("127.0.0.1", 8008))
val requestFactory = SimpleClientHttpRequestFactory().apply {
this.setConnectTimeout(TimeUnit.SECONDS.toMillis(10).toInt())
this.setReadTimeout(TimeUnit.SECONDS.toMillis(10).toInt())
this.setProxy(proxy)
this.setTaskExecutor(taskExecutor())
}
return AsyncRestTemplate(requestFactory)
}
Or I can simply set the corresponding system properties: -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=8008.
However, in the moment that I switch from the SimpleClientHttpRequestFactory to a Netty4ClientHttpRequestFactory there is no evident way to configure the proxy directly and it seems this client does not respect the system properties either.
val requestFactory = Netty4ClientHttpRequestFactory().apply {
this.setConnectTimeout(TimeUnit.SECONDS.toMillis(10).toInt())
this.setReadTimeout(TimeUnit.SECONDS.toMillis(10).toInt())
//this.setProxy(proxy) //???
}
Once I change for the netty client, I have no clue on how to make it go through the proxy.
My interest in using the netty client was that I not only wanted to make async requests, but also I wanted this to be non-blocking. I hope I'm not making a wrong assumption here.
Does anyone know how can I use a proxy server when using the Netty4ClientHttpRequestFactory or perhaps know of an alternative non-blocking client supported by Spring that I could use?
The Netty4ClientHttpRequestFactory (source) and related classes such as Netty4ClientHttpRequest (source) use SimpleChannelInboundHandler for the channel and do not use the proxy handler. Everything is private and unable to be overridden within the source, so there is no way to change it to support Proxies. You would have to almost rewrite the whole thing.
You have other async client options that will work very well and allow you more configuration options. The included Netty one is fairly basic anyway. OkHttpClientHttpRequestFactory and HttpComponentsAsyncClientHttpRequestFactory both let you pass in your own configured client.
To your interest, AsyncRestTemplate's different implementation:
SimpleClientHttpRequestFactory -> simple thread pool, blocking api, proxy supported
OkHttpClient (OkHttp3) -> blocking api, proxy supported
CloseableHttpAsyncClient -> non-blocking nio api, proxy supported
Netty4ClientHttpRequestFactory -> non-blocking nio api, proxy not supported
you can visit https://github.com/wuxudong/VariousAsyncHttpClientPerformance for more details
I'm using proxies with htmlunit, my proxy list contain mixture of both http and socks, I dont know if the next selected proxy to be passed to htmlunit is type http or socks, will htmlunit automatically determine the type and use the appropriate rule for connecting through that proxy?
I've made an application which used mixture of proxies as well, but it was a while ago. In that version of HtmlUnit, it required being explicitly told if the proxy was SOCKS or not, otherwise it assumed it was a HTTP proxy. I looked briefly in the change logs, and found nothing indicating that this had changed, so the answer should be no, it will assume that the proxy is HTTP if you don't tell HtmlUnit that the proxy is SOCKS.
To check what type a proxy is, one can use something like:
SocketAddress addr = new InetSocketAddress("proxyAddress", port);
Proxy proxy = new Proxy(Proxy.Type.HTTP, addr); //or Proxy.Type.SOCKS
URL url = new URL("http://google.com");
URConnection conn = url.openConnection(proxy);
If the code fails (i.e throws an exception), then the proxy is most likely either dead or SOCKS. (HtmlUnit will throw an exception in the first case anyway, or you can perform the same test again with Proxy.Type.SOCKS if you aren't certain the proxy is alive.)
Check out Java Networking and Proxies.
It talks about multiple strategies for setting proxies. It also gives option to provide multipe proxy to same connection through proxy selector.
The other two answers are about how to generically use a proxy in a Java program, but it is a little bit different with HtmlUnit. Setting the proxy on the Java process does nothing; instead you want the simulated browser to use the proxy.
int myProxyPort=8080;
WebClient webClient = new WebClient(BrowserVersion.INTERNET_EXPLORER_8, "myproxyhost.com", myProxyPort);
This framework detects the proxy type for you and can instantiate a HtmlUnit WebClient with correct Socks/Http/Https proxy configuration for you: https://github.com/subes/invesdwin-webproxy
Essentially it automates the trial&error approach when maintaining its proxy list for you. So you don't have to worry about that in your own code.