Elasticsearch Java client: can't connect to the remote server - java

I need to connect to the remotely located ElasticSearch index, using the url provided here:
http://api.exiletools.com/info/indexer.html
However, I can't figure out how to do this in Java.
Docs on ES Java Client don't really have much info at all.
I also did not find any JavaDocs for it, do they exist?
Now, there are working examples written in Python, which confirm that the server is up and running, connection part looks like this:
es = Elasticsearch([{
'host':'api.exiletools.com',
'port':80,
'http_auth':'apikey:DEVELOPMENT-Indexer'
}])
What I tried to do:
client = new TransportClient()
.addTransportAddress(new InetSocketTransportAddress("apikey:DEVELOPMENT-Indexer#api.exiletools.com/index", 9300));
also tried ports 9200 and 80
This results in:
java.nio.channels.UnresolvedAddressException
and NoNodeAvailableException

The Shop Indexer API offers an HTTP entry point on port 80 to communicate with their ES cluster via the HTTP protocol. The ES TransportClient is not the correct client to use for that as it will only communicate via TCP.
Since Elasticsearch doesn't provide an HTTP client out of the box, you need to either use a specific library for that (like e.g. Jest) or you can roll up your own REST client.
An easy way to achieve the latter would be using Spring's RestTemplate:
// create the REST template
RestTemplate rest = new RestTemplate()
// add the authorization header
HttpHeaders headers = new HttpHeaders();
headers.add("Authorization", "DEVELOPMENT-Indexer");
// define URL and query
String url = "http://api.exiletools.com/index/_search";
String allQuery = "{\"query\":{\"matchAll\":{}}}";
// make the request
HttpEntity<String> httpReq = new HttpEntity<String>(allQuery, headers);
ResponseEntity<String> resp = rest.exchange(url, HttpMethod.POST, httpReq, String.class)
// retrieve the JSON response
String body = resp.getBody();

Related

Accessing Individual SSE Payloads in Response using Java

We are currently using Apache HttpClient (5) to send a POST request to a server. The response is sent back to us using server side events (SSE) with multiple payloads using the standard format:
data: {...}
Currently we have code like this which sends the request and receives the response:
// Set the socket timeout
final ConnectionConfig connConfig = ConnectionConfig.custom()
.setSocketTimeout(socketTimeout, TimeUnit.MILLISECONDS)
.build();
// Custom config
final BasicHttpClientConnectionManager cm = new BasicHttpClientConnectionManager();
cm.setConnectionConfig(connConfig);
// Build the client
try (final CloseableHttpClient client = HttpClientBuilder.create().setConnectionManager(cm).build()) {
// Execute the request
return client.execute(request.getRequest(),
// Get and process the response
response -> HttpResponse.builder()
.withCode(response.getCode())
.withContent(EntityUtils.toByteArray(response.getEntity()))
.build()
);
}
This all works great except I need to access the individual incoming response payloads (data {...}) in the response as they arrive instead of waiting for them all to finish before being able to access the response.
If this isn't possible using Apache I am open to other options providing they can send a normal HTTP(S) POST.
Ok I couldn't get it to work with the Apache Client but I did find this video that showed how to do it using the native HttpClient.
// Create the client
final HttpClient client = HttpClient.newHttpClient();
// Make the request
final HttpResponse<Stream<String>> response = client.send(request, HttpResponse.BodyHandlers.ofLines());
// Status code check
if (response.statusCode() != 200) ...;
// This will consume individual events as they arrive
response.body().forEach(System.out::println);

how to create client TGT with java cxf

I'm new to the java rest CXF client. I will make various requests to a remote server, but first I need to create a Ticket Granting Ticket (TGT). I looked through various sources but I could not find a solution. The server requests that I will create a TGT are as follows:
Content-Type: text as parameter, application / x-www-form-urlencoded as value
username
password
I create TGT when I make this request with the example URL like below using Postman. (URL is example). But in the code below, I'm sending the request, but the response is null. Could you help me with the solution?
The example URL that I make a request with POST method using Postman: https://test.service.com/v1/tickets?format=text&username=user&password=pass
List<Object> providers = new ArrayList<Object>();
providers.add(new JacksonJsonProvider());
WebClient client = WebClient.create("https://test.service.com/v1/tickets?format=text&username=user&password=pass", providers);
Response response = client.getResponse();
You need to do a POST, yet you did not specify what your payload looks like?
Your RequestDTO and ResponseDTO have to have getters/setters.
An example of using JAX-RS 2.0 Client.
Client client = ClientBuilder.newBuilder().register(new JacksonJsonProvider()).build();
WebTarget target = client.target("https://test.service.com/v1/tickets");
target.queryParam("format", "text");
target.queryParam("username", "username");
target.queryParam("password", "password");
Response response = target.request().accept(MediaType.APPLICATION_FORM_URLENCODED).post(Entity.entity(yourPostDTO,
MediaType.APPLICATION_JSON));
YourResponseDTO responseDTO = response.readEntity(YourResponseDTO.class);
int status = response.getStatus();
Also something else that can help is if you copy the POST request from POSTMAN as cURL request. It might help to see the differences between your request and POSTMAN. Perhaps extra/different headers are added by postman?
Documentation: https://cxf.apache.org/docs/jax-rs-client-api.html#JAX-RSClientAPI-JAX-RS2.0andCXFspecificAPI
Similar Stackoverflow: Is there a way to configure the ClientBuilder POST request that would enable it to receive both a return code AND a JSON object?

Apache Http Client execute request without sending the enclosing entity

I do have the following scenario:
1) The Client sends a HTTP request with an enclosing entity to a Server, via a socket.
2) The Server uploads the enclosing entity to another location, let's call it Storage.
I am required to implement only the Server.
So far, I was able to implement it using Apache HTTP Components library using something like:
// The request from the client
org.apache.http.HttpRequest request = ...;
// The org.apache.http.entity.InputStreamEntity will
// read bytes from the socket and write to the Storage
HttpEntity entity = new InputStreamEntity(...)
BasicHttpEntityEnclosingRequest requestToStorage = new ......
requestToStorage.setEntity(entity);
CloseableHttpClient httpClient = ...
CloseableHttpResponse response = httpClient.execute(target, requestToStorage );
So far so good. Problem is, the Storage server requires authentication. When the Server makes the first request (via Apache Http Client API), the Storage responds with 407 Authentication Required. The Apache Http Client makes the initial handshake then resends the request, but now there is no entity since it has already been consumed for the first request.
One solution is to cache the entity from the Client, but it can be very big, over 1 GB.
Question Is there a better solution, like pre-sending only the request's headers?
Use the expect-continue handshake.
CloseableHttpClient client = HttpClients.custom()
.setDefaultRequestConfig(
RequestConfig.custom()
.setExpectContinueEnabled(true)
.build())
.build();

HTTP authentication with each HTTP request using Jersey 2 client + ApacheConnectorProvider

I would like to use HTTP authentication with a Jersey 2 client using ApacheConnectorProvider, but I want to set it for each request (not as ClientConfig property). The reason is that I use the client for multiple connections, only some of which require HTTP authentication. I assume it is better to not recreate the Client object each time I want to send an HTTP request.
What I found so far:
1) from https://github.com/jersey/jersey/blob/master/connectors/apache-connector/src/test/java/org/glassfish/jersey/apache/connector/AuthTest.java
CredentialsProvider credentialsProvider = new org.apache.http.impl.client.BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials("name", "password")
);
ClientConfig cc = new ClientConfig();
cc.property(ApacheClientProperties.CREDENTIALS_PROVIDER, credentialsProvider).property(ApacheClientProperties.PREEMPTIVE_BASIC_AUTHENTICATION, true);
cc.connectorProvider(new ApacheConnectorProvider());
Client client = ClientBuilder.newClient(cc);
WebTarget r = client.target(getBaseUri());
r.request().get(String.class);
This would probably work, but as I reuse the same Client for multiple HTTP requests, not all using HTTP authentication, I'd need to recreate a Client object each time I send a request. I do not know if that is an expensive operation, so it could be a solution.
2) from https://jersey.java.net/documentation/latest/client.html#d0e4833
Response response = client.target("http://localhost:8080/rest/homer/contact").request()
.property(HttpAuthenticationFeature.HTTP_AUTHENTICATION_BASIC_USERNAME, "homer")
.property(HttpAuthenticationFeature.HTTP_AUTHENTICATION_BASIC_PASSWORD, "p1swd745").get();
This does not seem to work with ApacheConnectorProvider.
What is the right solution to my problem?
Thanks!

Do Jersey client support NTLM proxy

I'm trying to make a jersey client call using NTLM proxy? is that possible as i was not able to get any clear information on the same. Did anyone tried before?
Yes it is possible to configure the Jersey Client to connect through a proxy server that requires NTLM authentication.
Here is a simplified code snippet that prepares a suitable ClientConfig that should work with Jersey v2.5+:
final ClientConfig config = new ClientConfig();
config.property(ClientProperties.PROXY_URI, "http://myproxy.com:8000");
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
final AuthScope ntlmAuthScope =
new AuthScope("myproxy.com", 8000, AuthScope.ANY_REALM, "NTLM");
credentialsProvider.setCredentials(
ntlmAuthScope,
new NTCredentials("user", "password", "hostname", "domain") );
config.property(
ApacheClientProperties.CREDENTIALS_PROVIDER, credentialsProvider);
config.connectorProvider(new ApacheConnectorProvider());
Client client = ClientBuilder.newClient(config);
Please note: I am using the Apache HttpClient connector with Jersey Client - you may require slightly different code if you are using another client transport connector.
You may also need to add the following line to your code if you want your POST/PUT requests to be buffered (and therefore repeatable) in response to any 407 authentication challenges that come back from your proxy server:
config.property(ClientProperties.REQUEST_ENTITY_PROCESSING,
RequestEntityProcessing.BUFFERED);

Categories