I have set up my Elastic Cloud Service through Google Cloud and have set up an Elastic Search Instance.
I can upload my data to Elastic search and query my data just fine. However, when I try to connect to the Elastic Search Instance through my Java Client, I keep getting a 'java.io.IOException' and a 'java.net.UnknownHostException' exceptions.
24-Jun-2020 18:55:52.657 SEVERE [http-nio-8181-exec-8] org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet [dispatcher] in context with path [] threw exception
java.io.IOException: <Elastic Search endpoint>
at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:828)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:248)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:235)
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1611)
Caused by: java.net.UnknownHostException: <Elastic Search Endpoint>
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:797)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1505)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1364)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298)
at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$InternalAddressResolver.resolveRemoteAddress(PoolingNHttpClientConnectionManager.java:664)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$InternalAddressResolver.resolveRemoteAddress(PoolingNHttpClientConnectionManager.java:635)
at org.apache.http.nio.pool.AbstractNIOConnPool.processPendingRequest(AbstractNIOConnPool.java:474)
at org.apache.http.nio.pool.AbstractNIOConnPool.lease(AbstractNIOConnPool.java:280)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.requestConnection(PoolingNHttpClientConnectionManager.java:295)
at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.requestConnection(AbstractClientExchangeHandler.java:377)
at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.start(DefaultClientExchangeHandlerImpl.java:129)
at org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:141)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:244)
... 49 more
And my Java Code:
String ELASTIC_SEARCH_USER_NAME = "elastic";
String ELASTIC_SEARCH_PASSWORD = <Password>;
String ELASTIC_SEARCH_ENDPOINT_URL = "https://92d5f385db294fb4b7ff335201d0a854.asia-northeast1.gcp.cloud.es.io";
Integer ELASTIC_SEARCH_PORT = 9243;
final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(ELASTIC_SEARCH_USER_NAME, ELASTIC_SEARCH_PASSWORD));
RestClientBuilder builder = RestClient.builder(new HttpHost(ELASTIC_SEARCH_ENDPOINT_URL, ELASTIC_SEARCH_PORT, "https"))
.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
#Override
public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
}
});
RestHighLevelClient highLevelClient = new RestHighLevelClient(builder);
Strangely, I have tried pinging my endpoint url in the command line but my cmd is unable to ping the url.
Is there something I need to set up in my Elastic Stack Console for my Java Client to request queries?
Thank you!
Are you sure, that your Elasticsearch is running on port 9243? This is in GCP but for AWS managed ES, there is no need to give the port number and only url is sufficient, make a change to below part of code and see if it works as it works in AWS ES where we don't have to mention the port.
RestClientBuilder builder = RestClient.builder(new HttpHost(ELASTIC_SEARCH_ENDPOINT_URL,, "https"))
Asking the guys over at elastic.co forums, they suggested that I drop the "https://" portion from the ELASTIC_SEARCH_ENDPOINT_URL.
String ELASTIC_SEARCH_ENDPOINT_URL = "https://92d5f385db294fb4b7ff335201d0a854.asia-northeast1.gcp.cloud.es.io";
to
String ELASTIC_SEARCH_ENDPOINT_URL = "92d5f385db294fb4b7ff335201d0a854.asia-northeast1.gcp.cloud.es.io";
and keeping the rest of the code, it worked!
The forum post I made if anyone wants to take a look
Related
Exception in thread "main" com.amazonaws.SdkClientException: Unable to find a region via the region provider chain. Must provide an explicit region in the builder or setup environment to supply a region.
at com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:371)
The Error message itself self sufficient and the error message clearly explains that you had not set the amazonAWSRegion when you are building Amazon service client. As your question does not clearly depicts which AWS service your are trying to connect through your code, I have shown a sample example to build AWS dynamoDB client.
when you were building AmazonDynamoDB client instance use the below code sample.
String amazonAWSAccessKey = "yourAmazonAWSAccessKey";
String amazonAWSSecretKey = "yourAmazonAWSSecretKey";
String amazonDynamoDBEndpoint = "AmazonDynamoDBEndpoint";
String amazonAWSRegion = "amazonAWSRegion"; //(ex: us-east-1/us-west-1)
AWSStaticCredentialsProvider awsCredentialsProvider = new AWSStaticCredentialsProvider(new BasicAWSCredentials(
amazonAWSAccessKey, amazonAWSSecretKey));
AmazonDynamoDB amazonDynamoDB = AmazonDynamoDBClientBuilder.standard()
.withCredentials(awsCredentialsProvider)
.withEndpointConfiguration(new AwsClientBuilder
.EndpointConfiguration(amazonDynamoDBEndpoint, amazonAWSRegion)).build();
If you want to connect the AWS service through Eclipse, the configuration setup is documented at Set up AWS Toolkit
I am using elasticsearch 7.3 version, I am very new to this, all i want is to connect to elasticsearch node, i have installed ealasticsearch in one of the google cloud instance now i want to connect to elasticsearch and create index, delete index using java, how can i achieve that, i used this code but its full of error
public class ElasticConnection {
/**
* #param args the command line arguments
*/
public static void main(String[] args) {
TransportClient client = new PreBuiltTransportClient(Settings.EMPTY)
.addTransportAddress(new TransportAddress(InetAddress.getByName("host1"), 9300))
.addTransportAddress(new TransportAddress(InetAddress.getByName("host2"), 9300));
// on shutdown
client.close();
}
}
tell what jar should i add or what should i do, please help me out with this.
Thanks.
public RestHighLevelClient createESRestClient() {
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(esUserName, esPassword));
RestClientBuilder restClientBuilder = RestClient
.builder(new HttpHost(esRestclientHost, 9200, "http"));
// Use this one if your ElasticSearch server is setup to use username & password authentication
if (esAuthentication) {
restClientBuilder.setHttpClientConfigCallback(h -> h.setDefaultCredentialsProvider(credentialsProvider));
}
return new RestHighLevelClient(restClientBuilder);
}
You can use the following code to create your ElasticSearch connection. RestHighLevelClient is the new client which replaced TransportClient, I suggest that you use RestHighLevelClient.
For your reference this is listed in the ElasticSearch documentation guide: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/7.3/java-rest-high-getting-started-initialization.html
Don't use the TransportClient - it is deprecated. use the high level rest client: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/7.3/java-rest-high.html.
Also, note that the rest port is 9200, for when using the java high level rest client.
another option that you can use and worth mentioning is Jest: https://github.com/searchbox-io/Jest/tree/master/jest
When I am trying to run java app to look the item from ebay api I am getting exp:
com.sun.xml.internal.ws.fault.ServerSOAPFaultException: Client received SOAP Fault from server: Authentication failed : Invalid Application:app key Please see the server log to find more detail regarding exact cause of the failure.
at com.sun.xml.internal.ws.fault.SOAP11Fault.getProtocolException(SOAP11Fault.java:178)
at com.sun.xml.internal.ws.fault.SOAPFaultBuilder.createException(SOAPFaultBuilder.java:124)
at com.sun.xml.internal.ws.client.sei.StubHandler.readResponse(StubHandler.java:238)
at com.sun.xml.internal.ws.db.DatabindingImpl.deserializeResponse(DatabindingImpl.java:189)
at com.sun.xml.internal.ws.db.DatabindingImpl.deserializeResponse(DatabindingImpl.java:276)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:104)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:77)
at com.sun.xml.internal.ws.client.sei.SEIStub.invoke(SEIStub.java:147)
at com.sun.proxy.$Proxy36.findItemsByKeywords(Unknown Source)
at FindItem.main(FindItem.java:59)
I imported 2 jars as written on tutorial but still have this issue,
I get the key from ebay web of app key.
My code is:
ClientConfig config = new ClientConfig();
config.setApplicationId("MY app key that taken from ebay");
//create a service client
FindingServicePortType serviceClient = FindingServiceClientFactory.getServiceClient(config);
//create request object
FindItemsByKeywordsRequest request = new FindItemsByKeywordsRequest();
//set request parameters
request.setKeywords("harry potter phoenix");
PaginationInput pi = new PaginationInput();
pi.setEntriesPerPage(2);
request.setPaginationInput(pi);
//call service
FindItemsByKeywordsResponse result = serviceClient.findItemsByKeywords(request);
and another question, how can I get specification of the item?
Hope you can help on this issue.
I think the AppID which you given to connect the API is Invalid.
My questions are
Do you have DevID, AppID and CertID provided by eBay ?
If you have then please try to connect from SOAP UI to check whether it is working.
It's not clicking how one is supposed to connect to a hosted Cloudant database using Ektorp. I'm using Ektorp 1.1 in Eclipse via the new m2eclipse Maven integration (which is pretty sweet). I'm struggling to find good CouchDB/Cloudant/Ektorp documentation other than javadocs.
I'm trying to get the sample Ektorp API example from their main page to work:
HttpClient httpClient = new StdHttpClient.Builder()
.host("localhost")
.port(5984)
.build();
CouchDbInstance dbInstance = new StdCouchDbInstance(httpClient);
CouchDbConnector db = new StdCouchDbConnector("mydatabase", dbInstance);
db.createDatabaseIfNotExists();
It doesn't matter what I use to build the httpClient with, I always get the UnknownHostException error below. I've tried these URLs for the host: https/http://cloudant.com/db/_session and https/http://[username].cloudant.com
What about the port number? Should the username and password be included in the StdHttpClient.Builder()?
Here's the full error - it's failing on the createDatabaseIfNotExists() call but I'm not confident the CouchDbConnector variable is correct.
Exception in thread "main" org.ektorp.DbAccessException: java.net.UnknownHostException: https://cloudant.com/db/_session
at org.ektorp.util.Exceptions.propagate(Exceptions.java:19)
at org.ektorp.http.StdHttpClient.executeRequest(StdHttpClient.java:104)
at org.ektorp.http.StdHttpClient.get(StdHttpClient.java:42)
at org.ektorp.http.RestTemplate.get(RestTemplate.java:21)
at org.ektorp.impl.StdCouchDbInstance.getAllDatabases(StdCouchDbInstance.java:61)
at org.ektorp.impl.StdCouchDbConnector.createDatabaseIfNotExists(StdCouchDbConnector.java:256)
at com.codegouge.examples.App.main(App.java:30)
Caused by: java.net.UnknownHostException: https://cloudant.com/db/_session
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:850)
at java.net.InetAddress.getAddressFromNameService(InetAddress.java:1201)
at java.net.InetAddress.getAllByName0(InetAddress.java:1154)
at java.net.InetAddress.getAllByName(InetAddress.java:1084)
at java.net.InetAddress.getAllByName(InetAddress.java:1020)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:126)
at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:149)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:108)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:576)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:554)
at org.ektorp.http.StdHttpClient.executeRequest(StdHttpClient.java:96)
So I was doing a couple things wrong. Using SSL requires additional parameters. Also, Ektorp 1.1.1 includes SSL-related bug fixes to 1.1.0. So this is my final HttpClient constructor:
HttpClient httpClient = new StdHttpClient.Builder()
.host("[username].cloudant.com")
.port(443)
.username("[username]")
.password("[password]")
.enableSSL(true)
.relaxedSSLSettings(true)
.build();
Also, be sure to update ektorp's dependency in pom.xml to look for version "1.1.1". I have a blog post covering this exercise here if interested.
You also can use the URL to connect with Ektorp:
JSONObject serviceAttr = val.getJSONObject(0);
JSONObject credentials = serviceAttr.getJSONObject("credentials");
httpClient = new StdHttpClient.Builder()
.url(credentials.getString("url"))
.build();
That's is easy way to connect. I found a tutorial to connect using Ektorp 1.4.2:
http://www.ibm.com/developerworks/java/library/j-hangman-app/index.html
I'm not very familiar with Ektorp, but you'll definitely need to get your username/password in there. I'd suggest creating the HttpClient with the following code:
HttpClient httpClient = new StdHttpClient.Builder()
.host("[username].cloudant.com")
.port(443)
.username("[username]")
.password("[password]")
.build();
I've changed the port to 443 (the default for HTTPS, which Cloudant listens on) and I've added a username and password. I don't see any way to let Ektorp know that you want to use HTTPS, but with luck that'll be handled internally.
I need to call axis2 web service with ws-security (username token) from xfire client over https. I could do the exercise via xfire dynamic client, but no luck with wsdl base client (i.e. generate java stub from wsdl). Could anybody point me out what could be wrong (stub, ws-security something else)?
Exception:
Exception in thread "main"
org.codehaus.xfire.XFireRuntimeException:
Could not invoke service.. Nested
exception is
org.codehaus.xfire.fault.XFireFault:
The endpoint reference (EPR) for the
Operation not found is
https://localhost/services/DataServiceSample2
and the WSA Action =
org.codehaus.xfire.fault.XFireFault:
The endpoint reference (EPR) for the
Operation not found is
https://localhost/services/DataServiceSample2
and the WSA Action =
Code:
public static void main(String[] args) throws MalformedURLException {
ProtocolSocketFactory easy = new EasySSLProtocolSocketFactory();
Protocol protocol = new Protocol("https", easy, 9443);
Protocol.registerProtocol("https", protocol);
ObjectServiceFactory serviceFactory = new ObjectServiceFactory();
serviceFactory.setStyle("message");
Service serviceModel = serviceFactory.create(DataServiceSample2PortType.class);
XFireProxyFactory factory = new XFireProxyFactory();
DataServiceSample2PortType service = (DataServiceSample2PortType) factory.create(serviceModel, "https://localhost:9443/services/DataServiceSample2");
Client client = Client.getInstance(service);
client.addOutHandler(new DOMOutHandler());
Properties properties = new Properties();
properties.setProperty(WSHandlerConstants.ACTION, WSHandlerConstants.USERNAME_TOKEN);
properties.setProperty(WSHandlerConstants.USER, "admin");
properties.setProperty(WSHandlerConstants.PASSWORD_TYPE, WSConstants.PW_TEXT);
properties.setProperty(WSHandlerConstants.PW_CALLBACK_CLASS, PasswordHandler.class.getName());
client.addOutHandler(new WSS4JOutHandler(properties));
sab.TopCustomerResponse topCustomersInCalifornia = service.topCustomersInCalifornia(null);
}
Please try replacing the localhost with ip address of the machine where your service is running. Instead of
factory.create(serviceModel,"https://localhost:9443/services/DataServiceSample2");
You can try specifying the ip address like this
factory.create(serviceModel,"https://192.168.2.18:9443/services/DataServiceSample2");
Please note that it is considered bad practice to ship code with ambiguous parameters. So after testing it you will need to replace hard coded ip address with some variable which can be easily configured.
I'm missing "SOAPAction" parameter in HTTP header. You could set it directly as
HttpsURLConnection conn;
...
conn.setRequestProperty("SOAPAction", "urn:executeXml");
AFAIK in XFire client it could be archived in such way:
Map m = new HashMap();
m.put("SOAPAction", "urn:executeXml");
client.setProperty(CommonsHttpMessageSender.HTTP_HEADERS, m);