We Keep Coding

Java Data Elasticsearch client appending port 9200 to url

Spring Data Elasticsearch version:
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-elasticsearch</artifactId>
<version>3.2.6.RELEASE</version>
</dependency>
I do not understand why my Elasticsearch Highlevel client is always forcing the port to 9200 even though I am specifying the port 443.
Here is how I am defining the RestHighLevelClient:
#Slf4j
#Configuration
public class ElasticsearchConfiguration {
#Value("${elasticsearch.host:127.0.0.1}")
private String elasticsearchHost;
#Value("${elasticsearch.port:9200}")
private String elasticsearchPort;
#Bean(destroyMethod = "close")
public RestHighLevelClient client() throws IOException {
log.info("Creating High level rest client for Elasticsearch with host: " + elasticsearchHost);
ClientConfiguration configuration = ClientConfiguration.builder()
.connectedTo(elasticsearchHost + ":" + elasticsearchPort)
.usingSsl()
.build();
return RestClients.create(configuration).rest();
}
#Bean
public ElasticsearchRestTemplate getElasticsearchTemplate() throws IOException {
return new ElasticsearchRestTemplate(client());
}
}
and how I have used the template:
private ElasticsearchRestTemplate elasticsearchTemplate;
#Autowired
public ElasticsearchServiceImpl(#Qualifier("getElasticsearchTemplate") ElasticsearchRestTemplate elasticsearchRestTemplate) {
this.elasticsearchTemplate = elasticsearchRestTemplate;
}
#PostConstruct
public void init() throws IOException {
create indexes...
}
I have verified that the host and the port are both set in the properties file:
--elasticsearch.host=https://vpc-example-ryphgjfwji3zonliyhhd3nu.eu-west-2.es.amazonaws.com
--elasticsearch.port=443
In the stacktrace error I am receiving: Caused by: java.io.IOException: https://vpc-example-ryphgjfwji3zonliyhhd3nu.eu-west-2.es.amazonaws.com:443:9200, it appears to be appending 9200 to the end of the url. Why is this happening?
I have verified that the connection is successfully made via curl.
Edit: Added another stacktrace
un 09 21:44:44 ip-10xxxx95.pre.xxx-prototype.xxx.uk bash[19738]: 2020-06-09 21:44:44.023 ERROR 19738 --- [ main] c.g.c.s.impl.ElasticsearchServiceImpl : https://vpc-example-ryphgjfwji3zonliyhhd3nu.eu-west-2.es.amazonaws.com: Name or service not known
Jun 09 21:44:44 ip-10-2xxxxx5.pre.xxx-prototype.xxx.uk bash[19738]: 2020-06-09 21:44:44.024 ERROR 19738 --- [ main] c.g.c.s.impl.ElasticsearchServiceImpl : [org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:964), org.elasticsearch.client.RestClient.performRequest(RestClient.java:233), org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1764),

Can you try creating the resthighleval client like below :
RestHighLevelClient restHighLevelClient = new RestHighLevelClient(
RestClient.builder(new HttpHost(configuration.getElasticsearchConfig().getHost(),
configuration.getElasticsearchConfig().getPort(),
"https")));
And use the below config, note I left the port config empty:
--elasticsearch.host=https://vpc-example-ryphgjfwji3zonliyhhd3nu.eu-west-2.es.amazonaws.com
--elasticsearch.port=
I've created a connection to AWS ES and ES hosted in AWS in past and let me know if you face issue with code, would be happy to help further.

Do not specify the protocol (https) when specifying the host. In your case, it would be:
--elasticsearch.host=vpc-example-ryphgjfwji3zonliyhhd3nu.eu-west-2.es.amazonaws.com
--elasticsearch.port=443

Java RMI connection over internet

I'm trying to get remote objects from a server hosted on different network. I'm able to connect on same machine and on same network, but when I try to get it from different network I get:
Connection refused to host: 192.168.1.131; nested exception is: java.net.ConnectException: Connection timed out: connect
It seems that lookup function is searching at wrong network. I tried to use System.setProperty but it doesn't work. Here the code:
Server
public class Main {
public static void main(String[] args) {
try{
System.out.println("Init server...\n");
TestInterface test = new TestImplement();
System.setProperty("java.rmi.server.hostname", "95.247.x.x");
System.out.println("Reg RMI...\n");
Registry rmiRegistry = LocateRegistry.createRegistry(5555);
rmiRegistry.rebind("Test" , test);
System.out.println("Reg completed!\n");
}catch(Exception e){
e.printStackTrace();
}
}
}
Client
...
registryRMI = LocateRegistry.getRegistry("95.247.x.x",5555);
TestInterface testClient = (TestInterface)registryRMI.lookup("Test");
...
Do I need to set java.rmi.server.hostname in client jar as well?

TestInterface test = new TestImplement();
System.setProperty("java.rmi.server.hostname", "95.247.x.x");
You need to set java.rmi.server.hostname before exporting any remote objects. Doing it afterwards is too late.
System.setProperty("java.rmi.server.hostname", "95.247.x.x");
TestInterface test = new TestImplement();

NoNodeAvailableException[None of the configured nodes are available] in Elasticsearch 5.1.2

On Google Compute Engine, I created 3 VMs and installed Elasticsearch 5.1.2 on them.
I installed GCE Discovery Plugin for unicast discovery.
From my local web browser(Win7), I could access these Elasticsearch nodes successfully.
On Google Cloud Platform, I have added the firewall rule accepting tcp:9300, tcp:9200.
Now I'd like to use Java transport client to talk to remote Elasticsearch nodes from my local java application.
I'm sure cluster.name is correct.
Code and Error are as follows:
public class NativeClient {
#SuppressWarnings({ "resource", "unchecked" })
public static Client createTransportClient() throws UnknownHostException {
Settings settings = Settings.builder().put("cluster.name", "elasticsearch").put("client.transport.sniff", true)
.build();
TransportClient client = new PreBuiltTransportClient(settings)
.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("104.100.100.96"), 9300));
return client;
}
private final Node node = null;
private final Client client = null;
}
public class IndicesOperations {
private final Client client;
public IndicesOperations(Client client) {
this.client = client;
}
public boolean checkIndexExists(String name){
IndicesExistsResponse response=client.admin().indices().prepareExists(name).execute().actionGet();
return response.isExists();
}
public static void main( String[] args ) throws InterruptedException, UnknownHostException {
Client client =NativeClient.createTransportClient();
IndicesOperations io=new IndicesOperations(client);
String myIndex = "test";
if(io.checkIndexExists(myIndex))
io.deleteIndex(myIndex);
io.createIndex(myIndex);
Thread.sleep(1000);
io.closeIndex(myIndex);
io.openIndex(myIndex);
io.deleteIndex(myIndex);
}
}
Exception in thread "main" NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{lu8DzekbSWOrNEgFgXxpgQ}{104.100.100.96}{104.100.100.96:9300}]]
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:328)
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:226)
at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:59)
at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:345)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:403)
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1226)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:80)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:54)
at com.packtpub.IndicesOperations.checkIndexExists(IndicesOperations.java:16)
at com.packtpub.IndicesOperations.main(IndicesOperations.java:49)
elasticsearch.yml
network.host: _gce_
cloud:
gce:
project_id: es-cloud
zone: asia-east1-b
discovery:
type: gce
Edit:
After deploying my java application on google compute engine, it can access the elasticsearch instance running on google compute engine. In this application, I just modified InetAddress.getByName("10.140.0.2"). When deploying on my local machine, I used the external ip of that VM.
What else do I have to modify to run it on my local machine?

I appended the external ip of my VM to network.publish_host property in elasticsearch.yml, then I could access the elasticsearch running on remote VM:
network.host: _gce_
network.publish_host: 104.100.100.96
cloud:
gce:
project_id: es-cloud
zone: asia-east1-b
discovery:
type: gce
I don't understand exactly, but fortunately it works.

eureka ribbon integration - Unable to list servers

I am getting the following exception when I am trying to execute the following code that uses ribbon and tries to get server list from eureka.
Exception
3122 [main] WARN com.netflix.loadbalancer.RoundRobinRule - No up servers available from load balancer: DynamicServerListLoadBalancer:{NFLoadBalancer:name=origin,current list of Servers=[],Load balancer stats=Zone stats: {},Server stats: []}ServerList:DiscoveryEnabledNIWSServerList:; clientName:origin; Effective vipAddresses:localhost; isSecure:false; datacenter:null
Exception in thread "main" com.netflix.client.ClientException: LoadBalancer returned null Server for :origin
at com.netflix.client.LoadBalancerContext.computeFinalUriWithLoadBalancer(LoadBalancerContext.java:418)
at com.netflix.client.AbstractLoadBalancerAwareClient.executeWithLoadBalancer(AbstractLoadBalancerAwareClient.java:166)
at com.netflix.zuul.RibbonTest.main(RibbonTest.java:23)
Code
public static void main(String[] args) throws Exception {
ConfigurationManager.loadPropertiesFromResources("ribbon-client.properties");
RestClient client = (RestClient) ClientFactory.getNamedClient("origin");
HttpRequest request = HttpRequest.newBuilder().uri(new URI("/serviceContext/api/v1/")).build();
for (int i = 0; i < 20; i++) {
HttpResponse response = client.executeWithLoadBalancer(request);
}
}
ribbon-client.properties
origin.ribbon.NIWSServerListClassName=com.netflix.niws.loadbalancer.DiscoveryEnabledNIWSServerList
#also tried unsuccessfully with localhost:8091, localhost:8091/serviceContext
origin.ribbon.DeploymentContextBasedVipAddresses=localhost
The service is a Spring-Boot (not using Spring-Cloud) app using eureka-client.properties:
eureka-client.properties
eureka.registration.enabled=true
eureka.preferSameZone=true
eureka.shouldUseDns=false
eureka.serviceUrl.default=http://localhost:9080/eureka/v2/
eureka.region=default
eureka.name=service-Eureka
eureka.vipAddress=localhost
eureka.port=8091
eureka.instanceId=CHNSHL119363L
The service gets registered successfully with eureka deployed in local tomcat#port 9080 and is discover-able # http://localhost:9080/eureka/ and http://localhost:9080/eureka/v2/apps/
W/O using Spring-Cloud what needs to be fixed in the above code/configuration to be able to get list of servers dynamically from eureka using ribbon?

The following post served useful.
The issue is solved by
1. Correcting vipAddress configuration for service and ribbon
2. Configuring and registering ribbon as eureka-client
1. Changes for vipAddress
eureka-client.properties (service)
eureka.vipAddress=my.eureka.local
ribbon-client.properties
origin.ribbon.DeploymentContextBasedVipAddresses=my.eureka.local
2. Changes to register ribbon as eureka-client
public static void main(String[] args) throws Exception {
ConfigurationManager.loadPropertiesFromResources("ribbon-client.properties");
**InstanceInfo instanceInfo = new EurekaConfigBasedInstanceInfoProvider(instanceConfig).get();
applicationInfoManager = new ApplicationInfoManager(instanceConfig, instanceInfo);
eurekaClient = new DiscoveryClient(applicationInfoManager, clientConfig);**
RestClient client = (RestClient) ClientFactory.getNamedClient("origin");
HttpRequest request = HttpRequest.newBuilder().uri(new URI("/serviceContext/api/v1/")).build();
for (int i = 0; i < 20; i++) {
HttpResponse response = client.executeWithLoadBalancer(request);
}
}

java.net.ConnectException while running webservice?

I am developing the web service where in I want to connect to the SOA server. It's giving connection exception.
Code:
public class ConnectSOA{
public static void main(String[] args){
Map<IWorkflowServiceClientConstants.CONNECTION_PROPERTY, String> connProperties = new HashMap<IWorkflowServiceClientConstants.CONNECTION_PROPERTY, String>();
connProperties.put(IWorkflowServiceClientConstants.CONNECTION_PROPERTY.CLIENT_TYPE,WorkflowServiceClientFactory.REMOTE_CLIENT);
connProperties.put(IWorkflowServiceClientConstants.CONNECTION_PROPERTY.EJB_PROVIDER_URL,"t3://10.10.78.79:8001");
connProperties.put(IWorkflowServiceClientConstants.CONNECTION_PROPERTY.EJB_INITIAL_CONTEXT_FACTORY,"weblogic.jndi.WLInitialContextFactory");
try {
workflowServiceClient = WorkflowServiceClientFactory
.getWorkflowServiceClient(connProperties, null, null);
itaskQueryService = workflowServiceClient.getTaskQueryService();
statePredicate = new Predicate(
TableConstants.WFTASK_STATE_COLUMN, Predicate.OP_EQ,
IWorkflowConstants.TASK_STATE_ASSIGNED);
iworkFlowContext = itaskQueryService.authenticate("demouser","demo1".toCharArray(), null);
} catch(Exception e ){
e.printStackTrace();
}
}
}
Exception :
java.net.ConnectException: t3://10.10.78.79:8001 Bootstrap to: hostname/'10.10.78.79:8001' over: 't3' got an error or timed out
I have check the soa server its up and running on the machine.

Can you ping your weblogic?
java weblogic.Admin -url t3://IP:8001 -username xxx -password xxx PING 10
I see that you have 10.10.78.79:8001 and in Exception IP:8001
Could be that your server is not ok. Check the config. See your config.xml and what the listen address for the server is and verify that you can ping it.