I'm trying to use ElasticSearch java API in a Dropwizard application.
I found the dropwizard-elasticsearch package: https://github.com/dropwizard/dropwizard-elasticsearch, that seems to be exactly what I need.
Unfortunately, it provides zero "useful" documentation, and no usage examples.
I still haven't understood how to connect to remote servers using the TransportClient, because, due to no documentation of drop wizard-elasticsearch configuration, I should try "randomly" until I find the correct configuration keys...
Does anyone have tried using dropwizard-elasticsearch? Or has someone a real usage example of this?
Thanks in advance,
Unless you really need to join the Elasticsearch cluster, I would avoid using the Java classes provided by Elasticsearch. If you do connect to Elasticsearch this way, you will need to keep the JVM versions used by Elasticsearch and your application in sync.
Instead, you can connect to Elasticsearch using the Jest client found on GitHub. This will allow you to connect to Elasticsearch over the REST interface, just like all of the other client libraries.
You will need to create a simple configuration block for Elasticsearch, to specify the URL of the REST interface. Also, you will need to create a Manager for starting and stopping the JestClient.
Update: You can find the Dropwizard bundle that I use for connecting to Elasticsearch on GitHub. Here are some basic usage instructions for Java 8:
Include the dependency for the bundle in your project's POM.
<dependency>
<groupId>com.meltmedia.dropwizard</groupId>
<artifactId>dropwizard-jest</artifactId>
<version>0.1.0</version>
</dependency>
Define the JestConfiguraion class somewhere in your application's configuration.
import com.meltmedia.dropwizard.jest.JestConfiguration;
...
#JsonProperty
protected JestConfiguration elasticsearch;
public JestConfiguration getElasticsearch() {
return jest;
}
Then include the bundle in the initialize method of your application.
import com.meltmedia.dropwizard.jest.JestBundle;
...
protected JestBundle jestBundle;
#Override
public void initialize(Bootstrap<ExampleConfiguration> bootstrap) {
bootstrap.addBundle(jestBundle = JestBundle.<ExampleConfiguration>builder()
.withConfiguration(ExampleConfiguration::getElasticsearch)
.build());
}
Finally, use the bundle to access the client supplier.
#Override
public void run(ExampleConfiguration config, Environment env) throws Exception {
JestClient client = jestBundle.getClientSupplier().get();
}
Too long for a comment.
Please check README.md ->"Usage" and "Configuration". If you want dropwizard to create managed TransportClient your configuration settings should be something like this.
nodeClient: false
clusterName: dropwizard_elasticsearch_test
servers:
- 127.23.42.1:9300
- 127.23.42.2:9300
- 127.23.42.3
How to obtain dropwizard-managed TransportClient? Example here: public void transportClientShouldBeCreatedFromConfig().
#Override
public void run(DemoConfiguration config, Environment environment) {
final ManagedEsClient managedClient = new ManagedEsClient(configuration.getEsConfiguration());
Client client = managedClient.getClient();
((TransportClient) client).transportAddresses().size();
// [...]
}
There is also a sample blog application using Dropwizard and ElasticSearch. See "Acknowledgements" section in README.md.
I have used Java api for elasticsearch and the thing is the bundle you are using I also explored but documentation part discouraged using it. Here you can use elastic without this bundle:-
Define your elastic configs in your .yml file.
elasticsearchHost: 127.0.0.1
elasticPort: 9300
clusterName: elasticsearch
Now in configuration file(which in my case is mkApiConfiguration) create static functions which will actually be the getter methods for these elastic configuration:-
#NotNull
private static String elasticsearchHost;
#NotNull
private static Integer elasticPort;
#NotNull
private static String clusterName;
#JsonProperty
public static String getElasticsearchHost() {
return elasticsearchHost;
}
//This function will be called while reading configurations from yml file
#JsonProperty
public void setElasticsearchHost(String elasticsearchHost) {
mkApiConfiguration.elasticsearchHost = elasticsearchHost;
}
#JsonProperty
public void setClusterName(String clusterName) {
mkApiConfiguration.clusterName = clusterName;
}
public void setElasticPort(Integer elasticPort) {
mkApiConfiguration.elasticPort = elasticPort;
}
#JsonProperty
public static String getClusterName() {
return clusterName;
}
#JsonProperty
public static Integer getElasticPort() {
return elasticPort;
}
Now create a elastic factory from where you can get a transport client better create it as a singleton class so that only one instance is created and shared for elastic configuration we can get configuration using getter method of configuration class as these are static method so we don't need to create an object to access these methods. So code of this factory goes like that
public class ElasticFactory {
//Private constructor
private ElasticFactory(){};
public static Client getElasticClient(){
try {
/*
* Creating Transport client Instance
*/
Client client = TransportClient.builder().build()
.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName(mkApiConfiguration.getElasticsearchHost()), mkApiConfiguration.getElasticPort()));
return client;
}
catch (Exception e){
e.printStackTrace();
return null;
}
}
Now you can call this elastic factory method from any class like below:-
/*As getInstance is a static method so we can access it directly without creating an object of ElasticFactory class */
Client elasticInstance= ElasticFactory.getElasticClient();
Related
I have a SpringBoot application which I am trying to test with the help of Testcontainers. I have something like:
#SpringBootTest
public class DummyIT {
#ClassRule
public static PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer();
static {
postgreSQLContainer.start();
String url = postgreSQLContainer.getJdbcUrl();
}
}
Unfortunately Testcontainers use random ports and if I understood correctly we are not supposed to have it any other way, which means that the result of postgreSQLContainer.getJdbcUrl() is not deterministic.
Moreover, my application retrieves the database url from its application.properties and I was aiming at replacing such value from the one provided by postgreSQLContainer.getJdbcUrl(), before it is first used during runtime. Is it possible to achieve this?
Thank you for your help.
You can use Testcontainers JDBC URL support.
For example,
#SpringBootTest(
properties = {
"spring.datasource.url=jdbc:tc:postgresql:9.6.8://localhost/your-database",
"spring.datasource.username=dummy-user",
"spring.datasource.password=dummy-password",
"spring.jpa.database-platform=org.hibernate.dialect.MySQLDialect",
"spring.datasource.driver-class-name=org.testcontainers.jdbc.ContainerDatabaseDriver"
})
public class DummyIT {
}
The Testcontainers driver ContainerDatabaseDriver will create and start a PostgreSQL server 9.6.8 with a database named your-database.
I´m setting up a Spring Boot application where certain configurations are being read from my application.yaml-file. I´ve done this a few times before and it works well, but I wondered whether there is a better way to access this configuration during runtime or whether I´m creating possible issues by not following some best practice.
Right now the class that extracts the configuration is simply defined as a Component like this:
#Component
#EnableConfigurationProperties
#ConfigurationProperties("myPrefix")
public class MyExternalConfiguration{
private HashMap<String, Boolean> entries= new HashMap<String, Boolean>();
public Boolean getConfigurationForKey(String key) {
return this.entries.get(key);
}
}
And then autowired to several other classes that need to access this configuration like this:
#Component
public class MyClass{
#Autowired
private MyExternalConfiguration myExternalConfiguration;
public void doSomething(){
//...
Boolean someEntry = myExternalConfiguration.getConfigurationForKey(someKey);
}
}
Now, this does work just fine. It´s just that I have seen examples of where configurations like this are handled as a singleton for example (although not in a Spring-Boot environment). I would just like to ask, whether there is some commonly accepted way to access external configurations or whether you see an issue with the way i access it in my project.
Thank you in advance!
There is a whole chapter about configuration in the Spring Boot Reference Manual:
https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#boot-features-external-config
Simply said there are two options to access configuration:
With the Value annotation:
#Value("${name}")
private String name;
Or typesafe with a configuration class:
#ConfigurationProperties(prefix="my")
public class Config {
private List<String> servers = new ArrayList<String>();
public List<String> getServers() {
return this.servers;
}
}
So there is no need to read the configuration file by your own.
I'm fairly new to Vertx, And trying to find some realistic examples of database usage.
I have a Verticle that creates a shared database object (And a number of classes that handle routing, but I would like to use the shared database outside the main class, obviously I could pass the database object in other classes constructors, but I'm sure Vertx has some better way to do this.
public void start() {
...
this.postgreSQLClient = PostgreSQLClient.createShared(vertx, sqlClientConfig);
...
}
Does anyone have any Java Vertx examples with realistic implementations of a database?
Thank you in advance.
Use Dependency Injection. I have used Guice
Here's the example of it:
Main.java
//within main function where you have object of vertx
Guice.createInjector(new AppInjector(vertx));
AppInjector.java
//Create an Injector file and bind your injections
PostgreSQLClient postgreSQLClient = PostgreSQLClient.createShared(vertx, sqlClientConfig, "my-shared-client");
bind(PostgreSQLClient.class).annotatedWith(Names.named("DBClient")).toInstance(postgreSQLClient);
UserService.java
public class UserService {
#Inject
#Named("DBClient")
private PostgreSQLClient client;
}
You can find the source code here
Just specify a pool name:
if different clients are created using the same Vert.x instance and
specifying the same pool name, they will share the same data source.
So updating your example:
public void start() {
this.postgreSQLClient = PostgreSQLClient.createShared(vertx, sqlClientConfig, "my-shared-client");
}
Note that when doing this, the configuration provided in the first call will be used. Subsequent calls will simply return the existing client.
Recently I am using play 2.4 framework for Java project.
In that I am using WsClient Library. That library is injected in my class.
#Inject WSClient wsClient
Now I am trying to write a test case for that class but test case fails because of null pointer error for wsClient variable.
wsClient.url("some url").get()
Can you please help me to resolve this?
following is the test code
// Class
public class ElasticSearch {
#Inject WSClient wsClient;
public Promise<WSResponse> createIndex() {
Logger.info("Entering ElasticSearch.createIndex()");
Logger.debug("WSClient: " + wsClient);
Promise<WSResponse> response =wsClient.url(this.getEsClient()+ "/" +this.getEsIndexName()).setContentType("application/json").put("");
Logger.info("Exiting ElasticSearch.createIndex()");
return response;
}
}
// Test function
#Test
public void testCreateIndex() {
running(fakeApplication(), new Runnable() {
public void run() {
ElasticSearch esearch= new ElasticSearch();
esearch.setEsIndexName("car_model");
assertNotNull(esearch.createIndex());
}
});
}
Before writing the options you have, I suggest to use elastic4s.
This third party lib will help you write a more functional code and give you very nice dsl to write your queries.
One more thing, I don't know what is your used case for using elasticsearch, but I recommend using a different client then the rest api, which will give you more secure connection and more efficient.
You get the NPE, because you instasiate ElasticSearch by your own with new, and don't let guice to the wiring for you, this is why the WSClient is null.
Now for your options,
You have 2 options:
Add WithApplication to your test, which will basically load your application, this will give you the access to Guice injector, from which you can take ElasticSearch class like this you have couple of ways to do it:
As described in play documentation using
import play.api.inject.guice.GuiceInjectorBuilder
import play.api.inject.bind
val injector = new GuiceInjectorBuilder()
.configure("key" -> "value")
.bindings(new ComponentModule)
.overrides(bind[Component].to[MockComponent])
.injector
val elasticsearch = injector.instanceOf[ElasticSearch]
By importing Play
import play.api.Play
val elasticsearch = Play.current.injector.instanceOf(classOf[ElasticSearch])
Using FakeApplication: just get hold of the fake application injector, and use it to get an instance of ElasticSearch class.
I don't like the above options, because you need an application running, which can make your tests very slow.
I suggest to create the WSClient by your self and instantiate ElasticSearch class with it, and run your tests.
val httpClient = NingWSClient.apply()
val elasticsearch = new ElasticSearch(httpClient)
This is a more light solution, and should make your tests run faster.
I have the following code deployed to an app engine server (the only
place where I can test JDO, unfortunately I cannot test JDO locally
because I don't have a local BigTable implementation).
final class PMF {
private static final PersistenceManagerFactory pmf = JDOHelper.getPersistenceManagerFactory("transactions-optional");
private PMF() { }
public static PersistenceManagerFactory get() { return pmf; }
}
#PersistenceCapable
class Data {
// ...
#Persistent
private static List<Store> stores = new ArrayList<Store>();
static List<Store> getStores() {
return stores;
}
}
...
Data.getStores().add(store);
writer.write("this line received OK by client.");
PMF.get().getPersistenceManager().makePersistent(Data.getStores());
writer.write("this line never received by client.");
As shown the first line of output from the server is received on the client and the second one is not which means makePersistent() is failing.
Anyone have any idea why this is happening?
Perhaps the simple fact that no standard persistence API for Java provides persistence of static fields.
You can mimic BigTable on your local machine by running your code locally using ant or the eclipse appengine plugin. The eclipse plugin also runs datanucleus in the background and will catch errors like this for you without having to upload to appengine whenever you make a change.