We Keep Coding

Spring data elasticsearch to create index dynamically based on request parameter, percolator support and create index via Elasticsearch Operations

I read through https://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/#reference to begin with
My requirements
I want to use percolator. Is there any support for it in spring data elasticsearch? I don't see any in the above link although I understand that percolating is same as indexing (technically from using spring data elasticsearch's perspective). So I can use the indexing part of spring data elasticsearch but just checking if there are any that are specific to percolator.
I want to create an index dynamically. I do understand I can achieve that using SpEL template expression as mentioned in https://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/#elasticsearch.mapping.meta-model.annotations but my case is slightly different, I will get the index name via the RequestParam as part of the API call. So this means as of my knowledge I cannot use SpEL or try something like https://stackoverflow.com/a/33520421/4068218
I see I can use ElasticsearchOperations or ElasticsearchRepository to create Index. Because of #2 (i.e index name via request parameter) I think ElasticsearchOperations better suits but I see IndexOperations facilitating createMapping, createSettings but not both together. I see putMapping too but I dont see anything that says both mapping and settings. The reason I want both is I want to create something like below to begin with
"settings" : {
"index" : {
"number_of_shards" : 1,
"number_of_replicas" : 0
}
},
"mappings": {
"properties": {
"message": {
"type": "text"
},
"query": {
"type": "percolator"
}
}
}
Bottom line :- How do I create an index (name of the index will be dynamic via request param) with mappings, settings using ElasticsearchOperations?
Any lead/help is much appreciated

First of all thank you very much #P.J.Meisch. Upvoted both your comments as a token of gratitude.
Below worked for me. Below might help others in future
Document mapping = Document.create().fromJson("{\n" +
"\n" +
" \"properties\": {\n" +
" \"message\": {\n" +
" \"type\": \"text\"\n" +
" },\n" +
" \"query\": {\n" +
" \"type\": \"percolator\"\n" +
" }\n" +
" }\n" +
"\n" +
"}");
Map<String, Object> settings = ImmutableMap.of( "number_of_shards" ,2,"number_of_replicas",1);
elasticsearchOperations.indexOps(IndexCoordinates.of("whatever-indexname-you-need")).create(settings,mapping);

How to show customized JSON in Swagger UI?

I'm using springfox-boot-starter 3.0 to build my API documents.
All things are going in a right way except one case. There seems to be no way to show a JSON example in Swagger UI model if my request parameter is a Map or JSONObject like this:
#PostMapping("/trigger")
#ApiOperation(value = "trigger something")
#ApiImplicitParams(
#ApiImplicitParam(name = "reqForm",
value = "{\"123123\":\"123123\"}", example = "{\"123123\":\"123123\"}"))
public StandardResult trigger(#RequestBody Map reqForm) throws Exception {
Object dagId = reqForm.get("dagId");
Object conf = reqForm.get("conf");
return StandardResult.succeed(dagService.trigger(dagId, conf));
}
I just want to show an example value and a example model of the JSON in Swagger UI and I don't want to write any of extra .java file to define the structure.
Other controllers with plenty of .java files to describe structure can be shown in Swagger UI like this:
But in this case, the Map shall be a dynamic parameter which would change frequently. So I hope to show a model of that JSON without too many .java files so that others who are reading my document would have a nice experience and I will not have to change .java file every day.
I know how to show the model and examples in Swagger UI by creating multiple Java beans using #ApiModel and #ApiModelProperty. But that may also lead to a dozens of .java files in order to create only one JSON and it is hard to find and update a property while something in JSON was changed.
For example, I'm going to tell others to send a JSON like this:
"dagInfo": {
"id": 17,
"tags": [
"test",
"task",
"dag"
],
"interval": "None",
"dagName": "testDagGenerate",
"dagCode": "test_dag_generate",
"dagDescription": "test"
}
by using #ApiImplicitParams shown below, I can show the example value but no model in Swagger UI.
#PostMapping("/trigger")
#ApiOperation(value = "trigger something")
#ApiImplicitParams(
#ApiImplicitParam(name = "reqForm",
value = "example json", example = "\"dagInfo\": {\n" +
" \"id\": 17,\n" +
" \"tags\": [\n" +
" \"test\",\n" +
" \"task\",\n" +
" \"dag\"\n" +
" ],\n" +
" \"interval\": \"None\",\n" +
" \"dagName\": \"testDagGenerate\",\n" +
" \"dagCode\": \"test_dag_generate\",\n" +
" \"dagDescription\": \"test\"\n" +
" }"))
public StandardResult trigger(#RequestBody JSONObject reqForm) throws Exception {
Long dagId = reqForm.getObject("dagId", Long.class);
JSONObject conf = reqForm.getJSONObject("conf");
return StandardResult.succeed(dagService.trigger(dagId, conf));
}
I have no idea how to write this model of example JSON directly to Swagger.
Or there is no way to define a example model in Swagger UI without any configuration?
The dependency of Maven is shown below:
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-boot-starter</artifactId>
<version>3.0.0</version>
</dependency>

elasticsearch wildcard search java

I am using java low level REST client api.
I am fetching particular fields on matching or related datas from elasticsearch by using wildcards.
I am struck with it getting data as well as searching data with wildcard.
How to fetch result in java list .
Here is my code:
HttpEntity entity1 = new NStringEntity( "{\n" + " \"query\" : {\n" + " \"wildcard\": { \"Content\":\"java *\"} \n" + "} \n"+ "}",ContentType.APPLICATION_JSON);
Response response = restClient.performRequest("GET", "/test/_search",Collections.singletonMap("pretty", "true"), entity1);
System.out.println(EntityUtils.toString(response.getEnti‌​ty()));
Could you please help me out? Thank you!

How to get a file from a curl get request in Java?

I'm trying to use an API to download some XBRL files. In order to do that I need to do a curl request, like this:
curl -XGET http://distribution.virk.dk/offentliggoerelser --data-binary #query_regnskaber.json
The idea is, as I understand it, that "#query_regnskaber.json" is a json file / json query that I need to send with my request and in return I get a XBRL file(s) / some data. I'm using Java with the play framework (not specifically using play framework for the curl request though, but maybe someone know some play features to do curl requests).
This is my current code:
String jsonStr =
"{" +
"\"query\": {" +
"\"bool\": {" +
"\"must\": [" +
"{" +
"\"term\": {" +
"\"offentliggoerelse.dokumenter.dokumentMimeType\": \"application\"" +
"}" +
"}," +
"{" +
"\"term\": {" +
"\"offentliggoerelse.dokumenter.dokumentMimeType\": \"xml\"" +
"}" +
"}," +
"{" +
"\"range\": {" +
"\"offentliggoerelse.offentliggoerelsesTidspunkt\": {" +
"\"from\": \"2016-12-01\"" +
"}" +
"}" +
"}" +
"]," +
"\"must_not\": []," +
"\"should\": []" +
"}" +
"}," +
"\"size\": 1000" +
"}";
String urlStr = "http://distribution.virk.dk/offentliggoerelser";
JSONObject jsonObj = new JSONObject(jsonStr);
URL myURL = new URL(urlStr);
HttpURLConnection urlCon = (HttpURLConnection)myURL.openConnection();
urlCon.setRequestProperty("Content-Type", "application/json; charset=UTF-8");
urlCon.setRequestMethod("GET");
urlCon.setDoInput(true);
urlCon.setDoOutput(true);
urlCon.connect();
OutputStream os = urlCon.getOutputStream();
os.write(jsonObj.toString().getBytes("UTF-8"));
os.close();
BufferedReader br = new BufferedReader(new InputStreamReader((urlCon.getInputStream())));
String output;
System.out.println("Output from Server .... \n");
while ((output = br.readLine()) != null) {
System.out.println(output);
}
urlCon.disconnect();
Something goes wrong and I'm not sure whether it's because of some missing settings, my code or both. I get the 403 error on the "urlCon.getInputStream()" call.
The only documentation I can find for the API is in Danish. It also mentions that it uses ElasticSearch, which I assume is used to find specific XBRL files that can be found on "http://distribution.virk.dk/offentliggoerelser/_search". Finding specific XBRL files is something I want to be able to do to. Just in case, here is a link to the API documentation.
I'm using the example json query that can be found in the documentation, in my code.
Thank you for your help.
My json test query:
{
"query": {
"bool": {
"must": [
{
"term": {
"offentliggoerelse.dokumenter.dokumentMimeType": "application"
}
},
{
"term": {
"offentliggoerelse.dokumenter.dokumentMimeType": "xml"
}
},
{
"range": {
"offentliggoerelse.offentliggoerelsesTidspunkt": {
"from": "2014-10-01"
}
}
}
],
"must_not": [],
"should": []
}
},
"size": 1000
}

it seems its a ElasticSearch at the backend and not much has changed,
to send query to http://distribution.virk.dk/offentliggoerelser is forbidden as is in ElasticSearch. (you can't query index directly)
But should work if you send POST (GET seems to work too) query to http://distribution.virk.dk/offentliggoerelser/_search (NOTE /_search)
so change
String urlStr = "http://distribution.virk.dk/offentliggoerelser";
to
String urlStr = "http://distribution.virk.dk/offentliggoerelser/_search";
Optionally change
urlCon.setRequestMethod("GET");
to
urlCon.setRequestMethod("POST");
NOTE:
in case you are wondering why your CURL works,
well it doesn't, because you use XGET instead of XPOST it simply ignores the query file you are sending thus spits out some information that don't correspond to your query
which is clearly wrong.

(Posted solution on behalf of the OP).
I added "/_search" to the site and changed the request method to POST. Explanation in nafas' answer.

Setting doOutput to TRUE turns it into a POST. Remove, along with whatever output you're sending. Request parameters should be in the URL for GET requests.

Java Mail Embed URL

When I'm adding an HTML URL into email body, it is not redirecting to the preferred location. This is the snippet, please tell me what am I doing wrong.
#location variable contains the URL
StringBuffer body = new StringBuffer("<html><body>Hi, <br/><br/>");
body.append("<p>"+cmts+"</p>");
#both the ways are not working, how to construct proper URL
body.append("<br/><br/>" + location + "<br/>");
body.append("<br/><br/>" +location + "<br/>");
#this is working as link only in OUTLOOK, but in other mail client it shows as plain text
body.append("<br/><br/>"+location);
URL:
http://host:port/weebApp/report/viewer.html#%2Fpublic%2FSamples%2FDashboards%2_FSample_report

It looks like a problem with the quotation marks. Try:
body.append("<br/><br/>" + location + "<br/>");

There could be many ways to add href in javamail for example:
1) InternetHeaders headers = new InternetHeaders();
headers.addHeader("Content-type", "text/html; charset=UTF-8");
String aHref = "some text\n" + text +
"\n<a href='http://google.com'>google.com</a>";
2) String aHref = "some text\n" + text +
"\n<a href='http://google.com'>google.com</a>";
messageBodyPart.setText(aHref,"UTF-8","html");
UPDATE:
Make sure the content-type is set to html or text/html because text/plain will display it as only text