I am using java low level REST client api.
I am fetching particular fields on matching or related datas from elasticsearch by using wildcards.
I am struck with it getting data as well as searching data with wildcard.
How to fetch result in java list .
Here is my code:
HttpEntity entity1 = new NStringEntity( "{\n" + " \"query\" : {\n" + " \"wildcard\": { \"Content\":\"java *\"} \n" + "} \n"+ "}",ContentType.APPLICATION_JSON);
Response response = restClient.performRequest("GET", "/test/_search",Collections.singletonMap("pretty", "true"), entity1);
System.out.println(EntityUtils.toString(response.getEntity()));
Could you please help me out? Thank you!
Related
I read through https://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/#reference to begin with
My requirements
I want to use percolator. Is there any support for it in spring data elasticsearch? I don't see any in the above link although I understand that percolating is same as indexing (technically from using spring data elasticsearch's perspective). So I can use the indexing part of spring data elasticsearch but just checking if there are any that are specific to percolator.
I want to create an index dynamically. I do understand I can achieve that using SpEL template expression as mentioned in https://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/#elasticsearch.mapping.meta-model.annotations but my case is slightly different, I will get the index name via the RequestParam as part of the API call. So this means as of my knowledge I cannot use SpEL or try something like https://stackoverflow.com/a/33520421/4068218
I see I can use ElasticsearchOperations or ElasticsearchRepository to create Index. Because of #2 (i.e index name via request parameter) I think ElasticsearchOperations better suits but I see IndexOperations facilitating createMapping, createSettings but not both together. I see putMapping too but I dont see anything that says both mapping and settings. The reason I want both is I want to create something like below to begin with
"settings" : {
"index" : {
"number_of_shards" : 1,
"number_of_replicas" : 0
}
},
"mappings": {
"properties": {
"message": {
"type": "text"
},
"query": {
"type": "percolator"
}
}
}
Bottom line :- How do I create an index (name of the index will be dynamic via request param) with mappings, settings using ElasticsearchOperations?
Any lead/help is much appreciated
First of all thank you very much #P.J.Meisch. Upvoted both your comments as a token of gratitude.
Below worked for me. Below might help others in future
Document mapping = Document.create().fromJson("{\n" +
"\n" +
" \"properties\": {\n" +
" \"message\": {\n" +
" \"type\": \"text\"\n" +
" },\n" +
" \"query\": {\n" +
" \"type\": \"percolator\"\n" +
" }\n" +
" }\n" +
"\n" +
"}");
Map<String, Object> settings = ImmutableMap.of( "number_of_shards" ,2,"number_of_replicas",1);
elasticsearchOperations.indexOps(IndexCoordinates.of("whatever-indexname-you-need")).create(settings,mapping);
I'm stuck with a SearchTemplate request, until now I've been able to write queries through the QueryBuilder and execute them calling the "search" method of the RestHighLevelClient java client, but when I try to execute the searchTemplate method I get a parse exception from the server "request body or source parameter is required".
So far I've been reading the documentation but nothing, the guide about the SearchTemplate is very poor.
This is how I'm currently trying to follow the basic steps from the official documentation:
SearchTemplateRequest request = new SearchTemplateRequest();
request.setRequest(new SearchRequest(indexPI));
request.setScriptType(ScriptType.INLINE);
request.setScript( "{" +
" \"query\": { \"match\" : { \"{{field}}\" : \"{{value}}\" } }," +
" \"size\" : \"{{size}}\"" +
"}");
Map<String, Object> scriptParams = new HashMap<>();
scriptParams.put("from","");
scriptParams.put("size","");
scriptParams.put("field", "title");
scriptParams.put("value", "elasticsearch");
scriptParams.put("size", 5);
request.setScriptParams(scriptParams);
SearchTemplateResponse response = elasticSearchClient.searchTemplate(request,
RequestOptions.DEFAULT);
The error I get:
Elasticsearch exception [type=parse_exception, reason=request body or source parameter is required]
At this point, I'm trying to understand how to set the request body for the above request and actually wondering if it is possible to do.
Any clue about what is missing besides the "unsettable" request body from the exception?
We have an API which take a file from system and shows on the application, for which I am trying to automate with rest assured and Java
I have tried, changing the image to binary code and then adding it as parameter that does not work.
Map<String, String> paramSample = new HashMap<>();
paramSample.put("api_key", "A813302*************");
paramSample.put("method", "my");
paramSample.put("body", "{\n" +
" \"to\":\"91xxxxxxxx\",\n" +
" \"type\": \"image\", \"image\" : {\"caption\" : \"{{caption}}\"},\n" +
"\"callback\":\"{{callback}}\"\n" +
"}");
paramSample.put("from", "91xxxxxxx");
paramSample.put("file","C:\\Users\\sobhit.s\\Pictures\\SMS-2047.png");
RequestSpecification request = given();
Response responseSample = request.params(paramSample).get(ExecutionConfig.BASE_URL).then().extract().response();
String res=responseSample.prettyPrint();
Response is-
{
"status": "xxxx",
"message": "Invalid file format. Upload valid file."
}
First if you are unsure, do this in Postman and then recreate the same in code.
This way you will have a post man to demonstrate your coding problem.
Use .queryParam() only for params and not for the body content. Body content should be under .body()
Use .multiPart() to upload the file as a multi part quest. Hope this helps.
given().queryParam(
"api_key", "A813302*************",
"method", "my",
"from", "91xxxxxxx")
.body("{\n" +
" \"to\":\"91xxxxxxxx\",\n" +
" \"type\": \"image\", \"image\" : {\"caption\" : \"{{caption}}\"},\n" +
"\"callback\":\"{{callback}}\"\n" +
"}")
.multiPart(new File("C:/Users/sobhit.s/Pictures/SMS-2047.png"))
.when()
.get(ExecutionConfig.BASE_URL)
.prettyPrint();
I'm trying to use an API to download some XBRL files. In order to do that I need to do a curl request, like this:
curl -XGET http://distribution.virk.dk/offentliggoerelser --data-binary #query_regnskaber.json
The idea is, as I understand it, that "#query_regnskaber.json" is a json file / json query that I need to send with my request and in return I get a XBRL file(s) / some data. I'm using Java with the play framework (not specifically using play framework for the curl request though, but maybe someone know some play features to do curl requests).
This is my current code:
String jsonStr =
"{" +
"\"query\": {" +
"\"bool\": {" +
"\"must\": [" +
"{" +
"\"term\": {" +
"\"offentliggoerelse.dokumenter.dokumentMimeType\": \"application\"" +
"}" +
"}," +
"{" +
"\"term\": {" +
"\"offentliggoerelse.dokumenter.dokumentMimeType\": \"xml\"" +
"}" +
"}," +
"{" +
"\"range\": {" +
"\"offentliggoerelse.offentliggoerelsesTidspunkt\": {" +
"\"from\": \"2016-12-01\"" +
"}" +
"}" +
"}" +
"]," +
"\"must_not\": []," +
"\"should\": []" +
"}" +
"}," +
"\"size\": 1000" +
"}";
String urlStr = "http://distribution.virk.dk/offentliggoerelser";
JSONObject jsonObj = new JSONObject(jsonStr);
URL myURL = new URL(urlStr);
HttpURLConnection urlCon = (HttpURLConnection)myURL.openConnection();
urlCon.setRequestProperty("Content-Type", "application/json; charset=UTF-8");
urlCon.setRequestMethod("GET");
urlCon.setDoInput(true);
urlCon.setDoOutput(true);
urlCon.connect();
OutputStream os = urlCon.getOutputStream();
os.write(jsonObj.toString().getBytes("UTF-8"));
os.close();
BufferedReader br = new BufferedReader(new InputStreamReader((urlCon.getInputStream())));
String output;
System.out.println("Output from Server .... \n");
while ((output = br.readLine()) != null) {
System.out.println(output);
}
urlCon.disconnect();
Something goes wrong and I'm not sure whether it's because of some missing settings, my code or both. I get the 403 error on the "urlCon.getInputStream()" call.
The only documentation I can find for the API is in Danish. It also mentions that it uses ElasticSearch, which I assume is used to find specific XBRL files that can be found on "http://distribution.virk.dk/offentliggoerelser/_search". Finding specific XBRL files is something I want to be able to do to. Just in case, here is a link to the API documentation.
I'm using the example json query that can be found in the documentation, in my code.
Thank you for your help.
My json test query:
{
"query": {
"bool": {
"must": [
{
"term": {
"offentliggoerelse.dokumenter.dokumentMimeType": "application"
}
},
{
"term": {
"offentliggoerelse.dokumenter.dokumentMimeType": "xml"
}
},
{
"range": {
"offentliggoerelse.offentliggoerelsesTidspunkt": {
"from": "2014-10-01"
}
}
}
],
"must_not": [],
"should": []
}
},
"size": 1000
}
it seems its a ElasticSearch at the backend and not much has changed,
to send query to http://distribution.virk.dk/offentliggoerelser is forbidden as is in ElasticSearch. (you can't query index directly)
But should work if you send POST (GET seems to work too) query to http://distribution.virk.dk/offentliggoerelser/_search (NOTE /_search)
so change
String urlStr = "http://distribution.virk.dk/offentliggoerelser";
to
String urlStr = "http://distribution.virk.dk/offentliggoerelser/_search";
Optionally change
urlCon.setRequestMethod("GET");
to
urlCon.setRequestMethod("POST");
NOTE:
in case you are wondering why your CURL works,
well it doesn't, because you use XGET instead of XPOST it simply ignores the query file you are sending thus spits out some information that don't correspond to your query
which is clearly wrong.
(Posted solution on behalf of the OP).
I added "/_search" to the site and changed the request method to POST. Explanation in nafas' answer.
Setting doOutput to TRUE turns it into a POST. Remove, along with whatever output you're sending. Request parameters should be in the URL for GET requests.
Thanks in advance for the help. I'm testing a GRAPHQL API using rest assured on testng in java.
I am trying to send agraphQL object disguised as JSON as a post, but when trying to package the string up as a JSON object I am getting the following error:
SyntaxError: Unexpected token B<br> at Object.parse (native)<br> at parse (/app/node_modules/body-parser/lib/types/json.js:88:17)<br> at /app/node_modules/body-parser/lib/read.js:116:18<br> at invokeCallback (/app/node_modules/raw-body/index.js:262:16)<br> at done (/app/node_modules/raw-body/index.js:251:7)<br> at IncomingMessage.onEnd (/app/node_modules/raw-body/index.js:307:7)<br> at emitNone (events.js:80:13)<br> at IncomingMessage.emit (events.js:179:7)<br> at endReadableNT (_stream_readable.js:913:12)<br> at _combinedTickCallback (internal/process/next_tick.js:74:11)<br> at process._tickDomainCallback (internal/process/next_tick.js:122:9)
The JSON object I am trying to create is here:
"{\"query\":\"{marketBySlug(slug: \"Boston\") {countryCode}}\"}"
I've figured out that my problem is the escaped quotes identifying Boston as a string are breaking the internally constructed Graphql query string, but I'm not sure how to get around this.
Try this:
"{\"query\":\"{marketBySlug(slug: \\\"Boston\\\") {countryCode}}\"}"
Instead of messing with strings and escaping, you should use parameterized queries. Your code will look like this:
String query =
"query MarketBySlug($slug: String!) {\n" +
" marketBySlug(slug: $slug) {\n" +
" countryCode\n" +
" }\n" +
"}";
Map<String, Object> variables = new HashMap<>();
variables.put("slug", slug);
given()
.body(new QueryDto(query, variables))
.when()
.post("/graphql")
.then()
.contentType(JSON)
.statusCode(HttpStatus.OK.value())
.body("data.marketBySlug.countryCode", equalTo(countryCode));
QueryDto is just a simple dto with the two fields (query and variables), setters and getters.