How to exclude source fields in search APIs? - java

Elasticsearch Java High Level REST Client's GET API provides a way to control which fields of the _source are fetched.
val request = GetRequest(index)
.id(id)
.fetchSourceContext(FetchSourceContext(true, includedFields, excludedFields))
elasticClient.get(request, RequestOptions.DEFAULT)
How can I achieve this with the Search APIs?
For example for the following search request:
val source = SearchSourceBuilder()
source.query(QueryBuilders.matchAllQuery())
val request = SearchRequest(index)
.source(source)
elasticClient.search(request, RequestOptions.DEFAULT)

Please refer this from official ES doc,
This method also accepts an array of one or more wildcard patterns to control which fields get included or excluded in a more fine-grained way:
code block
String[] includeFields = new String[] {"title", "innerObject.*"};
String[] excludeFields = new String[] {"user"};
sourceBuilder.fetchSource(includeFields, excludeFields);
Simlar to get API which you already mentioned, you can provide an array of
includeFields and excludeFields to control fetching of the fields from _source fields.

Related

How to get custom extension attributes with microsoft graph api

I'm trying to get user details with microsoft-graph
I'm looking for a custom extension element in my response, such as "extension_3a4189d71ad149c6ab5e65ac45bd6add_MyAttribute1"
when I retrieve the response with String, I can see all the elements.
final ResponseEntity<String> response = restTemplate.exchange("http://graph.windows.net/tenant.com/me?api-version=1.6, HttpMethod.GET, new HttpEntity(headers),String.class);
But when I retrieve the response with com.microsoft.graph.models.extensions.User I can't see the extention anymore.
final ResponseEntity<User> response = restTemplate.exchange("http://graph.windows.net/tenant.com/me?api-version=1.6, HttpMethod.GET, new HttpEntity(headers),User.class);
How can I retrieve the custom extension in more elegant way than getting in String object and look for elements one by one?
because the extension attributes are specific to your tenant, that means its non standard, no out of the box "object class" in the sdk would contain it since it has the app id appended to it. extension_appid_attribname.
so you would have to handle it yourself. you can try to extend the user class and add a method to read or deserialize/map the json return from the graph api similar to what Hury suggested, or something to that effect. there won't likely be an out of the box solution for this.
there are also json libraries out there that may help you deserialize to a dynamic object of some sort, if you really didn't want to map the object manually.
Update:
I dug into this a bit further. I don't think its in extensions.extension however, I did find that in the java sdk you can access it . Here's the documentation: https://github.com/microsoftgraph/msgraph-sdk-java/wiki/Working-with-Open-Types
You would do something like
String ext =
user
.additionalDataManager()
.get("extension_2lkj3l12jl3j2kj3_yourproperty")
.getAsString();
Give that a try
Hopefully that helps.
Please use api
https://graph.microsoft.com/beta/users instead of https://graph.microsoft.com/v1.0/users api https://graph.microsoft.com/beta/users will return all data with custom data of users.

twitter4J unable to get raw json when using search API

ConfigurationBuilder cb = new ConfigurationBuilder();
//... set your keys
String queryString = "Trump";
Twitter twitter = new TwitterFactory(cb.build()).getInstance();
result = twitter.search(query);
List<Status> tweets = result.getTweets();
String json = TwitterObjectFactory.getRawJSON(s);
and I get :
java.lang.IllegalStateException: Apparently jsonStoreEnabled is not
set to true.
Question 1:
I looked through the source code and looks like jsonStoreEnabled is set on ThreadLocal by a call to TwitterObjectFactory.registerJSONObject . I don't think search does that . Does it mean its impossible to get raw json when calling search API ?
Question2:
How do I intercept a call made inside the API to TwitterBaseImpl.httpResponseReceived to get the value of the field before its obscured by layers of the framework ? preferably without AspectJ
Seelenvirtuose is correct , I forgot to do cb.setJSONStoreEnabled(true)

Creating type inside index in elastic search through rest high level client

I am using rest high level client elastic search in my JAVA application. Document can be found here.
In my application at startup I am deleting index named "posts" where Elasticsearch datas are stored and creating again Index "posts" following this link
CreateIndexRequest request = new CreateIndexRequest("posts");
But, Inside index I need to create one type named "doc". Which is not mentioned in the website.
Temporary fix is when I am posting some data following this link it is creating type
Map<String, Object> jsonMap = new HashMap<>();
jsonMap.put("user", "kimchy");
jsonMap.put("postDate", new Date());
jsonMap.put("message", "trying out Elasticsearch");
IndexRequest indexRequest = new IndexRequest("posts", "doc", "1")
.source(jsonMap);
But, in this process when I am posting only then I can able to create type "doc". If I am not posting and trying to hit controller which calls data frmo index "posts" and type "doc". It gives error as "doc" type is not there.
Anyone hava any idea how to create type using rest high level client ES in java
By type you mean document type?
What about the second section Index Mappings in the link you provided?
Does this not work for you?
I needed to set type to "_doc" to make it working with ES7.6
If you know how to insert documents through API then this way will much more easy for you to do anything similar API (DELETE,POST,PUT...)
First, you will need RestHighLevelClient and all you have to do
String index = "/indexName/_doc"; <- Your path or type here
Request request = new Request("POST", index); <- Your method
request.setJsonEntity(
"{ \"message\": \" example add insert\" }" <- Your request body
);
client.getLowLevelClient().performRequest(request);
    
This will execute like how API does.

Java - processing Marklogic eval response giving JSON Documents

I have a MarkLogic XQuery eval call that returns a lists of strings. I use the below code to process the results. I have another call that returns a list of Json Documents but I can't see how to get EvalResult to give me a JsonDocument document. How do I change the below code to process Json Documents?
public static ArrayList<String> getStringList(DatabaseClient client, String query)
{
ArrayList<String> strings = new ArrayList<String>();
ServerEvaluationCall eval = client.newServerEval();
EvalResultIterator eri = eval.xquery(query).eval();
while (eri.hasNext())
{
EvalResult er = eri.next();
String s = er.getString();
strings.add(s);
}
return strings;
}
First, let me suggest that you only use eval as a last resort as it may open a security hole. Injection attacks aren't possible if you never send code from the client to be executed on the server. Start first with the out-of-the-box features, and if those aren't enough, consider writing a resource extension instead of using eval. Two examples are ResourceExtension and JavascriptResourceExtension.
But to answer your question, change this:
String s = er.getString();
to this:
JacksonHandle handle = er.get(new JacksonHandle());
JsonNode json = handle.get();
or this shortcut:
JsonNode json = er.getAs(JsonNode.class);
For a complete example, see handling of myArray and myObject EvalTest.evalAndInvokeXQuery (and of course, runAndTestXQuery) and evaltest.xqy.
These Jackson handles work the same whether you're getting JSON results from a document read, search, or eval. You can read more about the io shortcuts here. For more sample code with Jackson, see JacksonHandleExample, JacksonHandleTest, JacksonStreamTest, and JacksonDatabindTest.

Access fields of returned Java Object

I made a simple client call to the XML-RPC WordPress API/Posts using a xml-rpc client and according to their documentation here it returns a struct. How can i access the return values.
Here is a look at my code:
XmlRpcClient client = new XmlRpcClient("http://www.mywebsite.net/xmlrpc.php", false);
String[] fields = new String[4];
fields[0] = "post_id";
fields[1] = "post_title";
fields[2] = "post_date";
fields[3] = "post_content";
Object token = client.invoke("wp.getPost", new Object[]{"0","myusername", "mypassword", 1545, fields });
System.out.println(token);
When I print use
System.out.println(token);
I get the following out put:
{item_one=I am item number one, item_two=I am Item two...}
How can I access item_one and item_two?
There's a bit of information missing (what's the fully qualified name of XmlRpcClient?), but assuming that client.invoke actually returns just an Object and not something more specific that has accessor methods, you can parse the response using something like this:
Object token = client.invoke("wp.getPost", new Object[]{"0","myusername", "mypassword", 1545, fields });
String[] items = token.toString().split(",");
for (String item : items) {
String[] parts = item.split("=");
String key = parts[0];
String value = parts[1];
// do stuff with your keys and values here
}
Of course this isn't perfect code -- you may need to check for nulls, use String.trim(), etc, but it should get you started.
You don't have a true Java representation of the data returned, in that you don't have an object on which you can access
token.item_one
rather you have a string containing a representation of a set - that is something that (in concept) from which you could retrieve an value by its name
token.get("item_one")
This string format is probably JSON, which pretty much looks like JavaScript, and hence can represent quite complex data. In general you can have arrays of objects and objects containing objects (for example, a Customer might contain an Address object)
So you have two possibilities:
1). parse the string into a true Java representation such as one of the standard Java collection classes. You then use the get-by-name style I show above.
2). define a Java class that mimics the structure of the data and then parse the string to fill out such an object, you can then use the "dot" form of access - you really have a Java Object representing the data.
In the first case there are suitable libraries such as quickJson
For the second you can use implementations of standards such as JAX/B, which tends to be more work as you may need to construct the target Java Class by hand. Enterprise Java runtimes will give you these facilities and perhaps tooling to help, or look at implementaitons such as Jackson. You will see that JAX/B hada focus on mapping from XML to Java, but tutorials such as this show how to work with JSON instead.
My guess is that the first option, simple parsing to a collection may be enough for you.

Categories