How to implement logic using elasticsearch Pagination using ElasticsearchRepository in java - java

Is there any way to use ElasticsearchRepository which extends from PagingAndSortingRepository. This allows built-in support for pagination and sorting.
But I am not able to change my implementation to use ElasticsearchRepository. I just want to know how to apply:
How to use Post Search
How to use esQuery which basically providing Search query
String esQuery = String.format(searchTextQuery, startDate, endDate, formattedQueries);
How to use that Post URI which I m getting as below:
Request request = new Request("GET", "/" + user.getUserId() + "/_search");
So with all the above, how to use Pagination with ElasticsearchRepository
Below is my service code:
public List getResponses(ZonedDateTime startDate, ZonedDateTime endDate,
String cat, FieldFilterVM filter, String query) throws IOException {
User user = (User)
SecurityContextHolder.getContext().getAuthentication().getPrincipal();
Request request = new Request("GET", "/" + user.getUserId() + "/_search");
List<String> matchQueries = new ArrayList<>();
matchQueries.addAll(formatCategoryQuery(cat));
matchQueries.addAll(formatFilterQuery(filter, false));
if (query != null && query.length() > 0) {
matchQueries.add(String.format(textFilterQuery, query));
}
StringBuilder formattedQueries = new StringBuilder();
for (int i = 0; i < matchQueries.size(); i++) {
formattedQueries.append(',');
formattedQueries.append(matchQueries.get(i));
}
String esQuery = String.format(searchTextQuery, startDate, endDate,
formattedQueries);
request.setJsonEntity(esQuery);
Response response =
elasticSearchService.getClient().getLowLevelClient().performRequest(request);
String responseBody = IOUtils.toString(response.getEntity().getContent(),
"UTF-8");
ObjectMapper mapper = new ObjectMapper();
Map map = mapper.readValue(responseBody, new TypeReference<Map>() {
});
List matchedTextResponses = new ArrayList();
if (map != null) {
List<Map> textResponses = (List<Map>) ((Map)
map.get("hits")).get("hits");
for (Map textResponse : textResponses) {
}
}
return matchedTextResponses;
}

Related

unable to redirect response in rest controller

I made two RestController apis. On response of second api I wanted first api's response (which is a json response), so I tried to use HttpServletResponse.redirect. I also set required content type to it. But on second api response I got Unsupported Media Type Content type 'null' not supported.
first API
#GetMapping(value="checkStatus/{msisdn}",consumes=MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<CoreResponseHandler> fetchOcsByDate2(#PathVariable(value="msisdn",required=true)String msisdn){
long l_time_start = System.currentTimeMillis();
List<Object[]> list = repository.getSingleCallDetail(msisdn);
if(list==null || list.size()==0) {
System.out.println("NO RECORD FOUND");
}
JSONObject objMain = new JSONObject();
for(Object[] objArr: list) {
JSONObject obj = new JSONObject();
String msisdn_ = objArr[0]==null?null:objArr[0].toString();
String songId = objArr[1]==null?null:objArr[1].toString();
String songName = objArr[2]==null?null:objArr[2].toString();
String status = objArr[3]==null?null:objArr[3].toString();
String lang = objArr[4]==null?null:objArr[4].toString();
String startDate = objArr[5]==null?null:objArr[5].toString();
objMain.put("status", status);
objMain.put("language", lang);
obj.put("id", songId);
obj.put("msisdn", msisdn);
obj.put("songName", songName);
objMain.put("subscription", obj);
}
long l_time_end = System.currentTimeMillis();
long l_diff = l_time_end-l_time_start;
if(list!=null && list.size()>0) {
return new ResponseEntity<CoreResponseHandler>(new SuccessResponseBeanRefined(HttpStatus.OK, ResponseStatusEnum.SUCCESSFUL, ApplicationResponse.SUCCESSFUL, objMain,l_diff+" ms"),HttpStatus.OK);
}
if(list==null || list.size()==0) {
return new ResponseEntity<CoreResponseHandler>(new SuccessResponseBeanRefined(HttpStatus.NOT_FOUND, ResponseStatusEnum.FAILED, ApplicationResponse.Failed, "not found",l_diff+" ms"),HttpStatus.NOT_FOUND);
}
return new ResponseEntity<CoreResponseHandler>(new SuccessResponseBeanRefined(HttpStatus.BAD_REQUEST, ResponseStatusEnum.FAILED, ApplicationResponse.Failed," > Bad request",l_diff+" ms"),HttpStatus.BAD_REQUEST);
}
no problem in output. ran smooth
second API
#GetMapping(value="verifyOtp/{msisdn}/{otp}",consumes=MediaType.APPLICATION_JSON_VALUE)
public void verifyOtp(#PathVariable(value="msisdn",required=true)String msisdn,
#PathVariable(value="otp",required=true)String otp,HttpServletResponse response) throws Exception{
long l_time_start = System.currentTimeMillis();
long l_time_end = System.currentTimeMillis();
long l_diff = l_time_end-l_time_start;
List<Object[]> list = repository.verifyOtp(msisdn,otp);
SimpleDateFormat sdf = new SimpleDateFormat("YYYY-MM-dd HH:mm:ss");
if(list!=null && list.size()>0) {
for(Object[] obj:list) {
String strDate = obj[3]==null?null:obj[3].toString();
Date dtDb = sdf.parse(strDate);
Date dtNow = new Date();
String strDtNow = sdf.format(dtNow);
dtNow = sdf.parse(strDtNow);
long ldtDb = dtDb.getTime();
long ldtNow = dtNow.getTime();
if(ldtDb>ldtNow) {
System.out.println("success within time");
int ii = repository.updateIsActive(msisdn);
response.setContentType("application/json");
response.sendRedirect("http://localhost:9393/crbt/api/subscriber/ivr/checkStatus/"+msisdn);
}
else {
System.out.println("failure time over!");
}
}
}
else {
}
}
second Api Response in postman
What I expected was first API's response. But its giving me some 415 content type error
How can I get first API's success json response from second api's response.. I even tried org.springframework.http.HttpHeaders but couldn't get desired output. What changes I had to do in order to get first Api's response in my second api response.
I have a strange feeling answering your questions, because I dislike the solution I'll provided. But it might help, so I'll give a try.
Basically, your Controller are just Spring beans, which means you can do is having a dependency, and second controller will call first controller. This will also change your method verifyOtp to make it change the return type.
Something like that:
...
#Autowired
private FirstController firstController;
...
#GetMapping(value="verifyOtp/{msisdn}/{otp}",consumes=MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<CoreResponseHandler> verifyOtp(#PathVariable(value="msisdn",required=true)String msisdn,
#PathVariable(value="otp",required=true)String otp,HttpServletResponse response) throws Exception{
long l_time_start = System.currentTimeMillis();
long l_time_end = System.currentTimeMillis();
long l_diff = l_time_end-l_time_start;
List<Object[]> list = repository.verifyOtp(msisdn,otp);
SimpleDateFormat sdf = new SimpleDateFormat("YYYY-MM-dd HH:mm:ss");
if(list!=null && list.size()>0) {
for(Object[] obj:list) {
String strDate = obj[3]==null?null:obj[3].toString();
Date dtDb = sdf.parse(strDate);
Date dtNow = new Date();
String strDtNow = sdf.format(dtNow);
dtNow = sdf.parse(strDtNow);
long ldtDb = dtDb.getTime();
long ldtNow = dtNow.getTime();
if(ldtDb>ldtNow) {
System.out.println("success within time");
int ii = repository.updateIsActive(msisdn);
return firstController.fetchOcsByDate2(msidn);
}
else {
System.out.println("failure time over!");
return null;
}
}
}
else {
return null;
}
}
I think you are trying to achieve something uncommon, and to avoid having this dependency between controller, consider:
Change your use case. Make the second controller returning a HttpStatus.OK, and make the client do the next call to the first controller
Create a service in charge of loading the msidn, which will avoid duplicate code, and keep you in a more standard position to make our evolutions.
The issue occurred due to GetMapping .
#GetMapping(value="checkStatus/{msisdn}",consumes=MediaType.APPLICATION_JSON_VALUE)
replace with below in first Api:
#GetMapping(value="checkStatus/{msisdn}")

Elasticsearch Search Single Letter Java Api - 1.7

Hello Elasticsearch Friends.
I have a Problem with my settings and mappings in Elasticsearch Java API. I configured my Index and set the mapping and settings. My indexname is "orange11", type "profile". I want that when I'm typing in my search inputField after just 2 or 3 letters Elasticsearch gives me some results. I've read something about analyzer, mapping and all this stuff and so I tried out.
This is my IndexService code:
#Service
public class IndexService {
private Node node;
private Client client;
#Autowired
public IndexService(Node node) throws Exception {
this.node = node;
client = this.node.client();
ImmutableSettings.Builder indexSettings = ImmutableSettings.settingsBuilder();
indexSettings.put("orange11.analysis.filter.autocomplete_filter.type", "edge_ngram");
indexSettings.put("orange11.analysis.filter.autocomplete_filter.min.gram", 1);
indexSettings.put("orange11.analysis.filter.autocomplete_filter.max_gram", 20);
indexSettings.put("orange11.analysis.analyzer.autocomplete.type", "custom");
indexSettings.put("orange11.analysis.analyzer.tokenizer", "standard");
indexSettings.put("orange11.analysis.analyzer.filter", new String[]{"lowercase", "autocomplete_filter"});
IndicesExistsResponse res = client.admin().indices().prepareExists("orange11").execute().actionGet();
if (res.isExists()) {
DeleteIndexRequestBuilder delIdx = client.admin().indices().prepareDelete("orange11");
delIdx.execute().actionGet();
}
CreateIndexRequestBuilder createIndexRequestBuilder = client.admin().indices().prepareCreate("orange11").setSettings(indexSettings);
// MAPPING GOES HERE
XContentBuilder mappingBuilder = jsonBuilder().startObject().startObject("profile").startObject("properties")
.startObject("name").field("type", "string").field("analyzer", "autocomplete").endObject()
.endObject()
.endObject();
System.out.println(mappingBuilder.string());
createIndexRequestBuilder.addMapping("profile ", mappingBuilder);
createIndexRequestBuilder.execute().actionGet();
List<Accounts> accountsList = transformJsonFileToJavaObject();
//Get Data from jsonMap() function into a ListMap.
//List<Map<String, Object>> dataFromJson = jsonToMap();
createIndex(accountsList);
}
public List<Accounts> transformJsonFileToJavaObject() throws IOException {
ObjectMapper mapper = new ObjectMapper();
List<Accounts> list = mapper.readValue(new File("/Users/lucaarchidiacono/IdeaProjects/moap2/MP3_MoapSampleBuild/data/index/testAccount.json"), TypeFactory.defaultInstance().constructCollectionType(List.class, Accounts.class));
return list;
}
public void createIndex(List<Accounts> accountsList) {
for (int i = 0; i < accountsList.size(); ++i) {
Map<String, Object> accountMap = new HashMap<String, Object>();
accountMap.put("id", accountsList.get(i).getId());
accountMap.put("isActive", accountsList.get(i).isActive());
accountMap.put("balance", accountsList.get(i).getBalance());
accountMap.put("age", accountsList.get(i).getAge());
accountMap.put("eyeColor", accountsList.get(i).getEyeColor());
accountMap.put("name", accountsList.get(i).getName());
accountMap.put("gender", accountsList.get(i).getGender());
accountMap.put("company", accountsList.get(i).getCompany());
accountMap.put("email", accountsList.get(i).getEmail());
accountMap.put("phone", accountsList.get(i).getPhone());
accountMap.put("address", accountsList.get(i).getAddress());
accountMap.put("about", accountsList.get(i).getAbout());
accountMap.put("greeting", accountsList.get(i).getGreeting());
accountMap.put("favoriteFruit", accountsList.get(i).getFavoriteFruit());
accountMap.put("url", accountsList.get(i).getUrl());
//Request an Index for indexObject. Set the index specification such as indexName, indexType and ID.
IndexRequestBuilder indexRequest = client.prepareIndex("orange11", "profile", Integer.toString(i)).setSource(accountMap);
//Execute the indexRequest and get the result in indexResponse.
IndexResponse indexResponse = indexRequest.execute().actionGet();
if (indexResponse != null && indexResponse.isCreated()) {
//Print out result of indexResponse
System.out.println("Index has been created !");
System.out.println("------------------------------");
System.out.println("Index name: " + indexResponse.getIndex());
System.out.println("Type name: " + indexResponse.getType());
System.out.println("ID: " + indexResponse.getId());
System.out.println("Version: " + indexResponse.getVersion());
System.out.println("------------------------------");
} else {
System.err.println("Index creation failed.");
}
}
}
}
Every time I wanna run this code I become this exception:
Caused by: org.elasticsearch.index.mapper.MapperParsingException: Root type mapping not empty after parsing! Remaining fields: [profile : {properties={name={analyzer=autocomplete, type=string}}}]
at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:278)
at org.elasticsearch.index.mapper.DocumentMapperParser.parseCompressed(DocumentMapperParser.java:192)
at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:449)
at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:307)
at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$2.execute(MetaDataCreateIndexService.java:391)
I don't know how to continue, because I don't see any '.' missing in my indexSettings. Sorry for my bad English.
Change min.gram to min_gram.
(See indexSettings.put("orange11.analysis.filter.autocomplete_filter.min.gram", 1);)

How to update a record after extracting

private List<String> getSCFData(int trdCustomerKy, Date lastRunDate, Date currentDate) throws TradeException {
List<String> reportData = null;
String paymentDate = EMPTY_STRING;
String partyId = EMPTY_STRING;
YOWDAO hdDAO = new YOWDAO(mConnection);
List<YOWSCFExtractData> reportItems = hdDAO.getSCFData(trdCustomerKy, lastRunDate, currentDate);
if (null != reportItems && reportItems.size() > 0) {
reportData = new ArrayList<String>();
mTracer.log("Total records retrieved: " + reportItems.size());
for (YOWSCFExtractData data : reportItems) {
String Source = (null != data.getSource()) ? data.getSource() : BLANK_STRING;
String paymentCurrencyCd = (null != data.getPaymentCurrencyCd()) ? data.getPaymentCurrencyCd()
: BLANK_STRING;
String sellerName = (null != data.getSellerName()) ? data.getSellerName() : BLANK_STRING;
String paymentAmount = (null != data.getPaymentAmount()) ? data.getPaymentAmount() : BLANK_STRING;
if (null != data.getPaymentDate()) {
paymentDate = DateUtil.formatDate(data.getPaymentDate());
}
if (null != data.getapplCifId()) {
partyId = hdDAO.getPartyId(mConfiguration.getCustomerKy(), data.getapplCifId());
}
String dataRow = StringUtils.join(new String[] { Source, data.getBankRef(), partyId, sellerName,
data.getPartyId(), paymentAmount, paymentDate, paymentCurrencyCd}, COMMA);
reportData.add(dataRow);
}
}
return reportData;
}
I am extracting the data from oracle database. I want to update the record of a column once it is fetched to a string. for example when I had extracted data.getBanref() then I want to set it some string back in database. how would I do that? I am using hibernate........
What you can do is set the object data whatever values you want and then save it in the hibernate. If you want to update then use session.saveOrUpdate() or if you want to save a new record then use session.save(). Hope that helps!
You can write a hibernate query
Update table_Name column_Name and set it to whatever you want and call this query in your program. It will be easier i think so

neo4J java query parameters

I have a problem to pass list of string to my parameter {code}
String request = "START sdg=node:Sfamilly(master = {code}) MATCH t-[CONTAINS_SF]->sdg RETURN count(distinct t) as count"
Map<String, Object> params = new HashMap<String, Object>();
List<String> codes = new ArrayList<String>();
codes.add("1234");
codes.add("12345");
params.put("master", codes);
Result<Map<String, Object>> resultMag = neo4jTemplate.query(request,params);
it appears that my parameters are not considered ?
Any idea ?
I use spring data neo4j rest 2.3.0.M1.
Thanks.
Charles.
First of all, I think you wanted to say
params.put("code", codes); // on line 7
More importantly, it seems to me that passing lists is only supported when querying nodes by ID.
Not sure this is the best possible solution, but it should work. It builds a Lucene query first from your parameters, then passes it into your Neo4j query.
private void yourMethod() {
String request = "START sdg=node:Sfamilly({luceneQuery}) MATCH t-[CONTAINS_SF]->sdg RETURN count(distinct t) as count";
Map<String, Object> params = new HashMap<String, Object>();
List<String> codes = new ArrayList<String>();
codes.add("1234");
codes.add("12345");
params.put("luceneQuery", listToParams("master", codes));
Result<Map<String, Object>> resultMag = neo4jTemplate.query(request, params);
}
private String listToParams(String paramName, List<String> params) {
if (params.isEmpty()) {
throw new IllegalArgumentException("Empty params");
}
Iterator<String> paramsIterator = params.iterator();
StringBuilder builder = new StringBuilder(paramName).append(":").append(paramsIterator.next());
while (paramsIterator.hasNext()) {
builder.append(" OR ").append(paramName).append(":").append(paramsIterator.next());
}
return builder.toString();
}

Getting URL parameter in java and extract a specific text from that URL

I have a URL and I need to get the value of v from this URL.
Here is my URL: http://www.youtube.com/watch?v=_RCIP6OrQrE
How can I do that?
I think the one of the easiest ways out would be to parse the string returned by URL.getQuery() as
public static Map<String, String> getQueryMap(String query) {
String[] params = query.split("&");
Map<String, String> map = new HashMap<String, String>();
for (String param : params) {
String name = param.split("=")[0];
String value = param.split("=")[1];
map.put(name, value);
}
return map;
}
You can use the map returned by this function to retrieve the value keying in the parameter name.
If you're on Android, you can do this:
Uri uri = Uri.parse(url);
String v = uri.getQueryParameter("v");
I have something like this:
import org.apache.http.NameValuePair;
import org.apache.http.client.utils.URIBuilder;
private String getParamValue(String link, String paramName) throws URISyntaxException {
List<NameValuePair> queryParams = new URIBuilder(link).getQueryParams();
return queryParams.stream()
.filter(param -> param.getName().equalsIgnoreCase(paramName))
.map(NameValuePair::getValue)
.findFirst()
.orElse("");
}
I wrote this last month for Joomla Module when implementing youtube videos (with the Gdata API). I've since converted it to java.
Import These Libraries
import java.net.URL;
import java.util.regex.*;
Copy/Paste this function
public String getVideoId( String videoId ) throws Exception {
String pattern = "^(https?|ftp|file)://[-a-zA-Z0-9+&##/%?=~_|!:,.;]*[-a-zA-Z0-9+&##/%=~_|]";
Pattern p = Pattern.compile(pattern);
Matcher m = p.matcher(videoId);
int youtu = videoId.indexOf("youtu");
if(m.matches() && youtu != -1){
int ytu = videoId.indexOf("http://youtu.be/");
if(ytu != -1) {
String[] split = videoId.split(".be/");
return split[1];
}
URL youtube = new URL(videoId);
String[] split = youtube.getQuery().split("=");
int query = split[1].indexOf("&");
if(query != -1){
String[] nSplit = split[1].split("&");
return nSplit[0];
} else return split[1];
}
return null; //throw something or return what you want
}
URL's it will work with
http://www.youtube.com/watch?v=k0BWlvnBmIE (General URL)
http://youtu.be/k0BWlvnBmIE (Share URL)
http://www.youtube.com/watch?v=UWb5Qc-fBvk&list=FLzH5IF4Lwgv-DM3CupM3Zog&index=2 (Playlist URL)
Import these libraries
import org.apache.http.NameValuePair;
import org.apache.http.message.BasicNameValuePair;
Similar to the verisimilitude, but with the capabilities of handling multivalue parameters. Note: I've seen HTTP GET requests without a value, in this case the value will be null.
public static List<NameValuePair> getQueryMap(String query)
{
List<NameValuePair> queryMap = new ArrayList<NameValuePair>();
String[] params = query.split(Pattern.quote("&"));
for (String param : params)
{
String[] chunks = param.split(Pattern.quote("="));
String name = chunks[0], value = null;
if(chunks.length > 1) {
value = chunks[1];
}
queryMap.add(new BasicNameValuePair(name, value));
}
return queryMap;
}
Example:
GET /bottom.gif?e235c08=1509896923&%49%6E%...
Using pure Java 8
Assumming you want to extract param "v" from url:
String paramV = Stream.of(url.split("?")[1].split("&"))
.map(kv -> kv.split("="))
.filter(kv -> "v".equalsIgnoreCase(kv[0]))
.map(kv -> kv[1])
.findFirst()
.orElse("");
Assuming the URL syntax will always be http://www.youtube.com/watch?v= ...
String v = "http://www.youtube.com/watch?v=_RCIP6OrQrE".substring(31);
or disregarding the prefix syntax:
String url = "http://www.youtube.com/watch?v=_RCIP6OrQrE";
String v = url.substring(url.indexOf("v=") + 2);
I believe we have a better approach to answer this question.
1: Define a function that returns Map values.
Here we go.
public Map<String, String> getUrlValues(String url) throws UnsupportedEncodingException {
int i = url.indexOf("?");
Map<String, String> paramsMap = new HashMap<>();
if (i > -1) {
String searchURL = url.substring(url.indexOf("?") + 1);
String params[] = searchURL.split("&");
for (String param : params) {
String temp[] = param.split("=");
paramsMap.put(temp[0], java.net.URLDecoder.decode(temp[1], "UTF-8"));
}
}
return paramsMap;
}
2: Call your function surrounding with a try catch block
Here we go
try {
Map<String, String> values = getUrlValues("https://example.com/index.php?form_id=9&page=1&view_id=78");
String formId = values.get("form_id");
String page = values.get("page");
String viewId = values.get("view_id");
Log.d("FormID", formId);
Log.d("Page", page);
Log.d("ViewID", viewId);
} catch (UnsupportedEncodingException e) {
Log.e("Error", e.getMessage());
}
If you are using Jersey (which I was, my server component needs to make outbound HTTP requests) it contains the following public method:
var multiValueMap = UriComponent.decodeQuery(uri, true);
It is part of org.glassfish.jersey.uri.UriComponent, and the javadoc is here. Whilst you may not want all of Jersey, it is part of the Jersey common package which isn't too bad on dependencies...
I solved the problem like this
public static String getUrlParameterValue(String url, String paramName) {
String value = "";
List<NameValuePair> result = null;
try {
result = URLEncodedUtils.parse(new URI(url), UTF_8);
value = result.stream().filter(pair -> pair.getName().equals(paramName)).findFirst().get().getValue();
System.out.println("--------------> \n" + paramName + " : " + value + "\n");
} catch (URISyntaxException e) {
e.printStackTrace();
}
return value;
}
this will work for all sort of youtube url :
if url could be
youtube.com/?v=_RCIP6OrQrE
youtube.com/v/_RCIP6OrQrE
youtube.com/watch?v=_RCIP6OrQrE
youtube.com/watch?v=_RCIP6OrQrE&feature=whatever&this=that
Pattern p = Pattern.compile("http.*\\?v=([a-zA-Z0-9_\\-]+)(?:&.)*");
String url = "http://www.youtube.com/watch?v=_RCIP6OrQrE";
Matcher m = p.matcher(url.trim()); //trim to remove leading and trailing space if any
if (m.matches()) {
url = m.group(1);
}
System.out.println(url);
this will extract video id from your url
further reference
My solution mayble not good
String url = "https://www.youtube.com/watch?param=test&v=XcHJMiSy_1c&lis=test";
int start = url.indexOf("v=")+2;
// int start = url.indexOf("list=")+5; **5 is length of ("list=")**
int end = url.indexOf("&", start);
end = (end == -1 ? url.length() : end);
System.out.println(url.substring(start, end));
// result: XcHJMiSy_1c
work fine with:
https://www.youtube.com/watch?param=test&v=XcHJMiSy_1c&lis=test
https://www.youtube.com/watch?v=XcHJMiSy_1c
public static String getQueryMap(String query) {
String[] params = query.split("&");
for (String param : params) {
String name = param.split("=")[0];
if ("YourParam".equals(name)) {
return param.split("=")[1];
}
}
return null;
}

Categories