We Keep Coding

Should I map all my indexes with my analyzer on ElasticSearch

I'm "almost" new on Elastic Search. I've been using it for a while but never used Analyzers before.
I can make a full text search on my project but the problem is, when I try to find a name like "Alex", I should completely type down the name correcly. It doesn't work with "Al" or "Ale". It says something like "no match found".
I found some source codes from different sites, but it makes me confused.
What should I do is:
1) Creating a nGram tokenizer
2) Then mapping it with my all indexes?
I have lots of indexes already created and I got errors while creating a mapping on them.
Should I create my analyzer settings and mapping very in the beggining just before indexing my records ?
I'm working on a Java project, so answers on JAVA API will be very appreciated.
Thanks a lot!

mappings should always be created first and then the data should be indexed. if possible, delete your old indices and recreate with new mapping. if you are concerned about loosing your data, then just create a new type for an existing index. the new type can use the new mapping.
for example, here is a piece that uses the Java API to create a custom mapping
public class MappingCreator {
static Logger log = Logger.getLogger(MappingCreator.class.getName());
final static String indexName = "indexName";
final static String typeName = "typeName";
final static String mappingFileName = "pathToMapping.jsonFile";
final static String clusterName = "elasticsearch"; // or name of your cluster
final static String hostName = "localhost";
public static void main(String args[]) throws IOException
{
MappingCreator mapCreator = new MappingCreator();
Client myESclient = getClient();
IndicesExistsResponse res = myESclient.admin().indices().prepareExists(indexName).execute().actionGet();
if (res.isExists()) {
log.warn("Index "+indexName +" already exists. Will be deleted");
final DeleteIndexRequestBuilder deleteIndexBuilder = myESclient.admin().indices().prepareDelete(indexName);
deleteIndexBuilder.execute().actionGet();
}
final CreateIndexRequestBuilder createIndexBuilder = myESclient.admin().indices().prepareCreate(indexName)
.addMapping(typeName, mapCreator.getIndexFieldMapping());
CreateIndexResponse createIndexResponse = createIndexBuilder.execute().actionGet();
log.debug("Created mapping "+createIndexResponse.toString());
myESclient.close();
}
private String getIndexFieldMapping() throws IOException {
return IOUtils.toString(getClass().getClassLoader().getResourceAsStream(mappingFileName));
}
private static Client getClient() {
TransportClient transportClient = null;
try
{
Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name", clusterName).build();
transportClient = new TransportClient(settings);
transportClient = transportClient.addTransportAddress(new InetSocketTransportAddress(hostName, 9300));
/* be very careful about the port number here. by default its 9300. note that this is the TCP port which the java api will use. unlike the http port which is 9200 */
}
catch (Exception e)
{
log.error("Error in MappingCreator creating Elastic Search Client\n"
+ "Message "+e.getMessage()+"\n"
+ "StackTrace "+e.getStackTrace()
);
}
return (Client) transportClient;
}
}
i hope this helps. by the way its really cool that you are making your own nGram tokenizer. i would love to see the code for that and how it is done :)

How to perform Amazon Cloud Search with .net code?

I am learning Amazon Cloud Search but I couldn't find any code in either C# or Java (though I am creating in C# but if I can get code in Java then I can try converting in C#).
This is just 1 code I found in C#: https://github.com/Sitefinity-SDK/amazon-cloud-search-sample/tree/master/SitefinityWebApp.
This is 1 method i found in this code:
public IResultSet Search(ISearchQuery query)
{
AmazonCloudSearchDomainConfig config = new AmazonCloudSearchDomainConfig();
config.ServiceURL = "http://search-index2-cdduimbipgk3rpnfgny6posyzy.eu-west-1.cloudsearch.amazonaws.com/";
AmazonCloudSearchDomainClient domainClient = new AmazonCloudSearchDomainClient("AKIAJ6MPIX37TLIXW7HQ", "DnrFrw9ZEr7g4Svh0rh6z+s3PxMaypl607eEUehQ", config);
SearchRequest searchRequest = new SearchRequest();
List<string> suggestions = new List<string>();
StringBuilder highlights = new StringBuilder();
highlights.Append("{\'");
if (query == null)
throw new ArgumentNullException("query");
foreach (var field in query.HighlightedFields)
{
if (highlights.Length > 2)
{
highlights.Append(", \'");
}
highlights.Append(field.ToUpperInvariant());
highlights.Append("\':{} ");
SuggestRequest suggestRequest = new SuggestRequest();
Suggester suggester = new Suggester();
suggester.SuggesterName = field.ToUpperInvariant() + "_suggester";
suggestRequest.Suggester = suggester.SuggesterName;
suggestRequest.Size = query.Take;
suggestRequest.Query = query.Text;
SuggestResponse suggestion = domainClient.Suggest(suggestRequest);
foreach (var suggest in suggestion.Suggest.Suggestions)
{
suggestions.Add(suggest.Suggestion);
}
}
highlights.Append("}");
if (query.Filter != null)
{
searchRequest.FilterQuery = this.BuildQueryFilter(query.Filter);
}
if (query.OrderBy != null)
{
searchRequest.Sort = string.Join(",", query.OrderBy);
}
if (query.Take > 0)
{
searchRequest.Size = query.Take;
}
if (query.Skip > 0)
{
searchRequest.Start = query.Skip;
}
searchRequest.Highlight = highlights.ToString();
searchRequest.Query = query.Text;
searchRequest.QueryParser = QueryParser.Simple;
var result = domainClient.Search(searchRequest).SearchResult;
//var result = domainClient.Search(searchRequest).SearchResult;
return new AmazonResultSet(result, suggestions);
}
I have already created domain in Amazon Cloud Search using AWS console and uploaded document using Amazon predefine configuration option that is movie Imdb json file provided by Amazon for demo.
But in this method I am not getting how to use this method, like if I want to search Director name then how do I pass in this method as because this method parameter is of type ISearchQuery?

I'd suggest using the official AWS CloudSearch .NET SDK. The library you were looking at seems fine (although I haven't look at it any detail) but the official version is more likely to expose new CloudSearch features as soon as they're released, will be supported if you need to talk to AWS support, etc, etc.
Specifically, take a look at the SearchRequest class -- all its params are strings so I think that obviates your question about ISearchQuery.
I wasn't able to find an example of a query in .NET but this shows someone uploading docs using the AWS .NET SDK. It's essentially the same procedure as querying: creating and configuring a Request object and passing it to the client.
EDIT:
Since you're still having a hard time, here's an example. Bear in mind that I am unfamiliar with C# and have not attempted to run or even compile this but I think it should at least be close to working. It's based off looking at the docs at http://docs.aws.amazon.com/sdkfornet/v3/apidocs/
// Configure the Client that you'll use to make search requests
string queryUrl = #"http://search-<domainname>-xxxxxxxxxxxxxxxxxxxxxxxxxx.us-east-1.cloudsearch.amazonaws.com";
AmazonCloudSearchDomainClient searchClient = new AmazonCloudSearchDomainClient(queryUrl);
// Configure a search request with your query
SearchRequest searchRequest = new SearchRequest();
searchRequest.Query = "potato";
// TODO Set your other params like parser, suggester, etc
// Submit your request via the client and get back a response containing search results
SearchResponse searchResponse = searchClient.Search(searchRequest);

Call Yellowfin Business Intelligence services with C#

I'm trying to use Yellowfin services from my C# code. They are written on Java, so I've enabed JAX services as they recommend.
So JAX services are running at "localhost:8083/webservices/LegacyReportService?wsdl", and I cannot make them work as specified (I'm running RUNDASHBOARDREPORT method of the ReportService)
That's how I use it:
Web References to the YF services running on my localhost:
Here I call the service
public static reportServiceResponse RunDashboardReport(Int32 reportId, Int32 tabId)
{
var rq = CreateYfRequest("RUNDASHBOARDREPORT");
rq.reportId = reportId;
rq.dashboardTabId = tabId;
using (var srv = new LegacyReportServiceService())
{
var resp = srv.remoteReportCall(rq); // there is no "remoteAdministrationCall" as in the doc
return resp;
}
}
private static reportServiceRequest CreateYfRequest(String command)
{
var rq = new reportServiceRequest
{
loginId = "admin#yellowfin.com.au",
password = "test",
orgId = 1, // This is the primary organization
reportRequest = command
};
return rq;
}
And I get "An exception of type 'System.InvalidOperationException' occurred in System.Xml.dll "
when creating
new LegacyReportServiceService()
I've also tried to add it as a Service Reference, but the result is the same.
The YF team sais "We do have clients that are using .net and C# to develop their integration code. ...The support team has confirmed example code provided in development folder in the Yellowfin directory and WSDL code are accurate and are unable to replicate the errors you’ve identified in your original email."
Please help me to find out, what I'm doing wrong

I've found out that VS generates classes for the web service access from "localhost:8083/webservices/LegacyReportService?xsd=1" and it does that improperly. It makes String[] from original String type.
So editing the generated Reference.cs of the Web Reference did the thing.At least, I've got the response with errorCode 25 "Not authorized".

Try the following code to get rid of response code 25.
rsr.orgId = 1;
rsr.orgIdSpecified = true;
rsr.dashboardTabId = 11111;
rsr.dashboardTabIdSpecified = true;
rsr.reportId = 22222;
rsr.reportIdSpecified = true;
Certain parameters for yellowfin need to explicitly told to read.

Check if this works
reportServiceRequest rq = new reportServiceRequest();
rq.loginId = "admin#xxxxx.com";
rq.password = "xxxxxxx";
rq.orgId = 1;
rq.reportRequest = "RUNDASHBOARDREPORT";
rq.reportId = reportId;
rq.orgIdSpecified = true;
rq.dashboardTabId = 11111;
rq.dashboardTabIdSpecified = true;
rq.reportIdSpecified = true;
rq.dashboardTabId = tabId;
using (var srv = new ServiceReference2.LegacyReportServiceClient())
{
YellowFinIntegrationWithDotNet.ServiceReference2.reportServiceResponse resp = null;
try
{
resp = srv.remoteReportCall(rq); // there is no "remoteAdministrationCall" as in the doc
}
catch (System.Exception ex)
{
MessageBox.Show(ex.InnerException.ToString());
return resp;
}
return resp;
}

What is the error in the following HL7 encoding?

I am trying to encode an HL7 message of the type ORU_R01 using the HAPI 2.0 library for an OpenMRS module. I have followed the tutorials given in the HAPI documentation and according to that, I have populated the required fields of the ORU_R01 message. Now, I want to post this message using the following link:
http://localhost:8080/openmrs/remotecommunication/postHl7.form
I am using the following message for testing:
MSH|^~\&|||||20140713154042||ORU^R01|20140713154042|P|2.5|1
PID|||1
OBR|1||1234^SensorReading|88304
OBX|0|NM|1||45
OBX|1|NM|2||34
OBX|2|NM|3||23
I have properly ensured that all the parameters are correct. Once I have posted the HL7 message, I start the HL7 task from the scheduler. Then I go to the admin page and click on "Manage HL7 errors" in order to see if the message arrives there. I get the following stack trace:
ca.uhn.hl7v2.HL7Exception: HL7 encoding not supported
...
Caused by: ca.uhn.hl7v2.parser.EncodingNotSupportedException: Can't parse message beginning MSH|^~\
at ca.uhn.hl7v2.parser.Parser.parse(Parser.java:140)
The full stack trace is here: http://pastebin.com/ZnbFqfWC.
I have written the following code to encode the HL7 message (using the HAPI library):
public String createHL7Message(int p_id, int concept_id[], String val[])
throws HL7Exception {
ORU_R01 message = new ORU_R01();
SimpleDateFormat sdf = new SimpleDateFormat("yyyyMMddHHmmss",
Locale.ENGLISH);
MSH msh = message.getMSH();
msh.getFieldSeparator().setValue("|");
msh.getEncodingCharacters().setValue("^~\\&");
msh.getProcessingID().getProcessingID().setValue("P");
msh.getSequenceNumber().setValue("1");
msh.getMessageType().getTriggerEvent().setValue("R01");
msh.getMessageType().getMessageCode().setValue("ORU");
msh.getVersionID().getVersionID().setValue("2.5");
msh.getMessageControlID().setValue(
sdf.format(Calendar.getInstance().getTime()));
msh.getDateTimeOfMessage().getTime()
.setValue(sdf.format(Calendar.getInstance().getTime()));
ORU_R01_ORDER_OBSERVATION orderObservation = message
.getPATIENT_RESULT().getORDER_OBSERVATION();
ca.uhn.hl7v2.model.v25.segment.PID pid = message.getPATIENT_RESULT()
.getPATIENT().getPID();
Patient patient = (Patient) Context.getPatientService()
.getPatient(p_id);
System.out.println(String.valueOf(p_id) + " " + patient.getGivenName()
+ " " + patient.getFamilyName());
pid.getPatientName(0).getFamilyName().getSurname()
.setValue(patient.getFamilyName());
pid.getPatientName(0).getGivenName().setValue(patient.getGivenName());
pid.getPatientIdentifierList(0).getIDNumber()
.setValue(String.valueOf(p_id));
System.out.println();
// Parser parser = new PipeParser();
// String encodedMessage = null;
// encodedMessage = parser.encode(message);
// System.out.println(encodedMessage);
// Populate the OBR
OBR obr = orderObservation.getOBR();
obr.getSetIDOBR().setValue("1");
obr.getFillerOrderNumber().getEntityIdentifier().setValue("1234");
obr.getFillerOrderNumber().getNamespaceID().setValue("SensorReading");
obr.getUniversalServiceIdentifier().getIdentifier().setValue("88304");
Varies value = null;
// Varies value[] = new Varies[4];
for (int i = 0; i < concept_id.length; i++) {
ORU_R01_OBSERVATION observation = orderObservation
.getOBSERVATION(i);
OBX obx2 = observation.getOBX();
obx2.getSetIDOBX().setValue(String.valueOf(i));
obx2.getObservationIdentifier().getIdentifier()
.setValue(String.valueOf(concept_id[i]));
obx2.getValueType().setValue("NM");
NM nm = new NM(message);
nm.setValue(val[i]);
value = obx2.getObservationValue(0);
value.setData(nm);
}
Parser parser = new PipeParser();
String encodedMessage = null;
encodedMessage = parser.encode(message);
return encodedMessage;
}
In all likelihood, something is wrong with the MSH segment of the message, but I cannot seem to figure out what it is. What can I do to correct this error?

Why do you declare the Encoding Characters using double backslashes?
msh.getEncodingCharacters().setValue("^~\\&");
Shouldn't it be:
msh.getEncodingCharacters().setValue("^~\&");
...and because your message is using the default encoding characters maybe you don't even need to declare them at all? Extract from HAPI MSH Class reference
getENCODINGCHARACTERS
public ST getENCODINGCHARACTERS()
Returns MSH-2: "ENCODING CHARACTERS" - creates it if necessary
Update
I have no previous experience with HAPI. A quick google found an ORU example. Could you try initializing your MSH with initQuickstart("ORU", "R01", "P");
According to the comments in the example-code the initQuickstart method populates all of the mandatory fields in the MSH segment of the message, including the message type, the timestamp, and the control ID. (...and hopefully the default encoding chars as well :-)

Json From .Net Server to android client

I'm trying to call a ApiController from android apllication.
This is the api controller:
[AcceptVerbs("GET", "POST")]
public string Get(string coords)
{
using (var context = new Entities())
{
var records = from poi in context.Pois
where poi.Latitude >= fromLatitude &&
poi.Latitude <= toLatitude &&
poi.Longitude >= fromLongitude &&
poi.Longitude <= toLongitude
select new
{
poiName = poi.Name,
poiLatitude = poi.Latitude,
poiLongitude = poi.Longitude
};
return JsonConvert(records);
}
}
}
private string JsonConvert(object records)
{
return Newtonsoft.Json.JsonConvert.SerializeObject(records,);
}
At the android code, I'm creating json array with new JSON(string).
The problem is java throws an excetpion: the json string is not valid.
When i look at the debuuger, I see that the string have 2 backslash before ",
and java dont know how to parse that.
Where is the problem?
Thank you
Update: Solved. The WebApi returned XML with the json as string. changed the WebApi Not to return XML, then changed it to return object (and removed the JSONConvert) - and it works.

I know this is an old question, but i had a similar problem and found a solution.
In my case i had to pass a complex JSON object (nested) from a .NET Client to a Java Rest API and was using a string parameter which turned out to be an invalid JSON due to the double backslash (I seralized it so it was escaped and then .NET escaped it again before sending).
So, in order to avoid that i used StringContent
MyType obj = new MyType()
{
...
};
string obJSON = JsonConvert.SerializeObject(obj);
StringContent sc = new StringContent(obJSON, Encoding.UTF8,"application/json");
HttpResponseMessage response = client.PostAsync(ruta, sc).Result;
Hope this helps someone!