We Keep Coding

RestAssured with duplicate query parameters

Working on an automated test pack that uses REST-assured rest-assured-3.3.0. Have an endpoint to test that has a number of duplicate Query parameters used to drive it's search function:
/count?taskTypes=EAS&taskTypes=MBO&taskTypes=OTHER&taskAges=NEW&taskAges=EXISTING&ageSelection=ALL
REST-assured provides a params(Map<String, ?> var1) in it's request specification to pass in parameters. However this uses Map which can not contain duplicate keys. Therefore in order to construct the query in test code I am setting up the parameters and associated values in a Multimap:
import com.google.common.collect.Multimap;
Multimap<String, String> searchParams = ArrayListMultimap.create();
searchParams.put("taskTypes", "EAS");
searchParams.put("taskTypes", "MBO");
searchParams.put("taskTypes", "OTHER");
searchParams.put("taskAges", "NEW");
searchParams.put("taskAges", "EXISTING");
searchParams.put("ageSelection", "ALL");
The Multimap is then passed into a function to format the query and set up a GET request using REST-assured:
import org.apache.http.client.utils.URIBuilder;
import io.restassured.http.Method;
import io.restassured.response.ExtractableResponse;
import io.restassured.specification.RequestSpecification;
protected ExtractableResponse request(Method method, Multimap<String, ?> params, String url)
{
URIBuilder builder = new URIBuilder();
for (Map.Entry<String, ?> entry : params.entries()) {
builder.addParameter(entry.getKey(), entry.getValue().toString());
}
try {
url = url + builder.build().toString();
} catch(URISyntaxException e) {}
return getRequestSpec()
.when()
.headers("Content-Type", "application/json")
.request(method, url)
.then()
.spec(getResponseSpec(method))
.extract();
}
Running this code results in a request with the following parameters:
/count?taskTypes=OTHER&taskAges=EXISTING&ageSelection=ALL
The problem is that REST-assured appears to remove the duplicate query parameters passed to it in the url parameter. Interestingly REST-assured offers the following interface given().queryParam("param", 1,2,3)

Check that for loop here:
URIBuilder builder = new URIBuilder();
for (Map.Entry<String, ?> entry : params.entries()) {
builder.addParameter(entry.getKey(), entry.getValue().toString());
}
looks like despite using Multimap for params you are still ending with Map for the builder. Here is where you got rid of duplicated keys anyway.

queryParam(String var1, Collection<?> var2)
should work. I have used it in the past for rest-assured-3.*.
Example:
List<String> multiQuery = new ArrayList<>();
multiquery.add("EAS");
multiquery.add("MBO");
queryParam("taskTypes", multiQuery);

Using M2Doc programmatically : Error in the generated .docx document

I'm trying to use M2Doc programmatically, I managed to generate my .docx file without getting errors in the validation part but I'm getting the following Error in the generated document:
{m:self.Name} Couldn't find the 'aqlFeatureAccess(org.eclipse.emf.common.util.URI.Hierarchical,java.lang.String)' service
The "self.Name" part is what I wrote in my template.
I think I'm lacking some kind of reference to a service but I don't know how to fix it.
The self variable is a reference to a model based on a meta-model I created. But I'm not sure I imported it correctly in my code.
I based my code on the code I found on the M2Doc website + some code I found on their GitHub, especially concerning how to add a service in the queryEnvironment.
I searched in the source code of acceleo and M2Doc to see which services they add but it seems that they already import all the services I'm using.
As I said, the validation part is going well and doesn't generate a validation file.
public static void parseDocument(String templateName) throws Exception{
final URI templateURI = URI.createFileURI("Template/"+templateName+"."+M2DocUtils.DOCX_EXTENSION_FILE);
final IQueryEnvironment queryEnvironment =
org.eclipse.acceleo.query.runtime.Query.newEnvironmentWithDefaultServices(null);
final Map<String, String> options = new HashMap<>(); // can be empty
M2DocUtils.prepareEnvironmentServices(queryEnvironment, templateURI, options); // delegate to IServicesConfigurator
prepareEnvironmentServicesCustom(queryEnvironment, options);
final IClassProvider classProvider = new ClassProvider(ClassLoader.getSystemClassLoader()); // use M2DocPlugin.getClassProvider() when running inside Eclipse
try (DocumentTemplate template = M2DocUtils.parse(templateURI, queryEnvironment, classProvider)) {
ValidationMessageLevel validationLevel = validateDocument(template, queryEnvironment, templateName);
if(validationLevel == ValidationMessageLevel.OK){
generateDocument(template, queryEnvironment, templateName, "Model/ComplexKaosModel.kaos");
}
}
}
public static void prepareEnvironmentServicesCustom(IQueryEnvironment queryEnvironment, Map<String, String> options){
Set<IService> services = ServiceUtils.getServices(queryEnvironment, FilterService.class);
ServiceUtils.registerServices(queryEnvironment, services);
M2DocUtils.getConfigurators().forEach((configurator) -> {
ServiceUtils.registerServices(queryEnvironment, configurator.getServices(queryEnvironment, options));
});
}
public static void generateDocument(DocumentTemplate template, IQueryEnvironment queryEnvironment,
String templateName, String modelPath)throws Exception{
final Map<String, Object> variable = new HashMap<>();
variable.put("self", URI.createFileURI(modelPath));
final Monitor monitor = new BasicMonitor.Printing(System.out);
final URI outputURI = URI.createFileURI("Generated/"+templateName+".generated."+M2DocUtils.DOCX_EXTENSION_FILE);
M2DocUtils.generate(template, queryEnvironment, variable, outputURI, monitor);
}

The variable "self" contains an URI:
variable.put("self", URI.createFileURI(modelPath));
You have to load your model and set the value of self to an element from your model using something like:
final ResourceSet rs = new ResourceSetImpl();
final Resource r = rs.getResource(uri, true);
final EObject value = r.getContents()...;
variable.put("self", value);
You can get more details on resource loading in the EMF documentation.

How to use the PMML model in Java?

i have build a python file based on randomforestclassifier and made a PMML model, now we need to use this PMML in Java to classify the data into 2 catagories..but this is new to me and i don't know how to handle the java part..

Google: pmml java
Second Link is https://github.com/jpmml/jpmml-evaluator
There you have a library with examples. So try it and come back if you have problems.

Another choice is PMML4S that is implemented in Scala, but you are free to use it by Scala or Java API. It's very easy to use, for example:
import org.pmml4s.model.Model;
Model model = Model.fromFile("/the/pmml/file/path");
Object result = model.predict(data)
The data could be in Map, Array, A String in JSON, Series, the result type is same as the input. For details how to use PMML4S in Java, you could see the example: https://github.com/autodeployai/pmml4s/blob/master/src/test/java/org/pmml4s/model/JModelTest.java

There is a demo base on high performance and light framework Vert.x:
vertx-pmml
Just config router via Json like this:
"route": {
"/predict/iris": {
"access": "public",
"method": "POST",
"actor": "evaluter.predict",
"ext": {
"name": "IRIS_SVC",
"pmml": ".\\config\\model_svc.pmml",
"version": "1.0.0"
}
}
...
Then prefect your feature handler(not nescessary if feed a json map match your pmml inputfields):
//src/main/java/com/hirisklab/evaluate/evaluator/actor/EvaluateImpl.java
private Future<Map<String, Object>> Featurelize(JsonObject data) {
Promise<Map<String, Object>> promise = Promise.promise();
try {
Map<String, Object> feature = data.getMap();
// TODO: featurelize with pmml input fields...
promise.complete(feature);
} catch (Exception e) {
promise.fail(e);
}
return promise.future();
}
The action assigned in router(actor field) will handle at here:
//src/main/java/com/hirisklab/evaluate/evaluator/actor/EvaluateImpl.java
public void predict(JsonObject data, Handler<AsyncResult<EvaluateResponse<JsonObject>>> handler) {
Promise<EvaluateResponse<JsonObject>> promise = Promise.promise();
try {
JsonObject profile = Optional.ofNullable(data.getJsonObject("_EXT")).orElseThrow(() -> EvaluateException.InvalidConfigException);
EvaluaterFactory.getEvaluater(profile).onSuccess(evaluater -> {
Featurelize(data.getJsonObject("data")).onSuccess(feature -> {
evaluater.predict(feature).onSuccess(result -> {
promise.complete(new EvaluateResponse<JsonObject>(new JsonObject().put("raw", result)));
}).onFailure(f -> promise.fail(EvaluateException.FailedToPredict));
}).onFailure(f -> promise.fail(EvaluateException.FailedToFeaturelize));
}).onFailure(f -> promise.fail(EvaluateException.FailedToLoadEvaluator));
} catch (Exception e) {
e.printStackTrace();
promise.fail(e);
}
handler.handle(promise.future());
}

Here is the simple code that might help you to get the direction:
// This will load the PMML file
Evaluator evaluator = new LoadingModelEvaluatorBuilder()
.load(new File("path\\file.pmml"))
.build();
// The internal self check
evaluator.verify();
System.out.println("PMML Loaded");
// This will create the `actual` pipeline from the PMML and load in java
TransformerBuilder pmmlTransformerBuilder = new TransformerBuilder(evaluator)
.withTargetCols()
.withOutputCols()
.exploded(true);
System.out.println("Building in...");
Transformer pmmlTransformer = pmmlTransformerBuilder.build();
// Now we are loading the file and converting that into the data frame so that
// we can use it to transform into the prediction into the PMML model
Dataset<Row> DF = sparkSession.read()
.option("header", true)
.option("inferSchema", true)
.csv("path\\file.csv");
DF.printSchema();
// This will predict the new data points from the pipeline
Dataset<Row> result = pmmlTransformer.transform(DF);
NOTE: Make sure, you are loading the correct csv file there. There should not be any change in the column headers otherwise it will show an error.
Here is the link: JPMML-Evaluator (the same which is provided by Florian) to explore more about this.

How to use termVector in solrJ

Solr/SolrJ Version: 6.0.0
I've set termvector component in solrconfig.xml, and the request handler is "/tvrh", I test it in the browser and this works. Now I want to use it in solrJ, but it only returns the document. The following is my code:
SolrClient solrClient = new HttpSolrClient("http://localhost:8983/solr/test");
SolrQuery solrQuery = new SolrQuery();
solrQuery.setQuery(String.format("id:%s","clueweb12-0000tw-06-17744"));
solrQuery.setRequestHandler("/tvrh");
solrQuery.set("tv", true);
solrQuery.set("tv.all", true);
QueryResponse response = solrClient.query(solrQuery);
SolrDocumentList docs = response.getResults();
for (SolrDocument doc: docs){
for (String key: doc.keySet()){
System.out.println(key);
System.out.println(doc.getFieldValue(key));
}
}

Your question is how to use a non standard request handler in solr.
Be aware that the Term Vector Component belongs to a "non standard" request handler and is not supported from solrj:
https://cwiki.apache.org/confluence/display/solr/The+Term+Vector+Component#TheTermVectorComponent-SolrJandtheTermVectorComponent
You can call "/tvrh" via solrj in a generic mode. You can not use the method SolrClient#query(SolrParams params) for this, because in this case the "request handler" is only send as "qt"-Parameter and will not be part of the url-path (and qt-Parameter is ignored by default).
So please try the method "SolrClient#request" instead.

As #Karsten R says, we could not use SolrClient.query to send request. After I searched a lot and experimented a lot, the following code could work.
SolrClient solrClient = new HttpSolrClient("http://localhost:8983/solr/trecB13");
SolrQuery solrQuery = new SolrQuery();
solrQuery.setQuery(String.format("id:%s","clueweb12-0000tw-06-17744"));
solrQuery.setRequestHandler("/tvrh");
solrQuery.set("tv", true);
solrQuery.set("tv.all", true);
try {
NamedList<Object> response = solrClient.request(new QueryRequest(solrQuery));
TermVectorExtractor extractor = new TermVectorExtractor(response);
System.out.println(extractor.getTermVectorInfo());
} catch (Exception e) {
e.printStackTrace();
}
TermVectorExtractor.java reference Sakai-Solr Github code, the function of the class is to parse resonse object and get term info. A little different from original code. The different has been shown below.
import org.apache.solr.common.util.NamedList;
import java.util.*;
public class TermVectorExtractor {
private static final String TERM_VECTORS = "termVectors";
private Map<String, Map<String, Map<String, TermInfo>>> termVectorInfo = Collections.emptyMap();
/**
* Creates a TermVectorExtractor for the given query response sent by Solr.
*
* #param queryResponse response sent by the solr server for a search query.
*/
#SuppressWarnings("unchecked")
public TermVectorExtractor(NamedList<Object> queryResponse) {
NamedList<Object> res = (NamedList<Object>) queryResponse.get(TERM_VECTORS);
if (res != null)
termVectorInfo = extractTermVectorInfo(res);
}
}

How to create query string with a map in it? (Java)

I need to call this service in Java -
https://api.semantics3.com/test/v1/products?q={"cat_id": "13658", "brand": "Toshiba", "model": "Satellite"}
I've managed to do this in python as follows -
class Semantics:
def __init__(self):
self.service_address = 'https://api.semantics3.com/test/v1/products?'
self.api_key = 'SEM3158A71D4AB3A3715C2230B96943F46D0'
def query(self, params):
query = 'q=' + params.__dict__.__str__().replace("'",'"')
query = urlencode(urlparse.parse_qs(query), True)
req = Request(url = self.service_address + query)
req.add_header('api_key', self.api_key)
return urlopen(req).read()
class QueryParams:
def __init__(self, cat_id, model, brand):
self.cat_id = cat_id
self.model = model
self.brand = brand
qp = QueryParams('13658', 'Satellite', "Toshiba")
print Semantics().query(qp)
I have tried writing an equivalent Java program using Spring REST API and Apache HttpClient to no avail. I can't find a way to set a dictionary (i.e. Map) into the query String.
public static void main(String[] args) {
String uri = "https://api.semantics3.com/test/v1/products?";
HttpClient hc = new HttpClient();
GetMethod method = new GetMethod(uri);
method.getParams().setParameter("q", "Toshiba");//How do I insert a Map here?
method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
new DefaultHttpMethodRetryHandler(3, false));
method.setRequestHeader("api_key", "SEM2158A71D4AB3A3715C2435B96943F46D0");
try {
int statusCode = hc.executeMethod(method);
System.out.println(statusCode);
byte[] responseBody = method.getResponseBody();
System.out.println(new String(responseBody));
} catch (HttpException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
method.releaseConnection();
}
}
At the lowest level I can manually produce the query string manually via concatenation and then Url encode it. But is there a more elegant way to do it?

I think you can use external jar like GSON to convert the Map into JSON
Map<String, String> map = new HashMap<String,String>();
map.put("cat_id", "12345");
..
Gson gson = new Gson();
method.getParams().setParameter("q", gson.toJson(map));

Have a look at Google's Http Client
As you can see from the examples, it uses objects to build the request url and deserialise the response body. The docs also show you how to deserialise JSON specifically, and you can choose your JSON library of choice.