How to use termVector in solrJ - java

Solr/SolrJ Version: 6.0.0
I've set termvector component in solrconfig.xml, and the request handler is "/tvrh", I test it in the browser and this works. Now I want to use it in solrJ, but it only returns the document. The following is my code:
SolrClient solrClient = new HttpSolrClient("http://localhost:8983/solr/test");
SolrQuery solrQuery = new SolrQuery();
solrQuery.setQuery(String.format("id:%s","clueweb12-0000tw-06-17744"));
solrQuery.setRequestHandler("/tvrh");
solrQuery.set("tv", true);
solrQuery.set("tv.all", true);
QueryResponse response = solrClient.query(solrQuery);
SolrDocumentList docs = response.getResults();
for (SolrDocument doc: docs){
for (String key: doc.keySet()){
System.out.println(key);
System.out.println(doc.getFieldValue(key));
}
}

Your question is how to use a non standard request handler in solr.
Be aware that the Term Vector Component belongs to a "non standard" request handler and is not supported from solrj:
https://cwiki.apache.org/confluence/display/solr/The+Term+Vector+Component#TheTermVectorComponent-SolrJandtheTermVectorComponent
You can call "/tvrh" via solrj in a generic mode. You can not use the method SolrClient#query(SolrParams params) for this, because in this case the "request handler" is only send as "qt"-Parameter and will not be part of the url-path (and qt-Parameter is ignored by default).
So please try the method "SolrClient#request" instead.

As #Karsten R says, we could not use SolrClient.query to send request. After I searched a lot and experimented a lot, the following code could work.
SolrClient solrClient = new HttpSolrClient("http://localhost:8983/solr/trecB13");
SolrQuery solrQuery = new SolrQuery();
solrQuery.setQuery(String.format("id:%s","clueweb12-0000tw-06-17744"));
solrQuery.setRequestHandler("/tvrh");
solrQuery.set("tv", true);
solrQuery.set("tv.all", true);
try {
NamedList<Object> response = solrClient.request(new QueryRequest(solrQuery));
TermVectorExtractor extractor = new TermVectorExtractor(response);
System.out.println(extractor.getTermVectorInfo());
} catch (Exception e) {
e.printStackTrace();
}
TermVectorExtractor.java reference Sakai-Solr Github code, the function of the class is to parse resonse object and get term info. A little different from original code. The different has been shown below.
import org.apache.solr.common.util.NamedList;
import java.util.*;
public class TermVectorExtractor {
private static final String TERM_VECTORS = "termVectors";
private Map<String, Map<String, Map<String, TermInfo>>> termVectorInfo = Collections.emptyMap();
/**
* Creates a TermVectorExtractor for the given query response sent by Solr.
*
* #param queryResponse response sent by the solr server for a search query.
*/
#SuppressWarnings("unchecked")
public TermVectorExtractor(NamedList<Object> queryResponse) {
NamedList<Object> res = (NamedList<Object>) queryResponse.get(TERM_VECTORS);
if (res != null)
termVectorInfo = extractTermVectorInfo(res);
}
}

Related

Using M2Doc programmatically : Error in the generated .docx document

I'm trying to use M2Doc programmatically, I managed to generate my .docx file without getting errors in the validation part but I'm getting the following Error in the generated document:
{m:self.Name} Couldn't find the 'aqlFeatureAccess(org.eclipse.emf.common.util.URI.Hierarchical,java.lang.String)' service
The "self.Name" part is what I wrote in my template.
I think I'm lacking some kind of reference to a service but I don't know how to fix it.
The self variable is a reference to a model based on a meta-model I created. But I'm not sure I imported it correctly in my code.
I based my code on the code I found on the M2Doc website + some code I found on their GitHub, especially concerning how to add a service in the queryEnvironment.
I searched in the source code of acceleo and M2Doc to see which services they add but it seems that they already import all the services I'm using.
As I said, the validation part is going well and doesn't generate a validation file.
public static void parseDocument(String templateName) throws Exception{
final URI templateURI = URI.createFileURI("Template/"+templateName+"."+M2DocUtils.DOCX_EXTENSION_FILE);
final IQueryEnvironment queryEnvironment =
org.eclipse.acceleo.query.runtime.Query.newEnvironmentWithDefaultServices(null);
final Map<String, String> options = new HashMap<>(); // can be empty
M2DocUtils.prepareEnvironmentServices(queryEnvironment, templateURI, options); // delegate to IServicesConfigurator
prepareEnvironmentServicesCustom(queryEnvironment, options);
final IClassProvider classProvider = new ClassProvider(ClassLoader.getSystemClassLoader()); // use M2DocPlugin.getClassProvider() when running inside Eclipse
try (DocumentTemplate template = M2DocUtils.parse(templateURI, queryEnvironment, classProvider)) {
ValidationMessageLevel validationLevel = validateDocument(template, queryEnvironment, templateName);
if(validationLevel == ValidationMessageLevel.OK){
generateDocument(template, queryEnvironment, templateName, "Model/ComplexKaosModel.kaos");
}
}
}
public static void prepareEnvironmentServicesCustom(IQueryEnvironment queryEnvironment, Map<String, String> options){
Set<IService> services = ServiceUtils.getServices(queryEnvironment, FilterService.class);
ServiceUtils.registerServices(queryEnvironment, services);
M2DocUtils.getConfigurators().forEach((configurator) -> {
ServiceUtils.registerServices(queryEnvironment, configurator.getServices(queryEnvironment, options));
});
}
public static void generateDocument(DocumentTemplate template, IQueryEnvironment queryEnvironment,
String templateName, String modelPath)throws Exception{
final Map<String, Object> variable = new HashMap<>();
variable.put("self", URI.createFileURI(modelPath));
final Monitor monitor = new BasicMonitor.Printing(System.out);
final URI outputURI = URI.createFileURI("Generated/"+templateName+".generated."+M2DocUtils.DOCX_EXTENSION_FILE);
M2DocUtils.generate(template, queryEnvironment, variable, outputURI, monitor);
}
The variable "self" contains an URI:
variable.put("self", URI.createFileURI(modelPath));
You have to load your model and set the value of self to an element from your model using something like:
final ResourceSet rs = new ResourceSetImpl();
final Resource r = rs.getResource(uri, true);
final EObject value = r.getContents()...;
variable.put("self", value);
You can get more details on resource loading in the EMF documentation.

How can I update custom properties in alfresco workflow task using only Java?

First, I want to say thanks to everyone that took their time to help me figure this out because I was searching for more than a week for a solution to my problem. Here it is:
My goal is to start a custom workflow in Alfresco Community 5.2 and to set some custom properties in the first task trough a web script using only the Public Java API. My class is extending AbstractWebScript. Currently I have success with starting the workflow and setting properties like bpm:workflowDescription, but I'm not able to set my custom properties in the tasks.
Here is the code:
public class StartWorkflow extends AbstractWebScript {
/**
* The Alfresco Service Registry that gives access to all public content services in Alfresco.
*/
private ServiceRegistry serviceRegistry;
public void setServiceRegistry(ServiceRegistry serviceRegistry) {
this.serviceRegistry = serviceRegistry;
}
#Override
public void execute(WebScriptRequest req, WebScriptResponse res) throws IOException {
// Create JSON object for the response
JSONObject obj = new JSONObject();
try {
// Check if parameter defName is present in the request
String wfDefFromReq = req.getParameter("defName");
if (wfDefFromReq == null) {
obj.put("resultCode", "1 (Error)");
obj.put("errorMessage", "Parameter defName not found.");
return;
}
// Get the WFL Service
WorkflowService workflowService = serviceRegistry.getWorkflowService();
// Build WFL Definition name
String wfDefName = "activiti$" + wfDefFromReq;
// Get WorkflowDefinition object
WorkflowDefinition wfDef = workflowService.getDefinitionByName(wfDefName);
// Check if such WorkflowDefinition exists
if (wfDef == null) {
obj.put("resultCode", "1 (Error)");
obj.put("errorMessage", "No workflow definition found for defName = " + wfDefName);
return;
}
// Get parameters from the request
Content reqContent = req.getContent();
if (reqContent == null) {
throw new WebScriptException(Status.STATUS_BAD_REQUEST, "Missing request body.");
}
String content;
content = reqContent.getContent();
if (content.isEmpty()) {
throw new WebScriptException(Status.STATUS_BAD_REQUEST, "Content is empty");
}
JSONTokener jsonTokener = new JSONTokener(content);
JSONObject json = new JSONObject(jsonTokener);
// Set the workflow description
Map<QName, Serializable> params = new HashMap();
params.put(WorkflowModel.PROP_WORKFLOW_DESCRIPTION, "Workflow started from JAVA API");
// Start the workflow
WorkflowPath wfPath = workflowService.startWorkflow(wfDef.getId(), params);
// Get params from the POST request
Map<QName, Serializable> reqParams = new HashMap();
Iterator<String> i = json.keys();
while (i.hasNext()) {
String paramName = i.next();
QName qName = QName.createQName(paramName);
String value = json.getString(qName.getLocalName());
reqParams.put(qName, value);
}
// Try to update the task properties
// Get the next active task which contains the properties to update
WorkflowTask wfTask = workflowService.getTasksForWorkflowPath(wfPath.getId()).get(0);
// Update properties
WorkflowTask updatedTask = workflowService.updateTask(wfTask.getId(), reqParams, null, null);
obj.put("resultCode", "0 (Success)");
obj.put("workflowId", wfPath.getId());
} catch (JSONException e) {
throw new WebScriptException(Status.STATUS_BAD_REQUEST,
e.getLocalizedMessage());
} catch (IOException ioe) {
throw new WebScriptException(Status.STATUS_BAD_REQUEST,
"Error when parsing the request.",
ioe);
} finally {
// build a JSON string and send it back
String jsonString = obj.toString();
res.getWriter().write(jsonString);
}
}
}
Here is how I call the webscript:
curl -v -uadmin:admin -X POST -d #postParams.json localhost:8080/alfresco/s/workflow/startJava?defName=nameOfTheWFLDefinition -H "Content-Type:application/json"
In postParams.json file I have the required pairs for property/value which I need to update:
{
"cmprop:propOne" : "Value 1",
"cmprop:propTwo" : "Value 2",
"cmprop:propThree" : "Value 3"
}
The workflow is started, bpm:workflowDescription is set correctly, but the properties in the task are not visible to be set.
I made a JS script which I call when the workflow is started:
execution.setVariable('bpm_workflowDescription', 'Some String ' + execution.getVariable('cmprop:propOne'));
And actually the value for cmprop:propOne is used and the description is properly updated - which means that those properties are updated somewhere (on execution level maybe?) but I cannot figure out why they are not visible when I open the task.
I had success with starting the workflow and updating the properties using the JavaScript API with:
if (wfdef) {
// Get the params
wfparams = {};
if (jsonRequest) {
for ( var prop in jsonRequest) {
wfparams[prop] = jsonRequest[prop];
}
}
wfpackage = workflow.createPackage();
wfpath = wfdef.startWorkflow(wfpackage, wfparams);
The problem is that I only want to use the public Java API, please help.
Thanks!
Do you set your variables locally in your tasks? From what I see, it seems that you define your variables at the execution level, but not at the state level. If you take a look at the ootb adhoc.bpmn20.xml file (https://github.com/Activiti/Activiti-Designer/blob/master/org.activiti.designer.eclipse/src/main/resources/templates/adhoc.bpmn20.xml), you can notice an event listener that sets the variable locally:
<extensionElements>
<activiti:taskListener event="create" class="org.alfresco.repo.workflow.activiti.tasklistener.ScriptTaskListener">
<activiti:field name="script">
<activiti:string>
if (typeof bpm_workflowDueDate != 'undefined') task.setVariableLocal('bpm_dueDate', bpm_workflowDueDate);
if (typeof bpm_workflowPriority != 'undefined') task.priority = bpm_workflowPriority;
</activiti:string>
</activiti:field>
</activiti:taskListener>
</extensionElements>
Usually, I just try to import all tasks for my custom model prefix. So for you, it should look like that:
import java.util.Set;
import org.activiti.engine.delegate.DelegateExecution;
import org.activiti.engine.delegate.DelegateTask;
import org.apache.log4j.Logger;
public class ImportVariables extends AbstractTaskListener {
private Logger logger = Logger.getLogger(ImportVariables.class);
#Override
public void notify(DelegateTask task) {
logger.debug("Inside ImportVariables.notify()");
logger.debug("Task ID:" + task.getId());
logger.debug("Task name:" + task.getName());
logger.debug("Task proc ID:" + task.getProcessInstanceId());
logger.debug("Task def key:" + task.getTaskDefinitionKey());
DelegateExecution execution = task.getExecution();
Set<String> executionVariables = execution.getVariableNamesLocal();
for (String variableName : executionVariables) {
// If the variable starts by "cmprop_"
if (variableName.startsWith("cmprop_")) {
// Publish it at the task level
task.setVariableLocal(variableName, execution.getVariableLocal(variableName));
}
}
}
}

Post multipart form with google-http-java-client

It's not clear from the google-http-java-client* docs how you would go about posting a form that has a file field.
For example I'm trying to print a document using the Google Cloud Print API:
HttpRequestFactory httpRequestFactory = getHttpRequestFactory();
Map<String, Object> parameters = Maps.newHashMap();
parameters.put("printerId", printRequest.getPrinterId());
parameters.put("title", printRequest.getTitle());
parameters.put("contentType", printRequest.getContentType());
parameters.put("ticket", new Gson().toJson(printRequest.getOptions()));
MultipartContent content = new MultipartContent();
content.addPart(new MultipartContent.Part(new UrlEncodedContent(parameters)));
content.addPart(new MultipartContent.Part(
new FileContent(printRequest.getContentType(), printRequest.getFile())));
try {
HttpResponse response = httpRequestFactory.buildPostRequest(
SubmitUrl, content).execute();
System.out.println(IOUtils.toString(response.getContent()));
} catch (IOException e) {
String message = String.format();
System.out.println("Error submitting print job: " + e.getMessage());
}
Unfortunately this doesn't work. The API returns the error "Printer Id required for this request." which seems to me like the request isn't properly formed.
What am I doing wrong?
* I'm specifically using the google-http-java-client as it handles automatic refreshing of OAuth tokens etc for me. Please don't reply with solutions that involve using other HTTP clients.
So it looks like I misunderstood how form fields are added to multipart messages. The working code now looks like this
HttpRequestFactory httpRequestFactory = getHttpRequestFactory(username);
Map<String, String> parameters = Maps.newHashMap();
parameters.put("printerid", printRequest.getPrinterId());
parameters.put("title", printRequest.getTitle());
parameters.put("contentType", printRequest.getContentType());
// Map print options into CJT structure
Map<String, Object> options = Maps.newHashMap();
options.put("version", "1.0");
options.put("print", printRequest.getOptions());
parameters.put("ticket", new Gson().toJson(options));
// Add parameters
MultipartContent content = new MultipartContent().setMediaType(
new HttpMediaType("multipart/form-data")
.setParameter("boundary", "__END_OF_PART__"));
for (String name : parameters.keySet()) {
MultipartContent.Part part = new MultipartContent.Part(
new ByteArrayContent(null, parameters.get(name).getBytes()));
part.setHeaders(new HttpHeaders().set(
"Content-Disposition", String.format("form-data; name=\"%s\"", name)));
content.addPart(part);
}
// Add file
FileContent fileContent = new FileContent(
printRequest.getContentType(), printRequest.getFile());
MultipartContent.Part part = new MultipartContent.Part(fileContent);
part.setHeaders(new HttpHeaders().set(
"Content-Disposition",
String.format("form-data; name=\"content\"; filename=\"%s\"", printRequest.getFile().getName())));
content.addPart(part);
try {
HttpResponse response = httpRequestFactory.buildPostRequest(
SubmitUrl, content).execute();
System.out.println(IOUtils.toString(response.getContent()));
} catch (IOException e) {
...
}
The most important parts above were overriding the default HttpMediaType to specify "multipart/form-data" and adding each field as its own part with a "Content-Disposition" header to designate the form field name.

How to create query string with a map in it? (Java)

I need to call this service in Java -
https://api.semantics3.com/test/v1/products?q={"cat_id": "13658", "brand": "Toshiba", "model": "Satellite"}
I've managed to do this in python as follows -
class Semantics:
def __init__(self):
self.service_address = 'https://api.semantics3.com/test/v1/products?'
self.api_key = 'SEM3158A71D4AB3A3715C2230B96943F46D0'
def query(self, params):
query = 'q=' + params.__dict__.__str__().replace("'",'"')
query = urlencode(urlparse.parse_qs(query), True)
req = Request(url = self.service_address + query)
req.add_header('api_key', self.api_key)
return urlopen(req).read()
class QueryParams:
def __init__(self, cat_id, model, brand):
self.cat_id = cat_id
self.model = model
self.brand = brand
qp = QueryParams('13658', 'Satellite', "Toshiba")
print Semantics().query(qp)
I have tried writing an equivalent Java program using Spring REST API and Apache HttpClient to no avail. I can't find a way to set a dictionary (i.e. Map) into the query String.
public static void main(String[] args) {
String uri = "https://api.semantics3.com/test/v1/products?";
HttpClient hc = new HttpClient();
GetMethod method = new GetMethod(uri);
method.getParams().setParameter("q", "Toshiba");//How do I insert a Map here?
method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
new DefaultHttpMethodRetryHandler(3, false));
method.setRequestHeader("api_key", "SEM2158A71D4AB3A3715C2435B96943F46D0");
try {
int statusCode = hc.executeMethod(method);
System.out.println(statusCode);
byte[] responseBody = method.getResponseBody();
System.out.println(new String(responseBody));
} catch (HttpException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
method.releaseConnection();
}
}
At the lowest level I can manually produce the query string manually via concatenation and then Url encode it. But is there a more elegant way to do it?
I think you can use external jar like GSON to convert the Map into JSON
Map<String, String> map = new HashMap<String,String>();
map.put("cat_id", "12345");
..
Gson gson = new Gson();
method.getParams().setParameter("q", gson.toJson(map));
Have a look at Google's Http Client
As you can see from the examples, it uses objects to build the request url and deserialise the response body. The docs also show you how to deserialise JSON specifically, and you can choose your JSON library of choice.

How to use paging for retrieving Google docs?

I wand to receive google docs info with paging using Java Client Library.
My code:
private static final String URL_STRING = "https://docs.google.com/feeds/default/private/full/";
public List<DocumentListEntry> getAllDocs() throws Exception {
URL feedUri = new URL(URL_STRING);
DocumentQuery query = new DocumentQuery(feedUri);
query.setMaxResults(2);
query.setStartIndex(1);
DocumentListFeed feed = client.getFeed(query, DocumentListFeed.class);
return feed.getEntries();
}
And processing entries:
List<DocumentListEntry> docList = gDocumentsRetriever.getAllDocs();
for (DocumentListEntry entry : docList) {
processEntry(oAuthToken, gDocumentsRetriever, entry);
}
I get two entries.But if I change
query.setStartIndex(1);
to
query.setStartIndex(3);
i get same two entries.
I found how this issue can be realized:
http://code.google.com/apis/documents/docs/3.0/developers_guide_java.html#pagingThroughResults
For other services it realizes in the same way.

Categories