We Keep Coding

SolrJ Getting document score of all the resulting documents from solr query

I am able to fetch all the documents for a solr query in Solr 6.3.0 using the JAVA API SolrJ.I want an additional field of correct "score" calculated by solr(using tf,idf and field norm) to rank the documents.I am getting the score field as 1.0 for all the documents.Can you help me get the correct "score" field.
Below is my code snippet and the output.
String urlString = "http://localhost:8983/solr/mycore2";
SolrClient solr = new HttpSolrClient.Builder(urlString).build();
SolrQuery query = new SolrQuery();
query.setQuery( "*" );
query.set("fl", "id,house,postcode,score");
String s="house=".concat(address.getHouseNumber().getCoveredText());
query.addFilterQuery(s);
QueryResponse resp = solr.query(query);
SolrDocumentList list = resp.getResults();
if(list!=null)
{
System.out.println(list.toString());
}
Output
{numFound=4,start=0,maxScore=1.0,docs=[SolrDocument{id=1, house=[150-151], postcode=[641044], score=1.0}, SolrDocument{id=2, house=[150/151], postcode=[641044], score=1.0}, SolrDocument{id=3, house=[151/150], postcode=[641044], score=1.0}, SolrDocument{id=4, house=[151/150], postcode=[641044], score=1.0}]}
Edit
After MatsLindh's suggestion,here is the tweaked code and the output.
String urlString = "http://localhost:8983/solr/mycore2";
SolrClient solr = new HttpSolrClient.Builder(urlString).build();
SolrQuery query = new SolrQuery();
query.setQuery(address.getHouseNumber().getCoveredText().concat(" ").concat(address.getPostcode().getCoveredText()));
query.set("fl", "id,house,postcode,score");
QueryResponse resp = solr.query(query);
SolrDocumentList list = resp.getResults();
if(list!=null)
{
System.out.println(list.toString());
}
Output
{numFound=3,start=0,maxScore=2.4800222,docs=[SolrDocument{id=6, house=[34], postcode=[641006], score=2.4800222}, SolrDocument{id=5, house=[34], postcode=[641005], score=1.2400111}, SolrDocument{id=7, house=[2-11A], postcode=[641006], score=1.1138368}]}

Since you're not querying for anything, you're not getting a score (each score is the same, 1.0f). You're only applying a filter, which does not affect the score.
There is no tf/idf (but remember that Solr now uses BM25 as the default similarity model and not tf/idf) score to calculate if there are no tokens to match in the actual query.

How to get the suggested terms from Solr suggester response [duplicate]

What is the correct way of getting results from solrj using Solr Suggester?
This is my request:
SolrQuery query = new SolrQuery();
query.setRequestHandler("/suggest");
query.setParam("suggest", "true");
query.setParam("suggest.build", "true");
query.setParam("suggest.dictionary", "mySuggester");
query.setParam("suggest.q", "So");
QueryResponse response = server.query(query);
However I found it extremely difficult to get the response. The way I got the response is with this:
NamedList obj = (NamedList)((Map)response.getResponse().get("suggest")).get("mySuggester");
SimpleOrderedMap obj2 = (SimpleOrderedMap) obj.get("So");
List<SimpleOrderedMap> obj3 = (List<SimpleOrderedMap>) obj2.get("suggestions");
This seems to assume a lot about the objects I am getting from the response and will be difficult to anticipate errors.
Is there a better and cleaner way than this?

In new versions have a SuggesterResponse:
https://lucene.apache.org/solr/5_3_1/solr-solrj/org/apache/solr/client/solrj/response/SuggesterResponse.html

Best option is to get it as List, below code worked for me
HttpSolrClient solrClient = new HttpSolrClient(solrURL);
SolrQuery query = new SolrQuery();
query.setRequestHandler("/suggest");
query.setParam("suggest.q", "Ins");
query.setParam("wt", "json");
try {
QueryResponse response = solrClient.query(query);
System.out.println(response.getSuggesterResponse().getSuggestedTerms());
List<String> types=response.getSuggesterResponse().getSuggestedTerms().get("infixSuggester");
System.out.println(types);
} catch (SolrServerException | IOException e) {
e.printStackTrace();
}

You can get the suggestions via the SpellCheckResponse by doing the following
SpellCheckResponse spellCheckResponse=response.getSpellCheckResponse();
Check this link for more details

How to execute searchAndRank method in IBM Watson Retrieve And Rank API through Java wrapper

I am trying to perform a retrieveAndRank query using Java wrapper. I follow the online javadocs for Retrieve and Rank API.
The example there for SearchAndRank is:
https://www.ibm.com/watson/developercloud/retrieve-and-rank/api/v1/#query_ranker
RetrieveAndRank service = new RetrieveAndRank();
service.setUsernameAndPassword("{username}","{password}");
HttpSolrClient solrClient = new HttpSolrClient;
solrClient = getSolrClient(service.getSolrUrl("scfaaf8903_02c1_4297_84c6_76b79537d849"), "{username}","{password}");
SolrQuery query = new SolrQuery("what is the basic mechanism of the transonic aileron buzz");
QueryResponse response = solrClient.query("example_collection", query);
Ranking ranking = service.rank("B2E325-rank-67", response);
System.out.println(ranking);
but RetrieveAndRank class has no such rank(String rankerId, QueryResponse response) method. Just one getting a file or an InputStream as arguments (browsing IBM’s source code I see it expects CSV content not a java QueryResponse there).
How should I pass QueryResponse to the rank method?
I am using solr-solrj-5.5.2.jar and java-sdk-3.2.0-jar-with-dependencies.jar libraries

You need to use the /fcselect query handler and send the ranker_id as a parameter.
The code below assumes you have a Solr collection with documents and you have trained a ranker, otherwise follow this tutorial.
RetrieveAndRank service = new RetrieveAndRank();
service.setUsernameAndPassword(USERNAME, PASSWORD);
// create the solr client
String solrUrl = service.getSolrUrl(SOLR_CLUSTER_ID);
HttpClient client = createHttpClient(solrUrl, USERNAME, PASSWORD);
HttpSolrClient solrClient = new HttpSolrClient(solrUrl, client);
// build the query
SolrQuery query = new SolrQuery("*:*");
query.setRequestHandler("/fcselect");
query.set("ranker_id", RANKER_ID);
// execute the query
QueryResponse response = solrClient.query(SOLR_COLLECTION_NAME, query);
System.out.println("Found " + response.getResults().size() + " documents!");
System.out.println(response);
Make sure you update the service credentials for RetrieveAndRank(USERNAME and PASSWORD), SOLR_CLUSTER_ID, SOLR_COLLECTION_NAME and RANKER_ID.
The code for createHttpClient() can be found here.

MongoDB Java driver 3.x: How to pass allowDiskUse=true to aggregate() method?

I'm using mongo-java-driver 3.0.2.
I have a method that uses MongoCollection.aggregate(List<Bson> pipeline) to sort and limit:
private static MongoIterable<Document> selectTop(int n) {
BasicDBObject sortFields = new BasicDBObject("score", -1);
BasicDBObject sort = new BasicDBObject("$sort", sortFields);
BasicDBObject limit = new BasicDBObject("$limit", n);
List<BasicDBObject> pipeline = new ArrayList<>();
pipeline.add(sort);
pipeline.add(limit);
return playersCollection.aggregate(pipeline);
}
When n is big, it fails with:
com.mongodb.MongoCommandException: Command failed with error 16820: 'exception: Sort exceeded memory limit of 104857600 bytes, but did not opt in to external sorting. Aborting operation. Pass allowDiskUse:true to opt in.'
I've found that the MongoDB shell provides a method db.collection.aggregate(pipeline, options) (link) where options can contain an allowDiskUse field.
I can't find the equivalent to this in the Java API. Although there is an AggregationOptions class, the MongoCollection class doesn't provide an aggregate(List<Bson> pipeline, AggregationOptions options) method.

This still works on the 3.0.3 driver:
MongoClient client = new MongoClient(new ServerAddress("127.0.0.1", 27017));
DB test = client.getDB("test");
DBCollection sample = test.getCollection("sample");
List<DBObject> aggregationQuery = Arrays.<DBObject>asList(
new BasicDBObject("$sort",new BasicDBObject("score",-1)),
new BasicDBObject("$limit",1)
);
System.out.println(aggregationQuery);
Cursor aggregateOutput = sample.aggregate(
aggregationQuery,
AggregationOptions.builder()
.allowDiskUse(true)
.build()
);
while ( aggregateOutput.hasNext() ) {
DBObject doc = aggregateOutput.next();
System.out.println(doc);
}
Of course you can also use newer classes as well:
MongoClient client = new MongoClient(new ServerAddress("192.168.2.4", 27017));
MongoDatabase db = client.getDatabase("test");
MongoCollection<Document> collection = db.getCollection("sample");
AggregateIterable<Document> result = collection.aggregate(Arrays.asList(
new BasicDBObject("$sort", new BasicDBObject("score", -1)),
new BasicDBObject("$limit", 1)
)).allowDiskUse(true);
MongoCursor<Document> cursor = result.iterator();
while (cursor.hasNext()) {
Document doc = cursor.next();
System.out.println(doc);
}
So .aggregate() on MongoCollection returns an AggregateIterable class instance, which has an .allowDiskuse() method as well as others to set aggregation options.

How to set ResponseWriter to non-default in Solr

I have the following code, which is simply searching the Solr Server.
SolrServer server = new CommonsHttpSolrServer(url);
SolrQuery searchquery = new SolrQuery("company profile");
QueryResponse response = server.query(searchquery)
I want to have the response in json, other than the default which is xml. So I went into the solrconfig.xml file and enabled the following line:
<queryResponseWriter name="json" class="org.apache.solr.request.JSONResponseWriter" />
However, from the console, I'm still getting wt=javabin encoded to the search query request.
Also, I've modified the above code like this:
SolrServer server = new CommonsHttpSolrServer(url);
SolrQuery searchquery = new SolrQuery("company profile");
searchquery.setParam("wt", "json");
QueryResponse response = server.query(searchquery)
But I'm still getting wt=javabin encoded and wt=json also appended, such that the query now looks like this:
webapp/solr path=/select params={wt=javabin&wt=json}
Is there anything I'm doing wrong?
Thanks

SolrJ only supports the javabin and xml formats (configurable with CommonHttpSolrServer.setParser).
But why would you want to use JSON? Javabin is by far the format which provides the best decompression speed, its drawback being that is is not human-readable, on the contrary to JSON and XML (which is not a problem in this case since SolrJ is parsing the result for you).

With SolrJ and json-lib, I was able to get json response like this:
SolrServer server = new CommonsHttpSolrServer(url);
SolrQuery searchquery = new SolrQuery("company profile");
QueryResponse response = server.query(searchquery)
JSONArray jsonObject = JSONArray.fromObject( response.getResults() );
log.log(Level.INFO, "received jsonObject is {0}", jsonObject.toString());
Iterator data = jsonObject.iterator();
SolrResultBean bean = new SolrResultBean();
List output = new ArrayList();
while(data.hasNext()) {
output.add(data.next());
}
log.log(Level.INFO, "json items in the List are {0}", output);
bean.setObject(output);
// wicket page redirect
setResponsePage(SearchPage.class, new PageParameters());