We have a multivalued field in Solr that we want to reduce its length.
A sample result response is as follows:
response": {
"numFound": 1,
"start": 0,
"docs": [
{
"created_date": "2016-11-23T13:47:46.55Z",
"solr_index_date": "2016-12-01T08:21:59.78Z",
"modified_date": "2016-12-13T08:45:44.507Z",
"id": "FEAE38C2-ABFF-4F0C-8AFD-9B8F51036D8A",
"Field1": [
"false",
"true",
"true",
..... <= 1200 items
]
}
]
}
We have big data, a couple of TB and we are looking for an optimized way to alter all documents within Solr and to modify Field1 to contain only the first 100 items.
Can something like this be done without the need to write a script to manually fetch the document, make adjustments and push it back to solr? Has anyone had a similar experience? Thanks
We have faced this problem. But we use Two collections to solve this problem. Use SoleEntityProcessor to move the document from one collection to another.
[SolrEntityProcessor]
<dataConfig>
<document>
<entity name="sep" processor="SolrEntityProcessor" url="http://localhost:8983/solr/db" query="*:*"/>
</document>
</dataConfig>
While moving pass that document through updateRequestProcessorChain where we can write StatelessScriptUpdateProcessorFactory to edit our documents or to truncate the multivalued field.
In StatelessScriptUpdateProcessorFactory you can get the field and apply your operations and then reset that field.
[StatelessScriptUpdateProcessorFactory]
function processAdd(cmd) {
doc = cmd.solrDoc;
multiDate = doc.getFieldValue("multiValueField");
//Apply your operation to above field
//doc.setField("multiValueField",value);
}
function processDelete(cmd) {
// no-op
}
function processMergeIndexes(cmd) {
// no-op
}
function processCommit(cmd) {
// no-op
}
function processRollback(cmd) {
// no-op
}
function finish() {
// no-op
}
For More information on StatelessScriptUpdateProcessorFactory, you can refer to this question
On solr how can i copy selected values only from multi valued field to another multi valued field?
in which they edit the multivalued field using the script.
Related
I have a JSON string as follows (simplified):
{
"data":{
"fruit":[
{
"role":[
{
"role":"passive"
}
],
"objectType":"apple"
},
{
"role":[
{
"role":"active"
}
],
"objectType":"orange"
}
]
}
}
I would like to get objectType value where role is active. In this example the result would be orange. I've come up with the following code, however it doesn't return anything. How do I fix this?
$.data.fruit[?(#.role.role == 'active')]
Note that the number of elements in fruit array may vary and there is no guarantee that the last element would be correct.
At OP's request, here's the final, final answer!
$.data.fruit[?('active' in #..role..role)].objectType
On the off chance that it is a number, progressive levels in the stringification will each be indented by this many space characters (up to 10).
In the event that it is a string, progressive levels will be indented by this string (or its initial ten characters).
I am using spring data jpa for creating micro services. In repository I am using JPQL query. Using following code I am able to get set of data. But I want to iterate set of data for further logic.
For iteration I used for each but when I am using for each loop I am getting error "java.util.HashMap cannot be cast to com.spacestudy.model.RoomDepartmentMapping",
public Set<RoomDepartmentMapping> loadStatusOfRooms() {
Set<RoomDepartmentMapping> roomDeptMapping = roomDeptMappingRepo.findStaus();
return roomDeptMapping;
}
}
Repository
#Repository
public interface RoomDepartmentMappingRepository extends JpaRepository<RoomDepartmentMapping, Integer> {
#Query("select new map(roomDeptMap.sStatus as sStatus) from RoomDepartmentMapping as roomDeptMap")
Set<RoomDepartmentMapping> findStaus();
}
Result
[
{
"sStatus": "A"
},
{
"sStatus": "I"
},
{
"sStatus": "R"
}
]
Expecting Result
[
{
"sStatus": "Accepted"
},
{
"sStatus": "In Progress"
},
{
"sStatus": "Remaining"
}
]
For getting above expected result I am trying to iterate Set of data using for each and planning to use switch case. But I am not able to iterate using following loop.
For each loop
for(RoomDepartmentMapping roomDeptMappingObj:roomDeptMapping) {
System.out.println(roomDeptMappingObj);
}
Can any one tell me why I am not able to iterate set using for each loop?
Or please suggest any another way to do that.
Based on the name of your method I will assume that you need all available status values.
The query will be:
#Query("select distinct rdm.sStatus from RoomDepartmentMapping rdm")
Set<String> findStatus();
distinct => you need set
Set => RoomDepartmentMapping sStatus is a String
Usually, I avoid asking a new question as I always find a "close-enough" answer to my problem.
But this time I surprisingly have a fundamental and straightforward question - without any lead.
I have a straightforward MongoDB document that holds two arrays:
The 1st containing three numbers (ints) - each represent a code-number of a selected (predefined) question.
The 2nd was holding 3 Strings - that are the answers given by the user to these three correspondent questions.
For example, let's say subject code 12 is - "What's your 1st dog's name?", and the user's answer was: "Spiky", etc...
So I end up with something like:
{
"_id" : ObjectId("..."),
"questionCodesArray" : [
12,
56,
3
],
"answersArray" : [
"Spiky",
"go swimming",
"blabla.."
]
}
Now, I'd like to be able to allow the users to change their mind and choose a different question and supply a different answer to it.
For this, I need to have only the index of my element and access it via this index to change it using update() or findAndModify() [or any other method] and all the answers out there are "key-value style" which is not needed.
In simple Java I would've simply done something like:
questionsCodesArry[index] = newValue; and
answersArray[index] = newAnswerString;
All my attempts to write a descent query to do this simple index-based updating have failed.
Any help will be appreciated.
Thank you.
In plain MongoDB syntax what you need is this:
collection.update({
/*your filter goes here*/
}, {
$set: {
"questionCodesArray.<zero-based-index>": "new value"
}
})
I don't have a Java environment here to translate this into your driver's world. I might be able to do so tonight, though.
Still, I would definitely vote for a different, more natural and less error-prone design where you'd structure your data like so:
{
"_id" : ObjectId("59b635ffad44fad6662d8591"),
"answers" : [
{
"questionCode" : 12,
"answer" : "Spiky"
},
{
"questionCode" : 56,
"answer" : "go swimming"
},
{
"questionCode" : 3,
"answer" : "blabla.."
}
]
}
If that's an option for you I shall happily provide you the right update statement for this layout, too.
Well, after some trials here is a c&p complete and full method to change an array's element by a given index:
public boolean changeArrayValue(DB db, String collectionName, long id, String arrayNameKey, int index, Object newValue)
{
DBCollection collection = db.getCollection(collectionName);
DBObject query = new BasicDBObject("id",id);//unique id is recommended
DBObject update = new BasicDBObject("$set", new BasicDBObject(arrayNameKey+"."+index, newValue));
DBObject result = collection.findAndModify(query, null, null, false, update,true, true);
//Just for precaution
if(result == null)
return false;
return (((BasicDBList)result.get(arrayNameKey)).get(index)).equals(newValue);
}
As you can see the main issue lays in the $set line here: arrayNameKey+"."+index .
In the result I've put the needed flags to get the updated result, which is of type BasicDBList<Object>.
Pleases note that you'll have to add all the needed checks and validations according to your taste.
Enjoy..
I'm currently recovering JSON formated from HTTP CALL.
This function is launched with parameters provided by a form template and parameters are pushed on variables with submit event.
My function getusershttp is able to return some results that I'm able to see with console.log(results.content);:
"results" : [
{
"name" : {
"first" : "Billy",
"last" : "McKornic"
},
"id" : "a1c3fd06c71ccc50998baa02074976b4d639e4cf",
"situation" : "free",
},
{
"name" : {
"first" : "Dough",
"last" : "Wallas"
},
"id" : "5694c02beaf20d2d4b5747668b82264af8547e33",
"situation" : "occuped",
}
],
"status" : "OK"
}
I would like to put each result in my articles template with:
<template name="articles">
{{#each results}}
<header>
<p>{{name.first}} {{name.last}}</p>
<p>{{id}}</p>
<p>{{situation}}</p>
</header>
{{/each}}
</template>
What template event and function I must create in order to provide each results data?
Currently I have:
Template.articles.helpers({
results : function() {
return Meteor.call("getusershttp",FormParamX,FormParamY);
}
});
But I have a java error on page loading since my form is not submited and FormParamX and FormParamY are not populated.
How do I force my Template (event & function) to wait my form submited in order to start providing results?
Thanks!
Set the values for FormParamX and Y to reactive variables, then you can simply do:
results() {
if( Template.instance().FormParamX.get() && Template.instance().FormParamY.get() ) {
return Meteor.call("getusershttp",FormParamX,FormParamY);
}
}
Then any time the form parameters change, you'll recall your Meteor method and get the appropriate data.
Additional documentation on reactive variables are here, and there is also a nice walkthrough on TheMeteorChef.
After Stephen tips I took a look with session variables and it works!
In order to help, I provide the solution according my exemple.
First, my form template event function:
Template.search.events({
'submit form':function (event){
event.preventDefault();
var FormParamX = event.target.FormParamX.value;
var FormParamY = event.target.FormParamY.value;
Meteor.call("getusershttp",FormParamX,FormParamY,function(error, results) {
Session.set("MyUsers",results.content);
});
}
});
Here, my session variable MyUsers store my HTTP CALL JSON output.
Then, my articles template helpers:
Template.articles.helpers({
results : function() {
if (Session.get("MyUsers")) {
console.log(JSON.parse(Session.get("MyUsers")).results);
return JSON.parse(Session.get("MyUsers")).results;
};
}
});
Helpers only provide session variable MySearch value if it have some values.
I tested changing my form and on each submit, helpers provide different values as expected.
My template articles is not able now to provide each results but this is another issue.
Update: In order to provide each results value, I need to parse my JSON output. I updated the code and now my template is working.
Another time, thanks Stephen!
Here is my document
{
"_id":"1000",
"event_name":"Some name",
"tracks":
[
{
"id":"100"
"title":"Test title",
},
{
"id":"101"
"title":"Test title 2",
}
]
}
I want to directly acess track with id 101
So far i am doing this
Query query = new Query(Criteria.where("tracks.$._id").is(id));
// Execute the query and find one matching entry
Event event= mongoTemplate.findOne(query, Event.class,MongoDBCollections.EVENT);
I want some thing like this
Track track = event.getTrackById(id); how can i get track from that event object?
You are using the positional operator in the wrong place, it is a "projection" modifier and is not part of the query. You add the fields on the query spec with the .include() method:
Query query = new Query(Criteria.where("tracks._id").is(id);
query.fields().include("tracks.$");
Event event = mongoTemplate.findOne(query,Event.class,MongoDBCollections.EVENT);
But as with all projecton, the list of "fields" is all or nothing. So you either specify all of the fields to match your result class or modify that result class to just the fields you wish to return.