I have a json file that contains three datas together. I want to insert all three datas seperatly into the mongodB. Is that possible? if yes then how?
{
"docs": [
{
"_id": "First",
"count": 4,
"name": "Fish",
},
{
"_id": "Second",
"count": 6,
"name": "Meat"
},
{
"_id": "Third",
"count": 8,
"name": "Vegetables"
}
]
}
Inserting a group of documents from the mongo client shell:
let,
var input = {
"docs": [
{
"_id": "First",
"count": 4,
"name": "Fish",
},
{
"_id": "Second",
"count": 6,
"name": "Meat"
},
{
"_id": "Third",
"count": 8,
"name": "Vegetables"
}
]
}
Inserting the docs array:
db.collection.insert(input["docs"]);
This would insert each item in the docs array as separate documents in the collection.
db.collection.find();
would give us, three different documents that were inserted.
{ "_id" : "First", "count" : 4, "name" : "Fish" }
{ "_id" : "Second", "count" : 6, "name" : "Meat" }
{ "_id" : "Third", "count" : 8, "name" : "Vegetables" }
To do it in Java, you need to load and parse the JSON file using JSON parsing libraries such as Jackson parser, get the docs array and persist it.
Related
Dear MongoDb Experts out there! I am new to Mongo and I am currently working with a MongoDb with Java.
Having a MongoDb with a collection called "teams" with the following example structure:
{[
{
"id": "123",
"name": "Dev1",
"employees": [
{"name": "John", "age": 30},
{"name": "Jane", "age": 30}
]
},
{
"id": "456",
"name": "Dev2",
"employees": [
{"name": "Mike", "age": 30},
{"name": "Oscar", "age": 27}
]
}
]}
I want to have a query which returns an array with all employees, that are 30 years old. So the expected result would be:
{[
{"name": "John", "age": 30},
{"name": "Jane", "age": 30},
{"name": "Mike", "age": 30}
]}
It would be even better to only get the employees name (since I know the age I searched for), like:
{[
{"name": "John"},
{"name": "Jane"},
{"name": "Mike"}
]}
I have a MongoCollection object:
MongoCollection<Document> collection = mongoClient
.getDatabase(databaseName)
.getCollection("teams")
My question: Is it possible to retrieve my expected result from the MongoDb? If so, which operations do I have to call on my collection object?
Approach 1: Find with $elemMatch projection
db.getCollection('teams').find(
{"employees.age":30},
{ "employees": { "$elemMatch": { "age": 30 } },"_id":0 }
)
Output Format:
{
"employees" : [
{
"name" : "John",
"age" : 30.0
}
]
}
{
"employees" : [
{
"name" : "Mike",
"age" : 30.0
}
]
}
Approach 2: Aggregate with $unwind and key renaming using $projection
db.getCollection('teams').aggregate([
{"$match": {"employees.age":30}} ,
{"$unwind": "$employees"},
{"$match": {"employees.age":30}} ,
{"$project": {"name": "$employees.name","_id":0}}
])
Output format:
{
"name" : "John"
}
{
"name" : "Jane"
}
{
"name" : "Mike"
}
Approach 1 would be faster, but require additional work at app layer. It is recommended to offload formatting tasks to app servers rather than on db itself.
Approach 2 can instantly give to the formatted results, doing the querying and formatting both in the database servers
I'm trying to implement custom search in elastic search.
Problem statement is consider 3 documents inserted into elastic search with "names" field as array:
{
id:1,
names:["John Wick","Iron man"]
}
{
id:2,
names:["Wick Stone","Nick John"]
}
{
id:3,
names:["Manny Nick","Stone cold"]
}
when I search for "Nick" I want to boost or give priority to document starting with Nick so in this case document with id 2 should come first and then document with id 3 and also if I search for whole name "Manny Nick"
doc with id 3 should be given priority.
In such case, you may want to modify/boost the score of search matched result for required criteria. For example, match the documents with names "Nick" and at the same time modify and boost the score of documents which contains names that start with Nick so that documents that match Nick and also starts with Nick will have higher score.
One of the way to achieve this is using Function Score Query.
In the below query, search is made for keyword "Nick" and matched documents' score is modified and boosted for criteria "names that start with Nick" using Match Phrase Prefix Query with additional weight 20.
{
"query": {
"function_score": {
"query": {
"match": {
"names": "Nick"
}
},
"boost": "1",
"functions": [
{
"filter": {
"match_phrase_prefix": {
"names": "Nick"
}
},
"weight": 20
}
],
"boost_mode": "sum"
}
}
}
Testing:
Inserted data:
{
id:1,
names:["John Wick","Iron man"]
}
{
id:2,
names:["Wick Stone","Nick John"]
}
{
id:3,
names:["Manny Nick","Stone cold"]
}
Output:
{
"took": 10,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 20.693148,
"hits": [
{
"_index": "stack_1",
"_type": "1",
"_id": "T9kn5WsBrk7qsVCmKBGH",
"_score": 20.693148,
"_source": {
"id": 2,
"names": [
"Wick Stone",
"Nick John"
]
}
},
{
"_index": "stack_1",
"_type": "1",
"_id": "Ttkm5WsBrk7qsVCm2RF_",
"_score": 20.287682,
"_source": {
"id": 3,
"names": [
"Manny Nick",
"Stone cold"
]
}
}
]
}
}
I have indexed sample documents in elasticsearch and trying to search using fuzzy query. But am not getting any results when am search by using Java fuzzy query api.
Please find my below mapping script :
PUT productcatalog
{
"settings": {
"analysis": {
"analyzer": {
"attr_analyzer": {
"type": "custom",
"tokenizer": "letter",
"char_filter": [
"html_strip"
],
"filter": ["lowercase", "asciifolding", "stemmer_minimal_english"]
}
},
"filter" : {
"stemmer_minimal_english" : {
"type" : "stemmer",
"name" : "minimal_english"
}
}
}
},
"mappings": {
"doc": {
"properties": {
"values": {
"type": "text",
"analyzer": "attr_analyzer"
},
"catalog_type": {
"type": "text"
},
"catalog_id":{
"type": "long"
}
}
}
}
}
Please find my sample data.
PUT productcatalog/doc/1
{
"catalog_id" : "343",
"catalog_type" : "series",
"values" : "Activa Rooftop, valves, VG3000, VG3000FS, butterfly, ball"
}
PUT productcatalog/doc/2
{
"catalog_id" : "12717",
"catalog_type" : "product",
"values" : "Activa Rooftop, valves"
}
Please find my search script :
GET productcatalog/_search
{
"query": {
"match" : {
"values" : {
"query" : " activa rooftop VG3000",
"operator" : "and",
"boost": 1.0,
"fuzziness": 2,
"prefix_length": 0,
"max_expansions": 100
}
}
}
}
Am getting the below results for the above query :
{
"took": 239,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.970927,
"hits": [
{
"_index": "productcatalog",
"_type": "doc",
"_id": "1",
"_score": 0.970927,
"_source": {
"catalog_id": "343",
"catalog_type": "series",
"values": "Activa Rooftop, valves, VG3000, VG3000FS, butterfly, ball"
}
}
]
}
}
But if i use the below Java API for the same fuzzy search am not getting any results out of it.
Please find my below Java API query for fuzzy search :
QueryBuilder qb = QueryBuilders.boolQuery()
.must(QueryBuilders.fuzzyQuery("values", keyword).boost(1.0f).prefixLength(0).maxExpansions(100));
Update 1
I have tried with the below query
QueryBuilder qb = QueryBuilders.matchQuery(QueryBuilders.fuzzyQuery("values", keyword).boost(1.0f).prefixLength(0).maxExpansions(100));
But am not able to pass QueryBuilders inside matchQuery. Am getting this suggestion while am writing this query The method matchQuery(String, Object) in the type QueryBuilders is not applicable for the arguments (FuzzyQueryBuilder)
The mentioned java query is not a match query. It's a must query. you should use matchQuery instead of boolQuery().must(QueryBuilders.fuzzyQuery())
Update 1:
fuzzy query is a term query while match query is a full text query.
Also don't forget that in match query the default Operator is or operator which you should change it to and like your dsl query.
I am working with Elasticsearch recently, and I meet a problem that don't know how to solve it.
I have a Json like:
{
"objects": [
"object1": {
"id" : "12345",
"name":"abc"
},
"12345"
]
}
Object2 is a reference of object1, when I trying to saving(or called indexing) into elastic search, it says:
"org.elasticsearch.index.mapper.MapperParsingException: failed to parse"
After I google I found that because object1 is an object, but object 2 is considered as a string.
We cannot change our json in our project, so in this case how can I save it in the elasticsearch?
Thanks for any help and suggestion.
How do you do that?
I run this command and it works.
PUT test/t1/1
{
"objects": {
"object1": {
"id" : "12345",
"name":"abc"
},
"object2": "12345"
}
}
and the result is:
{
"_index": "test",
"_type": "t1",
"_id": "1",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 2,
"failed": 0
},
"created": true
}
UPDATE 1
Depending on your requirements one of these may solve your problem:
PUT test/t1/2
{
"objects": [
{
"object1": {
"id": "12345",
"name": "abc"
}
},
{
"object2": "12345"
}
]
}
PUT test/t1/2
{
"objects": [
{
"object1": {
"id": "12345",
"name": "abc"
},
"object2": "12345"
},
{
...
}
]
}
Can anyone help me with the following aggregate operation in mongodb: having a collection of items with ids and group ids, group them by group ids. For example, for collection of items:
{
"id": 1,
"group_id": 10,
"data": "some_data",
"name": "first"
},
{
"id": 2,
"group_id": 10,
"data": "some_data",
"name": "second"
},
{
"id": 3
"group_id": 20,
"data": "some_data",
"name": "third"
}
Create new collection of groups with the following structure:
{
"id": 10,
"items": [
{
"id": 1,
"group_id": 10,
"data": "some_data",
"name": "first"
},
{
"id": 2,
"group_id": 10,
"data": "some_data",
"name": "second"
}
]
},
{
"id": 10,
"items": [
{
"id": 2,
"group_id": 20,
"data": "some_data",
"name": "third"
}
]
}
The corresponding snippet with Java and spring-data-mongodb will also be appreciated.
In fact I'm doing the same right now with Java and want to move this logic to mongo for paging optimisation.
You can do it with the folowwing simple group aggregation:
db.table.aggregate(
[
{
$group: {
_id : "$group_id",
items : { "$push" : "$$ROOT" }
}
}
]
);
When you want to output the data from the aggregation into a new collection use the $out operator