I'm try to make a bulk update
Method: Post
Url: /customer/external/_bulk
Json Body:
{"index":{"_id":"1"}}
{"name": "John Doe" }
{"index":{"_id":"2"}}
{"name": "Jane Doe" }
Id 1 is updated but id 2 didnt update. I dont know why?
Response is here:
{
"took": 138,
"errors": false,
"items": [
{
"index": {
"_index": "customer",
"_type": "external",
"_id": "1",
"_version": 15,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"created": false,
"status": 200
}
}
]
}
As #Val mentioned, you should be having the new line character \n at the end of the last line in your json body:
{"index":{"_id":"1"}}
{"name": "John Doe" }
{"index":{"_id":"2"}}
{"name": "Jane Doe" }\n
as per mentioned in bulk_api. Hope it helps!
Related
I have a JSON output like this:
{
"items": [
{
"id": "1",
"name": "Anna",
"values": [
{
"code": "Latin",
"grade": 1
},
{
"code": "Maths",
"grade": 5
}
]
},
{
"id": "2",
"name": "Mark",
"values": [
{
"code": "Latin",
"grade": 5
},
{
"code": "Maths",
"grade": 5
}
]
}
]
}
I need to get field values for "name": "Anna". I am getting RestAssured Response and would like to use my beans to do that, but I can also use jsonPath() or jsonObject(), but I don't know how. I searched many topics but did not find anything.
I'm trying to validate the response schema with karate but facing issue with array.
Attaching the response and feature as well as my schema.json.
Response -
{
"page": 1,
"per_page": 6,
"total": 12,
"total_pages": 2,
"data": [
{
"id": 3,
"email": "emma.wong#reqres.in",
"first_name": "Emma",
"last_name": "Wong",
"avatar": "https://reqres.in/img/faces/3-image.jpg"
},
{
"id": 4,
"email": "eve.holt#reqres.in",
"first_name": "Eve",
"last_name": "Holt",
"avatar": "https://reqres.in/img/faces/4-image.jpg"
},
{
"id": 5,
"email": "charles.morris#reqres.in",
"first_name": "Charles",
"last_name": "Morris",
"avatar": "https://reqres.in/img/faces/5-image.jpg"
},
{
"id": 6,
"email": "tracey.ramos#reqres.in",
"first_name": "Tracey",
"last_name": "Ramos",
"avatar": "https://reqres.in/img/faces/6-image.jpg"
}
],
"support": {
"url": "https://reqres.in/#support-heading",
"text": "To keep ReqRes free, contributions towards server costs are appreciated!"
}
}
Scenario: Get all Users and validate schema
Given url getUrl
When method Get
Then status 200
And print response
Then match response == '#object'
* string jsonSchemaExpected = read('file:src/test/resources/features/sample/responseSchema.json')
And print response.data.length
And match response == jsonSchemaExpected
responseSchema.json
{
"page": "#number",
"per_page": "#number",
"total": "#number",
"total_pages": "#number",
"data": "#[] #object",
"support": "#object"
}
The only observation I have is if you cast to a string, you won't be able to do any matching.
Instead of * string jsonSchemaExpected do * def jsonSchemaExpected.
I'm trying to convert following Mongo shell aggregation query into Spring Data MongoDB Aggregation:
db.getCollection('Orders').aggregate([
{ $group: {
"_id": {"book":"$bookingId"},
"docs": {$push: '$$ROOT'}
}
},
{$project: {
latestOrd: {
$filter: {
input: "$docs",
as: "item",
cond: { $eq: ["$$item.bookingVersion", { $max: "$docs.bookingVersion" }] }
}
}
}
},
{ $unwind: "$latestOrd" },
{ $replaceRoot:{newRoot:"$latestOrd"}}
])
Query fetches all orders with highest booking version (for example one bookingId may have many documents with version 3).
This query works fine in Mongo shell but I have issue with Spring Data version of it:
Aggregation.group("bookingId").push(Aggregation.ROOT).as("docs");
Aggregation.project().and(filter("docs")
.as("item")
.by(valueOf("item.bookingVersion")
.equalToValue(AccumulatorOperators.Max.maxOf("docs.bookingVersion"))))
.as("latestOrd");
Aggregation.unwind("latestOrd");
Aggregation.replaceRoot("latestOrd");
Spring generates Mongo query similar to one that I've provided above except $max accumulator:
{ "$max" : "$$docs.bookingVersion" }
for some reason it adds double dollar sign instead of single dollar sign and as a result I have following error:
'Use of undefined variable: docs' on server 127.0.0.1:27017
I'm using spring-boot-starter 2.1.0.RELEASE and 4.2 version of Mongo server.
I appreciate any help on this.
Input documents:
[
{
"_id": "5f847811ebcd1a51a0196736",
"status": "REJECTED",
"bookingId": "1",
"bookingVersion": 4,
"operation": "CREATE"
},
{
"_id": "5f847811ebcd1a51a0196735",
"status": "CREATED",
"bookingId": "1",
"bookingVersion": 4,
"docNumber": "7",
"operation": "CREATE"
},
{
"_id": "5f847811ebcd1a51a0196734",
"status": "CREATED",
"bookingId": "1",
"bookingVersion": 3,
"docNumber": "6",
"operation": "CREATE"
},
{
"_id": "5f847811ebcd1a51a0196738",
"status": "CREATED",
"bookingId": "2",
"bookingVersion": 1,
"docNumber": "8",
"operation": "CREATE"
},
{
"_id": "5f847811ebcd1a51a0196737",
"status": "CREATED",
"bookingId": "2",
"bookingVersion": 2,
"docNumber": "9",
"operation": "CREATE"
}
]
Expected output:
[
{
"_id": "5f847811ebcd1a51a0196736",
"status": "REJECTED",
"bookingId": "1",
"bookingVersion": 4,
"operation": "CREATE"
},
{
"_id": "5f847811ebcd1a51a0196735",
"status": "CREATED",
"bookingId": "1",
"bookingVersion": 4,
"docNumber": "7",
"operation": "CREATE"
},
{
"_id": "5f847811ebcd1a51a0196737",
"status": "CREATED",
"bookingId": "2",
"bookingVersion": 2,
"docNumber": "9",
"operation": "CREATE"
}
]
Issue might be with 2.1.0.RELEASE version of spring boot but upgrading it is not an option for me.
To make this work I've created AggregationExpression instead of AccumulatorOperators.Max.maxOf("docs.bookingVersion") part.
Here is a method for it:
private AggregationExpression buildMaxExpression() {
return new AggregationExpression() {
#Override
public Document toDocument(final AggregationOperationContext context) {
return new Document("$max", "$docs.bookingVersion");
}
};
}
I am currently in the process of attempting to update an ElasticSearch document via the Java API. I have a groovy script with the following code:
static updateRequestById(String agencyIndex, String type, String id, def policy) {
UpdateRequest updateRequest = new UpdateRequest()
updateRequest.docAsUpsert(true);
updateRequest.parent("agentNumber");
updateRequest.index(agencyIndex)
updateRequest.type(type)
updateRequest.id(id)
updateRequest.doc("policies", policy)
elasticsearchClient.update(updateRequest).get()
}
The problem with I am having is that I want to update an array within the following document:
{
"took": 4,
"timed_out": false,
"_shards": {
"total": 10,
"successful": 10,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "int-b-agency",
"_type": "jacket",
"_id": "99808.1.27.09_4644",
"_score": 1,
"_source": {
"agentNumber": "99808.1.27.09",
"fileNumber": "4644",
"policies": [
{
"agentNumber": "99808.1.27.09",
"fileNumber": "4644",
"policyNumber": "2730609-91029084",
"checkNumber": "0",
"checkAmount": 0,
"createdOn": null,
"createdBy": "traxuser621",
"propertyTypeCode": "",
"propertyTypeDesc": "1-4 FAMILY RESIDENTIAL",
"ppaddress": "110 Allan Ct ",
"ppcity": "Jacksonville",
"ppstate": "FL",
"ppzip": "32226",
"ppcounty": "Duval",
"policytype": "",
"status": "Active",
"effectiveDate": "2015-04-01T00:00:00-05:00",
"formType": "BASIC OWNERS - ALTA Owners Policy 06_306_FL - FL Original Rate",
"rateCode": "FLOR",
"rateCodeDesc": "FL Original Rate",
"policyTypeCode": "1",
"policyTypeCodeDesc": "BASIC OWNERS",
"amount": 200000,
"hoiAgentNumber": "",
"proForma": false,
"pdfLocation": "\\\\10.212.61.206\\FNFCenter\\legacy_jacket_pdfs\\2015_4_FL6465\\Policy_2730609-91029084.pdf",
"legacyPolicy": "true",
"associatedPolNbr": null
}
]
}
}
]
}
}
In the document above I have a document that has an array called "policies" with a single object. I want to be able to update the "policies" array with additional objects. The end result should look something like the following:
{
"took": 4,
"timed_out": false,
"_shards": {
"total": 10,
"successful": 10,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "int-b-agency",
"_type": "jacket",
"_id": "41341.1.81.38_41340103",
"_score": 1,
"_source": {
"agentNumber": "41341.1.81.38",
"fileNumber": "41340103",
"policies": [
{
"agentNumber": "41341.1.81.38",
"fileNumber": "41340103",
"policyNumber": "8122638-91036874",
"checkNumber": "0",
"checkAmount": 0,
"createdOn": null,
"createdBy": "traxuser621",
"propertyTypeCode": "",
"propertyTypeDesc": "1-4 FAMILY RESIDENTIAL",
"ppaddress": "1800 Smith St ",
"ppcity": "sicklerville",
"ppstate": "PA",
"ppzip": "08105",
"ppcounty": "Dauphin",
"policytype": "",
"status": "Active",
"effectiveDate": "2016-02-01T00:00:00-06:00",
"formType": "TestData",
"rateCode": "PASALERATE",
"rateCodeDesc": "Sale Rate - Agent",
"policyTypeCode": "26",
"policyTypeCodeDesc": "SALE OWNERS",
"amount": 180000,
"hoiAgentNumber": "",
"proForma": false,
"pdfLocation": "SomeLocation1",
"legacyPolicy": "true",
"associatedPolNbr": null
},
{
"agentNumber": "41341.1.81.38",
"fileNumber": "41340103",
"policyNumber": "8122638-91036875",
"checkNumber": "0",
"checkAmount": 0,
"createdOn": null,
"createdBy": "traxuser621",
"propertyTypeCode": "",
"propertyTypeDesc": "1-4 FAMILY RESIDENTIAL",
"ppaddress": "1800 Smith St ",
"ppcity": "sicklerville",
"ppstate": "PA",
"ppzip": "08105",
"ppcounty": "Dauphin",
"policytype": "",
"status": "Active",
"effectiveDate": "2016-02-01T00:00:00-06:00",
"formType": "Test Data",
"rateCode": "PASALERATE",
"rateCodeDesc": "Sale Rate - Agent",
"policyTypeCode": "26",
"policyTypeCodeDesc": "SALE OWNERS",
"amount": 180000,
"hoiAgentNumber": "",
"proForma": false,
"pdfLocation": "SomeLocation2",
"legacyPolicy": "true",
"associatedPolNbr": null
}
]
}
}
]
}
}
What am I doing wrong?
You can use a scripted update:
Put your new policy in a parameter, for example policy
Use a script like the following :
if (!ctxt._source.policies) { ctxt._source.policies = [] }
ctxt._source.policies += policy
See this documentation : https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html
Updates in inverted indexes are deletes and replacements of documents. There is no in-place update like you find in a db. ES uses Lucene under the hood which in-turn implements a kick-ass inverted index.
I am trying to find a person by first or last name all having the same building_id.
For example someone will put in a name in a textfield which may be the first or last name. They'll all have the same building id. I cannot seem to find this in the documentation for elastic search.
My result data looks like this:
{
took: 9,
timed_out: false,
_shards: {
total: 5,
successful: 5,
failed: 0
},
hits: {
total: 56,
max_score: 1,
hits: [
{
_index: "jdbc",
_type: "jdbc",
_id: "5",
_score: 1,
_source: {
first_name: "some first name",
last_name: "some last name",
building_id: 1
}
},
...
...
One way is to use a term filter with a multi_match query. To illustrate, first create some docs:
curl -XPUT "http://localhost:9200/test_index"
curl -XPUT "http://localhost:9200/test_index/person/1" -d'
{
"first_name": "Bob",
"last_name": "Jones",
"building_id": 1
}'
curl -XPUT "http://localhost:9200/test_index/person/2" -d'
{
"first_name": "Bill",
"last_name": "Smith",
"building_id": 1
}'
curl -XPUT "http://localhost:9200/test_index/person/3" -d'
{
"first_name": "Joe",
"last_name": "Williams",
"building_id": 2
}'
curl -XPUT "http://localhost:9200/test_index/person/4" -d'
{
"first_name": "John",
"last_name": "Taylor",
"building_id": 2
}'
curl -XPUT "http://localhost:9200/test_index/person/5" -d'
{
"first_name": "Taylor",
"last_name": "Johnson",
"building_id": 2
}'
Then, if you want to search for all the "Taylor"s in Building 2, execute the following:
curl -XPOST "http://localhost:9200/test_index/person/_search" -d'
{
"query": {
"filtered": {
"query": {
"multi_match": {
"query": "taylor",
"fields": [
"first_name",
"last_name"
]
}
},
"filter": {
"term": {
"building_id": 2
}
}
}
}
}'
and you will get back the results:
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 2,
"successful": 2,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 0.94125634,
"hits": [
{
"_index": "test_index",
"_type": "person",
"_id": "5",
"_score": 0.94125634,
"_source": {
"first_name": "Taylor",
"last_name": "Johnson",
"building_id": 2
}
},
{
"_index": "test_index",
"_type": "person",
"_id": "4",
"_score": 0.5906161,
"_source": {
"first_name": "John",
"last_name": "Taylor",
"building_id": 2
}
}
]
}
}
Here is a runnable example (you will need Elasticsearch installed on your machine and running at localhost:9200):
http://be6c2e3260c3e2af000.qbox.io/gist/cc47b232ec60f09b93906a5c80028c1317dbdf28