We Keep Coding

Groovy DSL Spring cloud contract throwing java.lang.IllegalStateException for query parameter having Unicode Characters

I created a groovy DSL contract like below
import org.springframework.cloud.contract.spec.Contract
Contract.make {
final def NAME_REGEX = '[A-Za-z0-9\\u00C0-\\u00FF\'\\- ]{1,70}'
request {
method 'GET'
url('/api/getEmployess') {
queryParameters {
parameter 'name': $(c(regex(NAME_REGEX)), p('\u00CAdward J\u00F5hnson'))
}
}
headers {
contentType("application/json;charset=UTF-8")
}
}
response {
status OK()
body([
[
id : $(p(regex(nonBlank())), c('5a0eaf2012a9a12f1c98947a')),
name : fromRequest().query("name")
]
])
headers { contentType("application/json;charset=UTF-8") }
}
}
My service implementation returns 'name' and 'id' in response. In response, 'name' is Unicode value 'Êdward Jõhnson' which fails to match with the request parameter value.
I am getting below error -
Parsed JSON [[{"id":"5a0eaf2012a9a12f1c98947a","name":"Êdward Jõhnson"}]] doesn't match the JSON path [$[*][?(#.['name'] == 'Êdward Jõhnson')]]
java.lang.IllegalStateException: Parsed JSON [[{"id":"5a0eaf2012a9a12f1c98947a","name":"Êdward Jõhnson" }]] doesn't match the JSON path [$[*][?(#.['name'] == 'Êdward Jõhnson')]]
at com.toomuchcoding.jsonassert.JsonAsserter.check(JsonAsserter.java:228)
at com.toomuchcoding.jsonassert.JsonAsserter.checkBufferedJsonPathString(JsonAsserter.java:267)
I tried to pass the Unicode value in two ways in 'name' request query param -
Putting Unicode characters as Unicode numbers like in the above example -
parameter 'name': $(c(regex(NAME_REGEX)), p('\u00CAdward J\u00F5hnson'))
Putting Unicode characters as it is
parameter 'name': $(c(regex(NAME_REGEX)), p('Êdward Jõhnson'))
But for both cases, I am getting the same error. There looks some encoding issue because my value Êdward Jõhnson is changed to Êdward Jõhnson as mentioned in error.
Please help me to resolve this issue.

I found a workaround for this. In response 'name' field producer I put the same value which was in the request 'name' field producer. It was failing due to different encoding applied by groovy on unicode value. It's just a workaround to fix the problem, not a proper final solution.
import org.springframework.cloud.contract.spec.Contract
Contract.make {
final def NAME_REGEX = '^[A-Za-z0-9À-ÿ'\-\s]{1,70}$'
request {
method 'GET'
url('/api/getEmployess') {
queryParameters {
parameter 'name': $(c(regex(NAME_REGEX)), p('Êdward Jõhnson'))
}
}
headers {
contentType("application/json;charset=UTF-8")
}
}
response {
status OK()
body([
[
id : $(p(regex(nonBlank())), c('4b0eaf2012a9a12f1c98567c')),
name : $(p("Êdward Jõhnson"), c(fromRequest().body('$.name'))),
]
])
headers { contentType("application/json;charset=UTF-8") }
}
}

Using rest-assured get value using path method

I want to extract value of s.d.url from the below JSON.
I am using the below satement as shown using System.out.println
but I dont get the result. How do I do it when the field itself contains "."
JSON
{
"data":{
"H1":{
"com.abc.def":{
"a_enabled":false,
"b_config":true
},
"c.s.urls":{
"s.d.url":"https://url1.com",
"w.p.url":"https://url2.com",
"s.c.url":"https://url3.com"
},
"com.abc.con":{
"e_n":true,
"a_r":false,
"c_t":"XYZMB"
}
},
"dCId":"ABCD"
}
}
ExtractableResponse<Response> spec = given()
.request().log().all()
.expect().statusCode(200)
.when()
.get(EndpointsCloudServices.getConfigUrl() + "?" + params)
.then().log().body()
.extract();
//want to get value of s.d.url
System.out.println("Triage???? " + spec.path("data.H1.c.s.urls.s.d.url"));

Give a try for the following, it will return value of s.d.url (pay attention for square brackets and single quotes):
spec.path("data.H1.['c.s.urls'].['s.d.url']")
or even shorter, if you're sure that s.d.url is an unique name across the whole json document:
spec.path("$.['s.d.url']")
And next, this is just to illustrate common case of referring field which contains dots in its name by using JSONPath expression - all you need is to wrap the field name in [' and ']
Example JSON:
{
"field.name": "value",
"nested": {
"field.with.dot": {
"field.inside": "anotherValue"
}
}
}
Example valid JSONPath expressions to access corresponding field values:
$['field.name']
$.['field.inside']
nested.['field.with.dot'].['field.inside']
Hint: you can quickly test your JSONPaths agains your json using tools like online evaluator or expression tester

Elasticsearch custom plugin: Add extra query parameter before search

I'm new with Elasticsearch, so don't know how to start properly with the following task.
I have an index with documents that contain 2 types of fields:
address: the string including city and street;
houses: the list of houses' numbers (integers).
In usual case I could search this documents by followin query:
(1)
GET /_search
{
"query":{
"bool":{
"should": [
{"match": {"address": "Gotham Fourteenth street"}},
{"match": {"houses": 33}}
]
}
}
}
My goal is to match such records by single string, like:
(2)
GET /_search
{
"query":{
"bool":{
"should": [
{"match": {"address": "Gotham Fourteenth street 33"}}
]
}
}
}
or even:
curl -X GET 'http://localhost:9200/_search?q=Gotham+Fourteenth+street+33'
i.e. convert the query (2) to (1), that is cut the house number '33' from 'address' and put it as 'houses' match-parameter to the same query before search performed.
I think I could create a plugin in Java that would extract house number from 'address' (parsing is not a problem) and add an extra parameter 'houses' with this value.
Thus my problem is:
how in my plugin I can programmatically add an extra match-parameter 'houses' to my query before search performed?
how to cut the trailing house-number token from 'address' parameter?

curl -X GET 'http://localhost:9200/_search?q=address:"Gotham Fourteenth street"
curl -X GET 'http://localhost:9200/_search?q=address:"Gotham Fourteenth street" AND (houses=13)
Also checkout wildcards

In the end I've developed a plugin for ElasticSearch 2.4.2 and added a REST-action class (derived from BaseRestHandler) with the following handleRequest() method:
#Override
protected void handleRequest(RestRequest request, RestChannel channel, final Client client) throws Exception {
// Get the address parameter
String address = request.param("address");
// ... Code that parses address and extracts house number ...
int house = ...
// Now send both parameters as query to search engine
SearchResponse sr = client
.prepareSearch("myindex")
.setTypes("mytype")
// Query all shards, DFS==calculate global words' frequencies
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
// Address string without (cutted) house number
.setQuery(QueryBuilders.matchQuery("address", address))
// And extracted house number as second filtering parameter
.setPostFilter(QueryBuilders.termQuery("houses", house))
// Starting position of the first returning hit
.setFrom(0)
// Max number of hits
.setSize(10)
// Explain hit score
.setExplain(true)
.get();
// Convert the search response to rest response and send it
BytesRestResponse sr_rest = search2restResponse(sr);
channel.sendResponse(sr_rest);
}
}
The method search2restResponse() mentioned above converts SearchResponse to REST-response and looks as following:
private BytesRestResponse search2restResponse(SearchResponse sr) {
SearchHit[] searchHits = sr.getHits().getHits();
StringBuilder builder = new StringBuilder();
builder.append("[");
for (int i = 0; i < searchHits.length; i++) {
if (i > 0) {
builder.append(",");
}
builder.append(searchHits[i].getSourceAsString());
}
builder.append("]");
String res_json = builder.toString();
return new BytesRestResponse(RestStatus.OK, res_json);
}

check nested key existence in video object in youtube-api to avoid NULLPOINTEREXCEPTION

I want to check the existence of nested key in Video object returned as a Json response from youtube video search by using below code:-
YouTube.Videos.List searchvideostats = youtube.videos().list("snippet,statistics");
searchvideostats.setKey(apiKey);
Video v = searchvideostats.execute().getItems().get(0);
System.out.println(v.toPrettyString());
I got output like this:-
{
"etag" : "\"m2yskBQFythfE4irbTIeOgYYfBU/-TONXAYMx_10Caihcoac4XCyb4I\"",
"id" : "4Aa9GwWaRv0",
"kind" : "youtube#video",
"snippet" : {
"categoryId" : "10",
...................
.........
MY goal is:- how to check whether categoryId key is present in this response or not. coz if do v.getSnippet().getCategoryId() it gives NullPointerException if categotyId is not present in Json.
Tried:-
if (v.containsKey("id")) {
System.out.println("contains");
} else {
System.out.println("doesnt contains");
}
this returns contains as expected.
if (v.containsKey("categoryId")) {
System.out.println("contains");
} else {
System.out.println("doesnt contains");
}
This returns doesnt contains.. which is not expected. How would I check if this nested key is available?
P.S. -> I have to check many nested such keys.
Thanks for help.

You don't need String manipulations. Just use Youtube library.
if(video.getSnippet()!=null && video.getSnippet().getCategoryId()!=null)
will do the stuff.
note: checking for zero length categoryid might be necessary also.

regex match patern before and after a underscore

I have a file name convention {referenceId}_{flavor name}.mp4 in kaltura.
or if you are familiar with kaltura then tell me the slugRegex i could use for this naming convention that would support pre-encoded file ingestion
I have to extract referenceId and filename from it.
I'm using
/(?P)_(?P)[.]\w{3,}/

var filename = "referenceId_flavor-name.mp4";
var parts = filename.match(/([^_]+)_([^.]+)\.(\w{3})/i);
// parts is an array with 4 elements
// ["referenceId_flavor-name.mp4", "referenceId", "flavor-name", "mp4];

var file = 'refID_name.mp4',
parts = file.match(/^([^_]+)_(.+)\.mp4/, file);
Returns array:
[
'refID_name.mp4', //the whole match is always match 0
'refID', //sub-match 1
'name' //sub-match 2
]

/**
* Parse file name according to defined slugRegex and set the extracted parsedSlug and parsedFlavor.
* The following expressions are currently recognized and used:
* - (?P<referenceId>\w+) - will be used as the drop folder file's parsed slug.
* - (?P<flavorName>\w+) - will be used as the drop folder file's parsed flavor.
* - (?P<userId>\[\w\#\.]+) - will be used as the drop folder file entry's parsed user id.
* #return bool true if file name matches the slugRegex or false otherwise
*/
private function parseRegex(DropFolderContentFileHandlerConfig $fileHandlerConfig, $fileName, &$parsedSlug, &$parsedFlavor, &$parsedUserId)
{
$matches = null;
$slugRegex = $fileHandlerConfig->getSlugRegex();
if(is_null($slugRegex) || empty($slugRegex))
{
$slugRegex = self::DEFAULT_SLUG_REGEX;
}
$matchFound = preg_match($slugRegex, $fileName, $matches);
KalturaLog::debug('slug regex: ' . $slugRegex . ' file name:' . $fileName);
if ($matchFound)
{
$parsedSlug = isset($matches[self::REFERENCE_ID_WILDCARD]) ? $matches[self::REFERENCE_ID_WILDCARD] : null;
$parsedFlavor = isset($matches[self::FLAVOR_NAME_WILDCARD]) ? $matches[self::FLAVOR_NAME_WILDCARD] : null;
$parsedUserId = isset($matches[self::USER_ID_WILDCARD]) ? $matches[self::USER_ID_WILDCARD] : null;
KalturaLog::debug('Parsed slug ['.$parsedSlug.'], Parsed flavor ['.$parsedFlavor.'], parsed user id ['. $parsedUserId .']');
}
if(!$parsedSlug)
$matchFound = false;
return $matchFound;
}
is the code that deals with the regex. I used /(?P<referenceId>.+)_(?P<flavorName>.+)[.]\w{3,}/ and following this tutorial enter link description here