I'm trying to DRY my code and I used for that, for the first time, traits to enhance my enums.
What I want to do, is : for a given array of strings, find all the enums matching at least one keyword (non case sensitive)
The code below seems to works fine, but I think it generates me memory leaks when the method getSymbolFromIndustries is called thousands of times.
Here is a capture from VisualVM after about 10 minutes of run, the column Live Objects is always increasing after each snapshot and the number of items compared to the second line is so huge...
My heap size is always increasing too...
The trait :
trait BasedOnCategories {
String[] categories
static getSymbolFromIndustries(Collection<String> candidates) {
values().findAll {
value -> !value.categories.findAll {
categorie -> candidates.any {
candidate -> categorie.equalsIgnoreCase(candidate)
}
}
.unique()
.isEmpty()
}
}
}
One of the multiple enums I have implementing the trait
enum KTC implements BasedOnCategories, BasedOnValues {
KTC_01([
'industries': ['Artificial Intelligence','Machine Learning','Intelligent Systems','Natural Language Processing','Predictive Analytics','Google Glass','Image Recognition', 'Apps' ],
'keywords': ['AI','Voice recognition']
]),
// ... more values
KTC_43 ([
'industries': ['Fuel','Oil and Gas','Fossil Fuels'],
'keywords': ['Petroleum','Oil','Petrochemicals','Hydrocarbon','Refining']
]),
// ... more values
KTC_60([
'industries': ['App Discovery','Apps','Consumer Applications','Enterprise Applications','Mobile Apps','Reading Apps','Web Apps','App Marketing','Application Performance Management', 'Apps' ],
'keywords': ['App','Application']
])
KTC(value) {
this.categories = value.industries
this.keywords = value.keywords
}
My data-driven tests
def "GetKTCsFromIndustries"(Collection<String> actual, Enum[] expected) {
expect:
assert expected == KTC.getSymbolFromIndustries(actual)
where:
actual | expected
[ 'Oil and Gas' ] | [KTC.KTC_43]
[ 'oil and gas' ] | [KTC.KTC_43]
[ 'oil and gas', 'Fossil Fuels' ] | [KTC.KTC_43]
[ 'oil and gas', 'Natural Language Processing' ] | [KTC.KTC_01, KTC.KTC_43]
[ 'apps' ] | [KTC.KTC_01, KTC.KTC_60]
[ 'xyo' ] | []
}
My questions :
If someone have some clues to help me fix those leaks...
Is there a more elegant way to write the getSymbolFromIndustries method ?
thanks.
Not sure about performance issues, but I would redesign your trait like that:
https://groovyconsole.appspot.com/script/5205045624700928
trait BasedOnCategories {
Set<String> categories
void setCategories( Collection<String> cats ) {
categories = new HashSet( cats*.toLowerCase() ).asImmutable()
}
#groovy.transform.Memoized
static getSymbolFromIndustries(Collection<String> candidates) {
def lowers = candidates*.toLowerCase()
values().findAll{ value -> !lowers.disjoint( value.categories ) }
}
}
Now the rest of the context
trait BasedOnValues {
Set<String> keywords
}
enum KTC implements BasedOnCategories, BasedOnValues {
KTC_01([
'industries': ['Artificial Intelligence','Machine Learning','Intelligent Systems','Natural Language Processing','Predictive Analytics','Google Glass','Image Recognition'],
'keywords': ['AI','Voice recognition']
]),
// ... more values
KTC_43 ([
'industries': ['Fuel','Oil and Gas','Fossil Fuels'],
'keywords': ['Petroleum','Oil','Petrochemicals','Hydrocarbon','Refining']
]),
// ... more values
KTC_60([
'industries': ['App Discovery','Apps','Consumer Applications','Enterprise Applications','Mobile Apps','Reading Apps','Web Apps','App Marketing','Application Performance Management'],
'keywords': ['App','Application']
])
KTC(value) {
this.categories = value.industries
this.keywords = value.keywords
}
}
// some tests
[
[ [ 'Oil and Gas' ], [KTC.KTC_43] ],
[ [ 'oil and gas' ], [KTC.KTC_43] ],
[ [ 'oil and gas', 'Fossil Fuels' ], [KTC.KTC_43] ],
[ [ 'oil and gas', 'Natural Language Processing' ], [KTC.KTC_01, KTC.KTC_43] ],
[ [ 'xyo' ], [] ],
].each{
assert KTC.getSymbolFromIndustries( it[ 0 ] ) == it[ 1 ]
}
and then measure the performance
I am trying to validate a JSON which has multiple JSON objects nested.
example
Scenario: temp1
* def response1 =
"""
{
"productGroups": [
{
"dateLabel": "28 Aug, Wed",
"products": [
{
"id": 1439,
"product": "product 1"
},
{
"id": 1401,
"product": "product 2"
}
]
}
]
}
"""
* print response1.productGroups
Then match response1.productGroups[*] contains
"""
{
'dateLabel': #string,
'products': [
{
'id': #number,
'product': #string
}
]
}
"""
Getting the response as
reason: actual value does not contain expected
if I change the validate as
Then match response1.productGroups[0] contains
Getting the response as
reason: actual and expected arrays are not the same size - 2:1
What I wanted to do is verify the schema of "productGroups" object along with the inner objects of "products"
Please spend some time reading the docs, it is worth it: https://github.com/intuit/karate#schema-validation
* def product = { 'id': #number, 'product': #string }
Then match response1.productGroups[*] contains
"""
{
'dateLabel': #string,
'products': '#[] product'
}
"""
I need to compare two json strings by ignore some fields
I am currently using JSONAssert from org.SkyScreamer to do the comparison but does not ignore any attributes.
Json 1 :
{
"contributions": [
[
{
"order" : 1,
"contributorId" : "1980"
}
]
]
}
Json 2 :
{
"contributions": [
[
{
"order": 1,
"contributorId" : "5789"
}
]
]
}
ArrayValueMatcher<Object> arrValMatch1 = new ArrayValueMatcher<>(new CustomComparator(
JSONCompareMode.NON_EXTENSIBLE,
new Customization("contributions[0][*].contributorId",(o1, o2) -> true)));
Customization arrayValueMatchCustomization = new Customization("contributions", arrValMatch1);
CustomComparator customArrayValueComparator = new CustomComparator(
JSONCompareMode.NON_EXTENSIBLE,
arrayValueMatchCustomization);
assertEquals(subJson1, json2, customArrayValueComparator);
I expect the above scenario should be passed. But its failing with
Exception in thread "main" java.lang.AssertionError: contributions[0][contributorId=1980]
Expected: a JSON object
but none found
; contributions[0][contributorId=5789]
Unexpected: a JSON object
Use *.contributorId instead of contributions[0][*].contributorId
I am building a Java map of two different Json data streams. However if one of the streams contains a null value in the quantity, the flow will error with the following condition:
($.itemid) : '"' ++ skuLookup[$.sku][0].qty as :number as :string {format: "#.0000"} ++ '"'
^
Cannot coerce a :null to a :number
(com.mulesoft.weave.mule.exception.WeaveExecutionException). Message payload is of type: String
The ideal scenario would be ignore the null values within the iteration Here is the dataweave as untouched:
%output application/java
%var skuLookup = flowVars.SSRCreateStarshipItems groupBy $.sku
---
flowVars.SSRGetOrderItems map ({
($.itemid) : '"' ++ skuLookup[$.sku][0].qty as :number as :string
{format: "#.0000"} ++ '"'
})
Here is my ideal solution, however I cant seem to get it to work:
%output application/java
%var skuLookup = flowVars.SSRCreateStarshipItems groupBy $.sku
---
flowVars.SSRGetOrderItems map ({
($.itemid) : '"' ++ skuLookup[$.sku][0].qty as :number as :string
{format: "#.0000"} ++ '"'
}) when skuLookup[$.sku][0].qty != null
Setting the default value resolve your problem.
For example, If you were passing in a payload such as this:
{
"testMap": [
{
"testName": "TestName",
"Value": "3"
},
{
"testName": "TestName",
"Value": null
}
]
}
You could convert the Value key to a number or pass null with this:
%dw 1.0
%input payload application/json
%output application/json
---
payload.testMap map (data, index) -> {
testName: data.testName,
Value: data.Value as :number as :string {format: "#.0000"} default null
}
And this is the result:
[
{
"testName": "TestName",
"Value": "3.0000"
},
{
"testName": "TestName",
"Value": null
}
]
Also, if you want to remove the key completely, insert this after the output header for json:
skipNullOn="everywhere"
I would like to write a Json reader for such Json
{
"headers": [
{
"id": "time:monthly",
"type": "a"
},
{
"id": "Value1",
"type": "b"
},
{
"id": "Value2",
"type": "b"
}
],
"rows": [
[
"2013-01",
4,
5
],
[
"2013-02",
3,
6
]
]
}
I know (thanks to the header) that in the elements of rows the first element is of a type a, the second and the third will be of type b. My goal is to create an object row (List[a],List[b]) (
the number of element of type a and b varies that's why I use List).
My question is how can I parse rows or how can I read a Json array with different type of object and without an id ?
I'd be tempted to setup the model with cases classes and mix the play framework macro base json reader with a custom one for your rows like this.
import play.api.libs.json._
case class Header(id: String, `type`: String)
case class Row(a: String, b: Int, c: Int)
case class Data(headers: Seq[Header], rows: Seq[Row])
object RowReads extends Reads[Row] {
def reads(js: JsValue) = js match {
case JsArray(Seq(a,b,c)) =>
(a,b,c) match {
case (a: JsString, b: JsNumber, c: JsNumber) =>
JsSuccess(Row(a.value,b.value.toInt,c.value.toInt))
case _ => JsError("nope")
}
case _ => JsError("nope")
}
}
object Formats {
implicit val headerReads = Json.reads[Header]
implicit val rowReads = RowReads
implicit val dataReads = Json.reads[Data]
def readIt(js: JsValue) = {
Json.fromJson[Data](js: JsValue)
}
}
For more details.
https://playframework.com/documentation/2.4.x/ScalaJson