We Keep Coding

Liquid Template nested expression with special characters in the path

Let's say I have the following map:
{
context: {
'parent-name': { /* data */ },
'parent-name/http': { /* data */ },
}
self: {
id: 'parent-name'
}
}
Now I want to reference the parent-name/http but itself as a variable. I wrote the following expression:
{{context[self.id/http]}}
Using the https://github.com/bkiers/Liqp Java SDK, this throws an error:
parser error "mismatched input '/' expecting {'.', NEq, '==', '>=',
'>', '<=', '<', '[', ']', '?', 'contains', 'and', 'or'}" on line 1,
index 24
How do I write such expression?

AFAIK, you can't do that in one go. You'll first have to create the key, by using | append: "/http", and then retrieve the value with this key:
String json = "{\n" +
" \"context\": {\n" +
" \"parent-name\": \"mu\",\n" +
" \"parent-name/http\": \"foo\"\n" +
" }," +
" \"self\": {\n" +
" \"id\": \"parent-name\"\n" +
" }\n" +
"}";
Template t = Template.parse(
"{% assign key = self.id | append: \"/http\" %}" +
"{{context[key]}}");
System.out.println(t.render(json));

Is it possible to create a Mongo view using spring-data-mongo 1.10?

I have a simple requirement to be able to create a Mongo view from my Java app. We're using the 3.4 Mongo driver and spring-data-mongo 1.10.2. The Mongo docs for db.createCollection indicate you create a view by including viewOn in the options, but the CollectionOptions class that is used by mongoTemplate.createCollection doesn't have this property.
I've dug through source code up to s-d-m version 2.2 and still don't see it supported. How can I create a view?

I was able to get this working. Below is the method:
private void createView(BaseEntity model, String viewName, String viewDefinition) {
// Get the model's #Document annotation so we can determine its collection
Document doc = model.getClass().getAnnotation(Document.class);
Assert.notNull(doc, "Error - #Document annotation is null for model class: " + model.getClass().getSimpleName());
// Attempt to create the view
CommandResult result = mongoTemplate.executeCommand("{" +
"create: '" + viewName + "', " +
"viewOn: '" + doc.collection() + "', " +
"pipeline: [{$match: " + viewDefinition + "}]" +
"}");
if(result.ok()) {
LOGGER.info("Successfully created view '{}' on collection '{}'", viewName, doc.collection());
}
else {
throw new ViewCreationBeanException(
"Failed to create view '" + viewName + "' on collection '" + doc.collection() + "' - " + result.getErrorMessage(),
result.getException()
);
}
}
model is the Java class with the #Document annotation that specifies the mongo collection. What I was doing here was creating a view on a model based on its underlying collection. viewDefinition are the view constraints, a String such as: "{deleted: {$ne: true}}".

how to created REPEATED type in parquet file schema with avro?

We are creating a dataflow pipeline, we will read the data from postgres and write it to a parquet file. ParquetIO.Sink allows you to write a PCollection of GenericRecord into a Parquet file (from here https://beam.apache.org/releases/javadoc/2.5.0/org/apache/beam/sdk/io/parquet/ParquetIO.html). But the parquet file schema is not like what i expected
here is my schema:
schema = new org.apache.avro.Schema.Parser().parse("{\n" +
" \"type\": \"record\",\n" +
" \"namespace\": \"com.example\",\n" +
" \"name\": \"Patterns\",\n" +
" \"fields\": [\n" +
" { \"name\": \"id\", \"type\": \"string\" },\n" +
" { \"name\": \"name\", \"type\": \"string\" },\n" +
" { \"name\": \"createdAt\", \"type\": {\"type\":\"string\",\"logicalType\":\"timestamps-millis\"} },\n" +
" { \"name\": \"updatedAt\", \"type\": {\"type\":\"string\",\"logicalType\":\"timestamps-millis\"} },\n" +
" { \"name\": \"steps\", \"type\": [\"null\",{\"type\":\"array\",\"items\":{\"type\":\"string\",\"name\":\"json\"}}] },\n" +
" ]\n" +
"}");
this is my code so far:
Pipeline p = Pipeline.create(
PipelineOptionsFactory.fromArgs(args).withValidation().create());
p.apply(JdbcIO.<GenericRecord> read()
.withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
"org.postgresql.Driver", "jdbc:postgresql://localhost:port/database")
.withUsername("username")
.withPassword("password"))
.withQuery("select * from table limit(10)")
.withCoder(AvroCoder.of(schema))
.withRowMapper((JdbcIO.RowMapper<GenericRecord>) resultSet -> {
GenericRecord record = new GenericData.Record(schema);
ResultSetMetaData metadata = resultSet.getMetaData();
int columnsNumber = metadata.getColumnCount();
for(int i=0; i<columnsNumber; i++) {
Object columnValue = resultSet.getObject(i+1);
if(columnValue instanceof UUID) columnValue=columnValue.toString();
if(columnValue instanceof Timestamp) columnValue=columnValue.toString();
if(columnValue instanceof PgArray) {
Object[] array = (Object[]) ((PgArray) columnValue).getArray();
List list=new ArrayList();
for (Object d : array) {
if(d instanceof PGobject) {
list.add(((PGobject) d).getValue());
}
}
columnValue = list;
}
record.put(i, columnValue);
}
return record;
}))
.apply(FileIO.<GenericRecord>write()
.via(ParquetIO.sink(schema).withCompressionCodec(CompressionCodecName.SNAPPY))
.to("something.parquet")
);
p.run();
this is what i get:
message com.example.table {
required binary id (UTF8);
required binary name (UTF8);
required binary createdAt (UTF8);
required binary updatedAt (UTF8);
optional group someArray (LIST) {
repeated binary array (UTF8);
}
}
this is what i expected:
message com.example.table {
required binary id (UTF8);
required binary name (UTF8);
required binary createdAt (UTF8);
required binary updatedAt (UTF8);
optional repeated binary someArray(UTF8);
}
please help

I did not find a way to create a repeated element from Avro that isn't in a GroupType.
The ParquetIO in Beam uses a "standard" avro conversion defined in the parquet-mr project, which is implemented here.
It appears that there are two ways to turn an Avro ARRAY field to a Parquet message -- but neither of them create what you are looking for.
Currently, the avro conversion is the only way to interact with ParquetIO at the moment. I saw this JIRA Use Beam schema in ParquetIO that extend this to Beam Rows, which might permit a different parquet message strategy.
Alternatively, you could create a JIRA feature request for ParquetIO to support thrift structures, which should allow finer control over the parquet structure.

Is it a protobuf message you used to describe the expected schema? I think what you got is correctly generated from the specified JSON schema. optional repeated does not make sense in the protobuf language specification: https://developers.google.com/protocol-buffers/docs/reference/proto2-spec
You can remove null and square bracket to generate simply repeated field and it's semantically equivalent to optional repeated (since repeated means zero or more times).

Bean reference is coming as null

I have a context config file like so :
applicationContect.xml:
<bean id="CalcValidate" class="com.fmr.pmsvcs.orders.fixture.utility.CalcValidate" >
<property name="orderRestServiceL" >
<ref local="OrderRestService" />
</property>
</bean>
And my class looks like bellow :
CalcValidate.java
public class CalcValidate {
public static OrderRestService orderRestServiceL;
public static OrderRestService getOrderRestServiceL() {
return orderRestServiceL;
}
public static void setOrderRestServiceL(OrderRestService orderRestServiceL) {
CalcValidate.orderRestServiceL = orderRestServiceL;
}
public static String getNetAssets(String user, BigInteger fundid,
Order CalcOrder) throws Exception {
PortfolioReferenceParameter par =
orderRestServiceL.netAssets(user, fundid);
if (par.getPortfolios().get(0) == null
&& CalcOrder.getPortfolioTna() == null
&& CalcOrder.getPortfolioTnaUsd() == null) {
System.out
.println(" ##### PASS Portfolio Net Asset are null in service and DB");
return OrderFixtureConstants.TRUE;
}
// *** Validate against Net Asset in Fund Base Currency
if (!par.getPortfolios().get(0).getTotalNetAssets()
.equals(CalcOrder.getPortfolioTna())) {
return ("FAIL net Asset in response ["
+ CalcOrder.getPortfolioTna()
+ " ] doesn't match net Asset in DB ["
+ par.getPortfolios().get(0).getTotalNetAssets() + " ]");
}
System.out.println(" ##### PASS net Asset in response ["
+ CalcOrder.getPortfolioTna()
+ " ] does match net Asset in DB ["
+ par.getPortfolios().get(0).getTotalNetAssets() + " ]");
// *** Validate against Net Asset in Fund Base Currency
if (!par.getPortfolios().get(0).getTotalNetAssetsUSD()
.equals(CalcOrder.getPortfolioTnaUsd())) {
return ("FAIL net Asset USD in response ["
+ CalcOrder.getPortfolioTnaUsd()
+ " ] doesn't match net Asset in DB ["
+ par.getPortfolios().get(0).getTotalNetAssetsUSD() + " ]");
}
System.out.println(" ##### PASS net Asset in response ["
+ CalcOrder.getPortfolioTnaUsd()
+ " ] does match net Asset in DB ["
+ par.getPortfolios().get(0).getTotalNetAssetsUSD() + " ]");
return OrderFixtureConstants.TRUE;
}
}
In later part of my class I am calling a method like orderRestServiceL.getMethod();
Here "orderRestServiceL" coming as null. Anybody has any Idea how to resolve this?

Your issue seems to be related to the Java class field.
it is declared as static while it should be non-static:
public class CalcValidate {
private OrderRestService orderRestServiceL;
public OrderRestService getOrderRestServiceL() {
return orderRestServiceL;
}
public void setOrderRestServiceL(OrderRestService orderRestServiceL) {
this.orderRestServiceL = orderRestServiceL;
}

PlayFramework and jquery chart

i'm trying to use https://github.com/thegrubbsian/jquery.ganttView
in play 1.2.4
already check JQuery GanttChart - Apply Data
but my problem is how to renderJSON to the view,
i have try a simple one that is renderJSON("Hello"); and capture in the view, the problem is that i only manage to download a file like 4fporLKs.part1 that have inside Hello :(
can some one explain me how to do it
thanks

other guy is working in the same as i, but haven't got it
<script type="text/javascript">
$(function ()
{
$("#ganttChart").ganttView({
data : 'dataUrl: "data.json"',
slideWidth: 900,
behavior: {
onClick: function (data) {
var msg = "You clicked on an event: { start: " + data.start.toString("M/d/yyyy") + ", end: " + data.end.toString("M/d/yyyy") + " }";
$("#eventMessage").text(msg);
},
onResize: function (data) {
var msg = "You resized an event: { start: " + data.start.toString("M/d/yyyy") + ", end: " + data.end.toString("M/d/yyyy") + " }";
$("#eventMessage").text(msg);
},
onDrag: function (data) {
var msg = "You dragged an event: { start: " + data.start.toString("M/d/yyyy") + ", end: " + data.end.toString("M/d/yyyy") + " }";
$("#eventMessage").text(msg);
}
}
});
//$("#ganttChart").ganttView("setSlideWidth", 600);
});
</script>

"hello" is not well formated json string, try:
renderJSON("{\"hello\":\"world\"}");
Also check Play's documentation for controllers to see how render ie. json from List (paragraph: Return a JSON String)