Hi there I like to use at Apache Ignite a Pojo which has a HashMap attribute so I can work with dynamic models at runtime. Storing and Saving of such objects works fine.
However, I m wondering if there a way exist to access the key / values of such a Hashmap through a SQL query? If this is not supported any other ways I can work in Apache Ignite with dynamic objects?
POJO Class with dynamic attributes
#Data
public class Item {
private static final AtomicLong ID_GEN = new AtomicLong();
#QuerySqlField(index = true)
private Long id;
#QuerySqlField
public Map<String,Serializable> attributes = new HashMap<String,Serializable>();
public Item(Long id, String code, String name) {
this.id = id;
}
public Item() {
this(ID_GEN.incrementAndGet());
}
public void setAttribute(String name,Serializable value) {
attributes.put(name, value);
}
public Serializable getAttribute(String name) {
return attributes.get(name);
}
}
Example Query Feature illstrated
SqlFieldsQuery query = new SqlFieldsQuery("SELECT * FROM Item WHERE attributes('Price') > 100");
SQL in Ignite is not just syntactic sugar, it requires a schema of your models to be defined before you can run SQL queries and this won't work for a collection. Therefore you need to normalize the data just like with a regular DB or rework the model's structure somehow to avoid JOIN.
Apache Ignite has no support for destructuring/collections in its SQL, so you can't peek inside HashMap via SQL.
However, you may define your own SQL functions, so you can implement e.g. SELECT hashmap_get(ATTRIBUTES, 'subkey') FROM ITEM WHERE ID = ?
But you can't have indexes on function application so the usefulness is limited.
Related
I have a Ignite cache with the two classes below.
MyPerson class looks like this:
public class MyPerson implements Serializable
{
#QuerySqlField()
public String ID;
#QuerySqlField()
public Socials[] socials;
}
And the Socials class looks like this:
public class Socials implements Serializable
{
#QuerySqlField()
public String network;
#QuerySqlField()
public String login;
}
I create and populate the cache like this:
// creating person cache
CacheConfiguration<String, MyPerson> personCacheCfg = new CacheConfiguration<String, MyPerson>();
personCacheCfg.setName("Person");
personCacheCfg.setIndexedTypes(String.class, MyPerson.class);
IgniteCache<String, MyPerson> personCache = ignite.createCache(personCacheCfg);
// adding data
personCache.put("P1", new MyPerson("Person1", new Socials[] { new Socials("fb", "a#my.com"), new Socials("tw", "b.my.com") }));
personCache.put("P2", new MyPerson("Person2", new Socials[] { new Socials("tw", "c.my.com") }));
personCache.put("P3", new MyPerson("Person3", new Socials[] { new Socials("fb", "d.my.com") }));
Now, when I want to run a SqlFieldQuery that returns all Persons that have a specific Socials account, how would I do this with Ignite SQL? Would this be a join over the array field? Or is such case better done by storing the Socials in a separate cache?
You can't run SQL queries over nested collections or arrays. You should create an additional table of pairs (socials, personId) and run joins over these two tables.
Make sure to collocate data of these tables by personId. Otherwise you will need distributed joins, that impact performance dramatically. You can find documentation on affinity collocation here: https://apacheignite.readme.io/docs/affinity-collocation
Another option without use of SQL is a ScanQuery. But it requires a full data lookup. If you are going to select a big part of data anyway, scan query may be a good option too. Documentation: https://apacheignite.readme.io/docs/cache-queries#scan-queries
Is it possible to dynamically create a GraphQL schema ?
We store the data in mongoDB and there is a possibility of new fields getting added. We do not want any code change to happen for this newly added field in the mongoDB document.
Is there any way we can generate the schema dynamically ?
Schema is defined in code, but for java(schema as pojo), when new
attribute is added, you have to update and recompile code, then
archive and deploy the jar again. Any way to generate schema by the
data instead of pre-define it?
Currently we are using java related projects (graphql-java, graphql-java-annotations) for GraphQL development.
You could use graphql-spqr, it allows you auto-generate a schema based on your service classes. In your case, it would look like this:
public class Pojo {
private Long id;
private String name;
// whatever Ext is, any (complex) object would work fine
private List<Ext> exts;
}
public class Ext {
public String something;
public String somethingElse;
}
Presumably, you have a service class containing your business logic:
public class PojoService {
//this could also return List<Pojo> or whatever is applicable
#GraphQLQuery(name = "pojo")
public Pojo getPojo() {...}
}
To expose this service, you'd just do the following:
GraphQLSchema schema = new GraphQLSchemaGenerator()
.withOperationsFromSingleton(new PojoService())
.generate();
You could then fire a query such as:
query test {
pojo {
id
name
exts {
something
somethingElse
} } }
No need for strange wrappers or custom code of any kind, nor sacrificing type safety. Works with generics, dependency injection, or any other jazz you may have in your project.
Full disclosure: I'm the author of graphql-spqr.
After some days' investigation. I found it is hard to generate schema dynamically in Java (or cost is so high).
Well, from another way. I think we can use Map as a compromised way to accomplish that.
POJO/Entity
public class POJO{
#GraphQLField
private Long id;
#GraphQLField
private String name;
// ...
#GraphQLField
private GMap exts;
}
GMap is a customized Map (Because Map/HashMap is a JDK inner class which could not make as GraphQL Schema but only extend).
GMap
public class GMap extends HashMap<String, String> {
#GraphQLField
public String get(#GraphQLName("key") String key) {
return super.get(key);
}
}
Retrieve data from Client
// query script
query test
{
your_method
{
id
name
exts {
get(key: "ext") // Add a extended attribute someday
}
}
}
// result
{
"errors":[],
"data":
{
"list":
[
{"id":1, name: "name1", exts: {"get": "ext1"}},
{"id":2, name: "name2", exts: {"get": "ext2"}}
]
}
}
I'm trying to pass a List to Postgresql using Mule ESB.
A simplfied version of my pojo looks like:
public class NewEntry {
private String positionReference;
private String productID;
#XmlElement(required=true, name="PositionReference")
public String getPositionReference() {
return positionReference;
}
public void setPositionReference(String positionReference) {
this.positionReference = positionReference;
}
#XmlElement(required=true, name="ProductID")
public String getProductID() {
return productID;
}
public void setProductID(String productID) {
this.productID = productID;
}
}
This is passed in via a soap webservice as
List
until now i've had it simply being transformed into JSON and pumped it out to a file now I would like to insert it into a database, so in my database step I've put a in insert query in along the lines of:
<jdbc:query key="insertrecord" value="INSERT INTO f_intraday_rt(version) VALUES (#[message:productDescription]);"/>
Anyway no matter what message evaluation I use in the VALUES section, I can't get it to insert a value, just errors or blank rows.
How do I unbundle the loop and insert a row per list item?
Thanks
Tom
Use a collection splitter to split the list of objects into different messages just before your outbound JDBC endpoint. This will perform multiple inserts.
It's one INSERT per entry in the List, so you should loop over the collection and execute the SQL INSERT for each one.
Best to think about batching and transactions. Are they one unit of work?
Let's say I have a method in java, which looks up a user in a database and returns their address and the team they are on.
I want to return both values from the method, and don't want to split the method in two because it involves a database call and splitting involves twice the number of calls.
Given typical concerns in a moderate to large software project, what's the best option?
whatGoesHere getUserInfo(String name) {
// query the DB
}
I know the question smells of duplication with existing ones, but each other question had some element that made it different enough from this example that I thought it was worth asking again.
you have some options.
The most OOP it will be create a class to encapsulate those 2 properties, something like that
private class UserInfo {
private Address address;
private Team team;
}
Or if you want a simple solution you can return an array of objects:
Object[] getUserInfo(String name) {
// query the DB
return new Object[]{address,team};
}
Or if you want to expose this method to some library you can have some interface that it will consume those properties, something like this:
class APIClass{
interface UserInfo{
public Address getAddress();
public Team getTeam();
}
UserInfo getUserInfo(String name) {
// query the DB
return new UserInfo(){
public Address getAddress(){ return address; }
public Team getTeam(){ return team; }
};
}
}
cant a map help , A MultivalueMap. Where the key is the user name and the 2 values are the adress and the team name. I am assuming both your Address and team are String variables, You can know more about Multivalue Map here
http://commons.apache.org/collections/apidocs/org/apache/commons/collections/map/MultiValueMap.html
http://apachecommonstipsandtricks.blogspot.in/2009/01/multi-value-map-values-are-list.html
First model your abstractions, relationships and multiplicity well (see an e.g. below). Then you can model tables accordingly. Once these two steps are performed you can either leverage JPA that can be configured to load your object graph or you write JDBC code and create the graph your self by running a SQL query with proper SQL JOINs.
A User has an Address
A Team can have 1 or more Users (and can a User play for more teams?)
You can return a String array with user name and group name in it . The method looks like :
public String[] getUserInfo(String name) {
String[] result = new String[2];
// query the DB
...
result[0] = userName;
result[1] = groupName;
return result;
}
A common solution to this kind of issue is to create a custom object with as many attributes as the values you want to return.
If you can't create a new class for this, you can use a Map<String, Object>, but this approach is not type-safe.
I thought Guava had a generic Pair class already, but I cannot find it. You can build your own using generics if you're on Java 1.5+.
public class Pair<X,Y>
{
public final X first;
public final Y second;
public Pair(X first, Y second) {
this.first = first;
this.second = second;
}
}
Feel free to make the fields private and add getters. :) Using it is easy:
return new Pair<Address,Team>(address, team);
Update
Apache Commons Lang has Pair. See this SO question for more options.
Which ORM supports a domain model of immutable types?
I would like to write classes like the following (or the Scala equivalent):
class A {
private final C c; //not mutable
A(B b) {
//init c
}
A doSomething(B b) {
// build a new A
}
}
The ORM has to initialized the object with the constructor. So it is possible to check invariants in the constructor. Default constructor and field/setter access to intialize is not sufficient and complicates the class' implementation.
Working with collections should be supported. If a collection is changed it should create a copy from the user perspective. (Rendering the old collection state stale. But user code can still work on (or at least read) it.) Much like the persistent data structures work.
Some words about the motivation. Suppose you have a FP-style domain object model. Now you want to persist this to a database. Who do you do that? You want to do as much as you can in a pure functional style until the evil sides effect come in. If your domain object model is not immutable you can for example not share the objects between threads. You have to copy, cache or use locks. So unless your ORM supports immutable types your constrainted in your choice of solution.
UPDATE: I created a project focused on solving this problem called JIRM:
https://github.com/agentgt/jirm
I just found this question after implementing my own using Spring JDBC and Jackson Object Mapper. Basically I just needed some bare minimum SQL <-> immutable object mapping.
In short I just use Springs RowMapper and Jackson's ObjectMapper to map Objects back and forth from the database. I use JPA annotations just for metadata (like column name etc...). If people are interested I will clean it up and put it on github (right now its only in my startup's private repo).
Here is a rough idea how it works here is an example bean (notice how all the fields are final):
//skip imports for brevity
public class TestBean {
#Id
private final String stringProp;
private final long longProp;
#Column(name="timets")
private final Calendar timeTS;
#JsonCreator
public TestBean(
#JsonProperty("stringProp") String stringProp,
#JsonProperty("longProp") long longProp,
#JsonProperty("timeTS") Calendar timeTS ) {
super();
this.stringProp = stringProp;
this.longProp = longProp;
this.timeTS = timeTS;
}
public String getStringProp() {
return stringProp;
}
public long getLongProp() {
return longProp;
}
public Calendar getTimeTS() {
return timeTS;
}
}
Here what the RowMapper looks like (notice it mainly delegats to Springs ColumnMapRowMapper and then uses Jackson's objectmapper):
public class SqlObjectRowMapper<T> implements RowMapper<T> {
private final SqlObjectDefinition<T> definition;
private final ColumnMapRowMapper mapRowMapper;
private final ObjectMapper objectMapper;
public SqlObjectRowMapper(SqlObjectDefinition<T> definition, ObjectMapper objectMapper) {
super();
this.definition = definition;
this.mapRowMapper = new SqlObjectMapRowMapper(definition);
this.objectMapper = objectMapper;
}
public SqlObjectRowMapper(Class<T> k) {
this(SqlObjectDefinition.fromClass(k), new ObjectMapper());
}
#Override
public T mapRow(ResultSet rs, int rowNum) throws SQLException {
Map<String, Object> m = mapRowMapper.mapRow(rs, rowNum);
return objectMapper.convertValue(m, definition.getObjectType());
}
}
Now I just took Spring JDBCTemplate and gave it a fluent wrapper. Here are some examples:
#Before
public void setUp() throws Exception {
dao = new SqlObjectDao<TestBean>(new JdbcTemplate(ds), TestBean.class);
}
#Test
public void testAll() throws Exception {
TestBean t = new TestBean(IdUtils.generateRandomUUIDString(), 2L, Calendar.getInstance());
dao.insert(t);
List<TestBean> list = dao.queryForListByFilter("stringProp", "hello");
List<TestBean> otherList = dao.select().where("stringProp", "hello").forList();
assertEquals(list, otherList);
long count = dao.select().forCount();
assertTrue(count > 0);
TestBean newT = new TestBean(t.getStringProp(), 50, Calendar.getInstance());
dao.update(newT);
TestBean reloaded = dao.reload(newT);
assertTrue(reloaded != newT);
assertTrue(reloaded.getStringProp().equals(newT.getStringProp()));
assertNotNull(list);
}
#Test
public void testAdding() throws Exception {
//This will do a UPDATE test_bean SET longProp = longProp + 100
int i = dao.update().add("longProp", 100).update();
assertTrue(i > 0);
}
#Test
public void testRowMapper() throws Exception {
List<Crap> craps = dao.query("select string_prop as name from test_bean limit ?", Crap.class, 2);
System.out.println(craps.get(0).getName());
craps = dao.query("select string_prop as name from test_bean limit ?")
.with(2)
.forList(Crap.class);
Crap c = dao.query("select string_prop as name from test_bean limit ?")
.with(1)
.forObject(Crap.class);
Optional<Crap> absent
= dao.query("select string_prop as name from test_bean where string_prop = ? limit ?")
.with("never")
.with(1)
.forOptional(Crap.class);
assertTrue(! absent.isPresent());
}
public static class Crap {
private final String name;
#JsonCreator
public Crap(#JsonProperty ("name") String name) {
super();
this.name = name;
}
public String getName() {
return name;
}
}
Notice in the above how easy it is to map any query into immutable POJO's. That is you don't need it 1-to-1 of entity to table. Also notice the use of Guava's optionals (last query.. scroll down). I really hate how ORM's either throw exceptions or return null.
Let me know if you like it and I'll spend the time putting it on github (only teste with postgresql). Otherwise with the info above you can easily implement your own using Spring JDBC. I'm starting to really dig it because immutable objects are easier to understand and think about.
Hibernate has the #Immutable annotation.
And here is a guide.
Though not a real ORM, MyBatis may able to do this. I didn't try it though.
http://mybatis.org/java.html
AFAIK, there are no ORMs for .NET supporting this feature exactly as you wish. But you can take a look at BLTookit and LINQ to SQL - both provide update-by-comparison semantics and always return new objects on materialization. That's nearly what you need, but I'm not sure about collections there.
Btw, why you need this feature? I'm aware about pure functional languages & benefits of purely imutable objects (e.g. complete thread safety). But in case with ORM all the things you do with such objects are finally transformed to a sequence of SQL commands anyway. So I admit the benefits of using such objects are vaporous here.
You can do this with Ebean and OpenJPA (and I think you can do this with Hibernate but not sure). The ORM (Ebean/OpenJPA) will generate a default constructor (assuming the bean doesn't have one) and actually set the values of the 'final' fields. This sounds a bit odd but final fields are not always strictly final per say.
SORM is a new Scala ORM which does exactly what you want. The code below will explain it better than any words:
// Declare a model:
case class Artist ( name : String, genres : Set[Genre] )
case class Genre ( name : String )
// Initialize SORM, automatically generating schema:
import sorm._
object Db extends Instance (
entities = Set() + Entity[Artist]() + Entity[Genre](),
url = "jdbc:h2:mem:test"
)
// Store values in the db:
val metal = Db.save( Genre("Metal") )
val rock = Db.save( Genre("Rock") )
Db.save( Artist("Metallica", Set() + metal + rock) )
Db.save( Artist("Dire Straits", Set() + rock) )
// Retrieve values from the db:
val metallica = Db.query[Artist].whereEqual("name", "Metallica").fetchOne() // Option[Artist]
val rockArtists = Db.query[Artist].whereEqual("genres.name", "Rock").fetch() // Stream[Artist]