I have searched for two days and haven't seen any code that is closed to this. This is the only code in java that I seen and it's not exactly what I wanted.
conn.transact(list(list("db.fn/cas", datomic_id, "attribute you want to update", old value, new value))).get();
I have tried this code with a single value in the old value and a single value in the new value but it just stack the information instead of overlaying it.
Example: old value is chicken and new value is fish. After the transaction, it's [chicken, fish] instead of what I expected to be just [fish] and chicken will be move into archive(history).
So the question is, how do you ref the old array value and how do you give the new value an array so it'll update as what I expected to be as stated above.
I remember reading somewhere that under the hood it's just a series of values linking to one attribute. If this is the case does that mean that I have to find the datomic id of the string and change it? Also have to remove it if it's not on the new list?
FYI, these are the generic transaction functions I currently use for this kind of task (declared from Clojure, but should be fairly easy to adapt to Java if required):
[{:db/ident :bsu.fns/replace-to-many-scalars,
:db/doc "Given an entity's lookup ref, a to-many (scalar) attribute, and a list of new values,
yields a transaction that replaces the old values by new ones"
:db/id (d/tempid :db.part/user),
:db/fn (d/function
'{:lang :clojure,
:imports [],
:requires [[datomic.api :as d]],
:params [db entid attr new-vals],
:code (let [old-vals (if-let [e (d/entity db entid)] (get e attr) ())
to-remove (remove (set (seq new-vals)) old-vals)]
(concat
(for [ov to-remove] [:db/retract entid attr ov])
(for [nv new-vals] [:db/add entid attr nv]))
)}),
}
{:db/ident :bsu.fns/to-many-retract-all-but,
:db/doc "Given an entity lookup ref, a to-many (entity) attribute, and a list of lookup refs
expands to a transaction which will retract all the [origin `to-many-attr` target] relationships but those for which target is among the `to-spare-lookup-refs`"
:db/id (d/tempid :db.part/user),
:db/fn (d/function
'{:lang :clojure,
:imports [],
:requires [[datomic.api :as d]],
:params [db origin to-many-attr to-spare-lookup-refs],
:code (let [old-targets-ids (d/q '[:find [?t ...] :in $ ?to-many-attr ?origin :where [?origin ?to-many-attr ?t]]
db to-many-attr origin)
to-spare-ids (for [lr to-spare-lookup-refs] (:db/id (d/entity db lr)))
to-delete (->> old-targets-ids (remove (set to-spare-ids)))]
(for [eid to-delete] [:db/retract origin to-many-attr eid])
#_[old-targets-ids to-update-ids to-delete])}),
}]
I don't claim at all they're optimal performance or design-wise, but they've worked for me so far. HTH.
If you need a "last write wins" style consistent solution to replace all values for a particular entity for a card many attribute, your best bet is to go with a transaction function. You could take the following approach:
Get all datoms matching entity + attribute you want to retract all values for.
Generate retractions for all of them.
Create add transactions for all new values (e.g. from a passed collection)
Remove any conflicts (i.e. if you have the same EAV with both an add and a result)
Return the resulting transaction data.
Related
So I have been using jayway JSONPath to query JSON Objects much like the following:
{
"messageHeader" : {
"sentBy" : "someOne",
"userName" : "John Doe"
},
"payload" : []
}
And this is working fine for the most part, except now I wish to select the root level objects using the path $..* and preform separate tasks depending on the type of object present in the message, using their key/names as an identifier.
However, using said path, or $.* , will always produces a JSONArray much like this:
[{sentBy:someOne,userName:John Doe},[]]
The JSON objects appear to be anonymous, they have no keys. Is there anyway I can access the key for these objects directly as a String? If the data is not present, then why does the path: $.messageHeader.sentBy , work?
From README of the JsonPath :
When evaluating a path you need to understand the concept of when a
path is definite. A path is indefinite if it contains:
.. - a deep scan operator
?() - an expression
[, (, )] - multiple array indexes
Indefinite paths always returns a list (as represented by current JsonProvider).
This should explain the above phenomenon.
I'm trying to convert a large Map> to some JavaBean. The key of map corresponds to some property of JavaBean, and the value somehow is decoded to property value. So I decided to use some util for that, but don't know what will work. There are some requirements I have to this util (or framework):
all configuration must be in separate files
should be a possibility to map dynamic quantity of keys:
there is a map:
key | value
quan | n
key_1| value_1
key_2| value_2
........ | .............
key_n| value_n
where n - is any number
and the JavaBean has a List of some beans. They have a property. value_1, value_2, ... must be mapped in this property, and in the end there must be so much beans in list, as these keys and values in map.
3.. should be a possibility to set up custom decoder for property mapping, because in most cases the value in map is a List with 1 value, so I need to get the first item of list (if it's not empty).
4.. should be a possibility run some script to execute extraordinary mappings, for example:
there is a map, that is described in 2d point.
and the JavaBean has a property of type HashMap, where value_1 is mapped to Bean1 and some analogous value from input map is mapped to Bean2.
I've tried to use smooks, but when I've started, all these requirements were not clear yet and the smooks was something new, I haven't worked with it until now. So the smooks config doesn't contain the whole business-logic (because of second req.) and looks ugly, I don't like that. I can show the most ugliest fragment for 2d point:
<jb:bean beanId="javaBean" class="com.example.JavaBean" createOnElement="map">
<jb:wiring property="someBeans" beanIdRef="someBeanItems"/>
</jb:bean>
<jb:bean beanId="someBeanItems" class="java.util.ArrayList" createOnElement="map/entry">
<jb:wiring beanIdRef="someBeanItem"/>
</jb:bean>
<jb:bean beanId="someBeanItem" class="com.example.someBeanItem" createOnElement="map/entry">
<condition>map.quan[0]>0</condition>
<jb:expression property="property1">
index = map.quan[0]-1;
value = additionalProperties.property1_List[index];
map.quan[0] = map.quan[0] - 1;
return value;
</jb:expression>
</jb:bean>
Here "property1_List" is builded before executing smooks.
Now I look for something more nice and need your help: maybe you know how to make that better using smooks? Or what another frameworks for mapping can you recommend for my issue?
Maybe I'm really missing something.
I have indexed a bunch of key/value pairs in Lucene (v4.1 if it matters). Say I have
key1=value1 and key2=value2, e.g. as read from a properties file.
They get indexed both as specific fields and into a catchall "ALL" field, e.g.
new Field("key1", "value1", aFieldTypeMimickingKeywords);
new Field("key2", "value2", aFieldTypeMimickingKeywords);
new Field("ALL", "key1=value1", aFieldTypeMimickingKeywords);
new Field("ALL", "key2=value2", aFieldTypeMimickingKeywords);
// then get added to the Document of course...
I can then do a wildcard search, using
new WildcardQuery(new Term("ALL", "*alue1"));
and it will find the hit.
But, it would be nice to get more info, like "what was complete value (e.g. "key1=value1") that goes with that hit?".
The best I can figure out it to get the Document, then get the list of IndexableFields, then loop over all of them and see if the field.stringValue().contains("alue1"). (I can look at the data structures in the debugger and all the info is there)
This seems completely insane cause isn't that what Lucene just did? Shouldn't the Hit information return some of the Fields?
Is Lucene missing what seems like "obvious" functionality? Google and starting at the APIs hasn't revealed anything straightforward, but I feel like I must be searching on the wrong stuff.
You might want to try with IndexSearcher.explain() method. Once you get the ID of the matching document, prepare a query for each field (using the same search keywords) and invoke Explanation.isMatch() for each query: the ones that yield true will give you the matched field. Example:
for (String field: fields){
Query query = new WildcardQuery(new Term(field, "*alue1"));
Explanation ex = searcher.explain(query, docID);
if (ex.isMatch()){
//Your query matched field
}
}
Hy,
Hbase allows a column family to have different qualifiers in different rows. In my case a column family has the following specification
abc[cnt] # where cnt is an integer that can be any positive integer
what I want to achieve is to get all the data from a different column family, only if the value of the described qualifier (in a different column family) matches.
for narrowing the Scan down I just add those two families I need for the query. but that is as far as I could get for now.
I already achieved the same behaviour with a SingleColumnValueFilter, but then the qualifier was known in advance. but for this one the qualifier can be abc1, abc2 ... there would be too many options, thus too many SingleColumnValueFilter's.
Then I tried using the ValueFilter, but this filter only returns those columns that match the value, thus the wrong column family.
Can you think of any way to achieve my goal, querying for a value within a dynamically created qualifier in a column family and returning the contents of the column family and another column family (as specified when creating the Scan)? preferably only querying once.
Thanks in advance for any input.
UPDATE: (for clarification as discussed in the comments)
in a more graphical way, a row may have the following:
colfam1:aaa
colfam1:aab
colfam1:aac
colfam2:abc1
colfam2:abc2
whereas I want to get all of the family colfam1 if any value of colfam2 has e.g. the value x, with regard to the fact that colfam2:abc[cnt] is dynamically created with cnt being any positive integer
I see two approaches for this: client-side filtering or server-side filtering.
Client-side filtering is more straightforward. The Scan adds only the two families "colfam1" and "colfam2". Then, for each Result you get from scanner.next(), you must filter according to the qualifiers in "colfam2".
byte[] queryValue = Bytes.toBytes("x");
Scan scan = new Scan();
scan.addFamily(Bytes.toBytes("colfam1");
scan.addFamily(Bytes.toBytes("colfam2");
ResultScanner scanner = myTable.getScanner(scan);
Result res;
while((res = scanner.next()) != null) {
NavigableMap<byte[],byte[]> colfam2 = res.getFamilyMap(Bytes.toBytes("colfam2"));
boolean foundQueryValue = false;
SearchForQueryValue: while(!colfam2.isEmpty()) {
Entry<byte[], byte[]> cell = colfam2.pollFirstEntry();
if( Bytes.equals(cell.getValue(), queryValue) ) {
foundQueryValue = true;
break SearchForQueryValue;
}
}
if(foundQueryValue) {
NavigableMap<byte[],byte[]> colfam1 = res.getFamilyMap(Bytes.toBytes("colfam1"));
LinkedList<KeyValue> listKV = new LinkedList<KeyValue>();
while(!colfam1.isEmpty()) {
Entry<byte[], byte[]> cell = colfam1.pollFirstEntry();
listKV.add(new KeyValue(res.getRow(), Bytes.toBytes("colfam1"), cell.getKey(), cell.getValue());
}
Result filteredResult = new Result(listKV);
}
}
(This code was not tested)
And then finally filteredResult is what you want. This approach is not elegant and might also give you performance issues if you have a lot of data in those families. If "colfam1" has a lot of data, you don't want to transfer it to the client if it will end up not being used if value "x" is not in a qualifier of "colfam2".
Server-side filtering. This requires you to implement your own Filter class. I believe you cannot use the provided filter types to do this. Implementing your own Filter takes some work, you also need to compile it as a .jar and make it available to all RegionServers. But then, it helps you to avoid sending loads of data of "colfam1" in vain.
It is too much work for me to show you how to custom implement a Filter, so I recommend reading a good book (HBase: The Definitive Guide for example). However, the Filter code will look pretty much like the client-side filtering I showed you, so that's half of the work done.
I have a XML file with many copies of table node structure as below:
<databasetable TblID=”123” TblName=”Department1_mailbox”>
<SelectColumns>
<Slno>dept1_slno</Slno>
<To>dept1_to</To>
<From>dept1_from</From>
<Subject>dept1_sub</Subject>
<Body>dept1_body</Body>
<BCC>dept1_BCC</BCC>
<CC>dept1_CC</CC>
</SelectColumns>
<WhereCondition>MailSentStatus=’New’</WhereCondition>
<UpdateSuccess>
<MailSentStatus>’Yes’</MailSentStatus>
<MailSentFailedReason>’Mail Sent Successfully’</MailSentFailedReason>
</UpdateSuccess>
<UpdateFailure>
<MailSentStatus>’No’</MailSentStatus>
<MailSentFailedReason>’Mail Sending Failed ’</MailSentFailedReason>
</ UpdateFailure>
</databasetable>
As it is not an efficient manner to traverse the file for each time to fetch the details of each node for the queries in the program, I used the nested hashmap concept to store the details while traversing the XML file for the first time. The structure I used is as below:
MapMaster
Key Value
123 MapDetails
Key Value
TblName Department1_mailbox
SelectColumns mapSelect
Key Value
Slno dept1_slno
To dept1_to
From dept1_from
Subject dept1_sub
Body dept1_body
BCC dept1_BCC
CC dept1_CC
WhereCondition MailSentStatus=’New’
UpdateSuccess mapUS
MailSentStatus ’Yes’
MailSentFailedReason ’Mail Sent Successfully’
UpdateFailure mapUF
MailSentStatus ’No’
MailSentFailedReason ’Mail Sending Failed’
But the problem I’m facing now is regarding retrieving the Value part using the nested Keys. For example,
If I need the value of Slno Key, I have to specify TblID, SelectColumns, Slno in nested form like:
Stirng Slno = ((HashMap)((HashMap)mapMaster.get(“123”))mapDetails.get(“SelectColumns”))mapSelect.get(“Slno”);
This is unconvinent to use in a program. Please suggest a solution but don’t tell that iterators are available. As I’ve to fetch the individual value from the map according to the need of my program.
EDIT:my program has to fetch the IDs of the department for which there is privilege to send mails and then these IDs are compared with the IDs in XML file. Only information of those IDs are fetched from XML which returned true in comparison. This is all my program. Please help.
Thanks in advance,
Vishu
Never cast to specific Map implementation. Better use casting to Map interface, i.e.
((Map)one.get("foo")).get("bar")
Do not use casting in your case. You can define collection using generics, so compiler will do work for you:
Map<String, Map> one = new HashMap<String, Map>();
Map<String, Integer> two = new HashMap<String, Integer>();
Now your can say:
int n = one.get("foo").get("bar");
No casting, no problems.
But the better solution is not to use nested tables at all. Create your custom classes like SelectColumns, WhereCondition etc. Each class should have appropriate private fields, getters and setters. Now parse your XML creating instance of these classes. And then use getters to traverse the data structure.
BTW if you wish to use JAXB you do not have to do almost anything! Something like the following:
Unmarshaller u = JAXBContext.newInstance(SelectColumns.class, WhereCondition.class).createUnmarshaller();
SelectColumns[] columns = (SelectColumns[])u.unmarshal(in);
One approach to take would be to generate fully qualified keys that contain the XML path to the element or attribute. These keys would be unique, stored in a single hashmap and get you to the element quickly.
Your code would simply have to generate a unique textual representation of the path and store and retrieve the xml element based on the key.