I have a couple of files. Each file is for a month. e.g.
file-januar.csv
ID, From, To
1234, 2022-01-01, 2022-01-02
1235, 2022-07-01, 2022-08-20
file-februar.csv
ID, From, To
1234, 2022-01-01, 2022-01-02
1235, 2022-08-21, 2022-08-30
file-march.csv
...
The id is unique in each file. If the record did not change in the file for january and february is the same entry:
1234, 2022-01-01, 2022-01-02
If the record changes the entry for januar and februar are different
1235, 2022-07-01, 2022-08-20
1235, 2022-08-21, 2022-08-30
I need to create a single file without any duplicates in chronological order. My problem: I can not use a database.
Has sombody an idea howto create a single file januar-dezember without any duplicates? Each file has about 10.000 rows. How can i handle it? And how can i sort it chronological?
I would store each line in a Map<Integer, String> where the key is the unique ID (I suppose it is a number, but you also could use a string) and the value is the complete line.
Then read in each file in chronological order and store it into the map. Entries from later files will overwrite entries from earlier ones.
In the end you can write out the Map like this:
Set<Integer> keys = allValuesMap.keySet();
List<Integer> keyList = new ArrayList<>(keys);
Collections.sort(keyList);
for (Integer key: keyList) {
System.out.println(allValuesMap.get(key));
}
(Of course you would probably replace the System.out with another file.)
If that doesn't work, you could try to use an in-memory database like H2.
Related
I am trying to do txt based database system. I'm stuck here now. What I want to do is enter the location of the data and then update it. I separate the data with this character. "|"
Structure like this:
ID |Name |Job |Phone Number
---+-----+--------+------------
55 |John |Plumber |555444
The id part is to find out which row it is in, and the name part is in the column.
data_Update(filename, id, "Name", "Bob Ross");
I want to do a function like this.
You could do it in following manner:
Read the file and for each line of text add a entry in your HashMap
Map<Integer, Map<String,Object>> personMap
Where key represent Id of the person, And Value represent mapping of field name to field value for the current entry.
In your db_update method, locate the person by id and update e.g.
personMap.get(Id).put(fieldname,value)
I want to create a file in this format:
device1,t1,t2,t3,t4,t5
device2,t1,t2,t3,t4,t5
device3,t6,t7,t8,t9,t10
device4,t6,t7,t8,t9,t10
Here, t1, t2, ..., tn are time stamps.
Every value tn is generated based on one execution of JAR file along with that device name gets generated too.
I am able to generate a format like this using the JAR file now:
For example:
Current format in csv file:
device1,t1,device2,t2,device2,t3,device1,t4,device2,t5,device2,t6,device1,t7,device2,t8
I want this in this format in csv file:
device1-t1,t4,t7
device2-t2,t3,t5,t6,t8
So here, I have to put the time stamp belonging to specific devices on the right-hand side.
Please let me know how can I sort it in Java.
I will answer this question here as per my understanding of your question.
What you can do is to create a hashmap which stores device name as hashmap key.
And then for values create a sortedCollection.
Feed your timestamp in this sorted collection and keep updating this HashMap for the corresponding device name key.
As and when you will update your sorted timestamp collections, they will automatically be stored in sorted manner.
your hashmap will look like :
key : value (collection)
device1 : t1, t4, t7
device2 : t2, t5, t8 (add more timestamp in the end of this collection)
Then feed this hashmap data in the CSV file.
This is to do from java end.
If you want to sort in csv whenever a new timestamp is added for a device, then I dont think so that you can do this from java. Then you would have to keep some logic in csv file once all your data is added in csv file.
This is the solution:
I got output as:
Entire map:{Device1=[[t8], t9], Device2=[[[[[t2], t3], t5], t7], t10]}
BufferedReader reader = new BufferedReader(new FileReader("results.csv"));
String eachline;
// int i=2, j=2;
while((eachline = reader.readLine()) != null)
{
String[] fields = eachline.split(",");
if(Integer.parseInt(fields[2])==0)//data is = 0
{
if(tree.get(fields[0])!=null)//returns null if this key not present
{
values.add(tree.get(fields[0]));//get entire key value pair for that particular field
}
values.add(fields[1]);//to prev value, add next value
tree.put(fields[0], values.toString());// write to hashmap along with value
values.clear();
}
}
System.out.println("Entire map:"+tree);
I am trying to make a CouchDB view to obtain some document that is in set 1 and in set 2. For example, when I have a single key I can make some query like:
dbname/_design_doc/viewName?keys=[value1, value2, value3]
and it returns all the documents where it finds either the value1, 2 or 3. What I want is something like this but for a complex key.
For example,
dbname/_design_doc/viewName?keys=[[key1, key12, key13],[key21, key22]]
where key1x is a value for the first key and key2x is a value for the second key, meaning I would like to get every document that has key11 and key21, key11 and key22, key12 and key21, key12 and key22 and so on.
My view is this one:
"twokeys": {
"map": "function(doc) {\n if (doc.uid && doc.hid){\n
emit([doc.uid, doc.hid], doc);\n }\n}"
}
Is this possible?
Thanks in advance
You can query with the keys parameter using complex keys if you follow this answer.
Unfortunately, you can't query both the startkey or the endkey with the keys.
basically below is the java coding part, which the result will then be populated to a .csv file. However, I dont seem get the ordering part right (last line in below snippet).
Map<String, String> map = new HashMap<String, String>();
map.put("path", "/etc/crx-db/form-data/career");
map.put("type", "nt:unstructured");
map.put("p.limit", "-1");
map.put("daterange.property", "created");
map.put("daterange.lowerBound", from);
map.put("daterange.lowerOperation", ">=");
map.put("daterange.upperOperation", "<=");
map.put("daterange.upperBound", to);
map.put("orderby", "created"); //<--here
Providing that in crx repositry (/etc/crx-db/form-data/career), I have nodes: data1, data2, data3...
Then for each node, there is one property - Name: created | Type: Date | Value: 2014-01-28T23:21:15.029+08:00 (eg)
However my result in .csv is incorrect like (row 1 to 5):
2014-01-28T23:21:15.029+08:00
2014-01-28T23:48:12.219+08:00
2014-02-10T18:44:38.914+08:00 <-- unsorted
2014-02-10T18:43:32.426+08:00 <-- unsorted
2014-02-10T18:46:53.319+08:00
Pretty sure my code wasn't running. Any idea on how can I tweak my java code to make the sorting happen? As in returning sorted data1, data2, data3... based on the property created. Thanks.
You were almost there. It can be done as follows.
map.put("orderby", "#created");
map.put("orderby.sort", "desc"); // in case you want it descending
In case you need to check property within a child node, you can provide the relative path to that for the orderby value. For eg., if you are searching for dam:Asset and want to order them based on the jcr:lastModified property of its metadata, then your query would be something similar to this.
map.put("path", "/content/dam/geometrixx");
map.put("type", "dam:Asset");
map.put("orderby","#jcr:content/metadata/jcr:lastModified");
For further learning refer this
I have a XML file with many copies of table node structure as below:
<databasetable TblID=”123” TblName=”Department1_mailbox”>
<SelectColumns>
<Slno>dept1_slno</Slno>
<To>dept1_to</To>
<From>dept1_from</From>
<Subject>dept1_sub</Subject>
<Body>dept1_body</Body>
<BCC>dept1_BCC</BCC>
<CC>dept1_CC</CC>
</SelectColumns>
<WhereCondition>MailSentStatus=’New’</WhereCondition>
<UpdateSuccess>
<MailSentStatus>’Yes’</MailSentStatus>
<MailSentFailedReason>’Mail Sent Successfully’</MailSentFailedReason>
</UpdateSuccess>
<UpdateFailure>
<MailSentStatus>’No’</MailSentStatus>
<MailSentFailedReason>’Mail Sending Failed ’</MailSentFailedReason>
</ UpdateFailure>
</databasetable>
As it is not an efficient manner to traverse the file for each time to fetch the details of each node for the queries in the program, I used the nested hashmap concept to store the details while traversing the XML file for the first time. The structure I used is as below:
MapMaster
Key Value
123 MapDetails
Key Value
TblName Department1_mailbox
SelectColumns mapSelect
Key Value
Slno dept1_slno
To dept1_to
From dept1_from
Subject dept1_sub
Body dept1_body
BCC dept1_BCC
CC dept1_CC
WhereCondition MailSentStatus=’New’
UpdateSuccess mapUS
MailSentStatus ’Yes’
MailSentFailedReason ’Mail Sent Successfully’
UpdateFailure mapUF
MailSentStatus ’No’
MailSentFailedReason ’Mail Sending Failed’
But the problem I’m facing now is regarding retrieving the Value part using the nested Keys. For example,
If I need the value of Slno Key, I have to specify TblID, SelectColumns, Slno in nested form like:
Stirng Slno = ((HashMap)((HashMap)mapMaster.get(“123”))mapDetails.get(“SelectColumns”))mapSelect.get(“Slno”);
This is unconvinent to use in a program. Please suggest a solution but don’t tell that iterators are available. As I’ve to fetch the individual value from the map according to the need of my program.
EDIT:my program has to fetch the IDs of the department for which there is privilege to send mails and then these IDs are compared with the IDs in XML file. Only information of those IDs are fetched from XML which returned true in comparison. This is all my program. Please help.
Thanks in advance,
Vishu
Never cast to specific Map implementation. Better use casting to Map interface, i.e.
((Map)one.get("foo")).get("bar")
Do not use casting in your case. You can define collection using generics, so compiler will do work for you:
Map<String, Map> one = new HashMap<String, Map>();
Map<String, Integer> two = new HashMap<String, Integer>();
Now your can say:
int n = one.get("foo").get("bar");
No casting, no problems.
But the better solution is not to use nested tables at all. Create your custom classes like SelectColumns, WhereCondition etc. Each class should have appropriate private fields, getters and setters. Now parse your XML creating instance of these classes. And then use getters to traverse the data structure.
BTW if you wish to use JAXB you do not have to do almost anything! Something like the following:
Unmarshaller u = JAXBContext.newInstance(SelectColumns.class, WhereCondition.class).createUnmarshaller();
SelectColumns[] columns = (SelectColumns[])u.unmarshal(in);
One approach to take would be to generate fully qualified keys that contain the XML path to the element or attribute. These keys would be unique, stored in a single hashmap and get you to the element quickly.
Your code would simply have to generate a unique textual representation of the path and store and retrieve the xml element based on the key.