I want to create a file in this format:
device1,t1,t2,t3,t4,t5
device2,t1,t2,t3,t4,t5
device3,t6,t7,t8,t9,t10
device4,t6,t7,t8,t9,t10
Here, t1, t2, ..., tn are time stamps.
Every value tn is generated based on one execution of JAR file along with that device name gets generated too.
I am able to generate a format like this using the JAR file now:
For example:
Current format in csv file:
device1,t1,device2,t2,device2,t3,device1,t4,device2,t5,device2,t6,device1,t7,device2,t8
I want this in this format in csv file:
device1-t1,t4,t7
device2-t2,t3,t5,t6,t8
So here, I have to put the time stamp belonging to specific devices on the right-hand side.
Please let me know how can I sort it in Java.
I will answer this question here as per my understanding of your question.
What you can do is to create a hashmap which stores device name as hashmap key.
And then for values create a sortedCollection.
Feed your timestamp in this sorted collection and keep updating this HashMap for the corresponding device name key.
As and when you will update your sorted timestamp collections, they will automatically be stored in sorted manner.
your hashmap will look like :
key : value (collection)
device1 : t1, t4, t7
device2 : t2, t5, t8 (add more timestamp in the end of this collection)
Then feed this hashmap data in the CSV file.
This is to do from java end.
If you want to sort in csv whenever a new timestamp is added for a device, then I dont think so that you can do this from java. Then you would have to keep some logic in csv file once all your data is added in csv file.
This is the solution:
I got output as:
Entire map:{Device1=[[t8], t9], Device2=[[[[[t2], t3], t5], t7], t10]}
BufferedReader reader = new BufferedReader(new FileReader("results.csv"));
String eachline;
// int i=2, j=2;
while((eachline = reader.readLine()) != null)
{
String[] fields = eachline.split(",");
if(Integer.parseInt(fields[2])==0)//data is = 0
{
if(tree.get(fields[0])!=null)//returns null if this key not present
{
values.add(tree.get(fields[0]));//get entire key value pair for that particular field
}
values.add(fields[1]);//to prev value, add next value
tree.put(fields[0], values.toString());// write to hashmap along with value
values.clear();
}
}
System.out.println("Entire map:"+tree);
Related
I have a couple of files. Each file is for a month. e.g.
file-januar.csv
ID, From, To
1234, 2022-01-01, 2022-01-02
1235, 2022-07-01, 2022-08-20
file-februar.csv
ID, From, To
1234, 2022-01-01, 2022-01-02
1235, 2022-08-21, 2022-08-30
file-march.csv
...
The id is unique in each file. If the record did not change in the file for january and february is the same entry:
1234, 2022-01-01, 2022-01-02
If the record changes the entry for januar and februar are different
1235, 2022-07-01, 2022-08-20
1235, 2022-08-21, 2022-08-30
I need to create a single file without any duplicates in chronological order. My problem: I can not use a database.
Has sombody an idea howto create a single file januar-dezember without any duplicates? Each file has about 10.000 rows. How can i handle it? And how can i sort it chronological?
I would store each line in a Map<Integer, String> where the key is the unique ID (I suppose it is a number, but you also could use a string) and the value is the complete line.
Then read in each file in chronological order and store it into the map. Entries from later files will overwrite entries from earlier ones.
In the end you can write out the Map like this:
Set<Integer> keys = allValuesMap.keySet();
List<Integer> keyList = new ArrayList<>(keys);
Collections.sort(keyList);
for (Integer key: keyList) {
System.out.println(allValuesMap.get(key));
}
(Of course you would probably replace the System.out with another file.)
If that doesn't work, you could try to use an in-memory database like H2.
I am trying to do txt based database system. I'm stuck here now. What I want to do is enter the location of the data and then update it. I separate the data with this character. "|"
Structure like this:
ID |Name |Job |Phone Number
---+-----+--------+------------
55 |John |Plumber |555444
The id part is to find out which row it is in, and the name part is in the column.
data_Update(filename, id, "Name", "Bob Ross");
I want to do a function like this.
You could do it in following manner:
Read the file and for each line of text add a entry in your HashMap
Map<Integer, Map<String,Object>> personMap
Where key represent Id of the person, And Value represent mapping of field name to field value for the current entry.
In your db_update method, locate the person by id and update e.g.
personMap.get(Id).put(fieldname,value)
I'm writing an ORC file using Groovy.
One of the columns is a String. The ORC column type is:
.addField("Name", TypeDescription.createString())
The column vector is:
BytesColumnVector vName = (BytesColumnVector) batch.cols[1]
The values to be assigned to vName may include NULLs, but I can't get ORC to write a null value into its data.
Attempting to assign a null value through set(), setValue() or setRef() throws a null pointer error, either at the point of assignment, or when the batch row is written deeper within ORC.
The closest I can get is this:
byte[] b = new byte[0]
vName.setRef (i,b,0,0)
but this puts an empty string into the data file, as shown in the following dump snippet (see the second column, 'Name'):
{"ProductID":355,"Name":"","MakeFlag":false,"StandardCost":0,"Weight":null,"ModifiedDate":"2014-02-08 10:01:36.827"}
Any thoughts on how to set a null string?
EDIT: With the answer to this question, I was able to complete some code to write the contents of a database table to ORC. It may be useful to people searching for ORC-related examples.
https://www.linkedin.com/pulse/orc-adls-polybase-ron-dunn/enter link description here
An empty string is what I use. I don't think there's another way to do it.
Just make sure you mark the column as containing nulls.
Your code would ideally look like this:
BytesColumnVector vName = (BytesColumnVector) batch.cols[1];
byte[] EMPTY_BYTES = "".getBytes(StandardCharsets.UTF_8);
vName.setRef(i, EMPTY_BYTES, 0, 0);
vName.isNull[i] = true;
vName.noNulls = false;
I found this problem so interesting. I am using an awk 2D array that has a key,value,count of the same. and that is being printed to a file. This file is in the below format
A100|B100|3
A100|C100|2
A100|B100|5
Now I have a file like this .. My motive is to convert this file into a hash map so that the final output from the hash map is.
A100|B100|8
A100|C100|2
Just an aggregation
The challenge is, this one has 3 dimensions and not two. I did have an another file in the below format which is
D100|4
H100|5
D100|6
I easily aggregated the above as it is only 2D and I used the below code to do that
String[] fields= strLine.trim().split("\\|");
if(hashmap.containsKey(fields[0]))
{
//Update the value of the key here
hashmap.put(fields[0],hashmap.get(fields[0]) + Integer.parseInt(fields[1]));
}
else
{
//Inserting the key to the map
hashmap.put(fields[0],Integer.parseInt(fields[1]));
}
So this was quite simple for implementation.
But when it comes to 3D I have to have an another check inside.. My idea for this is to maintain a [B100,5(beanObject[5])]
Map<String,beanClassObject> hashmap=new Map<String,beanClassObject>();
secongField hash map which has been used in the code that has a mapping relation between the created ben Object subscript and the key as the second field "For instance it is "
This bean class would have the getter and setter method for the 2nd and 3rd fields of the file. I hope I am clear with this point. So the implementation of this would be
if(hashmap.containsKey(fields[0]))
{
**//Have to check whether the the particular key value pair already exists ... I dint find any method for this ... Just a normal iteration is there .. Could you ppl guide me regarding this**
//Update the value of the key here
secondFieldHashMap.get(fields[1]).COunt= secondFieldHashMap.get(fields[1]).getCOunt+ Integer.parseInt(fields[2]));
}
else
{
//Inserting the key to the map
hashmap.put(fields[0],Integer.parseInt(fields[1]));
secondFieldHashMap.get(fields[1]).COunt= Integer.parseInt(fields[2]));
}
else
{
// This meands there is no key field
// Hence insert the key field and also update the count of seconfFieldHashMap as done previously.
}
COuld you ppl please throw some ideas regarding this. Thank you
Consider using a Table available in the Google Guava libraries.
I have a XML file with many copies of table node structure as below:
<databasetable TblID=”123” TblName=”Department1_mailbox”>
<SelectColumns>
<Slno>dept1_slno</Slno>
<To>dept1_to</To>
<From>dept1_from</From>
<Subject>dept1_sub</Subject>
<Body>dept1_body</Body>
<BCC>dept1_BCC</BCC>
<CC>dept1_CC</CC>
</SelectColumns>
<WhereCondition>MailSentStatus=’New’</WhereCondition>
<UpdateSuccess>
<MailSentStatus>’Yes’</MailSentStatus>
<MailSentFailedReason>’Mail Sent Successfully’</MailSentFailedReason>
</UpdateSuccess>
<UpdateFailure>
<MailSentStatus>’No’</MailSentStatus>
<MailSentFailedReason>’Mail Sending Failed ’</MailSentFailedReason>
</ UpdateFailure>
</databasetable>
As it is not an efficient manner to traverse the file for each time to fetch the details of each node for the queries in the program, I used the nested hashmap concept to store the details while traversing the XML file for the first time. The structure I used is as below:
MapMaster
Key Value
123 MapDetails
Key Value
TblName Department1_mailbox
SelectColumns mapSelect
Key Value
Slno dept1_slno
To dept1_to
From dept1_from
Subject dept1_sub
Body dept1_body
BCC dept1_BCC
CC dept1_CC
WhereCondition MailSentStatus=’New’
UpdateSuccess mapUS
MailSentStatus ’Yes’
MailSentFailedReason ’Mail Sent Successfully’
UpdateFailure mapUF
MailSentStatus ’No’
MailSentFailedReason ’Mail Sending Failed’
But the problem I’m facing now is regarding retrieving the Value part using the nested Keys. For example,
If I need the value of Slno Key, I have to specify TblID, SelectColumns, Slno in nested form like:
Stirng Slno = ((HashMap)((HashMap)mapMaster.get(“123”))mapDetails.get(“SelectColumns”))mapSelect.get(“Slno”);
This is unconvinent to use in a program. Please suggest a solution but don’t tell that iterators are available. As I’ve to fetch the individual value from the map according to the need of my program.
EDIT:my program has to fetch the IDs of the department for which there is privilege to send mails and then these IDs are compared with the IDs in XML file. Only information of those IDs are fetched from XML which returned true in comparison. This is all my program. Please help.
Thanks in advance,
Vishu
Never cast to specific Map implementation. Better use casting to Map interface, i.e.
((Map)one.get("foo")).get("bar")
Do not use casting in your case. You can define collection using generics, so compiler will do work for you:
Map<String, Map> one = new HashMap<String, Map>();
Map<String, Integer> two = new HashMap<String, Integer>();
Now your can say:
int n = one.get("foo").get("bar");
No casting, no problems.
But the better solution is not to use nested tables at all. Create your custom classes like SelectColumns, WhereCondition etc. Each class should have appropriate private fields, getters and setters. Now parse your XML creating instance of these classes. And then use getters to traverse the data structure.
BTW if you wish to use JAXB you do not have to do almost anything! Something like the following:
Unmarshaller u = JAXBContext.newInstance(SelectColumns.class, WhereCondition.class).createUnmarshaller();
SelectColumns[] columns = (SelectColumns[])u.unmarshal(in);
One approach to take would be to generate fully qualified keys that contain the XML path to the element or attribute. These keys would be unique, stored in a single hashmap and get you to the element quickly.
Your code would simply have to generate a unique textual representation of the path and store and retrieve the xml element based on the key.