I have the following sample data in a .txt file
111, Sybil, 21
112, Edith, 22
113, Mathew, 30
114, Mary, 25
the required output is
[{"number":"111","name":"Sybil","age":"21" },
{"number":"112","name":"Edith","age":"22"},
{"number":"113","name":"Mathew","age":"30"},
"number":"114","name":"Mary","age":"25"]
Sadly, I have not gone far because I cant seem to get the values out of each line. instead, this is what is displayed
[one, two, three]
private void loadFile() throws FileNotFoundException, IOException {
File txt = new File("Users.txt");
try (Scanner scan = new Scanner(txt)) {
ArrayList data = new ArrayList<>() ;
while (scan.hasNextLine()) {
data.add(scan.nextLine());
System.out.print(scan.nextLine());
}
System.out.print(data);
}
I would appreciate any help. thank you
Not too sure about the requirements. If you just need to know how to get the values out, then use String.split() combined with Scanner.nextLine().
Codes below:
private void loadFile() throws FileNotFoundException, IOException {
File txt = new File("Users.txt");
try (Scanner scan = new Scanner(txt)) {
ArrayList data = new ArrayList<>();
while (scan.hasNextLine()) {
// split the data by ", " and split at most (3-1) times
String[] input = scan.nextLine().split(", ", 3);
data.add(input[0]);
data.add(input[1]);
data.add(input[2]);
System.out.print(scan.nextLine());
}
System.out.print(data);
}
}
The output would be as below and you can further modify it yourself:
[111, Sybil, 21, 112, Edith, 22, 113, Mathew, 30, 114, Mary, 25]
However, if you need the required format as well, the closest I can get is by using a HaspMap and put it into the ArrayList.
Codes below:
private void loadFile() throws FileNotFoundException, IOException {
File txt = new File("Users.txt");
try (Scanner scan = new Scanner(txt)) {
ArrayList data = new ArrayList<>();
while (scan.hasNextLine()) {
// Create a hashmap to store data in correct format,
HashMap<String, String> info = new HashMap();
String[] input = scan.nextLine().split(", ", 3);
info.put("number", input[0]);
info.put("name", input[1]);
info.put("age", input[2]);
// Put it inside the ArrayList
data.add(info);
}
System.out.print(data);
}
}
And the output would be:
[{number=111, name=Sybil, age=21}, {number=112, name=Edith, age=22}, {number=113, name=Mathew, age=30}, {number=114, name=Mary, age=25}]
Hope this answer helps you well.
Currently, you're skipping lines. A quote from the Scanner::nextLine documentation:
This method returns the rest of the current line, excluding any line separator at the end. The position is set to the beginning of the next line.
So you're adding one line to your list, and writing the next one to the console.
To get the data from each line, you can use the String::split method, which supports RegEx.
Example:
"line of my file".split(" ")
We can use streams to write some compact code.
First we define a record to hold our data.
Files.lines reads your file into memory, producing a stream of strings, one per line.
We call Stream#map to produce another stream, a series of string arrays. Each array has three elements, the three fields within each line.
We call map again, this time to produce a stream of Person objects. We construct each person object by parsing and passing to the constructor each of line’s three fields.
We call Stream#toList to collect those person objects into a list.
We call List#toString to generate text representing the contents of the list of person objects.
record Person ( int id , String name , int age ) {}
String output =
Files
.lines( Paths.of("/path/to/Users.txt" ) )
.map( line -> line.split( ", " ) )
.map( parts -> new Person(
Integer.parseInt( parts[ 0 ] ) ,
parts[ 1 ] ,
Integer.parseInt( parts[ 2 ] )
) )
.toList()
.toString()
;
If the format of the default Person#toString method does not suit you, add an override of that method to produce your desired output.
Related
Recently one of my data servers went down and a large number of video files are damaged (over 15,000 files, or more than 60TB). I wrote a script to check all files and put results in a very big log.txt file (almost 8GB).
I wrote code to find all lines starting with "Input #0" and lines which contain "damaged", then added their line numbers to ArrayList's. Next, I need to compare those two ArrayLists and find the closest line number in list2 to the number in list1 so I can get back file names from the log file.
For example:
if list1 contains numbers {1, 5, 45, 55, 100, 2000... etc}
and list2 contains numbers {50, 51, 53, 2010... etc} the result should be {45, 2000... etc}
This is my current code:
import java.io.*;
import java.util.*;
public class Log {
public static void main(String [] args) throws IOException{
ArrayList<Integer> list1 = new ArrayList<Integer>();
ArrayList<Integer> list2 = new ArrayList<Integer>();
File file = new File("C:\\log.txt");
try {
Scanner scanner = new Scanner(file);
Scanner scanner2 = new Scanner(file);
int lineNum = 0;
int lineNum2 = 0;
while (scanner.hasNextLine()){
String line = scanner.nextLine();
String line2 = scanner.nextLine();
lineNum++;
lineNum2++;
if((line.startsWith("Input #0"))) {
list1.add(lineNum);
}
if((line2.contains("damaged"))) {
list2.add(lineNum2);
}
}
This is what I'm getting from the code above:
list1 [5, 262, 304, 488, 523, 1189, 1796, 2503, 2722, 4052, 4201, 4230, 4298, 4312, 4559, 4887, 4903, 5067....]
list2 [1838, 1841, 1842, 1844, 1851, 1861, 1865, 1866, 1868, 1875, 1878, 1879, 1880, 1881, 1886, 1887, 1891....]
Some log data:
Input #0, mpegvideo, from '/cinegy/cinegy/VIDEO/BSF/BLOK 3 - 14. NOVHighb668ca7d201411141051110636.m2v':
.
.
.
.
.
.
Data with damage:
Input #0, mpegvideo, from '/cinegy/cinegy/VIDEO/BSF/BLOK 3 - 14. NOVHighb668ca7d201411141051110636.m2v':
.
.
.
.
.
[error 0x090010] file damaged at 16 09
[error 0x090010] file damaged at 19 15
The log for each individual file does not contain any pattern except for the first 5-6 lines or so. Both damaged and non-damaged files contain info written in 20 to 100+ lines.
So, from these numbers the first result should be number 1796.
I'm pretty much a novice in Java and I need help.
Here's a small code that will do the work, but I don't know if you want redundant values in the result, so I saved them in a list and in a set, choose the one you prefer:
public static void main(String[] args) {
int[] list1 = {5, 262, 304, 488, 523, 1189, 1796, 2503, 2722, 4052, 4201, 4230, 4298, 4312, 4559};
int[] list2 = {1838, 1841, 1842, 1844, 1851, 1861, 1865, 1866, 1868, 1875, 1878, 1879, 1880, 1881};
ArrayList<Integer> resultList = new ArrayList<Integer>();
Set<Integer> resultSet = new HashSet<Integer>();
int j = 0;
for(int i = 0; i < list2.length; i++){
for(; j < list1.length; j++){
if(list1[j] > list2[i])
break;
}
resultList.add(list1[j-1]);
resultSet.add(list1[j-1]);
}
System.out.println(resultList);
System.out.println(resultSet);
}
Output:
[1796, 1796, 1796, 1796, 1796, 1796, 1796, 1796, 1796, 1796, 1796, 1796, 1796, 1796]
[1796]
You defined two scanners (seems unnecessary) but you are only using one of them and calling nextline() twice on it. It looks like that is not intended and as a consequence the results you are getting are erroneous. It would be very helpful if you could post a sample excerpt from your logfile (you can filter the sensitive data) so that we can determine what the best approach is for this.
I think you should scrap your current approach because it does not seem like an efficient way to solve your problem of needing to find filenames of damaged files.
Depending on how your data looks, you can use regular expressions and possibly even extract the filenames directly into a Set.
Edit: Added some rough code that should do the job for you if you are indeed correct that each file starts with "Input #0". As long as there is a pattern in the log data for each file, then you should always be able to extract the data you need directly instead of going through the mess of matching entries from two separate arraylists.
public static void main(String [] args) throws FileNotFoundException{
Set<String> damagedFiles = new LinkedHashSet<String>();
File file = new File("C:\\log.txt");
Scanner scanner = new Scanner(file);
String filename = null;
try {
int lineNum = 0;
while (scanner.hasNextLine()){
String line = scanner.nextLine();
if(line.startsWith("Input #0")){
/*if desired, can use a regex lookahead to get only the path and filename
instead of the entire Input #0 line */
filename = line;
}
if(line.contains("damaged")){
if (filename != null){
damagedFiles.add(filename);
}
}
}
} finally {
scanner.close();
for (String s : damagedFiles){
System.out.println(s);
}
}
}
This is the result I got when running this code on a sample log file where I named the damaged files dmg#.m2v
Input #0, mpegvideo, from '/cinegy/cinegy/VIDEO/BSF/BLOK 3 - 14. dmg1.m2v':
Input #0, mpegvideo, from '/cinegy/cinegy/VIDEO/BSF/BLOK 3 - 14. dmg2.m2v':
Input #0, mpegvideo, from '/cinegy/cinegy/VIDEO/BSF/BLOK 3 - 14. dmg3.m2v':
Input #0, mpegvideo, from '/cinegy/cinegy/VIDEO/BSF/BLOK 3 - 14. dmg4.m2v':
So basically what I need to do is:
Read a text file like this:
[Student ID], [Student Name], Asg 1, 10, Asg 2, 10, Midterm, 40, Final, 40
01234567, Timture Choi, 99.5, 97, 100.0, 99.0
02345678, Elaine Tam, 89.5, 88.5, 99.0, 100
and present it like this (with calculations of rank and average):
ID Name Asg 1 Asg 2 Midterm Final Overall Rank
01234567 Timture Choi 99.5 97.0 100.0 99.0 99.3 1
02345678
Elaine Tam 89.5 88.5 99.0 100.0 97.4 2
Average: 94.5 92.75 99.5 99.5 98.3
Using printf() function
now this is what I have done so far:
import java.io.*;
import java.util.Scanner;
class AssignmentGrades {
public static void main(String args[]) throws Exception {
Scanner filename = new Scanner(System.in);
String fn = filename.nextLine(); //scannig the file name
System.out.println("Enter your name of file : ");
FileReader fr = new FileReader(fn+".txt");
BufferedReader br = new BufferedReader (fr);
String list;
while((list = br.readLine()) !=null) {
System.out.println(list);
}
fr.close();
}
}
So I can ask the user for the name of the file, then read it and print.
Now.. I'm stuck. I think I need to probably put it in to array and split?
String firstrow = br.readLine();
String[] firstrow = firstrow.split(", ");
something like that?.. ugh ive been stuck here for more than an hour
I really need help!! I appreciate your attention!! ( I started to learn java this week)
There are two ways for splitting the input line just read from the file
Using String object's split() method which would return an array. Read more about the split here.
StringTokenizer Class - This class can be used to divide the input string into separate tokens based on a set of delimeter. Here is a good tutorial to get started.
You should be able to get more examples using google :)
In case you want to parse integers from String. Check this.
Here I store the columns as an array of Strings and I store the record set as an ArrayList of String arrays. In the while loop if the column set is not initialized yet (first iteration) I initialize it with the split. Otherwise I add the split to the ArrayList. Import java.util.ArrayList.
String[] columns = null;
ArrayList<String[]> values = new ArrayList<String[]>();
String list;
while((list = br.readLine()) !=null) {
if (columns != null) {
columns = list.split(", ");
} else {
values.add(list.split(", "));
}
}
fr.close();
I am novice to java however, I cannot seem to figure this one out. I have a CSV file in the following format:
String1,String2
String1,String2
String1,String2
String1,String2
Each line are pairs. The 2nd line is a new record, same with the 3rd. In the real word the CSV file will change in size, sometimes it will be 3 records, or 4, or even 10.
My issues is how do I read the values into an array and dynamically adjust the size? I would imagine, first we would have to parse though the csv file, get the number of records/elements, then create the array based on that size, then go though the CSV again and store it in the array.
I'm just not sure how to accomplish this.
Any help would be appreciated.
You can use ArrayList instead of Array. An ArrayList is a dynamic array. ex.
Scanner scan = new Scanner(new File("yourfile"));
ArrayList<String[]> records = new ArrayList<String[]>();
String[] record = new String[2];
while(scan.hasNext())
{
record = scan.nextLine().split(",");
records.add(record);
}
//now records has your records.
//here is a way to loop through the records (process)
for(String[] temp : records)
{
for(String temp1 : temp)
{
System.out.print(temp1 + " ");
}
System.out.print("\n");
}
Just replace "yourfile" with the absolute path to your file.
You could do something like this.
More traditional for loop for processing the data if you don't like the first example:
for(int i = 0; i < records.size(); i++)
{
for(int j = 0; j < records.get(i).length; j++)
{
System.out.print(records.get(i)[j] + " ");
}
System.out.print("\n");
}
Both for loops are doing the same thing though.
You can simply read the CSV into a 2-dimensional array just in 2 lines with the open source library uniVocity-parsers.
Refer to the following code as an example:
public static void main(String[] args) throws FileNotFoundException {
/**
* ---------------------------------------
* Read CSV rows into 2-dimensional array
* ---------------------------------------
*/
// 1st, creates a CSV parser with the configs
CsvParser parser = new CsvParser(new CsvParserSettings());
// 2nd, parses all rows from the CSV file into a 2-dimensional array
List<String[]> resolvedData = parser.parseAll(new FileReader("/examples/example.csv"));
// 3rd, process the 2-dimensional array with business logic
// ......
}
tl;dr
Use the Java Collections rather than arrays, specifically a List or Set, to auto-expand as you add items.
Define a class to hold your data read from CSV, instantiating an object for each row read.
Use the Apache Commons CSV library to help with the chore of reading/writing CSV files.
Class to hold data
Define a class to hold the data of each row being read from your CSV. Let's use Person class with a given name and surname, to be more concrete than the example in your Question.
In Java 16 and later, more briefly define the class as a record.
record Person ( String givenName , String surname ) {}
In older Java, define a conventional class.
package work.basil.example;
public class Person {
public String givenName, surname;
public Person ( String givenName , String surname ) {
this.givenName = givenName;
this.surname = surname;
}
#Override
public String toString ( ) {
return "Person{ " +
"givenName='" + givenName + '\'' +
" | surname='" + surname + '\'' +
" }";
}
}
Collections, not arrays
Using the Java Collections is generally better than using mere arrays. The collections are more flexible and more powerful. See Oracle Tutorial.
Here we will use the List interface to collect each Person object instantiated from data read in from the CSV file. We use the concrete ArrayList implementation of List which uses arrays in the background. The important part here, related to your Question, is that you can add objects to a List without worrying about resizing. The List implementation is responsible for any needed resizing.
If you happen to know the approximate size of your list to be populated, you can supply an optional initial capacity as a hint when creating the List.
Apache Commons CSV
The Apache Commons CSV library does a nice job of reading and writing several variants of CSV and Tab-delimited formats.
Example app
Here is an example app, in a single PersoIo.java file. The Io is short for input-output.
Example data.
GivenName,Surname
Alice,Albert
Bob,Babin
Charlie,Comtois
Darlene,Deschamps
Source code.
package work.basil.example;
import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVRecord;
import java.io.BufferedReader;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;
public class PersonIo {
public static void main ( String[] args ) {
PersonIo app = new PersonIo();
app.doIt();
}
private void doIt ( ) {
Path path = Paths.get( "/Users/basilbourque/people.csv" );
List < Person > people = this.read( path );
System.out.println( "People: \n" + people );
}
private List < Person > read ( final Path path ) {
Objects.requireNonNull( path );
if ( Files.notExists( path ) ) {
System.out.println( "ERROR - no file found for path: " + path + ". Message # de1f0be7-901f-4b57-85ae-3eecac66c8f6." );
}
List < Person > people = List.of(); // Default to empty list.
try {
// Hold data read from file.
int initialCapacity = ( int ) Files.lines( path ).count();
people = new ArrayList <>( initialCapacity );
// Read CSV file.
BufferedReader reader = Files.newBufferedReader( path );
Iterable < CSVRecord > records = CSVFormat.RFC4180.withFirstRecordAsHeader().parse( reader );
for ( CSVRecord record : records ) {
// GivenName,Surname
// Alice,Albert
// Bob,Babin
// Charlie,Comtois
// Darlene,Deschamps
String givenName = record.get( "GivenName" );
String surname = record.get( "Surname" );
// Use read data to instantiate.
Person p = new Person( givenName , surname );
// Collect
people.add( p ); // For real work, you would define a class to hold these values.
}
} catch ( IOException e ) {
e.printStackTrace();
}
return people;
}
}
When run.
People:
[Person{ givenName='Alice' | surname='Albert' }, Person{ givenName='Bob' | surname='Babin' }, Person{ givenName='Charlie' | surname='Comtois' }, Person{ givenName='Darlene' | surname='Deschamps' }]
I have this CSV:
0,102000082,,2,125,'Battery location','Left-hand drive',2,2
0,300000029,102000082,3,895,'Behind the cab','Left',2,-7
0,102000082,,4,127,'Battery location','Right-hand drive',4,4
^-----
I use csvReader to map to a bean
public static List<BatteryBean> loadAndRead{
File csvFilename = new File("C:\\my.csv");
CSVReader csvReader = new CSVReader(new FileReader(csvFilename));
ColumnPositionMappingStrategy strat = new ColumnPositionMappingStrategy();
strat.setType(BatteryBean.class);
String[] columns = new String[] { "ktypnr","sentenceId","parentId","sortOrder", "adjItemNoteId","sentText","itemNoteText","parentSortOrder1","parentSortOrder10" };
strat.setColumnMapping(columns);
CsvToBean csv = new CsvToBean();
List<BatteryBean> list = csv.parse(strat, csvReader);
return list;
}
public static void main(String[] args) {
try {
List<BatteryBean> list = loadAndRead("C:\\work\\battery_report_raw.csv");
for (Object object : list) {
BatteryBean bb = (BatteryBean) object;
System.out.println(bb.getKtypnr());
}
}
So the problem is that the file contains empty strings between ,, and I get an exeption at parsing :
Caused by: java.lang.NumberFormatException: For input string: ""
I resolved. I have another question
Csv file
ktypnr sentence_id parent_id sort_order adj_item_note_id sent_text iem_note_text
0 102000082 2 125 Battery location' Left-hand drive'
0 300000029 102000082 3 895 Behind the cab' Left'
0 102000082 4 127 Battery location' Right-hand drive'
0 300000029 102000082 5 898 Behind the cab' Right'
So if one sentence_id = one parent_id i should combine those two so that looks like this(example first line and second line) but I should consider also the sort_order:
0, Battery location, Left-hand drive, Behind the cab, Left
I don't know how to proceed
Change parentId to String data type in BatteryBean. It seems it is integer.
I have problems with BufferedReader in java. I am reading line by line from large file, parsing lines and inserting into HashMap, but in the result only few lines are in HashMap
Map< Integer, String> data = new HashMap<>(1000000);
int completedTestsCount = 0;
BufferedReader reader = new BufferedReader(new FileReader("file.txt"), 120000);
String line = null;
while ((line = reader.readLine()) != null) {
if (line.contains("START executing FOR"))
{
String tempId = line.substring(42, line.length() - 38);
int startId = Integer.parseInt(tempId);
String dateTime = line.substring(6, 14);
data.put(startId, dateTime);
}
And it's an example of line from file that I want to parse "INFO 00:00:09 - START executing FOR test3625 at Mon Sep 23 00:00:09 GMT+00:00 2013", so keys are test id
HashMap saves data as , where key is unique, so may in your case,
String tempId = line.substring(42, line.length() - 38);
is the key, and as you are reading it from file, this might not be unique. This is the problem, you have to make sure key is unique.
The most likely explanation is that lots of lines in the file have the same startId. Each time you put a key/value pair with the same key, you will actually replace the previous map entry for that key. (A Map maps one key to exactly one value ...)
This might be because the ids are genuinely the same, or it might be that the way you are extracting the id from each line is incorrect; e.g. if the actual id doesn't always start at character 42.
By my counting, character 42 of your example line is the 2 in 3625 ... and that does not seem correct!
For using HashMaps, you will need all the stardId values to be unique.
You should be using a list, instead of a map in this case.
Define a custom KeyValuePair class and add the objects in the list.
class KeyValuePair{
int startId;
String dateTime;
}
List<KeyValuePair> data = new ArrayList<>();
String tempId = line.substring(42, line.length() - 38);
int startId = Integer.parseInt(tempId);
String dateTime = line.substring(6, 14);
data.add(new KeyValuePair(startId, dateTime))