For a project of mine, I'm trying to read a file of Integers and save each line into a file. Each of the files I'm reading have a different amount of lines.
The file would look like
17
72
61
11
63
95
100
Is there a way I can use a loop and save the value in a different variable for each line?
Create a Collection to capture your values. We can use a List so we maintain the input order.
List<Integer> list = new ArrayList<>();
Create a mechanism for reading from the file. A common option is a Scanner. The Scanner is Autoclosable so we'll put it in a try-with-resources block so we don't have to close it ourselves.
try (Scanner scanner = new Scanner(new File(fileName))) {
... do stuff ...
}
While the Scanner has values, read a value and add it to the List.
while (scanner.hasNextInt()) {
list.add(scanner.nextInt());
}
In sum...
List<Integer> list = new ArrayList<>();
try (Scanner scanner = new Scanner(yourFileName)) {
while (scanner.hasNextInt()) {
list.add(scanner.nextInt());
}
}
Assuming you just want to take a line and copy it into another file, here you go: Reading the file as long as there are lines and just change System.out.println(line); (line 16) to create and write to the file (and there just add an incrementing index (like index++ executed at the end of "while") to the filename and pass "line" to the write function).
The only thing that is important: close the BufferedReader and the FileWriter!!
Something like this:
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
public class ReadFileLineByLineUsingBufferedReader {
public static void main(String[] args) {
BufferedReader reader;
try {
reader = new BufferedReader(new FileReader("/path/to/file.txt"));
String line = reader.readLine();
int index = 0;
while (line != null) {
createFile(index);
writeToFile(line, index);
index++;
// read next line
line = reader.readLine();
}
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
public void CreateFile (int index) {
try {
File myObj = new File("filename" + index + ".txt");
} catch (IOException e) {
System.out.println("An error occurred.");
e.printStackTrace();
}
}
public void WriteToFile (String content, int index) {
try {
FileWriter myWriter = new FileWriter("filename" + index + ".txt");
myWriter.write(content);
myWriter.close();
} catch (IOException e) {
System.out.println("An error occurred.");
e.printStackTrace();
}
}
Related
I have a record split into multiple lines in a file. Only way to identify the end of the record is when new record starts with ABC. Below is the sample. File size could be 5-10 GB and I am looking for a efficient java logic ONLY to split the files(no need of reading every line), but splitting logic should a check to start a new file with new record, which should start with "ABC" in this case.
Added few more details, I am just looking for splitting the file and while splitting the last record should be ended correctly in a file.
Can someone please suggest?
HDR
ABCline1goesonforrecord1 //first record
line2goesonForRecord1
line3goesonForRecord1
line4goesonForRecord1
ABCline2goesOnForRecord2 //second record
line2goesonForRecord2
line3goesonForRecord2
line4goesonForRecord2
line5goesonForRecord2
ABCline2goesOnForRecord3 //third record
line2goesonForRecord3
line3goesonForRecord3
line4goesonForRecord3
TRL
So, this is the code that you need. I tested on a 10Gb file and it takes 64 seconds to split the file
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.UncheckedIOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;
import java.util.concurrent.TimeUnit;
public class FileSplitter {
private final Path filePath;
private BufferedWriter writer;
private int fileCounter = 1;
public static void main(String[] args) throws Exception {
long startTime = System.nanoTime();
new FileSplitter(Path.of("/tmp/bigfile.txt")).split();
System.out.println("Time to split " + TimeUnit.NANOSECONDS.toSeconds(System.nanoTime() - startTime));
}
private static void generateBigFile() throws Exception {
var writer = Files.newBufferedWriter(Path.of("/tmp/bigfile.txt"), StandardOpenOption.CREATE, StandardOpenOption.TRUNCATE_EXISTING);
for (int i = 0; i < 100_000; i++) {
writer.write(String.format("ABCline1goesonforrecord%d\n", i + 1));
for (int j = 0; j < 10_000; j++) {
writer.write(String.format("line%dgoesonForRecord%d\n", j + 2, i + 1));
}
}
writer.flush();
writer.close();
}
public FileSplitter(Path filePath) {
this.filePath = filePath;
}
void split() throws IOException {
try (var stream = Files.lines(filePath, StandardCharsets.UTF_8)) {
stream.forEach(line -> {
if (line.startsWith("ABC")) {
closeWriter();
openWriter();
}
writeLine(line);
});
}
closeWriter();
}
private void writeLine(String line) {
if (writer != null) {
try {
writer.write(line);
writer.write("\n");
} catch (IOException e) {
throw new UncheckedIOException("Failed to write line to file part", e);
}
}
}
private void openWriter() {
if (this.writer == null) {
var filePartName = filePath.getFileName().toString().replace(".", "_part" + fileCounter + ".");
try {
writer = Files.newBufferedWriter(Path.of("/tmp/split", filePartName), StandardOpenOption.CREATE, StandardOpenOption.TRUNCATE_EXISTING);
} catch (IOException e) {
throw new UncheckedIOException("Failed to write line to file", e);
}
fileCounter++;
}
}
private void closeWriter() {
if (writer != null) {
try {
writer.flush();
writer.close();
writer = null;
} catch (IOException e) {
throw new UncheckedIOException("Failed to close writer", e);
}
}
}
}
Btw, the solution with Scanner works too.
Regarding not reading all the lines, I don't see why don't you want this. If you choose not not read all the lines (it is possible) then, first you will overcomplicate the solution and second I'm pretty sure that you will loose from performance because of that logic that you have to incorporate in the splitting.
I didn't test this but something like this should work, you are not reading the whole file in memory just one line at a time so it should not be bad.
public void spiltRecords(String filename) {
/*
HDR
ABCline1goesonforrecord1 //first record
line2goesonForRecord1
line3goesonForRecord1
line4goesonForRecord1
ABCline2goesOnForRecord2 //second record
line2goesonForRecord2
line3goesonForRecord2
line4goesonForRecord2
line5goesonForRecord2
ABCline2goesOnForRecord3 //third record
line2goesonForRecord3
line3goesonForRecord3
line4goesonForRecord3
TRL
*/
try {
Scanner scanFile = new Scanner(new File(filename));
// now you do not want to edit the existing file in case things go wrong. one way is to get list of index
// where a new record starts.
LinkedList<Long> startOfRecordIndexes = new LinkedList<>();
long index = 0;
while (scanFile.hasNext()) {
if (scanFile.nextLine().startsWith("ABC")) {
startOfRecordIndexes.add(index);
}
index++;
}
// Once you have the starting index for all records you can iterate through the list and create new records
scanFile = scanFile.reset();
index = 0;
BufferedWriter writer = null;
while (scanFile.hasNext()) {
if (!startOfRecordIndexes.isEmpty() && index == startOfRecordIndexes.peek()) {
if(writer != null) {
writer.write("TRL");
writer.close();
}
writer = new BufferedWriter(new OutputStreamWriter(
new FileOutputStream("Give unique filename"), StandardCharsets.UTF_8));
writer.write("HDR");
writer.write(scanFile.nextLine());
startOfRecordIndexes.remove();
} else {
writer.write(scanFile.nextLine());
}
}
// Close the last record
if(writer != null) {
writer.write("TRL");
writer.close();
}
} catch (IOException e) {
// deal with exception
}
}
I would need to parse a text file with below format and extract only the required values from the text file. the content of the text file is
4564444 FALSE / TRUE 0 name k0LiuME5Q3
4342222 TRUE / TRUE 0 id ab4454jj
i need to get the values after name and id. what is the best way. I used Scanner Class in java but could not get the values. tried with below code.
Scanner scanner = new Scanner(new File("test.txt"));
while(scanner.hasNext()){
String[] tokens = scanner.nextLine().split(" ");
String last = tokens[tokens.length - 1];
System.out.println(last);
}
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.ArrayList;
public class Read_Text_File {
public static void main(String[] args) {
System.out.println(getValues());
}
public static ArrayList<String> getValues() {
FileInputStream stream = null;
try {
stream = new FileInputStream("src/resources/java_txt_file.txt");
} catch (FileNotFoundException e) {
e.printStackTrace();
}
BufferedReader reader = new BufferedReader(new InputStreamReader(stream));
String strLine;
ArrayList<String> lines = new ArrayList<String>();
try {
while ((strLine = reader.readLine()) != null) {
String lastWord = strLine.substring(strLine.lastIndexOf(" ")+1);
lines.add(lastWord);
}
} catch (IOException e) {
e.printStackTrace();
}
try {
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
return lines;
}
}
Output:
[k0LiuME5Q3, ab4454jj]
You need to split by space, not semicolon:
String[] tokens = scanner.nextLine().split(" ");
read the data line by line, on each line use String.split("\s*") to get the parts without the whitespace into an array with 7 elements. the last one of these elements is what you are looking for.
I have one input.txt file which consist on let suppose 520 lines.
I have to make a code in java which will act like this.
Create first file named file-001.txt from first 200 lines. then create another file-002 from 201-400 lines. then file-003.txt from remaining lines.
I have coded this, it just write first 200 lines. What changes I need to make in order to update its working to above scenario.
public class DataMaker {
public static void main(String args[]) throws IOException{
DataMaker dm=new DataMaker();
String file= "D:\\input.txt";
int roll=1;
String rollnum ="file-00"+roll;
String outputfilename="D:\\output\\"+rollnum+".txt";
String urduwords;
String path;
ArrayList<String> where = new ArrayList<String>();
int temp=0;
try(BufferedReader br = new BufferedReader(new FileReader(file))) {
for(String line; (line = br.readLine()) != null; ) {
++temp;
if(temp<201){ //may be i need some changes here
dm.filewriter(line+" "+temp+")",outputfilename);
}
}
} catch (FileNotFoundException e) {
System.out.println("File not found");
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
void filewriter(String linetoline,String filename) throws IOException{
BufferedWriter fbw =null;
try{
OutputStreamWriter writer = new OutputStreamWriter(
new FileOutputStream(filename, true), "UTF-8");
fbw = new BufferedWriter(writer);
fbw.write(linetoline);
fbw.newLine();
}catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
finally {
fbw.close();
}
}
}
One way can be use of if else but I cant just use it because my actual file is 6000+ lines.
I want this code to work like I run the code and give me 30+ output files.
You can change the following bit:
if(temp<201){ //may be i need some changes here
dm.filewriter(line+" "+temp+")",outputfilename);
}
to this:
dm.filewriter(line, "D:\\output\\file-00" + ((temp/200)+1) + ".txt");
This will make sure first 200 lines go to first file, next 200 lines go to next file and so on.
Also, you might want to batch 200 lines together and write them in one go rather than creating a writer everytime and write to file.
You may have a method that creates the Writer to the current File, reads up to limit number of lines, closes the Writer to the current File, then returns true if it had enough to read , false if it couldn't read the limit number of lines (i.e, abort next call, don't attempt to read more lines or write next file).
Then you would call this in a loop , passing the Reader, the new file name, and the limit number.
Here is an example :
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.FileReader;
import java.io.IOException;
import java.io.OutputStreamWriter;
public class DataMaker {
public static void main(final String args[]) throws IOException {
DataMaker dm = new DataMaker();
String file = "D:\\input.txt";
int roll = 1;
String rollnum = null;
String outputfilename = null;
boolean shouldContinue = false;
try (BufferedReader br = new BufferedReader(new FileReader(file))) {
do {
rollnum = "file-00" + roll;
outputfilename = "D:\\output\\" + rollnum + ".txt";
shouldContinue = dm.fillFile(outputfilename, br, 200);
roll++;
} while (shouldContinue);
} catch (FileNotFoundException e) {
System.out.println("File not found");
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
private boolean fillFile(final String outputfilename, final BufferedReader reader, final int limit)
throws IOException {
boolean result = false;
String line = null;
BufferedWriter fbw = null;
int temp = 0;
try {
OutputStreamWriter writer = new OutputStreamWriter(
new FileOutputStream(outputfilename, true), "UTF-8");
fbw = new BufferedWriter(writer);
while (temp < limit && ((line = reader.readLine()) != null)) {
temp++;
fbw.write(line);
fbw.newLine();
}
// abort if we didn't manage to read the "limit" number of lines
result = (temp == limit);
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
} finally {
fbw.close();
}
return result;
}
}
I'm looking to sort the contacts read from a file in alphabetical order by last name to the console? How would I go about doing so? The contacts are already written to file starting with the last name, I just want to read them back into the application in alphabetical order when a user wants to view the contacts in the console.
// Read from file, print to console. by XXXXX
// ----------------------------------------------------------
int counter = 0;
String line = null;
// Location of file to read
File file = new File("contactlist.csv");
try {
Scanner scanner = new Scanner(file);
while (scanner.hasNextLine()) {
line = scanner.nextLine();
System.out.println(line);
counter++;
}
scanner.close();
} catch (FileNotFoundException e) {
}
System.out.println("\n" + counter + " contacts in records.");
}
break;
// ----------------------------------------------------------
// End read file to console. by XXXX
Before printing, add each line to a sorted set, as a TreeSet:
Set<String> lines = new TreeSet<>();
while (scanner.hasNextLine()) {
line = scanner.nextLine();
lines.add(line);
counter++;
}
for (String fileLine : lines) {
System.out.println(fileLine);
}
package main.java.com.example;
import java.io.*;
import java.net.URL;
import java.util.Set;
import java.util.TreeSet;
public class ReadFromCSV {
public static void main(String[] args) {
try {
final ClassLoader loader = ReadFromCSV.class.getClassLoader();
URL url = loader.getResource("csv/contacts.csv");
if (null != url) {
File f = new File(url.getPath());
BufferedReader br = new BufferedReader(new FileReader(f));
Set<String> set = new TreeSet<String>();
String str;
while ((str = br.readLine()) != null) {
set.add(str);
}
for (String key : set) {
System.out.println(key);
}
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Read names from the file, put them in an object of class SortedSet.
So this is what I have so far :
public String[] findStudentInfo(String studentNumber) {
Student student = new Student();
Scanner scanner = new Scanner("Student.txt");
// Find the line that contains student Id
// If not found keep on going through the file
// If it finds it stop
// Call parseStudentInfoFromLine get the number of courses
// Create an array (lines) of size of the number of courses plus one
// assign the line that the student Id was found to the first index value of the array
//assign each next line to the following index of the array up to the amount of classes - 1
// return string array
}
I know how to find if a file contains the string I am trying to find but I don't know how to retrieve the whole line that its in.
This is my first time posting so If I have done anything wrong please let me know.
You can do something like this:
File file = new File("Student.txt");
try {
Scanner scanner = new Scanner(file);
//now read the file line by line...
int lineNum = 0;
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
lineNum++;
if(<some condition is met for the line>) {
System.out.println("ho hum, i found it on line " +lineNum);
}
}
} catch(FileNotFoundException e) {
//handle this
}
Using the Apache Commons IO API https://commons.apache.org/proper/commons-io/ I was able to establish this using FileUtils.readFileToString(file).contains(stringToFind)
The documentation for this function is at https://commons.apache.org/proper/commons-io/javadocs/api-2.4/org/apache/commons/io/FileUtils.html#readFileToString(java.io.File)
Here is a java 8 method to find a string in a text file:
for (String toFindUrl : urlsToTest) {
streamService(toFindUrl);
}
private void streamService(String item) {
try (Stream<String> stream = Files.lines(Paths.get(fileName))) {
stream.filter(lines -> lines.contains(item))
.forEach(System.out::println);
} catch (IOException e) {
e.printStackTrace();
}
}
When you are reading the file, have you considered reading it line by line? This would allow you to check if your line contains the file as your are reading, and you could then perform whatever logic you needed based on that?
Scanner scanner = new Scanner("Student.txt");
String currentLine;
while((currentLine = scanner.readLine()) != null)
{
if(currentLine.indexOf("Your String"))
{
//Perform logic
}
}
You could use a variable to hold the line number, or you could also have a boolean indicating if you have passed the line that contains your string:
Scanner scanner = new Scanner("Student.txt");
String currentLine;
int lineNumber = 0;
Boolean passedLine = false;
while((currentLine = scanner.readLine()) != null)
{
if(currentLine.indexOf("Your String"))
{
//Do task
passedLine = true;
}
if(passedLine)
{
//Do other task after passing the line.
}
lineNumber++;
}
This will find "Mark Sagal" in Student.txt. Assuming Student.txt contains
Student.txt
Amir Amiri
Mark Sagal
Juan Delacruz
Main.java
import java.io.BufferedReader;
import java.io.FileReader;
import java.util.ArrayList;
public class Main {
public static void main(String[] args) {
final String file = "Student.txt";
String line = null;
ArrayList<String> fileContents = new ArrayList<>();
try {
FileReader fReader = new FileReader(file);
BufferedReader fileBuff = new BufferedReader(fReader);
while ((line = fileBuff.readLine()) != null) {
fileContents.add(line);
}
fileBuff.close();
} catch (Exception e) {
System.out.println(e.getMessage());
}
System.out.println(fileContents.contains("Mark Sagal"));
}
}
I am doing something similar but in C++. What you need to do is read the lines in one at a time and parse them (go over the words one by one). I have an outter loop that goes over all the lines and inside that is another loop that goes over all the words. Once the word you need is found, just exit the loop and return a counter or whatever you want.
This is my code. It basically parses out all the words and adds them to the "index". The line that word was in is then added to a vector and used to reference the line (contains the name of the file, the entire line and the line number) from the indexed words.
ifstream txtFile;
txtFile.open(path, ifstream::in);
char line[200];
//if path is valid AND is not already in the list then add it
if(txtFile.is_open() && (find(textFilePaths.begin(), textFilePaths.end(), path) == textFilePaths.end())) //the path is valid
{
//Add the path to the list of file paths
textFilePaths.push_back(path);
int lineNumber = 1;
while(!txtFile.eof())
{
txtFile.getline(line, 200);
Line * ln = new Line(line, path, lineNumber);
lineNumber++;
myList.push_back(ln);
vector<string> words = lineParser(ln);
for(unsigned int i = 0; i < words.size(); i++)
{
index->addWord(words[i], ln);
}
}
result = true;
}
Here is the code of TextScanner
public class TextScanner {
private static void readFile(String fileName) {
try {
File file = new File("/opt/pol/data22/ds_data118/0001/0025090290/2014/12/12/0029057983.ds");
Scanner scanner = new Scanner(file);
while (scanner.hasNext()) {
System.out.println(scanner.next());
}
scanner.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
if (args.length != 1) {
System.err.println("usage: java TextScanner1"
+ "file location");
System.exit(0);
}
readFile(args[0]);
}
}
It will print text with delimeters