Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I have file filled with text lines. File's size is about 150GB. What the best solution to sort all of the lines by alphabet? For example:
INPUT:
We’re pretty sure someone famous once said that. breathe in the
millennial generation. Anyway, It’s the phrase that we live by
here at Alphabet. have an unrivalled energy and passion for
creating brands that live and
OUTPUT:
Anyway, It’s the phrase that we live by here at Alphabet. breathe
in the millennial generation. have an unrivalled energy and
passion for creating brands that live and We’re pretty sure
someone famous once said that.
We usually don’t need all of the lines in the file in memory at once – instead, we just need to be able to iterate through each one, do some processing and throw it away. So, this is exactly what we’re going to do – iterate through the lines without holding the in memory.
FileInputStream inputStream = null;
Scanner sc = null;
try {
inputStream = new FileInputStream(path);
sc = new Scanner(inputStream, "UTF-8");
while (sc.hasNextLine()) {
String line = sc.nextLine();
// System.out.println(line);
}
// note that Scanner suppresses exceptions
if (sc.ioException() != null) {
throw sc.ioException();
}
} finally {
if (inputStream != null) {
inputStream.close();
}
if (sc != null) {
sc.close();
}
}
This solution will iterate through all the lines in the file – allowing for processing of each line – without keeping references to them – and in conclusion, without keeping them in memory. http://www.baeldung.com/java-read-lines-large-file
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
I need to edit a lot of text files effectively and fast! What is the best thing I can do?
I already come up with this function:
private boolean edit(File source)
{
if (!source.getAbsolutePath().endsWith(".java")) //Java text files only
return false;
String l, str = "", orig = "";
try
{
BufferedReader r = new BufferedReader(new FileReader(source));
while ((l = r.readLine()) != null)
{
orig = str += l+"\n";
}
r.close();
for (Entry<String, String> e : mappings.entrySet()) //Replacing string by HashMap mappings!
str = fastReplace(str, e.getKey(), e.getValue()); //Faster alterntive to String#replaceAll
if (!str.equals(orig))
{
BufferedWriter bf = new BufferedWriter(new FileWriter(source));
bf.write(str);
bf.close();
return true;
}
}
catch (Exception e)
{
doLog(e.toString()); //Logging exception but unimportant for us...
}
return false;
}
I found my function a little bit clumsy because it first needs to read text file into string then edit it and write it back after that. So the question is. Is there any better and faster way to edit text file? I mean for example without necessity to turning it into string and then writing it back. For example is there a way to edit file directly as a text file or writing it without overriding the same unchanged parts of the file or any faster way to read and write file? Or is my function actually already fastest as it can be?
In case somebody is wondering what my "fastReplace" function does then check this Faster alternatives to replace method in a Java String? but I do not think it is important.
If you need to replace a string by another one of the exact same size byte for byte, then you could read the data sequentially in chunks with multiple block size, replace the desired spots and write the data back if a change has been made. If no change was made then there is no need to write back the block. In the best scenario, you will save few I/O operations, at the cost of significant code complexity.
If your editing is more complex and involves string insertions, then you have no way out of reading and writing back the entire text.
Early optimization is a bad idea. Source code files hardly span a single block and in your case you will probably save no time or space.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I am stuck on some homework as i am new to java and still learning. I am wondering if it is possible to find a word in a .txt file and output the line that the word is on. I also need to allow a user to make a choice based on what is displayed back.
Example :
Word is Details
Txt file contains
Details on lion
Details on tiger
Output : "Details on tiger"
Thank in advanced for any help
This question has been answered before, but anyway You can go with this apporach.
Simply put:
Create a Scanner object and pass the required file into the
constructor as a new file object.
Iterate over the file with a
while loop until you find the specified string.
In order to store the lines that contains the desired word,we decalre a new string variable
Here's the code snippet:
Scanner scanner= new Scanner(new File("filename.txt"));
String lines = "";
while(scanner.hasNextLine()){
String stringLine = scanner.nextLine();
if(stringLine.indexOf("YOUR_WORD") != -1){
//print whatever you want here
System.out.println(stringLine);
//add every line that contains stringLine into another string;
lines+=stringLine;
}
}
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 5 years ago.
Improve this question
I am reading a text file in Java that looks like this,
"
Q1. You are given a train data set having 1000 columns and 1 million rows. The data set is based on a classification problem. Your manager has asked you to reduce the dimension of this data so that model computation time can be reduced. Your machine has memory constraints. What would you do? (You are free to make practical assumptions.)
Q2. Is rotation necessary in PCA? If yes, Why? What will happen if you don’t rotate the components?
Q3. You are given a data set. The data set has missing values which spread along 1 standard deviation from the median. What percentage of data would remain unaffected? Why? "
Now, I want to read this file and then store each of these sentences(questions) in a string array. How can I do that in java?
I tried this,
String mlq = new String(Files.readAllBytes(Paths.get("MLques.txt")));
String[] mlq1=mlq.split("\n\n");
But this is not working.
Try this
String mlq = new String(Files.readAllBytes(Paths.get("MLQ.txt")));
String[] mlq1=mlq.split("\r\n\r\n");
System.out.println(mlq1.length);
System.out.println(Arrays.toString(mlq1));
This should do it by line gap of 2 lines.
File file = new File("C:\\MLques.txt");
BufferedReader br = new BufferedReader(new FileReader(file));
String st;
while ((st = br.readLine()) != null) {
System.out.println(st + "\n");
}
I think it will work.
This is a piece of code from one of my project.
public static List<String> readStreamByLines(InputStream in) throws IOException {
return IOUtils.readLines(in, StandardCharsets.UTF_8).stream()
.map(String::trim)
.collect(Collectors.toList());
}
But!!! If you have really big file, then collecting all content into a List is not good. You have to read InputStream line by line and do all you need for every single row.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I am making a some large project. In that I am stuck at one place. I am using the following code to read from the file:
String str = null;
FileReader fr = new FileReader("H:\\Eclipse\\Emulator\\progin8085.txt");
BufferedReader br = new BufferedReader(fr);
str = br.readLine(); //using it in a loop from 0 to total_no_of_lines
Now I when I reached suppose at line 8(counting lines numbers while entering data to the file), I want to go back to line 3 or 4 or any and again want to read and execute each statement. How to read previous statements using BufferedReader only? If it is not possible, any other solution?
Some Readers support marks. You can use those to rewind the file. A particularly useful Reader (imo) is the LineNumberReader and it supports marks. This sort of code might suit your needs.
public static int final READ_AHEAD_LIMIT = 100000;
LineNumberReader lnReader = new LineNumberReader(reader);
while (youWantToRead) {
...
if (mightBeInterestingLater) {
lnReader.mark(READ_AHEAD_LIMIT);
}
...
if (nowWantToRewind) {
lnReader.reset();
// We're now at whatever place mark() was last called at.
}
Save all the lines of text file in String[] StrArray and do whatever you want.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I have two files and I want to individually read them then use the the first file to create 5 objects and then use the second file to add the parameters to be passed to a constructor, question is is I am not quite sure how to do that.
What I wanted to do was to loop the hasNextLine and assign the next line to a string and have an object created from that string name, then passing variables in the same way but I see that may not be possible in Java. If it isn't what is another way I could approach this?
Well I was trying to do something like this
while(salesPersonScanner.hasNextLine()){
String personName = salesPersonScanner.nextLine();
SalesPerson personName = new SalesPerson();
}
You can use a BufferedReader to iterate over lines:
final BufferedReader reader = new BufferedReader(new FileReader("/path/to/file"));
String line;
while ((line = reader.readLine()) != null) {
// Create your object from the string
}
You have two options:
You can read all of the required parameters from files and then pass them all to the constructor to create a new object.
You can create an object with your constructor and use setters to set each of the instance variables.
I personally suggest the first approach unless you have to set lots of variables.