We Keep Coding

How to split single text file into multiple with character as delimiter

I have a text document that has multiple separate entries all compiled into one .log file.
The format of the file looks something like this.
$#UserID#$
Date
User
UserInfo
SteamFriendID
=========================
<p>Message</p>
$#UserID#$
Date
User
UserInfo
SteamFriendID
========================
<p>Message</p>
$#UserID#$
Date
User
UserInfo
SteamFriendID
========================
<p>Message</p>
I'm trying to take everything in between the instances of "$#UserID$#", and print them into separate text files.
So far, with the looking that I've done, I tried implementing it using StringBuilder in something like this.
FileReader fr = new FileReader(“Path to raw file.”);
int idCount = 1;
FileWriter fw = new FileWriter("Path to parsed files" + idCount);
BufferedReader br = new BufferedReader(fr);
//String line, date, user, userInfo, steamID;
StringBuilder sb = new StringBuilder();
//br.readLine();
while ((line = br.readLine()) != null) {
if(line.substring(0,1).contains("$#")) {
if (sb.length() != 0) {
File file = new File("Path to parsed logs" + idCount);
PrintWriter pw = new PrintWriter(file, "UTF-8");
pw.println(sb.toString());
pw.close();
//System.out.println(sb.toString());
Sb.delete(0, sb.length());
idCount++;
}
continue;
}
sb.append(line + "\r\n");
}
But this only gives me the first 2 of the entries in separate parsed files. Leaving the 3rd one out for some reason.
The other way I was thinking about doing it was reading in all the lines using .readAllLines(), store the list as an array, loop through the lines to find "$#", get that line's index & then recursively write the lines starting at the index given.
Does anyone know of a better way to do this, or would be willing to explain to me why I'm only getting two of the three entries parsed?

Short / quick fix is to write the contents of the StringBuilder once after your while loop like this:
public static void main(String[] args) {
try {
int idCount = 1;
FileReader fr = new FileReader("<path to desired file>");
BufferedReader br = new BufferedReader(fr);
//String line, date, user, userInfo, steamID;
StringBuilder sb = new StringBuilder();
//br.readLine();
String line = "";
while ((line = br.readLine()) != null) {
if(line.startsWith("$#")) {
if (sb.length() != 0) {
writeFile(sb.toString(), idCount);
System.out.println(sb);
sb.setLength(0);
idCount++;
}
continue;
}
sb.append(line + "\r\n");
}
if (sb.length() != 0) {
writeFile(sb.toString(), idCount);
System.out.println(sb);
idCount++;
}
} catch (IOException e) {
e.printStackTrace();
}
}
private static void writeFile(String content, int id) throws IOException
{
File file = new File("<path to desired dir>\\ID_" + id + ".txt");
file.createNewFile();
PrintWriter pw = new PrintWriter(file, "UTF-8");
pw.println(content);
pw.close();
}
I've changed two additional things:
the condition "line.substring(0,1).contains("$#")" did not work properly, the substring call only returns one character, but is compared to two characters -> never true. I changed that to use the 'startsWith' method.
After the content of the StringBuilder is written to file, you did not reset or empty it, resulting in the second and third file containing every previous blocks aswell (thrid file equals input then...). So thats done with "sb.setLength(0);".

Creating an inverted index with limited memory in java

Im curious on how create an Inverted Index on data that doesn't fit into memory. So right now I'm reading a file directory and indexing the files based on the contents inside the file, I am using a HashMap to store the index. The code below is a snippet from a function I use and I call the function on an entire directory. What do I do if this directory was just massive and the HashMap can't fit all the entries. Yes, This does sound like premature optimization. Im just having fun. I don't want to use Lucene so don't even mention it because I'm tired as to seeing that as the majority answer to "Index" stuff. This HashMap is my only constraint everything else is stored in files to easily reference stuff later on.
Im just curious how I can do this since it stores it in the map like so
keyword -> file1,file2,file3,etc..(locations)
keyword2 -> file9,file11,file13,etc..(locations)
My thoughts were to create a file which would some how be able to update itself to be like the format above but I feel thats not efficient.
Code Snippet
br = new BufferedReader(new FileReader(file));
while ((line = br.readLine()) != null) {
for (String _word : line.split("\\W+")) {
word = _word.toLowerCase();
if (!ignore_words.contains(word)) {
fileLocations = index.get(word);
if (fileLocations == null) {
fileLocations = new LinkedList<Long>();
index.put(word, fileLocations);
}
fileLocations.add(file_offset);
}
}
}
br.close();
Update:
So I managed to come up with something, but performance wise I feel this is slow, especially if there was a large amount of data. I basically created a file that would just have to word and its offset on each line the word appeared.Lets name it index.txt.
It had the format of like so
word1:offset
word2:offset
word1:offset <-encountered again.
word3:offset
etc...
I then created multiple files for each word and appended the offset to that file each time it was encountered in the index.txt file.
So basically the format of the word files are like so
word1.txt -- Format
word1:offset1:offset2:offset3:offset4...and so on
each time word1 is encountered in the index.txt file it would append it to the word1.txt file and add to end.
Then finally, I go through all the word files I created and overwrite the index.txt file with the final output in the index file looking like so
word1:offset1:offset2:offset3:offset4:...
word2:offset9:offset11:offset13:offset14:...
etc..
Then to finish it up, I delete all the word files.
The nasty code snippet for this is below, its a fair amount.
public void createIndex(String word, long file_offset)
{
PrintWriter writer;
try {
writer = new PrintWriter(new FileWriter(this.file,true));
writer.write(word + ":" + file_offset + "\n");
writer.close();
}
catch (IOException ioe)
{
ioe.printStackTrace();
}
}
public void mergeFiles()
{
String line;
String wordLine;
String[] contents;
String[] wordContents;
BufferedReader reader;
BufferedReader mergeReader;
PrintWriter writer;
PrintWriter mergeWriter;
try {
reader = new BufferedReader(new FileReader(this.file));
while((line = reader.readLine()) != null)
{
contents = line.split(":");
writer = new PrintWriter(new FileWriter(
new File(contents[0] + ".txt"),true));
if(this.words.get(contents[0]) == null)
{
this.words.put(contents[0], contents[0]);
writer.write(contents[0] + ":");
}
writer.write(contents[1] + ":");
writer.close();
}
//This could be put in its own method below.
mergeWriter = new PrintWriter(new FileWriter(this.file));
for(String word : this.words.keySet())
{
mergeReader = new BufferedReader(
new FileReader(new File(word + ".txt")));
while((wordLine = mergeReader.readLine()) != null)
{
mergeWriter.write(wordLine + "\n");
}
}
mergeWriter.close();
deleteFiles();
}
catch(IOException ioe)
{
ioe.printStackTrace();
}
}
public void deleteFiles()
{
File toDelete;
for(String word : this.words.keySet())
{
toDelete = new File(word + ".txt");
if(toDelete.exists())
{
toDelete.delete();
}
}
}

Remove a Specific Line From text file

I trying to remove a specific line from a file. But I have a problem in deleting a particular line from the text file. Let's said, my text file I want to remove Blueberry in the file following:
Old List Text file:
Chocolate
Strawberry
Blueberry
Mango
New List Text file:
Chocolate
Strawberry
Mango
I tried to run my Java program, when I input for delete and it didn't remove the line from the text file.
Output:
Please delete:
d
Blueberry
Remove:Blueberry
When I open my text file, it keep on looping with the word "Blueberry" only.
Text file:
Blueberry
Blueberry
Blueberry
Blueberry
Blueberry
Blueberry
Blueberry
Blueberry
My question is how to delete the specific line from the text file?
Here is my Java code:
String input="Please delete: ";
System.out.println(input);
try
{
BufferedReader reader = new BufferedReader
(new InputStreamReader (System.in));
line = reader.readLine();
String inFile="list.txt";
String line = "";
while(!line.equals("x"))
{
switch(line)
{
case "d":
line = reader.readLine();
System.out.println("Remove: " + line);
String lineToRemove="";
FileWriter removeLine=new FileWriter(inFile);
BufferedWriter change=new BufferedWriter(removeLine);
PrintWriter replace=new PrintWriter(change);
while (line != null) {
if (!line.trim().equals(lineToRemove))
{
replace.println(line);
replace.flush();
}
}
replace.close();
change.close();
break;
}
System.out.println(input);
line = reader.readLine();
}
}
catch(Exception e){
System.out.println("Error!");
}

Let's take a quick look at your code...
line = reader.readLine();
//...
while (line != null) {
if (!line.trim().equals(lineToRemove))
{
replace.println(line);
replace.flush();
}
}
Basically, you read the first line of the file and then repeatedly compare it with the lineToRemove, forever. This loop is never going to exit
This is a proof of concept, you will need to modify it to your needs.
Basically, what you need to ensure you're doing, is you're reading each line of the input file until there are no more lines
// All the important information
String inputFileName = "...";
String outputFileName = "...";
String lineToRemove = "...";
// The traps any possible read/write exceptions which might occur
try {
File inputFile = new File(inputFileName);
File outputFile = new File(outputFileName);
// Open the reader/writer, this ensure that's encapsulated
// in a try-with-resource block, automatically closing
// the resources regardless of how the block exists
try (BufferedReader reader = new BufferedReader(new FileReader(inputFile));
BufferedWriter writer = new BufferedWriter(new FileWriter(outputFile))) {
// Read each line from the reader and compare it with
// with the line to remove and write if required
String line = null;
while ((line = reader.readLine()) != null) {
if (!line.equals(lineToRemove)) {
writer.write(line);
writer.newLine();
}
}
}
// This is some magic, because of the compounding try blocks
// this section will only be called if the above try block
// exited without throwing an exception, so we're now safe
// to update the input file
// If you want two files at the end of his process, don't do
// this, this assumes you want to update and replace the
// original file
// Delete the original file, you might consider renaming it
// to some backup file
if (inputFile.delete()) {
// Rename the output file to the input file
if (!outputFile.renameTo(inputFile)) {
throw new IOException("Could not rename " + outputFileName + " to " + inputFileName);
}
} else {
throw new IOException("Could not delete original input file " + inputFileName);
}
} catch (IOException ex) {
// Handle any exceptions
ex.printStackTrace();
}
Have a look at Basic I/O and The try-with-resources Statement for some more details

Reading input from console, reading file and writing to a file needs to be distinguished and done separately. you can not read and write file at the same time. you are not even reading your file. you are just comparing your console input indefinitely in your while loop.In fact, you are not even setting your lineTobeRemoved to the input line. Here is one way of doing it.
Algorithm:
Read the console input (your line to delete) then start reading the file and looking for line to delete by comparing it with your input line. if the lines do not match match then store the read line in a variable otherwise throw this line since you want to delete it.
Once finished reading, start writing the stored lines on the file. Now you will have updated file with one line removed.
public static void main(String args[]) {
String input = "Please delete: ";
System.out.println(input);
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(
System.in));
String line = reader.readLine();
reader.close();
String inFile = "list.txt";
System.out.println("Remove: " + line);
String lineToRemove = line;
StringBuffer newContent = new StringBuffer();
BufferedReader br = new BufferedReader(new FileReader(inFile));
while ((line = br.readLine()) != null) {
if (!line.trim().equals(lineToRemove)) {
newContent.append(line);
newContent.append("\n"); // new line
}
}
br.close();
FileWriter removeLine = new FileWriter(inFile);
BufferedWriter change = new BufferedWriter(removeLine);
PrintWriter replace = new PrintWriter(change);
replace.write(newContent.toString());
replace.close();
}
catch (Exception e) {
e.printStackTrace();
}
}

jTextArea saves only first line of text in text file using BufferedReader?

I am trying to save the multiple line output in a text file from my jTextArea(named as "outputarea" in a code) to my desired path, Everything is OK but the file being saved do not contain the whole output, but only first line oof text. I am using "\n" to break the line in jtextarea while giving multiple line output, does that make any difference or any other problem in this code, This code is just the code on saveAs button, output is coming from another methods I've created. Thanks in Advance!
private void saveAs() {
FileDialog fd = new FileDialog(home.this, "Save", FileDialog.SAVE);
fd.show();
if(fd.getFile()!=null)
{
fn=fd.getFile();
dir=fd.getDirectory();
filename = dir + fn +".txt";
setTitle(filename);
try
{
DataOutputStream d=new DataOutputStream(new FileOutputStream(filename));
holdText = outputarea.getText();
BufferedReader br = new BufferedReader(new StringReader(holdText));
while((holdText = br.readLine())!=null)
{
d.writeBytes(holdText+"\r\n");
d.close();
}
}
catch (Exception e)
{
System.out.println("File not found");
}
outputarea.requestFocus();
save(filename);
}
}

You should put the d.close(); after the completion of while loop, because just after writing the first line in the file using DataOutputStream, you are closing it and you don't let it to fulfill the whole job.
You can see even an error is wrote in your console:
File not found
This is not because it doesn't find your file, it's because in the iterations after the first, it tries to write into a closed stream. So only the first line is wrote then. So change you code like this:
while ((holdText = br.readLine()) != null) {
d.writeBytes(holdText + "\r\n");
}
d.close();
Also I can advise to use a PrintWriter instead of DataOutputStream. Then you can easily change the writeBytes into println method. In this way you don't need to append \r\n manually to each line you write.
Another good hint is to use a try-with-resource (in case you use java 7 or later) or at least a finally block to close your streams either way:
String holdText = outputarea.getText();
try (PrintWriter w = new PrintWriter(new File(filename));
BufferedReader br = new BufferedReader(new StringReader(holdText))) {
while ((holdText = br.readLine()) != null) {
w.println(holdText);
}
} catch (Exception e) {
System.out.println("File not found");
}
Good Luck.

Deleting and renaming file

So I'm trying to delete a line of data from a file, which I have successfully done by opening a new file and writing all the information that doesn't match with the data that I would like to remove. The problem is, after I have done that, I would like to delete my original file, and then rename the new file with excludes the information I wanted to delete, to the same name as the original file. I have added in the code to do this, but for some reason it's not working.
public static void delete() throws IOException
{
File inputFile = new File("Elements.txt");
File tempFile = new File("myTempFile.txt");
BufferedReader reader = new BufferedReader(new FileReader(inputFile));
BufferedWriter writer = new BufferedWriter(new FileWriter(tempFile));
String element = JOptionPane.showInputDialog(null, "Enter the name of the Element you wish to delete.", "Remove an Element.", JOptionPane.INFORMATION_MESSAGE);;
String currentLine;
while((currentLine = reader.readLine()) != null) {
String trimmedLine = currentLine.trim();
if(trimmedLine.startsWith(element)) continue;
writer.write(currentLine + System.getProperty("line.separator"));
}
writer.close();
reader.close();
inputFile.delete();
tempFile.renameTo(inputFile);
JOptionPane.showMessageDialog(null, "Data has been removed from the file: Elements.txt");
}
As you can see near the bottom, I have these lines:
inputFile.delete();
tempFile.renameTo(inputFile);
These lines are meant to delete my original file(inputFile) and then rename my new file(tempFile) to the file name that the original file had. After running the code however, I simply get a file called "myTempFile.txt" which has succesfully deleted the line of data that I wanted, but my original file is still present and it wasn't deleted, neither was the new file renamed to the original file.
Any idea why this is happening?

Use the java.nio.file API. This is 2015.
final Path src = Paths.get("Elements.txt").toAbsolutePath();
final Path tmp = src.resolveSibling("Elements.txt.new");
try (
final BufferedReader reader = Files.newBufferedReader(src, StandardCharsets.UTF_8);
final BufferedWriter writer = Files.newBufferedWriter(tmp, StandardCharsets.UTF_8,
StandardOpenOption.CREATE_NEW);
) {
// yadda yadda
}
Files.move(tmp, src, StandardCopyOption.REPLACE_EXISTING);
File is unreliable. It has always been.

in such a case i would start fiddling around, reading documentation and maybe googling for a bit. But i will give you an answer, too!
inputFile.delete();
This could go wrong, for example if you have your file opened in a text editor.
Luckily delete() returns a boolean, try checking that!
Also as Niels correctly mentioned File.renameTo() is quite unrelieble if you have access to Java 7 use the files.nio alternative. In Java 7 you can use Files.move(Path source, Path target, CopyOption... options)
Docs for Java 7 Files: http://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html

But your very code works correctly for me. I only change the path to the file and I make sure the file is not opened in editor
public class NewClass {
public static void main(String[] args) {
try {
delete();
} catch (IOException ex) {
Logger.getLogger(NewClass.class.getName()).log(Level.SEVERE, null, ex);
}
}
public static void delete() throws IOException {
File inputFile = new File("C:\\Users\\olyjosh\\Desktop\\Elements.txt");
File tempFile = new File("C:\\Users\\olyjosh\\Desktop\\myTempFile.txt");
BufferedReader reader = new BufferedReader(new FileReader(inputFile));
BufferedWriter writer = new BufferedWriter(new FileWriter(tempFile));
String element = JOptionPane.showInputDialog(null, "Enter the name of the Element you wish to delete.", "Remove an Element.", JOptionPane.INFORMATION_MESSAGE);;
String currentLine;
while ((currentLine = reader.readLine()) != null) {
String trimmedLine = currentLine.trim();
if (trimmedLine.startsWith(element)) {
continue;
}
writer.write(currentLine + System.getProperty("line.separator"));
}
writer.close();
reader.close();
inputFile.delete();
tempFile.renameTo(inputFile);
JOptionPane.showMessageDialog(null, "Data has been removed from the file: Elements.txt");
}
}