I'm looking for a small code snippet that will find a line in file and remove that line (not content but line) but could not find. So for example I have in a file following:
myFile.txt:
aaa
bbb
ccc
ddd
Need to have a function like this: public void removeLine(String lineContent), and if I pass
removeLine("bbb"), I get file like this:
myFile.txt:
aaa
ccc
ddd
This solution may not be optimal or pretty, but it works. It reads in an input file line by line, writing each line out to a temporary output file. Whenever it encounters a line that matches what you are looking for, it skips writing that one out. It then renames the output file. I have omitted error handling, closing of readers/writers, etc. from the example. I also assume there is no leading or trailing whitespace in the line you are looking for. Change the code around trim() as needed so you can find a match.
File inputFile = new File("myFile.txt");
File tempFile = new File("myTempFile.txt");
BufferedReader reader = new BufferedReader(new FileReader(inputFile));
BufferedWriter writer = new BufferedWriter(new FileWriter(tempFile));
String lineToRemove = "bbb";
String currentLine;
while((currentLine = reader.readLine()) != null) {
// trim newline when comparing with lineToRemove
String trimmedLine = currentLine.trim();
if(trimmedLine.equals(lineToRemove)) continue;
writer.write(currentLine + System.getProperty("line.separator"));
}
writer.close();
reader.close();
boolean successful = tempFile.renameTo(inputFile);
public void removeLineFromFile(String file, String lineToRemove) {
try {
File inFile = new File(file);
if (!inFile.isFile()) {
System.out.println("Parameter is not an existing file");
return;
}
//Construct the new file that will later be renamed to the original filename.
File tempFile = new File(inFile.getAbsolutePath() + ".tmp");
BufferedReader br = new BufferedReader(new FileReader(file));
PrintWriter pw = new PrintWriter(new FileWriter(tempFile));
String line = null;
//Read from the original file and write to the new
//unless content matches data to be removed.
while ((line = br.readLine()) != null) {
if (!line.trim().equals(lineToRemove)) {
pw.println(line);
pw.flush();
}
}
pw.close();
br.close();
//Delete the original file
if (!inFile.delete()) {
System.out.println("Could not delete file");
return;
}
//Rename the new file to the filename the original file had.
if (!tempFile.renameTo(inFile))
System.out.println("Could not rename file");
}
catch (FileNotFoundException ex) {
ex.printStackTrace();
}
catch (IOException ex) {
ex.printStackTrace();
}
}
This I have found on the internet.
You want to do something like the following:
Open the old file for reading
Open a new (temporary) file for writing
Iterate over the lines in the old file (probably using a BufferedReader)
For each line, check if it matches what you are supposed to remove
If it matches, do nothing
If it doesn't match, write it to the temporary file
When done, close both files
Delete the old file
Rename the temporary file to the name of the original file
(I won't write the actual code, since this looks like homework, but feel free to post other questions on specific bits that you have trouble with)
So, whenever I hear someone mention that they want to filter out text, I immediately think to go to Streams (mainly because there is a method called filter which filters exactly as you need it to). Another answer mentions using Streams with the Apache commons-io library, but I thought it would be worthwhile to show how this can be done in standard Java 8. Here is the simplest form:
public void removeLine(String lineContent) throws IOException
{
File file = new File("myFile.txt");
List<String> out = Files.lines(file.toPath())
.filter(line -> !line.contains(lineContent))
.collect(Collectors.toList());
Files.write(file.toPath(), out, StandardOpenOption.WRITE, StandardOpenOption.TRUNCATE_EXISTING);
}
I think there isn't too much to explain there, basically Files.lines gets a Stream<String> of the lines of the file, filter takes out the lines we don't want, then collect puts all of the lines of the new file into a List. We then write the list over top of the existing file with Files.write, using the additional option TRUNCATE so the old contents of the file are replaced.
Of course, this approach has the downside of loading every line into memory as they all get stored into a List before being written back out. If we wanted to simply modify without storing, we would need to use some form of OutputStream to write each new line to a file as it passes through the stream, like this:
public void removeLine(String lineContent) throws IOException
{
File file = new File("myFile.txt");
File temp = new File("_temp_");
PrintWriter out = new PrintWriter(new FileWriter(temp));
Files.lines(file.toPath())
.filter(line -> !line.contains(lineContent))
.forEach(out::println);
out.flush();
out.close();
temp.renameTo(file);
}
Not much has been changed in this example. Basically, instead of using collect to gather the file contents into memory, we use forEach so that each line that makes it through the filter gets sent to the PrintWriter to be written out to the file immediately and not stored. We have to save it to a temporary file, because we can't overwrite the existing file at the same time as we are still reading from it, so then at the end, we rename the temp file to replace the existing file.
Using apache commons-io and Java 8 you can use
List<String> lines = FileUtils.readLines(file);
List<String> updatedLines = lines.stream().filter(s -> !s.contains(searchString)).collect(Collectors.toList());
FileUtils.writeLines(file, updatedLines, false);
public static void deleteLine() throws IOException {
RandomAccessFile file = new RandomAccessFile("me.txt", "rw");
String delete;
String task="";
byte []tasking;
while ((delete = file.readLine()) != null) {
if (delete.startsWith("BAD")) {
continue;
}
task+=delete+"\n";
}
System.out.println(task);
BufferedWriter writer = new BufferedWriter(new FileWriter("me.txt"));
writer.write(task);
file.close();
writer.close();
}
Here you go. This solution uses a DataInputStream to scan for the position of the string you want replaced and uses a FileChannel to replace the text at that exact position. It only replaces the first occurrence of the string that it finds. This solution doesn't store a copy of the entire file somewhere, (either the RAM or a temp file), it just edits the portion of the file that it finds.
public static long scanForString(String text, File file) throws IOException {
if (text.isEmpty())
return file.exists() ? 0 : -1;
// First of all, get a byte array off of this string:
byte[] bytes = text.getBytes(/* StandardCharsets.your_charset */);
// Next, search the file for the byte array.
try (DataInputStream dis = new DataInputStream(new FileInputStream(file))) {
List<Integer> matches = new LinkedList<>();
for (long pos = 0; pos < file.length(); pos++) {
byte bite = dis.readByte();
for (int i = 0; i < matches.size(); i++) {
Integer m = matches.get(i);
if (bytes[m] != bite)
matches.remove(i--);
else if (++m == bytes.length)
return pos - m + 1;
else
matches.set(i, m);
}
if (bytes[0] == bite)
matches.add(1);
}
}
return -1;
}
public static void replaceText(String text, String replacement, File file) throws IOException {
// Open a FileChannel with writing ability. You don't really need the read
// ability for this specific case, but there it is in case you need it for
// something else.
try (FileChannel channel = FileChannel.open(file.toPath(), StandardOpenOption.WRITE, StandardOpenOption.READ)) {
long scanForString = scanForString(text, file);
if (scanForString == -1) {
System.out.println("String not found.");
return;
}
channel.position(scanForString);
channel.write(ByteBuffer.wrap(replacement.getBytes(/* StandardCharsets.your_charset */)));
}
}
Example
Input: ABCDEFGHIJKLMNOPQRSTUVWXYZ
Method Call:
replaceText("QRS", "000", new File("path/to/file");
Resulting File: ABCDEFGHIJKLMNOP000TUVWXYZ
Here is the complete Class. In the below file "somelocation" refers to the actual path of the file.
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
public class FileProcess
{
public static void main(String[] args) throws IOException
{
File inputFile = new File("C://somelocation//Demographics.txt");
File tempFile = new File("C://somelocation//Demographics_report.txt");
BufferedReader reader = new BufferedReader(new FileReader(inputFile));
BufferedWriter writer = new BufferedWriter(new FileWriter(tempFile));
String currentLine;
while((currentLine = reader.readLine()) != null) {
if(null!=currentLine && !currentLine.equalsIgnoreCase("BBB")){
writer.write(currentLine + System.getProperty("line.separator"));
}
}
writer.close();
reader.close();
boolean successful = tempFile.renameTo(inputFile);
System.out.println(successful);
}
}
This solution reads in an input file line by line, writing each line out to a StringBuilder variable. Whenever it encounters a line that matches what you are looking for, it skips writing that one out. Then it deletes file content and put the StringBuilder variable content.
public void removeLineFromFile(String lineToRemove, File f) throws FileNotFoundException, IOException{
//Reading File Content and storing it to a StringBuilder variable ( skips lineToRemove)
StringBuilder sb = new StringBuilder();
try (Scanner sc = new Scanner(f)) {
String currentLine;
while(sc.hasNext()){
currentLine = sc.nextLine();
if(currentLine.equals(lineToRemove)){
continue; //skips lineToRemove
}
sb.append(currentLine).append("\n");
}
}
//Delete File Content
PrintWriter pw = new PrintWriter(f);
pw.close();
BufferedWriter writer = new BufferedWriter(new FileWriter(f, true));
writer.append(sb.toString());
writer.close();
}
Super simple method using maven/gradle+groovy.
public void deleteConfig(String text) {
File config = new File("/the/path/config.txt")
def lines = config.readLines()
lines.remove(text);
config.write("")
lines.each {line -> {
config.append(line+"\n")
}}
}
public static void deleteLine(String line, String filePath) {
File file = new File(filePath);
File file2 = new File(file.getParent() + "\\temp" + file.getName());
PrintWriter pw = null;
Scanner read = null;
FileInputStream fis = null;
FileOutputStream fos = null;
FileChannel src = null;
FileChannel dest = null;
try {
pw = new PrintWriter(file2);
read = new Scanner(file);
while (read.hasNextLine()) {
String currline = read.nextLine();
if (line.equalsIgnoreCase(currline)) {
continue;
} else {
pw.println(currline);
}
}
pw.flush();
fis = new FileInputStream(file2);
src = fis.getChannel();
fos = new FileOutputStream(file);
dest = fos.getChannel();
dest.transferFrom(src, 0, src.size());
} catch (IOException e) {
e.printStackTrace();
} finally {
pw.close();
read.close();
try {
fis.close();
fos.close();
src.close();
dest.close();
} catch (IOException e) {
e.printStackTrace();
}
if (file2.delete()) {
System.out.println("File is deleted");
} else {
System.out.println("Error occured! File: " + file2.getName() + " is not deleted!");
}
}
}
package com.ncs.cache;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.File;
import java.io.FileWriter;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.PrintWriter;
public class FileUtil {
public void removeLineFromFile(String file, String lineToRemove) {
try {
File inFile = new File(file);
if (!inFile.isFile()) {
System.out.println("Parameter is not an existing file");
return;
}
// Construct the new file that will later be renamed to the original
// filename.
File tempFile = new File(inFile.getAbsolutePath() + ".tmp");
BufferedReader br = new BufferedReader(new FileReader(file));
PrintWriter pw = new PrintWriter(new FileWriter(tempFile));
String line = null;
// Read from the original file and write to the new
// unless content matches data to be removed.
while ((line = br.readLine()) != null) {
if (!line.trim().equals(lineToRemove)) {
pw.println(line);
pw.flush();
}
}
pw.close();
br.close();
// Delete the original file
if (!inFile.delete()) {
System.out.println("Could not delete file");
return;
}
// Rename the new file to the filename the original file had.
if (!tempFile.renameTo(inFile))
System.out.println("Could not rename file");
} catch (FileNotFoundException ex) {
ex.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
}
}
public static void main(String[] args) {
FileUtil util = new FileUtil();
util.removeLineFromFile("test.txt", "bbbbb");
}
}
src : http://www.javadb.com/remove-a-line-from-a-text-file/
This solution requires the Apache Commons IO library to be added to the build path. It works by reading the entire file and writing each line back but only if the search term is not contained.
public static void removeLineFromFile(File targetFile, String searchTerm)
throws IOException
{
StringBuffer fileContents = new StringBuffer(
FileUtils.readFileToString(targetFile));
String[] fileContentLines = fileContents.toString().split(
System.lineSeparator());
emptyFile(targetFile);
fileContents = new StringBuffer();
for (int fileContentLinesIndex = 0; fileContentLinesIndex < fileContentLines.length; fileContentLinesIndex++)
{
if (fileContentLines[fileContentLinesIndex].contains(searchTerm))
{
continue;
}
fileContents.append(fileContentLines[fileContentLinesIndex] + System.lineSeparator());
}
FileUtils.writeStringToFile(targetFile, fileContents.toString().trim());
}
private static void emptyFile(File targetFile) throws FileNotFoundException,
IOException
{
RandomAccessFile randomAccessFile = new RandomAccessFile(targetFile, "rw");
randomAccessFile.setLength(0);
randomAccessFile.close();
}
I refactored the solution that Narek had to create (according to me) a slightly more efficient and easy to understand code. I used embedded Automatic Resource Management, a recent feature in Java and used a Scanner class which according to me is more easier to understand and use.
Here is the code with edited Comments:
public class RemoveLineInFile {
private static File file;
public static void main(String[] args) {
//create a new File
file = new File("hello.txt");
//takes in String that you want to get rid off
removeLineFromFile("Hello");
}
public static void removeLineFromFile(String lineToRemove) {
//if file does not exist, a file is created
if (!file.exists()) {
try {
file.createNewFile();
} catch (IOException e) {
System.out.println("File "+file.getName()+" not created successfully");
}
}
// Construct the new temporary file that will later be renamed to the original
// filename.
File tempFile = new File(file.getAbsolutePath() + ".tmp");
//Two Embedded Automatic Resource Managers used
// to effectivey handle IO Responses
try(Scanner scanner = new Scanner(file)) {
try (PrintWriter pw = new PrintWriter(new FileWriter(tempFile))) {
//a declaration of a String Line Which Will Be assigned Later
String line;
// Read from the original file and write to the new
// unless content matches data to be removed.
while (scanner.hasNextLine()) {
line = scanner.nextLine();
if (!line.trim().equals(lineToRemove)) {
pw.println(line);
pw.flush();
}
}
// Delete the original file
if (!file.delete()) {
System.out.println("Could not delete file");
return;
}
// Rename the new file to the filename the original file had.
if (!tempFile.renameTo(file))
System.out.println("Could not rename file");
}
}
catch (IOException e)
{
System.out.println("IO Exception Occurred");
}
}
}
Try this:
public static void main(String[] args) throws IOException {
File file = new File("file.csv");
CSVReader csvFileReader = new CSVReader(new FileReader(file));
List<String[]> list = csvFileReader.readAll();
for (int i = 0; i < list.size(); i++) {
String[] filter = list.get(i);
if (filter[0].equalsIgnoreCase("bbb")) {
list.remove(i);
}
}
csvFileReader.close();
CSVWriter csvOutput = new CSVWriter(new FileWriter(file));
csvOutput.writeAll(list);
csvOutput.flush();
csvOutput.close();
}
Old question, but an easy way is to:
Iterate through file, adding each line to an new array list
iterate through the array, find matching String, then call the remove method.
iterate through array again, printing each line to the file, boolean for append should be false, which basically replaces the file
This solution uses a RandomAccessFile to only cache the portion of the file subsequent to the string to remove. It scans until it finds the String you want to remove. Then it copies all of the data after the found string, then writes it over the found string, and everything after. Last, it truncates the file size to remove the excess data.
public static long scanForString(String text, File file) throws IOException {
if (text.isEmpty())
return file.exists() ? 0 : -1;
// First of all, get a byte array off of this string:
byte[] bytes = text.getBytes(/* StandardCharsets.your_charset */);
// Next, search the file for the byte array.
try (DataInputStream dis = new DataInputStream(new FileInputStream(file))) {
List<Integer> matches = new LinkedList<>();
for (long pos = 0; pos < file.length(); pos++) {
byte bite = dis.readByte();
for (int i = 0; i < matches.size(); i++) {
Integer m = matches.get(i);
if (bytes[m] != bite)
matches.remove(i--);
else if (++m == bytes.length)
return pos - m + 1;
else
matches.set(i, m);
}
if (bytes[0] == bite)
matches.add(1);
}
}
return -1;
}
public static void remove(String text, File file) throws IOException {
try (RandomAccessFile rafile = new RandomAccessFile(file, "rw");) {
long scanForString = scanForString(text, file);
if (scanForString == -1) {
System.out.println("String not found.");
return;
}
long remainderStartPos = scanForString + text.getBytes().length;
rafile.seek(remainderStartPos);
int remainderSize = (int) (rafile.length() - rafile.getFilePointer());
byte[] bytes = new byte[remainderSize];
rafile.read(bytes);
rafile.seek(scanForString);
rafile.write(bytes);
rafile.setLength(rafile.length() - (text.length()));
}
}
Usage:
File Contents: ABCDEFGHIJKLMNOPQRSTUVWXYZ
Method Call: remove("ABC", new File("Drive:/Path/File.extension"));
Resulting Contents: DEFGHIJKLMNOPQRSTUVWXYZ
This solution could easily be modified to remove with a certain, specifiable cacheSize, if memory is a concern. This would just involve iterating over the rest of the file to continually replace portions of size, cacheSize. Regardless, this solution is generally much better than caching an entire file in memory, or copying it to a temporary directory, etc.
I'm trying to write compressed data to a file and then read in the data and decompress it using the GZIP library. I've tried changing all formatting to StandardCharsets.UTF-8 and ISO-8859-1 and neither have fixed the GZIP format error. I'm wondering if it could possible have to do with the file I'm reading in? Here's the compression function:
public static byte[] compress(String originalFile, String compressFile) throws IOException {
// read in data from text file
// The name of the file to open.
String fileName = originalFile;
// This will reference one line at a time
String line = null;
String original = "";
try {
// FileReader reads text files in the default encoding.
FileReader fileReader =
new FileReader(fileName);
// Always wrap FileReader in BufferedReader.
BufferedReader bufferedReader =
new BufferedReader(fileReader);
while((line = bufferedReader.readLine()) != null) {
original.concat(line);
}
// Always close files.
bufferedReader.close();
}
catch(FileNotFoundException ex) {
System.out.println(
"Unable to open file '" +
fileName + "'");
}
catch(IOException ex) {
System.out.println(
"Error reading file '"
+ fileName + "'");
// Or we could just do this:
// ex.printStackTrace();
}
// create a new output stream for original string
try (ByteArrayOutputStream out = new ByteArrayOutputStream())
{
try (GZIPOutputStream gzip = new GZIPOutputStream(out))
{
gzip.write(original.getBytes(StandardCharsets.UTF_8));
}
byte[] compressed = out.toByteArray();
out.close();
String compressedFileName = compressFile;
try {
// Assume default encoding.
FileWriter fileWriter =
new FileWriter(compressedFileName);
// Always wrap FileWriter in BufferedWriter.
BufferedWriter bufferedWriter =
new BufferedWriter(fileWriter);
// Note that write() does not automatically
// append a newline character.
String compressedStr = compressed.toString();
bufferedWriter.write(compressedStr);
// Always close files.
bufferedWriter.close();
}
catch(IOException ex) {
System.out.println(
"Error writing to file '"
+ fileName + "'");
// Or we could just do this:
// ex.printStackTrace();
}
return compressed;
}
}
(I'm receiving the error on the line in the following decompression function) -
GZIPInputStream compressedByteArrayStream = new GZIPInputStream(new ByteArrayInputStream(s.getBytes(StandardCharsets.UTF_8)));
Decompression Function:
public static String decompress(String file) throws IOException {
byte[] compressed = {};
String s = "";
File fileName = new File(file);
FileInputStream fin = null;
try {
// create FileInputStream object
fin = new FileInputStream(fileName);
// Reads up to certain bytes of data from this input stream into an array of bytes.
fin.read(compressed);
//create string from byte array
s = new String(compressed);
System.out.println("File content: " + s);
}
catch (FileNotFoundException e) {
System.out.println("File not found" + e);
}
catch (IOException ioe) {
System.out.println("Exception while reading file " + ioe);
}
finally {
// close the streams using close method
try {
if (fin != null) {
fin.close();
}
}
catch (IOException ioe) {
System.out.println("Error while closing stream: " + ioe);
}
}
// create a new input string for compressed byte array
GZIPInputStream compressedByteArrayStream = new GZIPInputStream(new ByteArrayInputStream(s.getBytes(StandardCharsets.UTF_8)));
ByteArrayOutputStream byteOutput = new ByteArrayOutputStream();
byte[] buffer = new byte[8192];
// create a string builder and byte reader for the compressed byte array
BufferedReader decompressionBr = new BufferedReader(new InputStreamReader(compressedByteArrayStream, StandardCharsets.UTF_8));
StringBuilder decompressionSb = new StringBuilder();
// write data to decompressed string
String line1;
while((line1 = decompressionBr.readLine()) != null) {
decompressionSb.append(line1);
}
decompressionBr.close();
int len;
String uncompressedStr = "";
while((len = compressedByteArrayStream.read(buffer)) > 0) {
uncompressedStr = byteOutput.toString();
}
compressedByteArrayStream.close();
return uncompressedStr;
}
Here's the error message that i am receiving:
[B#7852e922
File content:
java.io.EOFException
at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:268)
at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:258)
at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:164)
at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:79)
at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:91)
at org.kingswoodoxford.Compression.decompress(Compression.java:136)
at org.kingswoodoxford.Compression.main(Compression.java:183)
Any suggestions as to how I might be able to fix this?
When you read the file you discard the new line at the end of each line.
A more efficient option which does do this is to copy a block i.e. char[] at a time. You can also convert the text as you go rather than creating a String or a byte[].
BTW original.concat(line); returns the concatenated string which you are discarding.
The real problem is you write to one stream and close a different one. This means that if there is any buffered data at the end of the file (and this is highly likely) the end of the file will be truncated and when you read it it will complain that your file is incomplete or EOFException.
Here is a shorter example
public static void compress(String originalFile, String compressFile) throws IOException {
char[] buffer = new char[8192];
try (
FileReader reader = new FileReader(originalFile);
Writer writer = new OutputStreamWriter(
new GZIPOutputStream(new FileOutputStream(compressFile)));
) {
for (int len; (len = reader.read(buffer)) > 0; )
writer.write(buffer, 0, len);
}
}
In the decompress, don't encode binary as text and attempt to get back the same data. It will almost certainly be corrupted. Try to use a buffer and a loop like I did for compress. i.e. it shouldn't be any more complicated.
I need some help on below problem. I am working on a project where I need to deal with files.
I get the handle of input stream from the user from which before writing it to disk I need to perform certain steps.
calculate the file digest
check for only 1 zip file present, unzip the data if zipped
dos 2 unix conversion
record length validation
and encrypt and save the file to disk
Also need to break the flow if there is any exception in the process
I tried to use piped output and input stream, but the constraint is Java recommends it to run in 2 separate threads. Once I read from input stream I am not able to use it from other processing steps. Files can be very big so cannot cache all the data in buffer.
Please provide your suggestions or is there any third party lib I can use for same.
The biggest issue is that you'll need to peek ahead in the provided InputStream to decide if you received a zipfile or not.
private boolean isZipped(InputStream is) throws IOException {
try {
return new ZipInputStream(is).getNextEntry() != null;
} catch (final ZipException ze) {
return false;
}
}
After this you need to reset the inputstream to the initial position before setting up a DigestInputStream.
Then read a ZipInputstream or the DigestInputstream directly.
After you've done your processing, read the DigestInputStream to the end so you can obtain the digest.
Below code has been validated through a wrapping "CountingInputstream" that keeps track of the total number of bytes read from the provided FileInputStream.
final FileInputStream fis = new FileInputStream(filename);
final CountingInputStream countIs = new CountingInputStream(fis);
final boolean isZipped = isZipped(countIs);
// make sure we reset the inputstream before calculating the digest
fis.getChannel().position(0);
final DigestInputStream dis = new DigestInputStream(countIs, MessageDigest.getInstance("SHA-256"));
// decide which inputStream to use
InputStream is = null;
ZipInputStream zis = null;
if (isZipped) {
zis = new ZipInputStream(dis);
zis.getNextEntry();
is = zis;
} else {
is = dis;
}
final File tmpFile = File.createTempFile("Encrypted_", ".tmp");
final OutputStream os = new CipherOutputStream(new FileOutputStream(tmpFile), obtainCipher());
try {
readValidateAndWriteRecords(is, os);
failIf2ndZipEntryExists(zis);
} catch (final Exception e) {
os.close();
tmpFile.delete();
throw e;
}
System.out.println("Digest: " + obtainDigest(dis));
dis.close();
System.out.println("\nValidating bytes read and calculated digest");
final DigestInputStream dis2 = new DigestInputStream(new CountingInputStream(new FileInputStream(filename)), MessageDigest.getInstance("SHA-256"));
System.out.println("Digest: " + obtainDigest(dis2));
dis2.close();
Not really relevant, but these are the helper methods:
private String obtainDigest(DigestInputStream dis) throws IOException {
final byte[] buff = new byte[1024];
while (dis.read(buff) > 0) {
dis.read(buff);
}
return DatatypeConverter.printBase64Binary(dis.getMessageDigest().digest());
}
private void readValidateAndWriteRecords(InputStream is, final OutputStream os) throws IOException {
final BufferedReader br = new BufferedReader(new InputStreamReader(is));
// do2unix is done automatically by readline
for (String line = br.readLine(); line != null; line = br.readLine()) {
// record length validation
if (line.length() < 1) {
throw new RuntimeException("RecordLengthValidationFailed");
}
os.write((line + "\n").getBytes());
}
}
private void failIf2ndZipEntryExists(ZipInputStream zis) throws IOException {
if (zis != null && zis.getNextEntry() != null) {
throw new RuntimeException("Zip File contains multiple entries");
}
}
==> output:
Digest: jIisvDleAttKiPkyU/hDvbzzottAMn6n7inh4RKxPOc=
CountingInputStream closed. Total number of bytes read: 1100
Validating bytes read and calculated digest
Digest: jIisvDleAttKiPkyU/hDvbzzottAMn6n7inh4RKxPOc=
CountingInputStream closed. Total number of bytes read: 1072
Fun question, I may have gone overboard with my answer :)
I have few text files. Each text file contains some path and/or the reference of some other file.
File1
#file#>D:/FilePath/File2.txt
Mod1>/home/admin1/mod1
Mod2>/home/admin1/mod2
File2
Mod3>/home/admin1/mod3
Mod4>/home/admin1/mod4
All I want is, copy all the paths Mod1, Mod2, Mod3, Mod4 in another text file by supplying only File1.txt as input to my java program.
What I have done till now?
public void readTextFile(String fileName){
try {
br = new BufferedReader(new FileReader(new File(fileName)));
String line = br.readLine();
while(line!=null){
if(line.startsWith("#file#>")){
String string[] = line.split(">");
readTextFile(string[1]);
}
else if(line.contains(">")){
String string[] = line.split(">");
svnLinks.put(string[0], string[1]);
}
line=br.readLine();
}
} catch (Exception e) {
e.printStackTrace();
}
}
Currently my code reads the contents of File2.txt only, control does not come back to File1.txt.
Please ask if more inputs are required.
First of all you are jumping to another file without closing the current reader and when you come back you lose the cursor. Read one file first and then write all its contents that match to another file. Close the current reader (Don't close the writer) and then open the next file to read and so on.
Seems pretty simple. You need to write your file once your svnLinks Map is populated, assuming your present code works (haven't seen anything too weird in it).
So, once the Map is populated, you could use something along the lines of:
File newFile = new File("myPath/myNewFile.txt");
// TODO check file can be written
// TODO check file exists or create
FileOutputStream fos = null;
OutputStreamWriter osw = null;
BufferedWriter bw = null;
try {
fos = new FileOutputStream(newFile);
osw = new OutputStreamWriter(fos);
bw = new BufferedWriter(osw);
for (String key: svnLinks.keySet()) {
bw.write(key.concat(" my separator ").concat(svnLinks.get(key)).concat("myNewLine"));
}
}
catch (Throwable t) {
// TODO handle more gracefully
t.printStackTrace();
if (bw != null) {
try {
bw.close();
}
catch (Throwable t) {
t.printStackTrace();
}
}
Here is an non-recursive implementation of your method :
public static void readTextFile(String fileName) throws IOException {
LinkedList<String> list = new LinkedList<String>();
list.add(fileName);
while (!list.isEmpty()) {
BufferedReader br = null;
try {
br = new BufferedReader(new FileReader(new File(list.pop())));
String line;
while ((line = br.readLine()) != null) {
if (line.startsWith("#file#>")) {
String string[] = line.split(">");
list.add(string[1]);
} else if (line.contains(">")) {
String string[] = line.split(">");
svnLinks.put(string[0], string[1]);
}
}
} catch (Exception e) {
e.printStackTrace();
} finally {
br.close();
}
}
}
Just used a LinkedList to maintain the order. I suggest you to add some counter if you to limit the reading of files to a certain number(depth). eg:
while (!list.isEmpty() && readCount < 10 )
This will eliminate the chance of running the code to infinity(in case of circular reference).
I was wondering how do you manipulate big Textfiles in Java, if we assume that the Filesize is larger than the memory. I googled that topic and it shows that most people recommend java.niofor such a task.
Unfortunately I haven't found any documentation on how to manipulate the File. For example read every Line, modify it, write it. I tried something like this, but this doesn't work:
FileChannel fileChannel = null;
try {
fileChannel = new RandomAccessFile(file, "rw").getChannel();
ByteBuffer buffer = ByteBuffer.allocate(256);
while (fileChannel.read(buffer) != -1) {
buffer.rewind();
buffer.flip();
String nextLine = buffer.asCharBuffer().toString();
if (replaceBackSlashes) {
nextLine = nextLine.replace("\\\\", "/");
}
if (!(removeEmptyLines && StringUtils.isEmpty(nextLine))) {
buffer.flip();
buffer.asCharBuffer().put(nextLine);
}
buffer.clear();
}
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
if (fileChannel != null) {
try {
fileChannel.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
So what are your recommendations? Also the String nextline, doesn't match anything in my File. Maybe I need to set the encoding?
Line by line. Something like this ...
public static void main(String[] args) throws Exception {
File someFile = new File("someFile.txt");
File temp = File.createTempFile(someFile.getName(), null);
BufferedReader reader = null;
PrintStream writer = null;
try {
reader = new BufferedReader(new FileReader(someFile));
writer = new PrintStream(temp);
String line;
while ((line = reader.readLine())!=null) {
// manipulate line
writer.println(line);
}
}
finally {
if (writer!=null) writer.close();
if (reader!=null) reader.close();
}
if (!someFile.delete()) throw new Exception("Failed to remove " + someFile.getName());
if (!temp.renameTo(someFile)) throw new Exception("Failed to replace " + someFile.getName());
}
Kudos to xagyg for a nice, clean answer! The following just didn't fit into a comment:
If you're running Java 7 already, you can save a lot of boilerplate code by using try-with-resources for the processing loop:
File source = ...
File target = ...
try (BufferedReader in = new BufferedReader(new FileReader(source));
PrintStream out = new PrintStream(target)) {
String line;
while ((line = in.readLine()) != null) {
// manipulate line
out.println(line);
}
}
// no catch or finally clause!
No more of that initalize-to-null-try-catch-finally-close-if-not-null mess, Java will take care of that for you now. Less code, less potential to forget or screw up that crucial call to close().