We Keep Coding

Remove word after line from txt file is read

I have this code which is used to read lines from a file and insert it into Postgre:
try {
BufferedReader reader;
try {
reader = new BufferedReader(new FileReader(
"C:\\in_progress\\test.txt"));
String line = reader.readLine();
while (line != null) {
System.out.println(line);
Thread.sleep(100);
Optional<ProcessedWords> isFound = processedWordsService.findByKeyword(line);
if(!isFound.isPresent()){
ProcessedWords obj = ProcessedWords.builder()
.keyword(line)
.createdAt(LocalDateTime.now())
.build();
processedWordsService.save(obj);
}
// read next line
line = reader.readLine();
}
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
catch (Exception e) {
e.printStackTrace();
}
How I can remove a line from the file after the line is inserted into SQL database?

The issues with the current code:
Adhere to the Single responsibility principle. Your code is doing too many things: reads from a file, performs findByKeyword() call, prepares the data and hands it out to store in the database. It's hardly can be thoroughly tested, and it's very difficult to maintain.
Always use try-with-recourses to get your recourses closed at any circumstances.
Don't catch the general Exception type - your code should only catch thous exceptions, which are more or less expected and for which there's a clear scenario on how to handle them. But don't catch all the exceptions.
How I can remove a line from the file after the line is inserted into SQL database?
It is not possible to remove a line from a file in the literal sense. You can override the contents of the file or replace it with another file.
My advice would be to file data in memory, process it, and then write the lines which should be retained into the same file (i.e. override the file contents).
You can argue that the file is huge and dumping it into memory would result in an OutOfMemoryError. And you want to read a line from a file, process it somehow, then store the processed data into the database and then write the line into a file... So that everything is done line by line, all actions in one go for a single line, and as a consequence all the code is crammed in one method. I hope that's not the case because otherwise it's a clear XY-problem.
Firstly, File System isn't a reliable mean of storing data, and it's not very fast. If the file is massive, then reading and writing it will a take a considerable amount of time, and it's done just it in order to use a tinny bit of information then this approach is wrong - this information should be stored and structured differently (i.e. consider placing into a DB) so that it would be possible to retrieve the required data, and there would be no problem with removing entries that are no longer needed.
But if the file is lean, and it doesn't contain critical data. Then it's totally fine, I will proceed assuming that it's the case.
The overall approach is to generate a map Map<String, Optional<ProcessedWords>> based on the file contents, process the non-empty optionals and prepare a list of lines to override the previous file content.
The code below is based on the NIO2 file system API.
public void readProcessAndRemove(ProcessedWordsService service, Path path) {
Map<String, Optional<ProcessedWords>> result;
try (var lines = Files.lines(path)) {
result = processLines(service, lines);
} catch (IOException e) {
result = Collections.emptyMap();
logger.log();
e.printStackTrace();
}
List<String> linesToRetain = prepareAndSave(service, result);
writeToFile(linesToRetain, path);
}
Processing the stream of lines from a file returned Files.lines():
private static Map<String, Optional<ProcessedWords>> processLines(ProcessedWordsService service,
Stream<String> lines) {
return lines.collect(Collectors.toMap(
Function.identity(),
service::findByKeyword
));
}
Saving the words for which findByKeyword() returned an empty optional:
private static List<String> prepareAndSave(ProcessedWordsService service,
Map<String, Optional<ProcessedWords>> wordByLine) {
wordByLine.forEach((k, v) -> {
if (v.isEmpty()) saveWord(service, k);
});
return getLinesToRetain(wordByLine);
}
private static void saveWord(ProcessedWordsService service, String line) {
ProcessedWords obj = ProcessedWords.builder()
.keyword(line)
.createdAt(LocalDateTime.now())
.build();
service.save(obj);
}
Generating a list of lines to retain:
private static List<String> getLinesToRetain(Map<String, Optional<ProcessedWords>> wordByLine) {
return wordByLine.entrySet().stream()
.filter(entry -> entry.getValue().isPresent())
.map(Map.Entry::getKey)
.collect(Collectors.toList());
}
Overriding the file contents using Files.write(). Note: since varargs OpenOption isn't provided with any arguments, this call would be treated as if the CREATE, TRUNCATE_EXISTING, and WRITE options are present.
private static void writeToFile(List<String> lines, Path path) {
try {
Files.write(path, lines);
} catch (IOException e) {
logger.log();
e.printStackTrace();
}
}

For Reference
import java.io.*;
public class RemoveLinesFromAfterProcessed {
public static void main(String[] args) throws Exception {
String fileName = "TestFile.txt";
String tempFileName = "tempFile";
File mainFile = new File(fileName);
File tempFile = new File(tempFileName);
try (BufferedReader br = new BufferedReader(new FileReader(mainFile));
PrintWriter pw = new PrintWriter(new FileWriter(tempFile))
) {
String line;
while ((line = br.readLine()) != null) {
if (toProcess(line)) { // #1
// process the code and add it to DB
// ignore the line (i.e, not add to temp file)
} else {
// add to temp file.
pw.write(line + "\n"); // #2
}
}
} catch (Exception e) {
e.printStackTrace();
}
// delete the old file
boolean hasDeleted = mainFile.delete(); // #3
if (!hasDeleted) {
throw new Exception("Can't delete file!");
}
boolean hasRenamed = tempFile.renameTo(mainFile); // #4
if (!hasRenamed) {
throw new Exception("Can't rename file!");
}
System.out.println("Done!");
}
private static boolean toProcess(String line) {
// any condition
// sample condition for example
return line.contains("aa");
}
}
Read the file.
1: The condition to decide whether to delete the line or to retain it.
2: Write those line which you don't want to delete into the temporary file.
3: Delete the original file.
4: Rename the temporary file to original file name.

The basic idea is the same as what #Shiva Rahul said in his answer.
However another approach can be , store all the line numbers you want to delete in a list. After you have all the required line numbers that you want to delete you can use LineNumberReader to check and duplicate your main file.
Mostly I have used this technique in batch-insert where I was unsure how many lines may have a particular file plus before removal of lines had to do lot of processing.
It may not be suitable for your case ,just posting the suggestion here if any one bumps to this thread.
private void deleteLines(String inputFilePath,String outputDirectory,List<Integer> lineNumbers) throws IOException{
File tempFile = new File("temp.txt");
File inputFile = new File(inputFilePath);
// using LineNumberReader we can fetch the line numbers of each line
LineNumberReader lineReader = new LineNumberReader(new FileReader(inputFile));
//writter for writing the lines into new file
BufferedWriter bufferedWriter = new BufferedWriter(new FileWriter(tempFile));
String currentLine;
while((currentLine = lineReader.readLine()) != null){
//if current line number is present in removeList then put empty line in new file
if(lineNumbers.contains(lineReader.getLineNumber())){
currentLine="";
}
bufferedWriter.write(currentLine + System.getProperty("line.separator"));
}
//closing statements
bufferedWriter.close();
lineReader.close();
//delete the main file and rename the tempfile to original file Name
boolean delete = inputFile.delete();
//boolean b = tempFile.renameTo(inputFile); // use this to save the temp file in same directory;
boolean b = tempFile.renameTo(new File(outputDirectory+inputFile.getName()));
}
To use this function all you have to do is gather all the required line numbers.inputFilePath is the path of the source file and outputDirectory is where I want store the file after processing.

Unable to modify or delete text file created by my .jar executable after closing and relaunching the executable

I have a fairly simple Java project that opens (creates if doesn't exist) a text file in the current directory (somewhere within my Documents folder), reads the data with BufferedReader, then modifies with PrintWriter. The code works fine in Eclipse. But when exported to a runnable .jar file the resulting .jar executable can only modify a file that it itself created (created because it didn't exist before). If I then close and re-launch the .jar as a new instance to modify the file it created last launch I get
java.io.FileNotFoundException: Data.txt (Access is denied)
at java.base/java.io.FileOutputStream.open0(Native Method)
at java.base/java.io.FileOutputStream.open(FileOutputStream.java:293)
at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:235)
at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:184)
at java.base/java.io.PrintWriter.<init>(PrintWriter.java:309)
at project.file.LocalStorageFile.setList(LocalStorageFile.java:136)
...
So to reiterate:
Run in eclipse - works fine
Export to runnable jar
Run jar
If file doesn't exist it creates one
Reads file
if it created the file in step 4 - writes successfully, but if file already existed - exception
When looking into this exception the answers suggest:
restricted directory on disk C (which can't be it for me, as mine is a perfectly accessible (at least once) folder in Documents, and the folder's permissions seem to be in order)
forgetting to close streams (I close both reader and writer streams in a finally block and make sure they were closed with some console print lines)
file is being used (the problem persists after a computer restart and even after an OS re-installation I had to do earlier)
lack of admin rights (I've launched CMD as administrator and used it to "java -jar project.jar" with same results)
tried deleting file using file.delete() which equals false
tried flushing the print writer
tried setting file to readable and writable with file.setReadable(true) and file.setWritable(true)
I am using gradle to build my .jar because I need some libs I get from there for other functions. but even if I "export as runnable jar" using Eclipse itself I get the same issue. And of course when I run the program directly from Eclipse multiple times I get no problem accessing and modifying the same file.
Here are is my LocalStorageFile class with the read and write functions if you want to test it on your machine:
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.io.PrintWriter;
import java.util.TreeMap;
/**
* Store keywords and strings in a local file.
*
* Format: keyword1, keyword2 \t string1 \n keyword3, keyword4 \t string2
* \n
*/
public class LocalStorageFile {
private static final String FILE_PATH = "Data.txt"; // current directory
private static final String KEY_SEP = "\t"; // key/value separator
private static final String LINE_SEP = "\n"; // line separator
private static File file;
public LocalStorageFile() {
file = new File(FILE_PATH);
// Create file if it doesn't exist
if (!file.exists()) {
try {
file.createNewFile();
} catch (IOException e) {
e.printStackTrace();
}
}
}
/**
* Retrieve list from file "Data.txt" in local directory.
*
* #return TreeMap of keys and strings.
*/
public static TreeMap<String, String> getList() {
System.out.println("Retrieving list data.");
TreeMap<String, String> myList = new TreeMap<String, String>();
File file = new File(FILE_PATH);
if (!file.exists()) {
try {
file.createNewFile(); // if file already exists will do nothing
} catch (IOException e1) {
e1.printStackTrace();
}
}
BufferedReader br = null;
// Read file line by line and add to myList
try {
file.setReadable(true);
file.setWritable(true);
br = new BufferedReader(new FileReader(file));
String line;
while ((line = br.readLine()) != null) {
System.out.println(" Now reading line:" + line);
String[] keyValuePair = line.split(KEY_SEP);
if (keyValuePair.length == 2) { // avoid any error lines
// Re-insert tabs and newlines
String key = keyValuePair[0].replaceAll("\\\\t", "\t")
.replaceAll("\\\\n", "\n");
String value = keyValuePair[1].replaceAll("\\\\t", "\t")
.replaceAll("\\\\n", "\n");
// Put data into map
myList.put(key, value);
}
}
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
if (br != null) {
br.close();
System.out.println("Buffered reader closed.");
}
} catch (Exception e) {
e.printStackTrace();
}
}
return myList;
}
/**
* Rewrite list to file "Data.txt" in local directory.
*
* #param myList
* TreeMap of keys and strings.
* #return 1 on success, 0 on fail.
*/
public static int setList(TreeMap<String, String> myList) {
System.out.println("Saving list data.");
String textData = "";
int result = 0;
// Construct textData using myList
for (String key : myList.keySet()) {
String value = myList.get(key);
// Sanitize strings
String keyClean = key.replaceAll("\t", "\\\\t").replaceAll("\n",
"\\\\n");
String valueClean = value.replaceAll("\t", "\\\\t").replaceAll(
"\n", "\\\\n");
// Assemble line with separators
String line = keyClean + KEY_SEP + valueClean + LINE_SEP;
System.out.println(" Now saving line:" + line);
textData += line;
}
// Replace file content with textData
PrintWriter prw = null;
File file = new File(FILE_PATH);
if (file.exists()) {
boolean delStatus = file.delete();
System.out.println("File deleted? " + delStatus);
// file.setReadable(true);
// file.setWritable(true);
} else {
System.out.println("File doesn't exist");
}
try {
file.createNewFile();
prw = new PrintWriter(file); // <- this is line 136 from the exception
prw.println(textData);
prw.flush();
result = 1;
} catch (Exception e) {
e.printStackTrace();
} finally {
if (prw != null) {
prw.close();
System.out.println("Print writer closed.");
}
}
return result;
}
}
It doesn't seem to be an issue with the code, but perhaps something with my system?
Any clues on where I should be digging will be greatly appreciated.

OK. I've gotten it to work finally. I had to turn off my Avast Antivirus temporarily and I was able to edit existing files. Specifically "Ransomware Protection" was protecting my Documents folder.
Thank you, commenters, for the help, couldn't have reached this conclusion without you! An answer in this question mentioned Comodo antivirus causing this same issue. I will create a new working directory in an unprotected folder, because Avast doesn't allow me to add a non-exe file as an exception.

reading and writing to same file with jar

So, here's the problem: I'm working on a Java program that reads from a .csv file, and constructs objects out of it. I'm using InputStream, InputStreamReader, and BufferedReader to read the file. The IDE I'm using is NetBeans and the file being read is in the src directory. A quick note, for your convenience, I hardcoded the filename, so that you would understand how it's actually being read. In my actual program, the filename is being passed in as a parameter of the method. Anyways, it seems to work fine in the IDE. But when I create a JAR, it doesn't do what I want it to do.
public void readFile(filename) throws IOException, FileNotFoundException {
is = getClass().getResourceAsStream("file.csv");
isr = new InputStreamReader(is);
//fr = new FileReader(filename);
br = new BufferedReader(isr);
String info;
while ((info = br.readLine()) != null)
{
String[] tokens = info.split(",");
Object object = new Object();
object.setProperty(tokens[0]);
object.setAnotherProperty(tokens[1]);
object.setSomeOtherProperty(tokens[2]);
}
}
catch (FileNotFoundException f)
{
f.getMessage();
}
catch (IOException ioe)
{
ioe.getMessage();
}
catch (ArrayIndexOutOfBoundsException oob)
{
//;
}
catch (NullPointerException npe)
{
//;
}
finally
{
br.close();
isr.close();
is.close();
}
My method to update the file looks like this(once again, the filename has been hardcoded so you could better understand what's going on):
public void updateRoom(String filename, String property1, string property2, string property3) throws FileNotFoundException
{
for (Objects o : objects)
{
if (o.getProperty().equals(property1))
{
o.setProperty(property1);
o.setAnotherProperty(property2);
o.setSomeOtherProperty(property3);
}
}
File file = new File("file.csv");
PrintWriter pr = new PrintWriter(file);
for (Object o : objects)
{
pr.println(o.getProperty() + "," +
o.getAnotherProperty() + "," +
o.getSomeOtherProperty())
}
pr.close();
}
The problem is that the .JAR reads the file when I run it, but instead of writing to the SAME file, it simply creates a new one and writes to that one. It's a problem because every time I run the program again, the properties and values remain unchanged. It's NOT reading from the newly-created file. It's still reading from the original file, but it's writing to a new file.
I want to READ AND WRITE to the same file. That way, if I close the program and run it again, it will have the new properties/values already loaded in.

How to read a file in Android

I have a text file called "high.txt". I need the data inside for my Android app. But I have absolutely no idea how to read it into an ArrayList of the Strings. I tried the normal way of doing it in Java but apparently that doesn't work in Android since it cant find the file. So how do I go about doing this? I have put it in my res folder. But how do you take the input stream that you get from opening the file within Android and read it into an ArrayList of Strings. I am stuck on that part.
Basically it would look something like this:
3. What do you do for an upcoming test?
L: make sure I know what I'm studying and really review and study for this thing. Its what Im good at. Understand the material really well.
CL: Time to study. I got this, but I really need to make sure I know it,
M: Tests can be tough, but there are tips and tricks. Focus on the important, interesting stuff. Cram in all the little details just to get past this test.
CR: -sigh- I don't like these tests. Hope I've studied enough to pass or maybe do well.
R: Screw the test. I'll study later, day before should be good.
This is for a sample question and all the lines will be stored as separate strings in the array list.

If you put the text file in your assets folder you can use code like this which I've taken and modified from one of my projects:
public static void importData(Context context) {
try {
BufferedReader br = new BufferedReader(new InputStreamReader(context.getAssets().open("high.txt")));
String line;
while ((line = br.readLine()) != null) {
String[] columns = line.split(",");
Model model = new Model();
model.date = DateUtil.getCalendar(columns[0], "MM/dd/yyyy");
model.name = columns[1];
dbHelper.insertModel(model);
}
} catch (IOException e) {
e.printStackTrace();
}
}
Within the loop you can do anything you need with the columns, what this example is doing is creating an object from each row and saving it in the database.
For this example the text file would look something like this:
15/04/2013,Bob
03/03/2013,John
21/04/2013,Steve

If you want to read file from External storage than use below method.
public void readFileFromExternal(){
String path = Environment.getExternalStorageDirectory().getPath()
+ "/AppTextFile.txt";
try {
BufferedReader reader = new BufferedReader(new FileReader(path));
String line, results = "";
while( ( line = reader.readLine() ) != null)
{
results += line;
}
reader.close();
Log.d("FILE","Data in your file : " + results);
} catch (Exception e) {
}
}

//find all files from folder /assets/txt/
String[] elements;
try {
elements = getAssets().list("txt");
} catch (IOException e) {
e.printStackTrace();
}
//for every files read text per line
for (String fileName : elements) {
Log.d("xxx", "File: " + fileName);
try {
InputStream open = getAssets().open("txt/" + fileName);
InputStreamReader inputStreamReader = new InputStreamReader(open);
BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
String line = "";
while ((line = bufferedReader.readLine()) != null) {
Log.d("xxx", line);
}
} catch (IOException e) {
e.printStackTrace();
}
}

How to read an external text file from jar

I have a text file that gets written to a network folder and I want my users to be able to click on a jar file which will read in the text file, sort it, and output a sorted file to the same folder. But I'm having trouble formatting the syntax of the InputStream to read the file in.
When I use a FileReader instead of an InputStreamReader the following code works fine in eclipse, but returns empty when run from the jar. When I change it to InputStream like my research suggests - I get a NullPointerException like it can't find the file.
Where did I go wrong? :)
public class sort {
/**
* #param args
*/
public static void main(String[] args) {
sort s = new sort();
ArrayList<String> farmRecords = new ArrayList<String>();
farmRecords = s.getRecords();
String testString = new String();
if(farmRecords.size() > 0){
//do some work to sort the file
}else{
testString = "it didn't really work";
}
writeThis(testString);
}
public ArrayList<String> getRecords(){
ArrayList<String> records = new ArrayList();
BufferedReader br;
InputStream recordsStream = this.getClass().getResourceAsStream("./input.IDX");
try {
String sCurrentLine;
InputStreamReader recordsStreamReader = new InputStreamReader(recordsStream);
br = new BufferedReader(recordsStreamReader);
while ((sCurrentLine = br.readLine()) != null) {
System.out.println(sCurrentLine);
}
br.close();
} catch (IOException e) {
e.printStackTrace();
}
return records;
}
private static void writeThis(String payload){
String filename = "./output.IDX";
try {
BufferedWriter fr = new BufferedWriter(new FileWriter(filename));
fr.write(payload);
fr.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}

getResourceAsStream() loads files from the classpath. If you are running this from the command line, you would need the current directory (.) on the classpath. If you want to load arbitrary files from the file system, you should use FileInputStream (or FileReader to save having to subsequently wrap the input stream in a reader).

Using a FIS to get a file inside a jar will not work since the file is not on the file system per se. You should use getResrouceAsStream() for that.
Also, to access a file inside a jar, you must add an "!" to the file path. Is the file inside the jar? If not, then try a script to start the jar after passing the classpath:
start.sh
java -cp .:your.jar com.main.class.example.run
Execute this script (on linux) or modify it as per your platform.
Also, you can use the following code to print out the classpath. This way you can check whether your classpath contains the file?
ClassLoader sysClassLoader = ClassLoader.getSystemClassLoader();
// Get the URLs
URL[] urls = ((URLClassLoader) sysClassLoader).getURLs();
for (int i = 0; i < urls.length; i++) {
System.out.println(urls[i].getFile());
}
}