We Keep Coding

Java Reading in text file and outputting it to new file with removed duplicates

I have a text file with an integer on each line, ordered from least to greatest, and I want to put them in a new text file with any duplicate numbers removed.
I've managed to read in the text file and print the numbers on the screen, but I'm unsure on how to actually write them in a new file, with duplicates removed?
public static void main(String[] args)
{
try
{
FileReader fr = new FileReader("sample.txt");
BufferedReader br = new BufferedReader(fr);
String str;
while ((str = br.readLine()) != null) {
out.println(str + "\n");
}
br.close();
}
catch (IOException e) {
out.println("File not found");
}
}

When reading the file, you could add the numbers to a Set, which is a data structure that doesn't allow duplicate values (just Google for "java collections" for more details)
Then you iterate through this Set, writing the numbers to a FileOutputStream (google for "java io" for more details)

Instead of printing each of the numbers, add them to an Array. After you've added all the integers, you can cycle through the array to remove duplicates (sample code for this can be found fairly easily).
Once you have an array, use BufferedWriter to write to an output file. Example code for how to do this can be found here: https://www.mkyong.com/java/how-to-write-to-file-in-java-bufferedwriter-example/
Alternatively, use a Set, and BufferedWriter should still work in the same way.

assuming the input file is already ordered:
public class Question42475459 {
public static void main(final String[] args) throws IOException {
final String inFile = "sample.txt";
try (final Scanner scanner = new Scanner(new BufferedInputStream(new FileInputStream("")), "UTF-8");
BufferedWriter writer = new BufferedWriter(new FileWriter(inFile + ".out", false))) {
String lastLine = null;
while (scanner.hasNext()) {
final String line = scanner.next();
if (!line.equals(lastLine)) {
writer.write(line);
writer.newLine();
lastLine = line;
}
}
}
}
}

Java code optimization, replacing all chars in a file

I have tried doing it like this:
import java.io.*;
public class ConvertChar {
public static void main(String args[]) {
Long now = System.nanoTime();
String nomCompletFichier = "C:\\Users\\aahamed\\Desktop\\test\\test.xml";
Convert(nomCompletFichier);
Long inter = System.nanoTime() - now;
System.out.println(inter);
}
public static void Convert(String nomCompletFichier) {
FileWriter writer = null;
BufferedReader reader = null;
try {
File file = new File(nomCompletFichier);
reader = new BufferedReader(new FileReader(file));
String oldtext = "";
while (reader.ready()) {
oldtext += reader.readLine() + "\n";
}
reader.close();
// replace a word in a file
// String newtext = oldtext.replaceAll("drink", "Love");
// To replace a line in a file
String newtext = oldtext.replaceAll("&(?!amp;)", "&");
writer = new FileWriter(file);
writer.write(newtext);
writer.close();
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
}
However the code above takes more time to execute than creating two different files:
import java.io.*;
public class ConvertChar {
public static void main(String args[]) {
Long now = System.nanoTime();
String nomCompletFichier = "C:\\Users\\aahamed\\Desktop\\test\\test.xml";
Convert(nomCompletFichier);
Long inter = System.nanoTime() - now;
System.out.println(inter);
}
private static void Convert(String nomCompletFichier) {
BufferedReader br = null;
BufferedWriter bw = null;
try {
File file = new File(nomCompletFichier);
File tempFile = File.createTempFile("buffer", ".tmp");
bw = new BufferedWriter(new FileWriter(tempFile, true));
br = new BufferedReader(new FileReader(file));
while (br.ready()) {
bw.write(br.readLine().replaceAll("&(?!amp;)", "&") + "\n");
}
bw.close();
br.close();
file.delete();
tempFile.renameTo(file);
} catch (IOException e) {
// writeLog("Erreur lors de la conversion des caractères : " + e.getMessage(), 0);
} finally {
try {
bw.close();
} catch (Exception ignore) {
}
try {
br.close();
} catch (Exception ignore) {
}
}
}
}
Is there any way to do the 2nd code without creating a temp file and reducing the execution time? I am doing a code optimization.

The main reason why your first program is slow is probably that it's building up the string oldtext incrementally. The problem with that is that each time you add another line to it it may need to make a copy of it. Since each copy takes time roughly proportional to the length of the string being copied, your execution time will scale like the square of the size of your input file.
You can check whether this is your problem by trying with files of different lengths and seeing how the runtime depends on the file size.
If so, one easy way to get around the problem is Java's StringBuilder class which is intended for exactly this task: building up a large string incrementally.

The main culprit in your first example is that you're building oldtext inefficiently using String concatenations, as explained here. This allocates a new string for every concatenation. Java provides you StringBuilder for building strings:
StringBuilder builder = new StringBuilder;
while(reader.ready()){
builder.append(reader.readLine());
builder.append("\n");
}
String oldtext = builder.toString();
You can also do the replacement when you're building your text in StringBuilder. Another problem with your code is that you shouldn't use ready() to check if there is some content left in the file - check the result of readLine(). Finally, closing the stream should be in a finally or try-with-resources block. The result could look like this:
StringBuilder builder = new StringBuilder();
try (BufferedReader reader = new BufferedReader(new FileReader(file))) {
String line = reader.readLine();
while (line != null) {
builder.append(line.replaceAll("&(?!amp;)", "&"));
builder.append('\n');
line = reader.readLine();
}
}
String newText = builder.toString();
Writing to a temporary file is a good solution too, though. The amount of I/O, which is the slowest to handle, is the same in both cases - read the full content once, write result once.

Count number of Characters in a CSV file in Java

I have this CSV File.
City,Job,Salary
Delhi,Doctors,500
Delhi,Lawyers,400
Delhi,Plumbers,100
London,Doctors,800
London,Lawyers,700
London,Plumbers,300
Tokyo,Doctors,900
Tokyo,Lawyers,800
Tokyo,Plumbers,400
Lawyers,Doctors,300
Lawyers,Lawyers,400
Lawyers,Plumbers,500
Hong Kong,Doctors,1800
Hong Kong,Lawyers,1100
Hong Kong,Plumbers,1000
Moscow,Doctors,300
Moscow,Lawyers,200
Moscow,Plumbers,100
Berlin,Doctors,800
Berlin,Plumbers,900
Paris,Doctors,900
Paris,Lawyers,800
Paris,Plumbers,500
Paris,Dog catchers,400
I have the code, which does multiple operations in the CSV File. Now I want to count the characters and also want to append a line to CSV File at the bottom.
import java.io.*;
import java.util.Arrays;
public class Main {
public static void main(String args[]) throws IOException
{
String csv="C:\\Users\\Dipayan\\Desktop\\salaries.csv";
BufferedReader br= new BufferedReader(new FileReader(csv));
String line="";
int count=0;
String str[]=new String[200];
int[] a=new int[24];
try {
br.readLine();
while((line = br.readLine())!=null)
{
String[] f=line.split(",");
a[count]=Integer.parseInt(f[2]);
str[count]=f[1];
count++;
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Arrays.sort(a);
for(int i:a)
{
System.out.println(i);
}
System.out.println("min="+a[0]);
System.out.println("Max="+a[count-1]);
// String sorting
String min1=str[0];
for(int i=0;i<count;i++)
{
if(str[i].compareTo(min1)<0)
min1=str[i];
}
System.out.println(min1);
/*String Sort end*/
/*Finding the Median*/
int median;
if(a.length%2==0)
median = a[a.length/2]+a[(a.length/2)+1];
else
median=a[a.length/2];
System.out.println("Median"+median);
/*Median*/
System.out.println("Line"+count);
}
}

java.io.File.length() gets you the length of the file denoted by the abstract pathname. A better approach, as my erroneous assumption was pointed out in #Tilo's comment would be to first read the file into a String, and use it's getChars().length method.
As to your second question, appending a line to the end of a file necessitates opening the file in append mode and writing your string to it, as in the following example:
FileWriter f = new FileWriter(csvFileObject, true);
f.write(ourString);
f.close();
Hope that helps... And, there are no external jars necessary.

Well, you're almost there. You're already reading each line
while((line = br.readLine())!=null) {
//bla bla
}
You'll have to declare an int characterSum = 0; before that and add the length of each line to it. Something like
int characterSum = 0;
while((line = br.readLine())!=null) {
characterSum += line.length();
//bla bla
}
As for appending a line to your file... there's already a question on this.

You can try CSVWriter java package to add a line to your CSV file.
Here is the link that shows an example of how to append a line in CSV.
http://www.csvreader.com/java_csv_samples.php

To count the characters modify your while loop like this:
int numChars = 0;
while((line = br.readLine())!=null) {
numChars += line.length();
// leave your other code here
}
To append a line use this constructor of the java.util.FileWriter:
try (FileWriter fw = new FileWriter("file.name", true);
BufferedWriter bw = new BufferedWriter(fw);
PrintWriter pw = new PrintWriter(bw)) {
pw.println("the,new,line,to,append");
} catch (IOException e) {
// handle
}
Note that the "try with resources" statement this is Java 7 syntax.

How to read data from a file and put it into variable and output to a different file. Java

Intro java class tard here. I'm trying to read data from a file and then manipulate to a different file and save it. I think i'm close but having issues using scanner and .IO together. Any help would be great.
import java.util.Scanner;
import java.io.*;
public class fileswitch
{
public static void main(String[] Args) throws IOException
{
String filename;
String filename2;
String text;
Scanner keyboard = new Scanner(System.in);
System.out.print("Enter the name of a file: ");
filename = keyboard.nextLine();
PrintWriter outputFile = new PrintWriter(filename);
System.out.print("Enter the name of a second file: ");
filename2 = keyboard.nextLine();
PrintWriter outputFile2 = new PrintWriter(filename2);
while (filename.hasNextLine())
{
text = filename.readAllLines();
text = text.toUpperCase();
outputFile2.print(text);
outputFile2.close();
}
}
}

You can also use for creating a new file
package test;
import java.io.File;
import java.io.IOException;
import org.apache.commons.io.FileUtils;
public class WriteStringToFile {
public static void main(String[] args) throws IOException {
String string = "This is\na test";
File file = new File("test.txt");
FileUtils.writeStringToFile(file, string);
}
}
And that is a good practice because you do not have to close streams.
This generates the test.txt file with the expected output

Try using BufferedReader
BufferedReader pw = new BufferedReader(new FileReader(fileName));
String s = null;
s = pw.readLine();
Working example
public static void main(String[] args) throws IOException {
Scanner keyboard = new Scanner(System.in);
String filePath = keyboard.next();
BufferedReader bufferedReader = new BufferedReader(new FileReader(filePath));
String line = bufferedReader.readLine();
System.out.println(line);
}
Enter path on console as
C:\Users\path\Desktop\1.txt
You can use PrintWriter to write
PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(fileName2)));

Your code does not compile.
while (filename.hasNextLine()) // String#hasNextLine() does not exist
hasNextLine() belongs to Scanner which isn't being used for reading the file but just your console keystrokes from the keyboard.
If you meant to use outputFile there; that won't work either because you can't use a PrintWriter as a file reader. Well, the name kind of makes that obvious. Doesn't it?
But, you should avoid using PrintWriter for writing as well unless you're formatting the output. For plain character output prefer a FileWriter (enclosed within a BufferedWriter for performance). Similarly, for reading the files prefer a FileReader (enclosed within a BufferedReader again).
Here's how your code would look:
public static void main(String[] Args) throws IOException
{
// create the scanner for console
Scanner keyboard = new Scanner(System.in);
// read the input/output file names
System.out.print("Enter the name of a file: ");
String inFile = keyboard.nextLine();
System.out.print("Enter the name of a second file: ");
String outFile = keyboard.nextLine();
// close the scanner
keyboard.close();
// open file streams
BufferedReader reader = new BufferedReader(new FileReader(inFile));
BufferedWriter writer = new BufferedWriter(new FileWriter(outFile));
// copy the data (line by line)
String text = null;
while ((text = reader.readLine()) != null)
{
writer.write(text);
writer.newLine();
}
// close the file streams
reader.close();
writer.close();
}

Creating a file and writing to it

I created a file named 'test.txt' and then took input from the user to write the input to the file. Everything runs fine. The program doesn't show any error at all. The file is created and the program takes input from the user but when I checked the content of the file, it was empty. Can anyone figure out what is wrong with my code? The code is as follows.
package InputOutput;
import java.io.*;
public class CharacterFileReaderAndFileWriter{
private BufferedReader br = null;
private BufferedWriter bw = null;
private PrintWriter pw = new PrintWriter(System.out, true);
public File createFile() throws IOException{
File f = new File("test.txt");
return f;
}
public void writeToFile() throws IOException{
try{
bw = new BufferedWriter(new FileWriter(createFile()));
}
catch(FileNotFoundException ex){
ex.printStackTrace();
}
//take input from the console (user)
br = new BufferedReader(new InputStreamReader(System.in));
String s;
pw.println("Please enter something");
pw.println("To stop the program, enter 'stop'");
do{
s = br.readLine();
if(s.compareTo("stop")==0)
break;
s+= "\r\n";//adding an new line to the string s
bw.write(s);
}
while(s.compareTo("stop")!=0);
br.close();
bw.close();
}
public static void main(String[] args) throws IOException{
CharacterFileReaderAndFileWriter cfr = new CharacterFileReaderAndFileWriter();
cfr.writeToFile();
}
}

Most example programs show that you have to call.flush() on your BufferedWriter before the .close(). This should not be required, .close() should call .flush() automatically, but it doesn't hurt. Also you should call all the Stream/Writer objects .close() methods in reverse order as well, again correctly written classes should call .close() on all the object they wrap, but it doesn't hurt to do it anyway.
Other things that might catch you out later:
if(s.compareTo("stop")==0)
should be
if ("stop".equalsIgnoreCase(s))
it is more efficient, eliminates the possibility of a NullPointerException on s, handles any case of stop and most importantly more idiomatic Java than using .compareTo()
s+= "\r\n";//adding an new line to the string s
bw.write(s);
should be
bw.write(System.getProperty("line.separator"));
bw.write(s);
The s+= creates intermediate objects and garbage that needs to be collected. Hard coding line endings is bad as well.

You need close the outputstream.
file.close();