We Keep Coding

Writing an extra line when reading a text file from MySQL database and writing into project folder

I am storing text files in MySQL and reading it from Java into the project file. I am facing a problem in this. I am saving 3 files which contain 992 lines each. But when I am reading and saving it back in java project folder, 2 files are written as 993 lines each, the last line is an empty string.
How to resolve this?
Here are my codes to read and write into project folder.
and I am attaching the links where the files can be accessed.
file1
file2
file3
In these files file1 and file2 are writing extra lines.
Here is the code
import java.io.File;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
public class ReadBlob
{
public static void main(String[] args) throws Exception
{
Connection myConn = null;
Statement myStmt = null;
ResultSet myRs = null;
InputStream input = null;
FileOutputStream output = null;
try {
// 1. Get a connection to database
myConn = DriverManager.getConnection(
"jdbc:mysql://localhost:3306/database", "username", "password");
// 2. Execute statement
myStmt = myConn.createStatement();
String sql = "select file from tablename where id='1'";//I am mention the ID related to the files
myRs = myStmt.executeQuery(sql);
// 3. Set up a handle to the file
File theFile = new File("data2.txt");
output = new FileOutputStream(theFile);
if (myRs.next())
{
input = myRs.getBinaryStream("file");
System.out.println("Reading data from database...");
System.out.println(sql);
byte[] buffer = new byte[2];
while (input.read(buffer) > 0) {
output.write(buffer);
}
System.out.println("\nSaved to file: " + theFile.getAbsolutePath());
System.out.println("\nCompleted successfully!");
}
}
catch (Exception exc)
{
exc.printStackTrace();
}
finally
{
if (input != null)
{
input.close();
}
if (output != null)
{
output.close();
}
close(myConn, myStmt);
}
}
private static void close(Connection myConn, Statement myStmt)
throws SQLException {
if (myStmt != null) {
myStmt.close();
}
if (myConn != null) {
myConn.close();
}
}
}

File 1 is 20579 bytes long, file 2 is 20585 bytes long and file 3 is 20612 bytes long. The working file has an even length. In your code you read and write bytes 2 by 2. My guess is that when you write the last byte of file 1 and 2 you add an extra byte in your array and write it to your file.
Try to read bytes one by one to see if it works better.

You are reading the stream incorrectly by ignoring the actual number of bytes read returned by the read method. This leads to off-by-one errors in the number of bytes you write, which results in repeating bytes from earlier reads, corrupting your written file.
You need to change your code to:
byte[] buffer = new byte[4096];
int bytesRead;
while ((bytesRead = input.read(buffer)) != -1) {
output.write(buffer, 0, bytesRead);
}
Notice that I also increased the buffer size, as this will be more efficient than reading per two bytes.
Your code can be further simplified by using try-with-resources to close the streams.
Or even more concise, don't create an output stream, and instead use:
Files.copy(input, file.toPath());
The files may still be incorrect, as it is possible you made similar mistakes writing the file.

use Java to convert ANY file to hex and back again

I have some files i would like to convert to hex, alter, and then reverse again, but i have a problem trying to do jars, zips, and rars. It seems to only work on files containing normally readable text. I have looked all around but cant find anything that would allow jars or bats to do this correctly. Does anyone have an answer that does both? converts to hex then back again, not just to hex?

You can convert any file to hex. It's just a matter of obtaining a byte stream, and mapping every byte to two hexadecimal numbers.
Here's a utility class that lets you convert from a binary stream to a hex stream and back:
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.io.Reader;
import java.io.Writer;
public class Hex {
public static void binaryToHex(InputStream is, OutputStream os) {
Writer writer = new BufferedWriter(new OutputStreamWriter(os));
try {
int value;
while ((value = is.read()) != -1) {
writer.write(String.format("%02X", value));
}
writer.flush();
} catch (IOException e) {
System.err.println("An error occurred");
}
}
public static void hexToBinary(InputStream is, OutputStream os) {
Reader reader = new BufferedReader(new InputStreamReader(is));
try {
char buffer[] = new char[2];
while (reader.read(buffer) != -1) {
os.write((Character.digit(buffer[0], 16) << 4)
+ Character.digit(buffer[1], 16));
}
} catch (IOException e) {
System.err.println("An error occurred");
}
}
}
Partly inspired by this sample from Mykong and this answer.

Don't use a Reader to read String / char / char[], use an InputStream to read byte / byte[].

How to find the end of file (EOF) for a zip file in java [duplicate]

This question already has answers here:
How to check if a generated zip file is corrupted?
(8 answers)
Closed 7 years ago.
I am trying to download a ZIP file from a http url connection. How do i confirm if the zip file is completely downloaded. Is there anyway i can get the end of zip file.

EOF will reach when stream return -1 (as mentioned by MadConan also). You need to read the data until
inputStream.read == -1
A simple example shown below:
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.net.URLConnection;
public class ZipFileDownloader {
public static void downloadZipFile(String urlPath, String saveTo) {
try {
URLConnection connection = new URL(urlPath).openConnection();
InputStream in = connection.getInputStream();
FileOutputStream out = new FileOutputStream(saveTo + "myFile.zip");
byte[] b = new byte[1024];
int count;
while ((count = in.read(b)) > 0) {
out.write(b, 0, count);
}
out.flush();
out.close();
in.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}

Reading contents of excel file

I am looking for some help with reading contents from a excel file.
Here is the code, I want it to read from the .csv file and
generate the result for the same.
My error shows
java.lang.NumberFormatException: For input string: "companycode"
at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043)
at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
at java.lang.Double.parseDouble(Double.java:538)
at java.lang.Double.valueOf(Double.java:502)
at testgbt.TestGBT.main(TestGBT.java:40)
Please help and thanks in advance!!
package testgbt;
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.FileWriter;
import java.io.InputStreamReader;
import java.util.Vector;
import java.io.FileOutputStream;
import testgbt.BoostingTree;
import testgbt.BoostingTree.ResultFunction;
public class TestGBT {
public static void main(String[] args) {
String data_name = "C:\\Mihir\\NetBeans\\TestGBT\\data.csv";
Vector<Vector<Double>> x = new Vector<Vector<Double>>();
Vector<Double> y = new Vector<Double>();
try {
FileInputStream fis = new FileInputStream("C:\\Mihir\\NetBeans\\TestGBT\\data.csv");
// -------- load data from the file -------------
// get object of data input stream
BufferedReader b_reader = new BufferedReader(new
InputStreamReader(new FileInputStream(data_name))); // buffer
String read_line;
while ((read_line = b_reader.readLine()) != null) {
String[] str_list = read_line.trim().split(",");
Vector<Double> buffer_x = new Vector<Double>();
buffer_x.add(Double.valueOf(str_list[0]));
x.add(buffer_x);
y.add(Double.valueOf(str_list[1]));
}
b_reader.close();
// --------- learn a function of y=f(x) -------
BoostingTree gbt_ranker = new BoostingTree();
ResultFunction res_fun = gbt_ranker.gradient_boosting_tree(x, y);
// --------- save the curve fitting result y=f(x) ------
FileWriter file_writer = new FileWriter("C:\\Mihir\\NetBeans\\TestGBT\\result.txt");
FileOutputStream fos=new FileOutputStream("C:\\Mihir\\NetBeans\\TestGBT\\result.csv");
for (int i = 0; i < x.size(); i ++) {
file_writer.append(String.format("%f,%f,%f\n",
x.get(i).get(0),
res_fun.get_value(x.get(i)),
y.get(i)));
}
file_writer.close();
} catch (Exception err) {
err.printStackTrace();
System.exit(0);
}
}
}

Does your input CSV have a header row? Remove the header row from your input data or change your code to skip the first line.
You can skip the first row easily by changing this line:
String read_line;
to this:
String read_line = b_reader.readLine();

Looks like str_list[1] isn't the data you're looking for. In fact, it looks like you're trying to convert one of your column headers into a double. You probably just need to skip past the headers.

Getting NULL value at the end while writing to the file

I have problem while writing to the file. I want to write contents of my input file to output file but while writing to the file, I am getting NULL value written at the end of file.
What's the reason behind that?
My code is:
import java.io.BufferedReader;
import java.io.DataInputStream;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
public class FileReading {
/**
* #param args
* #throws IOException
*/
public static void main(String[] args) throws IOException {
// TODO Auto-generated method stub
FileInputStream fi=new
FileInputStream("E:\\Tejas\\NewData_FromNov\\New_Folder\\bb.txt");
DataInputStream di=new DataInputStream(fi);
BufferedReader buf=new BufferedReader(new InputStreamReader(di));
FileOutputStream fout=new FileOutputStream("E:\\Tejas\\NewData_FromNov\\New_Folder\\Out_bb.txt");
int ss;
byte[] input=new byte[500];
int len=input.length;
while((ss=di.read(input,0,len))!=-1)
{
System.out.println(ss);
//fout.write(ss);
fout.write(input,0,len);
}
fout.close();
}
}

You're always writing out the full buffer, even if you've only read part of it because the third argument to write is len (the length of the buffer) instead of ss (the number of bytes read). Your loop should look like this:
int bytesRead; // Easier to understand than "ss"
byte[] buffer = new byte[500];
while((bytesRead = di.read(buffer, 0, buffer.length)) != -1)
{
System.out.println(bytesRead);
fout.write(buffer, 0, bytesRead);
}
Additionally:
You should close both the input and output streams in finally blocks to ensure they're always closed (even if there's an exception).
You don't need a DataInputStream - just a FileInputStream is fine here.
You're not using your BufferedReader at all.
Consider using Guava or a similar third-party library which contains utility methods to do all of this.

The read method returns the number of actually read bytes, or -1 if the end of the stream has been reached. So you should only write ss bytes, and not len bytes:
while ((ss = di.read(input, 0, len)) != -1) {
System.out.println(ss);
fout.write(input, 0, ss);
}
Note thet the DataInputStream and the BufferedReader are completely unnecessary here.