I'm trying to send a deflated string over http, when I use compression and decompression on the server side, without using streams, it's ok but when I write it to stream like this:
byte[] deflatedData = mtext.getByte();
try {
t.sendResponseHeaders(200,deflatedData.length);
} catch (IOException e1) {
display(e1);
e1.printStackTrace();
if(closeafter){
t.close();
}
return;
}
DeflaterOutputStream os = new DeflaterOutputStream(t.getResponseBody());
try {
os.write(deflatedData ,0,deflatedData .length);
} catch (IOException e1) {
mByte = null;
display(e1);
if(closeafter){
t.close();
}
return;
}
os.flush();
os.close();
and read from client side like this:
InflaterInputStream ini = new
InflaterInputStream(response.body().byteStream());
ByteArrayOutputStream bout =new ByteArrayOutputStream(512);
int b;
while ((b = ini.read()) != -1) {
bout.write(b);
}
ini.close();
bout.close();
String s=new String(bout.toByteArray());
android decompresses like this:
public static byte[] decompress(byte[] data) throws IOException, DataFormatException{
Inflater inflater = new Inflater();
inflater.setInput(data);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(data.length);
byte[] buffer = new byte[1024];
while (!inflater.finished()) {
int count = inflater.inflate(buffer);
outputStream.write(buffer, 0, count);
}
byte[] output = outputStream.toByteArray();
outputStream.close();
inflater.end();
return output;
}
so I get the following exception:
java.util.zip.DataFormatException: data error
Where am I going wrong?
The sending part was totally ok , The answer was to Use InflaterInputStream Directly from the input stream , like this:
public static String ReadDeflatedData(InputStream input){
InflaterInputStream in = new InflaterInputStream(input, new Inflater());
int bytesRead=0;
StringBuilder sb = new StringBuilder();
byte[] contents = null;
try {
contents = new byte[in.available()];
} catch (IOException e2) {
// TODO Auto-generated catch block
e2.printStackTrace();
}
try {
while( (bytesRead = in.read(contents)) != -1){
sb.append(new String(contents, 0, bytesRead));
}
} catch (IOException e) {
System.out.println(e.toString());
e.printStackTrace();
}
try {
return new String(sb.toString().getBytes(),"UTF-8");
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return null;
}
Related
I am using Opencsv v5.3, I am encoding InputStream to OutputStream in CSV format. In the scenario, InputStream contains 24051 records while it writes 24037 correct records with 24038th corrupted/incomplete record.
Could you please help?
Below is my code:
private static void doCsvEncoding(Socket clientSocket, InputStream inputStream, OutputStream outputStream) {
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
try {
byte[] byteChunk = new byte[131072];
try {
int readBytes = inputStream.read(byteChunk);
int total = 0;
while (readBytes != -1) {
total += readBytes;
byteArrayOutputStream.write(byteChunk, 0, readBytes);
readBytes = inputStream.read(byteChunk);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
clientSocket.shutdownInput();
}
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(byteArrayOutputStream.toByteArray());
ObjectInputStream objectInputStream = new ObjectInputStream(byteArrayInputStream);
int cnt = 0;
ByteArrayOutputStream mediatorOutputStream = new ByteArrayOutputStream();
StatefulBeanToCsv<T> statefulBeanToCsv =
new StatefulBeanToCsvBuilder<T>(new OutputStreamWriter(mediatorOutputStream))
.withIgnoreField(T.class, T.class.getField("tag"))
.withIgnoreField(T.class, T.class.getDeclaredField("code")).build();
try {
while (true) {
T t = (T)objectInputStream.readObject();
statefulBeanToCsv.write(t);
cnt ++;
}
} catch (IOException | NullPointerException e) {
e.printStackTrace();
} catch (CsvRequiredFieldEmptyException e) {
e.printStackTrace();
} catch (CsvDataTypeMismatchException e) {
e.printStackTrace();
}
outputStream.write(("" + cnt + " ").getBytes());
System.out.println(cnt);
mediatorOutputStream.close();
ByteArrayInputStream mediatorInputStream = new ByteArrayInputStream(mediatorOutputStream.toByteArray());
try {
int readBytes = mediatorInputStream.read(byteChunk);
while (readBytes != -1) {
outputStream.write(byteChunk, 0, readBytes);
readBytes = mediatorInputStream.read(byteChunk);
}
clientSocket.shutdownOutput();
} catch (IOException e) {
e.printStackTrace();
}
} catch(IOException | ClassNotFoundException | NoSuchFieldException e) {
e.printStackTrace();
}
}
Instead of using StatefulBeanToCsvBuilder, using CSVWiter and adding header through Java Reflection worked.
Currently am facing a problem with FileOutputStream in my code used FileOutputStream for creating file in my disk .Once file created there is no way for opening , deleting or moving file from its location ,getting error message already locked by other user When stopped web server it working properly .how can i solve this issue.
private void writeToFile(InputStream uploadedInputStream,
String uploadedFileLocation) {
try {
OutputStream out = new FileOutputStream(new File(
uploadedFileLocation));
int read = 0;
byte[] bytes = new byte[1024];
out = new FileOutputStream(new File(uploadedFileLocation));
while ((read = uploadedInputStream.read(bytes)) != -1) {
out.write(bytes, 0, read);
}
out.flush();
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}
You are creating two instances of FileOutputStream and assigning both to out, but only closing one.
Remove the second out = new FileOutputStream(new File(uploadedFileLocation));.
Also, you should consider using a try-with-resources block
private void writeToFile(InputStream uploadedInputStream,
String uploadedFileLocation) {
try (OutputStream out = new FileOutputStream(new File(uploadedFileLocation))) {
int read = 0;
byte[] bytes = new byte[1024];
while ((read = uploadedInputStream.read(bytes)) != -1) {
out.write(bytes, 0, read);
}
} catch (IOException e) {
e.printStackTrace();
}
}
or finally block to ensure that the streams are closed
private void writeToFile(InputStream uploadedInputStream,
String uploadedFileLocation) {
OutputStream out = null;
try {
out = new FileOutputStream(new File(uploadedFileLocation))
int read = 0;
byte[] bytes = new byte[1024];
while ((read = uploadedInputStream.read(bytes)) != -1) {
out.write(bytes, 0, read);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
if (out != null) {
try {
out.close();
} catch (IOException exp) {
}
}
}
}
See The try-with-resources Statement for more details
I can marhsall XML from a file when I read it from disk, but when I download it via the web I get this error.
[org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature end of file.]
at javax.xml.bind.helpers.AbstractUnmarshallerImpl.createUnmarshalException
I assume the web input stream contains additional information or something?
Works
InputStream inputStream = null;
try {
inputStream = new FileInputStream(filePath);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
Doesnt Work
InputStream inputStream = null;
try {
inputStream = new URL(url).openStream();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
BulkDataRecordType bulkDataRecordType = getObjectFromXml(inputStream);
In another class
public BulkDataRecordType getObjectFromXml(InputStream inputStream)
{
try {
JAXBContext jc = JAXBContext.newInstance(BulkDataRecordType.class);
Unmarshaller unmarshaller = jc.createUnmarshaller();
bulkDataRecordType = (BulkDataRecordType) unmarshaller.unmarshal(inputStream);
} catch (JAXBException e1) {
e1.printStackTrace();
}
I am first checking the checksum of the string. Once I commented this out it worked. I found a solution to create two new streams and it worked. If you have a better solution let me know.
public byte[] getCheckSumFromFile(InputStream inputStream)
{
MessageDigest md = null;
try {
md = MessageDigest.getInstance("MD5");
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
return null;
}
InputStream is = null;
try {
is = new DigestInputStream(inputStream, md);
}
finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
return null;
}
}
return md.digest();
}
Create Two Streams From Original
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
try {
byte[] buffer = new byte[1024];
int len;
while ((len = inputStream.read(buffer)) > -1 ) {
byteArrayOutputStream.write(buffer, 0, len);
}
byteArrayOutputStream.flush();
} catch (IOException e1) {
e1.printStackTrace();
}
inputStream = new ByteArrayInputStream(byteArrayOutputStream.toByteArray());
// Get check sum of downloaded file
byte[] fileCheckSum = getCheckSumFromFile(new ByteArrayInputStream(byteArrayOutputStream.toByteArray()));
I want a efficient and a fast way of sending a stop message in a socket.
I have a method that send files from one pc to another. All of the files from the sender appear on the receiver's PC. However all of the data is being written to the first file (only). The other files exist, but are empty. This happen because the receiver method doesn't know when to start writing to the next file.
Sender
public static void sendFile (final Socket sock, File source)
{
FileInputStream fileIn = null;
try
{
//Read bytes from the source file
fileIn = new FileInputStream(source);
//Write bytes to the receive
//No need to use a buffered class, we make our own buffer.
OutputStream netOut = sock.getOutputStream();
byte[] buffer = new byte[BUFFER_SIZE];
int read;
while ((read = fileIn.read(buffer)) != -1)
{
netOut.write(buffer, 0, read);
netOut.flush ();
}
//Send some stop message here
}
catch (Exception e)
{
e.printStackTrace ();
}
finally
{
if (fileIn != null)
{
try
{
fileIn.close ();
}
catch (IOException e)
{
e.printStackTrace ();
}
}
}
}
//Send files via socket
public static void sendFile (final Socket sock, File[] source)
{
for (int i = 0; i < source.length; i++)
sendFile (sock, source[i]);
}
Receiver:
public static void receiveFile (final Socket sock, File destination)
{
BufferedOutputStream out = null;
try
{
//Receive data from socket
InputStream clientInputStream = sock.getInputStream();
//Write bytes to a file
out = new BufferedOutputStream (new FileOutputStream (destination));
byte[] buffer = new byte[BUFFER_SIZE];
int read;
while (true)
{
read = clientInputStream.read(buffer);
out.write(buffer, 0, read);
out.flush ();
}
}
catch (IOException e)
{
e.printStackTrace ();
}
finally
{
if (out != null)
{
try
{
out.close ();
}
catch (IOException e)
{
e.printStackTrace();
}
}
}
}
//Receive files via socket
public static void receiveFile (final Socket sock, File[] destination)
{
for (int i = 0; i < destination.length; i++)
receiveFile (sock, destination[i]);
}
You need to modify your send/receive protocol to include at least a minimal header before sending the file. Your header should include at least the size of the data to follow, and anything else you might want (such as file name).
I tried with a header like you suggested, but it doesn't work. The receiver still don't know when to stop(so I get EOFException). All the received data get written to the first file.
public static void sendFile (Socket sock, File source)
{
FileInputStream fileIn = null;
try
{
//Read bytes from the source file
fileIn = new FileInputStream(source);
//Write bytes to the receive
//No need to use a buffered class, we make our own buffer.
OutputStream netOut = sock.getOutputStream();
byte[] buffer = new byte[BUFFER_SIZE];
int readBytes = 0;
long fileSize = source.length();
long counter = 0;
//Send the file size
DataOutputStream objOut = new DataOutputStream (netOut);
System.out.println ("Writing: " + source.length ());
objOut.writeLong (fileSize);
objOut.flush ();
while ((counter += readBytes) < fileSize)
{
readBytes = fileIn.read(buffer);
netOut.write(buffer, 0, readBytes);
netOut.flush ();
}
fileIn.close();
}
catch (Exception e)
{
e.printStackTrace ();
}
finally
{
if (fileIn != null)
{
try
{
fileIn.close ();
}
catch (IOException e)
{
e.printStackTrace ();
}
}
}
}
[]
public static void receiveFile (Socket sock, File destination)
{
BufferedOutputStream fileOut = null;
try
{
//Receive data from socket
InputStream netIn = sock.getInputStream();
//Write bytes to a file
fileOut = new BufferedOutputStream (new FileOutputStream (destination));
byte[] buffer = new byte[BUFFER_SIZE];
int readBytes = 0;
long fileSize;
long counter = 0;
//Receive the file size
DataInputStream objIn = new DataInputStream (netIn);
fileSize = objIn.readLong ();
System.out.println ("Receiving: " + fileSize);
while (true)
{
readBytes = netIn.read (buffer);
fileOut.write (buffer, 0, readBytes);
fileOut.flush ();
counter += readBytes;
if (counter > fileSize)
break;
}
}
catch (IOException e)
{
e.printStackTrace ();
}
finally
{
if (fileOut != null)
{
try
{
fileOut.close ();
}
catch (IOException e)
{
e.printStackTrace();
}
}
}
System.out.println ("Ending method");
}
I always use this method to easily read the content of a file. Is it efficient enough? Is 1024 good for the buffer size?
public static String read(File file) {
FileInputStream stream = null;
StringBuilder str = new StringBuilder();
try {
stream = new FileInputStream(file);
} catch (FileNotFoundException e) {
}
FileChannel channel = stream.getChannel();
ByteBuffer buffer = ByteBuffer.allocate(1024);
try {
while (channel.read(buffer) != -1) {
buffer.flip();
while (buffer.hasRemaining()) {
str.append((char) buffer.get());
}
buffer.rewind();
}
} catch (IOException e) {
} finally {
try {
channel.close();
stream.close();
} catch (IOException e) {
}
}
return str.toString();
}
You may find that this is fast enough.
String text = FileUtils.readFileToString(file);
AFAIK, this uses the default buffer size of 8K. However I have found larger sizes such as 64K can make a slight difference.
I would always look to FileUtils http://commons.apache.org/io/api-1.4/org/apache/commons/io/FileUtils.html to see if they had a method. In this case I would use readFileToString(File)
http://commons.apache.org/io/api-1.4/org/apache/commons/io/FileUtils.html#readFileToString%28java.io.File%29
They have already dealt with almost all the problem cases...
Try the following, it should work (well):
public static String read(File file)
{
StringBuilder str = new StringBuilder();
BufferedReader in = null;
String line = null;
try
{
in = new BufferedReader(new FileReader(file));
while ((line = in.readLine()) != null)
str.append(line);
in.close();
}
catch (FileNotFoundException e)
{
e.printStackTrace();
}
catch (IOException e)
{
e.printStackTrace();
}
return str.toString();
}