We Keep Coding

Socket data transfer, "MZ" Appended to buffer? C++ / Java

I'm sending a file from Server (Java) to client (C++).
While everywhere else the data is recevied just fine. In this area, Sometimes there is "MZ[]" appended to the filename.
Here is the Receving code (C++):
else if(command == "freceive")
{
// Receive Filename
int fsize, l = 0;
WIN32_FIND_DATA FindFileData;
HANDLE hFind;
char filetoSave[BUFFER] = { 0 };
std::ostringstream response;
memset(filetoSave, '\0', BUFFER);
int fl = recv(sockfd, filetoSave, BUFFER, 0);
if (fl == SOCKET_ERROR && WSAGetLastError() == WSAECONNRESET)
{
connected = false;
}
//===================================================
std::ofstream File(filetoSave, std::ios::app | std::ios::binary);
memset(recvbuf, '\0', BUFFER);
// Recevie File
while ((fsize = recv(sockfd, recvbuf + l, sizeof(recvbuf) - l, 0)) > 0)
{
if(fsize > 0){
File.write(recvbuf, fsize);
}
}
File.close();
if(fsize == SOCKET_ERROR && WSAGetLastError() == WSAECONNRESET)
{
connected = false;
}
File.close();
if(IsFile((char*)filetoSave) == TRUE){
hFind = FindFirstFile(filetoSave , &FindFileData);
if (hFind == INVALID_HANDLE_VALUE)
{
response << "ERROR Receiving file (" << filetoSave << ") : " << GetLastError();
}
else
{
ULONGLONG FileSize = FindFileData.nFileSizeHigh;
FileSize <<= sizeof( FindFileData.nFileSizeHigh ) * 8;
FileSize |= FindFileData.nFileSizeLow;
response << "Saved : '" << filetoSave << "'\nSize : " << FileSize << " bytes.\nSaved as : " << filetoSave;
FindClose(hFind);
}
} else {
response << "Error File not found (" << filetoSave << ") : " << GetLastError();
}
EncSend(response.str().c_str());
}
and here is the part of the server which sends the filename (Java):
public static void SendRaw(Socket sock, String data)
{
try {
OutputStream outputStream = sock.getOutputStream();
DataOutputStream dataOutputStream = new DataOutputStream(outputStream);
dataOutputStream.write(data.getBytes());
dataOutputStream.flush();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (NullPointerException ne) {
ne.printStackTrace();
}
}
Everything is going ok except the fact that the filename has MZ[] appended to it which I do not want.
Why is this happening? It has given me a headache! Where is this MZ[] coming from!?
Here's a screenshot, Note after MZ there is a wierd ascii char like [] in absolute filename, But os ignores it.
screenshot here

file not found exception (No such directory)

I am trying to write to eeprom (at24) from user space in android.
I got the exception
com.ipil.ipilMdm.service.lib I/System.out: java.io.FileNotFoundException: /sys/devices/soc.0/78ba000.i2c/i2c-6/6-0050/eeprom (No such file or directory)
The path mentioned works perfect from echo and cat in command line. It is also tested in linux based application where it is opened as a read-write file.
I have added the permissions in manifest file. What else might be the issue?
Below is my code:
#Override
public void prod_eeprom_write(String data, int id) {
String someText = "Writesomething";
//String fileName = Environment.getRootDirectory().getAbsolutePath();
final File permFile = new File("/sys/devices/soc.0/78ba000.i2c/i2c-6/6-0050/eeprom");
try {
/* if (file == null) {
System.out.print("error open\n");
return;
}*/
//=================== Condition for action==========================
switch (id) {
case IMEI_NUMBER:
offset = IMEI_NUMBER_OFFSET;
length = IMEI_NUMBER_LENGTH;
break;
case HARDWARE_REVISION:
offset = HW_REV_OFFSET;
length = HW_REV_LENGTH;
break;
case WIFI_MAC_ADDRESS:
offset = WIFI_MAC_OFFSET;
length = WIFI_MAC_LENGTH;
break;
case BT_MAC_ADDRESS:
offset = BT_MAC_OFFSET;
length = BT_MAC_LENGTH;
break;
default : return;
}
/* now got offset and length, so update max buffer */
//strncpy(data,max_data.length + offset, length);
if (data.length() < length)
length = data.length();
if(data.length() > length){
System.out.print("LENGTH OF DATA IS TOO BIG : CAN NOT WRITE\nMAX LENGTH IS %d BYTES\n"+ length);
return;
}
//strncpy(data, max_data.length + offset, data.length());
max_data[INFO_MAX_LENGTH] = '\0';
System.out.println("DATA TO BE WRITTEN TO EEPROM: "+ data);
//if(file.canWrite()){
fileOutputStream = new FileOutputStream(permFile);
byte[] b = data.getBytes(); //new byte[max_data.length];
//for (int i = 0; i < b.length; i++)
// b[i] = (byte) max_data[i];
fileOutputStream.write(b, max_data.length + offset, data.length());
Log.i(TAG, " prod_eeprom_written-----------ok : hogya ");
Thread.sleep(10);
fileOutputStream.close();
System.out.println("success...");
}catch (Exception io){
io.printStackTrace();
}
}

Java IIOException caused by ZipException while trying to convert from String to PNG

My Java Servlet has a String coming from a client (all I know is that he is actually sending me a PNG as a parameter in a request with contentType = application/x-www-form-urlencoded), and I'm trying to convert it back to a File and save it. Here is my code (based on this):
byte[] data = org.apache.commons.codec.binary.Base64.decodeBase64(request.getParameter("image").getBytes());
String filename = request.getParameter("filename");
try{
java.awt.image.BufferedImage image = javax.imageio.ImageIO.read(new ByteArrayInputStream(data));
javax.imageio.ImageIO.write(image, "png", new File(getUploadFolder() + filename + ".png"));
}catch(Exception e){
System.out.println("Exception saving image: " + e.getMessage());
e.printStackTrace();
}
And here is my error stacktrace:
Exception saving image: Error reading PNG image data
javax.imageio.IIOException: Error reading PNG image data
at com.sun.imageio.plugins.png.PNGImageReader.readImage(PNGImageReader.java:1308)
at com.sun.imageio.plugins.png.PNGImageReader.read(PNGImageReader.java:1577)
at javax.imageio.ImageIO.read(ImageIO.java:1448)
at javax.imageio.ImageIO.read(ImageIO.java:1352)
at com.mypackage.servlet.UploadServlet.doPost(UploadServlet.java:113)
Caused by: java.util.zip.ZipException: invalid code lengths set
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:164)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
at java.io.FilterInputStream.read(FilterInputStream.java:83)
at com.sun.imageio.plugins.png.PNGImageReader.decodePass(PNGImageReader.java:1085)
at com.sun.imageio.plugins.png.PNGImageReader.decodeImage(PNGImageReader.java:1196)
at com.sun.imageio.plugins.png.PNGImageReader.readImage(PNGImageReader.java:1301)
... 4 more
Looking at the InflaterInputStream class code, I see that it is actually a DataFormatException:
public int read(byte[] b, int off, int len) throws IOException {
ensureOpen();
if (b == null) {
throw new NullPointerException();
} else if (off < 0 || len < 0 || len > b.length - off) {
throw new IndexOutOfBoundsException();
} else if (len == 0) {
return 0;
}
try {
int n;
while ((n = inf.inflate(b, off, len)) == 0) {
if (inf.finished() || inf.needsDictionary()) {
reachEOF = true;
return -1;
}
if (inf.needsInput()) {
fill();
}
}
return n;
} catch (DataFormatException e) {
String s = e.getMessage();
throw new ZipException(s != null ? s : "Invalid ZLIB data format");
}
}
But this is not really so enlightening.

As #leonbloy says, there's no need to decode and then re-encode the data.
If you are sure the PNG file is good*, you can simply write the data to file, as follows, and you should be done.
byte[] data = org.apache.commons.codec.binary.Base64.decodeBase64(request.getParameter("image").getBytes());
String filename = request.getParameter("filename");
OutputStream output = new FileOutputStream(new File(getUploadFolder() + filename + ".png"));
try {
output.write(data);
}
finally {
output.close();
}
*) The evidence provided so far suggests otherwise, and the code you were using should work (even if it's a waste of CPU cycles). It would be very interesting to see a sample of the data producing the exception you see, to figure out the cause of the exception.

Replace nio sockets to exec'ed software for binary protocol

I have to make an abstaction in my software - replace direct unblockable NIO sockets ( client/server ) to software abstraction.
For example, instead of connecting via tcp client would exec openssl s_client -connect xxx.xxx.xxx.xxx . I have written a little demo, and it even works. Sometimes :(
The first trouble is that Process's streams can't be used with Selector, so I can't replace socketchannel with any other type of channel, so I have to read/write without any chance to avoid blocking.
The second one is that a protocol is a duplex binary file-transfer protocol ( binkp ), so process's buffered streams are unusabe. I've tried to avoid that converting in/out data to base64 and it works, but also sometimes.
I can't understant why it works or not sometimes. I put a piece of test code below. The first word is frame's length, but first bit is ignored. Please, tell me your guesses. Thanks.
public class BufferedSocketBase64 {
static class InToOut implements Runnable {
InputStream is;
OutputStream os;
boolean direction; //
public InToOut(InputStream is, OutputStream os, boolean direction) {
super();
this.is = is;
this.os = os;
this.direction = direction;
}
#Override
public void run() {
System.out.println(Thread.currentThread().getId() + " start "
+ ((direction) ? "encode from to" : "decode from to"));
boolean eof = false;
while (true) {
if (direction) {
// encode to base64 data
try {
int[] head = new int[2];
for (int i = 0; i < 2; i++) {
head[i] = is.read();
}
int len = (head[0] & 0xff << 8 | head[1] & 0xff) & 0x7FFF;
byte[] buf = new byte[len + 2];
buf[0] = (byte) (head[0] & 0xff);
buf[1] = (byte) (head[1] & 0xff);
for (int i = 2; i < len; i++) {
buf[i] = (byte) (is.read() & 0xff);
}
System.out.println(Thread.currentThread()
.getId() + " << " + new String(buf));
if (len > 0) {
String send = Base64Util.encode(buf, len);
send += "\n";
os.write(send.getBytes());
os.flush();
}
} catch (IOException e) {
eof = true;
}
} else { // decode from base64
try {
StringBuilder sb = new StringBuilder(1024);
byte c = 0x0a;
do {
c = (byte) is.read();
if (c >= 0 && c != 0x0a) {
sb.append(new String(new byte[] { c }));
}
} while (c != 0x0a && c >= 0);
if (sb.length() != 0) {
try {
byte[] buf = Base64Util.decode(sb.toString());
System.out.println(Thread.currentThread()
.getId() + " >> " + buf.length);
os.write(buf);
os.flush();
} catch (StringIndexOutOfBoundsException e) {
System.out
.println(Thread.currentThread().getId()
+ " error on " + sb.toString());
}
}
} catch (IOException e) {
eof = true;
}
}
if (eof) {
System.out.println(Thread.currentThread().getId() + " EOF");
break;
}
}
try {
is.close();
os.close();
} catch (IOException e) {
}
}
}
public static void main(String[] args) throws Exception {
Process proc2 = Runtime.getRuntime().exec("nc -l -p 2020");
Process proc1 = Runtime.getRuntime().exec("nc 127.0.0.1 2020");
Socket sock1 = new Socket();
sock1.connect(new InetSocketAddress("127.0.0.1", 24554), 30);
Socket sock2 = new Socket();
sock2.connect(new InetSocketAddress("127.0.0.1", 24557), 30);
new Thread(new InToOut(sock1.getInputStream(), proc1.getOutputStream(),
true)).start();
new Thread(new InToOut(proc1.getInputStream(), sock1.getOutputStream(),
false)).start();
new Thread(new InToOut(sock2.getInputStream(), proc2.getOutputStream(),
true)).start();
new Thread(new InToOut(proc2.getInputStream(), sock2.getOutputStream(),
false)).start();
}
UPDATED:
I've found right way. I uses syncchronized queries for each stream and synchronized threads to fill or erase that queries. All threads mutually blocks themselves. And it works! :)
Sorry for bother.

I've found right way. I uses syncchronized queries for each stream and synchronized threads to fill or erase that queries. All threads mutually blocks themselves. And it works! :) Sorry for bother.

Speed up my Java tcp transfer!

I need to speed up transfers across my gigabit ethernet connection. Right now, I'm doing something almost exactly like this, but I'm only seeing about 40% of that when I run this code below.
I also ran this script on all of my (Mac Pro) machines before testing
#!/bin/bash
sudo sysctl -w net.inet.tcp.win_scale_factor=8
sudo sysctl -w kern.ipc.maxsockbuf=16777216
sudo sysctl -w net.inet.tcp.sendspace=8388608
sudo sysctl -w net.inet.tcp.recvspace=8388608
The actual code follows:
import java.io.*;
import java.nio.*;
import java.net.*;
public class BandwidthTester {
private static final int OUT_BUF = (1 << 17),
IN_BUF = (1 << 17), SEND_BUF = (1 << 22), RECV_BUF = (1 << 22);
public static void main(String[] args) {
try {
// server
if (args.length == 0) {
ServerSocket sock = new ServerSocket();
sock.bind(new InetSocketAddress(41887));
// wait for connection
Socket s = sock.accept();
s.setSendBufferSize(SEND_BUF);
System.out.println("Buffers: " + s.getSendBufferSize() + " and " + s.getReceiveBufferSize());
sock.close();
BufferedOutputStream bOut = new BufferedOutputStream(s.getOutputStream(), OUT_BUF);
// send lots of data
sendLotsOfData(bOut);
} else if (args.length == 2) {
String host = args[0];
int port = Integer.parseInt(args[1]);
System.out.println("Connecting to " + args[0] + ":" + args[1]);
Socket sock = new Socket();
sock.setReceiveBufferSize(RECV_BUF);
sock.connect(new InetSocketAddress(host, port));
System.out.println("Buffers: " + sock.getSendBufferSize() + " and " + sock.getReceiveBufferSize());
BufferedInputStream bIn = new BufferedInputStream(sock.getInputStream(), IN_BUF);
getLotsOfData(bIn);
}
} catch (Exception e) {
e.printStackTrace();
}
}
public static void getLotsOfData(InputStream in) {
System.out.println("Getting data...");
try {
long start = System.currentTimeMillis();
ByteBuffer intConv = ByteBuffer.allocate(4);
in.read(intConv.array());
int len = intConv.getInt(0);
for (int i=0; i < len; i++) {
in.read(intConv.array());
int val = intConv.getInt(0);
}
long end = System.currentTimeMillis();
double elapsed = ((double)(end - start)) / (1000.0);
System.out.println("Read in " + elapsed + " seconds: " + ( (4.0*8.0*len/elapsed) + " bits per second"));
} catch (Exception e) {
e.printStackTrace();
}
}
public static void sendLotsOfData(OutputStream out) {
System.out.println("Sending data...");
try {
long start = System.currentTimeMillis();
int len = (1 << 29);
ByteBuffer intConv = ByteBuffer.allocate(4);
intConv.putInt(0, len);
out.write(intConv.array());
for (int i=0; i < len; i++) {
intConv.putInt(0, i);
out.write(intConv.array());
}
out.flush();
long end = System.currentTimeMillis();
double elapsed = ((double)(end - start)) / (1000.0);
System.out.println("Sent in " + elapsed + " seconds: " + ( (4.0*8.0*len/elapsed) + " bits per second"));
}
catch (Exception e) {
e.printStackTrace();
}
}
}
Any suggestions? It's taking about 42 seconds to send all of that data, but even a 10% improvement here would have a dramatic impact on my program.

One thing you might try is using a larger buffer for the ByteBuffer. Going from 4 bytes to 16, I went from a 12 second transfer time to a 9 second transfer time. (tested using 2^26 rather then 2^29 for length)
That said, it was being run locally; so no actual network issues should have been encountered.
Somewhat dirty modified code for sending:
ByteBuffer intConv = ByteBuffer.allocate(16);
intConv.putInt(0, len);
out.write(intConv.array(),0,4);
for (int i=0; i < len; i+=4) {
for(int j=0; j<4; j++)
intConv.putInt(4*j, i);
out.write(intConv.array());
}
And Receiving:
ByteBuffer intConv = ByteBuffer.allocate(16);
in.read(intConv.array(),0,4);
int len = intConv.getInt(0);
for (int i=0; i < len; i+=4) {
in.read(intConv.array());
for(int j=0; j<4; j++)
{
int val=intConv.getInt(j*4);
}
}
Clearly the receiving end would need some modification to handle strange and odd cases like 'what if there were only 3 ints remaining/read from the stream', but I think this would be enough to see if it improves performance.