We Keep Coding

Deserialization of an image from string with unknown encoding

I have a set of serialized images which I want to deserialize. The data looks like the following:
GIF89ax\000\364\001\367\000\000\000\000\000\001\001\001\002\002\002\003\003\003\004\004\004\005\005\005\006\006\006\a\a\a\b\b\b\t\t\t\n\n\n\v\v\v\f\f\f\r\r\ ...
Due to data privacy I cann't post an full image. I have trouble to find out what kind of encoding this is. Now I'm struggeling to find a away to convert this to an BufferedImage in Java.

It turned out that the images were serialzed with Google-Protocoll Buffer TextFormat. So the code which converted the code correctly is
import com.google.protobuf.ByteString;
import com.google.protobuf.TextFormat;
String imgStr = "GIF89ax\000\364\001\367\000\000\000\000\000\001\001\001\002\002\002\003\003\003\004\004\004\005\005\005\006\006\006\a\a\a\b\b\b\t\t\t\n\n\n\v\v\v\f\f\f\r\r\ ..."; // must an full image string
ByteString unescapeBytes = TextFormat.unescapeBytes(imgStr);
byte[] bytes = new byte[unescapeBytes.size()];
unescapeBytes.copyTo(bytes, 0);
final BufferedOutputStream bw = new BufferedOutputStream(new FileOutputStream("result.gif"));
bw.write(bytes);
bw.close();
Thats it.

Store and Get byte[] array in Elasticsearch

I have a byte[] array. Suppose-
byte[] b = new byte[] {'11', '22', '33', '44'};
I want to store this array in Elasticsearch and get this byte array from elasticsearch by java. How can I do it? Please someone show me an example.
For example I want to see something like this-
SearchHit hit = searchResponse.getHits().getHits();
byte[] b = (byte[]) hit.getSource().get("byte_array");
But I am getting this error. Please someone help me
java.lang.ClassCastException: java.util.ArrayList cannot be cast to [B

How to convert byte array to buffered image

I have a server-side java code that gets a byte array from the client. In order to do some image processing, I need to convert the byte array into a BufferedImage. I have a code that's supposed to do that here:
public void processImage(byte[] data) {
ByteArrayInputStream stream = new ByteArrayInputStream(data);
BufferedImage bufferedImage;
bufferedImage = ImageIO.read(stream);
// bufferedImage is null
//...
}
But this doesn't work; bufferedImage is null. According to the ImageIO documentation:
If no registered ImageReader claims to be able to read the resulting stream, null is returned.
How do I tell the ImageReader what image type it is. For instance, if I know the image to be JPEG (which it is, in my case), what am I supposed to do?
EDIT: Thanks for the suggestion that the file is most likely not in JPEG format. This is the client-side code I have that sends the data as String over to the server:
import org.json.JSONObject;
// Client-side code that sends image to server as String
public void sendImage() {
FileInputStream inputStream = new FileInputStream(new File("myImage.jpg"));
ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
byte[] b = new byte[1024];
while ((bytesRead = inputStream.read(b)) != -1) {
byteStream.write(b,0,bytesRead);
}
byte[] byteArray = byteStream.toByteArray();
JSONObject jsonObject = new JSONObject();
jsonObject.put("data",new String(byteArray));
// ... more code here that sends jsonObject in HTTP post body
}
And this is the server-side code that calls the processImage() function:
// Server-side code that calls processImage() function
public void handleRequest(String jsonData) {
JSONObject jsonObject = new JSONObject(jsonData);
processImage(jsonObject.getString("data").getBytes());
}

The most likely explanation is that the byte array doesn't contain a JPEG image. (For instance, if you've just attempted to download it, you may have an HTML document giving an error diagnostic.) If that's the case, you'll need to find what is causing this and fix it.
However, if you "know" that the byte array contains an image with a given format, you could do something like this:
Use ImageIO.getImageReadersByFormatName or ImageIO.getImageReadersByMIMEType to get an Iterator<ImageReader>.
Pull the first ImageReader from the Iterator.
Create an MemoryCacheImageInputStream wrapping a ByteArrayInputStream for the types.
Use ImageReader.setInput to connect the reader to the ImageInputStream.
Use ImageReader.read to get the BufferedImage.

Unable to convert bytes from charset 65535 in to Japanese (5035)

I have a character set conversion issue:
I am updating Japanese Kanji characters in DB2 in iSeries system with the following conversion method:
AS400 sys = new AS400("<host>","username","password");
CharConverter charConv = new CharConverter(5035, sys);
byte[] b = charConv.stringToByteArray(5035, sys, "試験");
AS400Text textConverter = new AS400Text(b.length, 65535,sys);
While retrieving, I use the following code to convert & display:
CharConverter charConv = new CharConverter(5035, sys);
byte[] bytes = charConv.stringToByteArray(5035, sys, dbRemarks);
String s = new String(bytes);
System.out.println("Remarks after conversion to AS400Text :"+s);
But, the system is displaying garbled characters while displaying. Can anybody help me to decode Japanese characters from binary storage?

Well I don't know anything about CharConverter or AS400Text, but code like this is almost always a mistake:
String s = new String(bytes);
That uses the platform default encoding to convert the binary data to text.
Usually storage and retrieval should go through opposite processes - so while you've started with a string and then converted it to bytes, and converted that to an AS400Text object when storing it, I'd expect you to start with an AS400Text object, convert that to a byte array, and then convert that to a String using CharConverter when fetching. The fact that you're calling stringToByteArray in both cases suggests there's something amiss.
(It would also help if you'd tell us what dbRemarks is, and how you've fetched it.)
I do note that having checked some documentation for AS400Text, I've seen this:
Due to recent changes in the behavior of the character conversion routines, this system object is no longer necessary, except when the AS400Text object is to be passed as a parameter on a Toolbox Proxy connection.
There's similar documentation for CharConverter. Are you sure you actually need to go through this at all? Have you tried just storing the string directly and retrieving it directly, without going through intermediate steps?

Thank you Jon Skeet!
Yes. I have committed a mistake, not encoding the string while declaration.
My issue is to get the data stored in DB2, convert it into Japanese and provide for editing in web page. I am getting dbRemarks from the result set. I have missed another thing in my post:
While inserting, I am converting to text like:
String text = (String) textConverter.toObject(b);
PreparedStatement prepareStatementUpdate = connection.prepareStatement(updateSql);
prepareStatementUpdate.setString(1, text);
int count = prepareStatementUpdate.executeUpdate();
I am able to retrieve and display clearly with this code:
String selectSQL = "SELECT remarks FROM empTable WHERE emp_id = ? AND dep_id=? AND join_date='2013-11-15' ";
prepareStatement = connection.prepareStatement(selectSQL);
prepareStatement.setInt(1, 1);
prepareStatement.setString(2, 1);
ResultSet resultSet = prepareStatement.executeQuery();
while ( resultSet.next() ) {
byte[] bytedata = resultSet.getBytes( "remarks" );
AS400Text textConverter2 = new AS400Text(bytedata.length, 5035,sys);
String javaText = (String) textConverter2.toObject(bytedata);
System.out.println("Remarks after conversion to AS400Text :"+javaText);
}
It is working fine with JDBC, but for working with JPA, I need to convert to string for editing in web page or store in table. So, I have tried this way, but could not succeed:
String remarks = resultSet.getString( "remarks" );
byte[] bytedata = remarks.getBytes();
AS400Text textConverter2 = new AS400Text(bytedata.length, 5035,sys);
String javaText = (String) textConverter2.toObject(bytedata);
System.out.println("Remarks after conversion to AS400Text :"+javaText);

Thanks a lot Jon and Buck Calabro !
With your clues, I have succeeded with the following approach:
String remarks = new String(resultSet.getBytes("remarks"),"SJIS");
byte[] byteData = remarks.getBytes("SJIS");
CharConverter charConv = new CharConverter(5035, sys);
String convertedStr = charConv.byteArrayToString(5035, sys, byteData);
I am able to convert from string. I am planning to implement the same with JPA, and started coding.

Java - Image encoding in XML

I thought I would find a solution to this problem relatively easily, but here I am calling upon the help from ye gods to pull me out of this conundrum.
So, I've got an image and I want to store it in an XML document using Java. I have previously achieved this in VisualBasic by saving the image to a stream, converting the stream to an array, and then VB's xml class was able to encode the array as a base64 string. But, after a couple of hours of scouring the net for an equivalent solution in Java, I've come back empty handed. The only success I have had has been by:
import it.sauronsoftware.base64.*;
import java.awt.image.BufferedImage;
import org.w3c.dom.*;
...
BufferedImage img;
Element node;
...
java.io.ByteArrayOutputStream os = new java.io.ByteArrayOutputStream();
ImageIO.write(img, "png", os);
byte[] array = Base64.encode(os.toByteArray());
String ss = arrayToString(array, ",");
node.setTextContent(ss);
...
private static String arrayToString(byte[] a, String separator) {
StringBuffer result = new StringBuffer();
if (a.length > 0) {
result.append(a[0]);
for (int i=1; i<a.length; i++) {
result.append(separator);
result.append(a[i]);
}
}
return result.toString();
}
Which is okay I guess, but reversing the process to get it back to an image when I load the XML file has proved impossible. If anyone has a better way to encode/decode an image in an XML file, please step forward, even if it's just a link to another thread that would be fine.
Cheers in advance,
Hoopla.

I've done something similar (encoding and decoding in Base64) and it worked like a charm. Here's what I think you should do, using the class Base64 from the Apache Commons project:
// ENCODING
BufferedImage img = ImageIO.read(new File("image.png"));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ImageIO.write(img, "png", baos);
baos.flush();
String encodedImage = Base64.encodeToString(baos.toByteArray());
baos.close(); // should be inside a finally block
node.setTextContent(encodedImage); // store it inside node
// DECODING
String encodedImage = node.getTextContent();
byte[] bytes = Base64.decode(encodedImage);
BufferedImage image = ImageIO.read(new ByteArrayInputStream(bytes));
Hope it helps.

Apache Commons has a Base64 class that should be helpful to you:
From there, you can just write out the bytes (they are already in a readable format)

After you get your byte array
byte[] array = Base64.encode(os.toByteArray());
use an encoded String :
String encodedImg = new String( array, "utf-8");
Then you can do fun things in your xml like
<binImg string-encoding="utf-8" bin-encoding="base64" img-type="png"><![CDATA[ encodedIImg here ]]></binImg>

With Java 6, you can use DatatypeConverter to convert a byte array to a Base64 string:
byte[] imageData = ...
String base64String = DatatypeConverter.printBase64Binary(imageData);
And to convert it back:
String base64String = ...
byte[] imageData = DatatypeConverter.parseBase64Binary(base64String);

Your arrayToString() method is rather bizarre (what's the point of that separator?). Why not simply say
String s = new String(array, "US-ASCII");
The reverse operation is
byte[] array = s.getBytes("US-ASCII");
Use the ASCII encoding, which should be sufficient when dealing with Base64 encoded data. Also, I'd prefer a Base64 encoder from a reputable source like Apache Commons.

You don't need to invent your own XML data type for this. XML schema defines standard binary data types, such as base64Binary, which is exactly what you are trying to do.
Once you use the standard types, it can be converted into binary automatically by some parsers (like XMLBeans). If your parser doesn't handle it, you can find classes for base64Binary in many places since the datatype is widely used in SOAP, XMLSec etc.

most easy implementation I was able to made is as below, And this is from Server to Server XML transfer containing binary data Base64 is from the Apache Codec library:
- Reading binary data from DB and create XML
Blob blobData = oRs.getBlob("ClassByteCode");
byte[] bData = blobData.getBytes(1, (int)blobData.length());
bData = Base64.encodeBase64(bData);
String strClassByteCode = new String(bData,"US-ASCII");
on requesting server read the tag and save it in DB
byte[] bData = strClassByteCode.getBytes("US-ASCII");
bData = Base64.decodeBase64(bData);
oPrStmt.setBytes( ++nParam, bData );
easy as it can be..
I'm still working on implementing the streaming of the XML as it is generated from the first server where the XML is created and stream it to the response object, this is to take care when the XML with binary data is too large.
Vishesh Sahu

The basic problem is that you cannot have an arbitrary bytestream in an XML document, so you need to encode it somehow. A frequent encoding scheme is BASE64, but any will do as long as the recipient knows about it.

I know that the question was aking how to encode an image via XML, but it is also possible to just stream the bytes via an HTTP GET request instead of using XML and encoding an image. Note that input is a FileInputStream.
Server Code:
File f = new File(uri_string);
FileInputStream input = new FileInputStream(f);
OutputStream output = exchange.getResponseBody();
int c = 0;
while ((c = input.read()) != -1) {
output.write(c); //writes each byte to the exchange.getResponseBody();
}
result = new DownloadFileResult(int_list);
if (input != null) {input.close();}
if (output != null){ output.close();}
Client Code:
InputStream input = connection.getInputStream();
List<Integer> l = new ArrayList<>();
int b = 0;
while((b = input.read()) != -1){
l.add(b);//you can do what you wish with this list of ints ie- write them to a file. see code below.
}
Here is how you would write the Integer list to a file:
FileOutputStream out = new FileOutputStream("path/to/file.png");
for(int i : result_bytes_list){
out.write(i);
}
out.close();

node.setTextContent( base64.encodeAsString( fileBytes ) )
using org.apache.commons.codec.binary.Base64