I have a glitch in my program that causes a question mark (\u003f) to appear in the sixth (index = 5) slot in strings when I encrypt them. Normally, this is reversed upon decryption. However, it is not reversed if I save the string to a file first. I have determined that when I save a string containing Unicode characters to a file, I will not be able to determine the correct length for the file. I have managed to reproduce the glitch in the following function...
public static void testFileIO(String[] args)
{
System.out.println("TESTING FILE IO FUNCTIONS...");
try
{
String filename = "test.txt";
String testString = "UB\u4781ERBLAH\u037f\u8746";
System.out.println("Output: " + testString);
FileWriter fw = new FileWriter(filename);
fw.write(testString);
fw.close();
FileReader fr = new FileReader(filename);
int length;
for(length = 0; fr.read() != -1; length++);
if(length != testString.length())
System.out.println("Failure on file length measurement.");
fr.close();
fr = new FileReader(filename);
char[] buffer = new char[length];
fr.read(buffer);
String result = new String(buffer);
fr.close();
System.out.println("Input: " + result);
if(result.equals(testString)) System.out.println("SUCCESS!");
else System.out.println("FAILURE.");
}
catch (Throwable e)
{
e.printStackTrace();
System.out.println("FAILURE.");
return;
}
}
As an additional note, a failure in file length measurement is also caught.
Here is the Crypto class that I use to encrypt and decrypt Strings...
abstract public class Crypto
{
/**
* Encrypt the plaintext with a bitwise xor cipher
* #param plainText The plaintext to encrypt
* #param password The key for the bitwise xor cipher
* #return Ciphertext yielded by given plaintext and password
*/
public static String encrypt(String plainText, String key)
{
char[] data = plainText.toCharArray();
Random rand = new Random();
rand.setSeed(key.hashCode());
char[] pass = new char[data.length];
for(int i = 0; i < pass.length; i++)
{
pass[i] = (char)rand.nextInt();
}
for(int i = 0; i < data.length; i++)
{
data[i] ^= pass[i % pass.length];
}
return new String(data);
}
/**
* Decrypt an encrypted message using the same key as for encryption
* #param cipherText The cipherText message to be deciphered
* #param password The seed for the random generator to get the right keys
* #return The plaintext message corresponding to 'cipherText'
*/
public static String decrypt(String cipherText, String key)
{
char[] data = cipherText.toCharArray();
Random rand = new Random();
rand.setSeed(key.hashCode());
char[] pass = new char[data.length];// = key.getBytes("ASCII");
for(int i = 0; i < pass.length; i++)
{
pass[i] = (char)rand.nextInt();
}
for(int i = 0; i < data.length; i++)
{
data[i] ^= pass[i % pass.length];
}
return new String(data);
}
}
The code is correct but almost never works - As a rule of thumb, avoid FileReader and FileWriter and build your own readers/writers using InputStreamReader and OutputStreamWriter which allow you to specify the encoding to use (and hence how to protect 16bit Unicode characters when you write 8bit data).
I use a helper class for this because I need it all the time:
private static final String FILE = "file";
private static final String CHARSET = "charset";
public static BufferedReader createReader( File file, Encoding charset ) throws IOException {
JavaUtils.notNull( FILE, file );
JavaUtils.notNull( CHARSET, charset );
FileInputStream stream = null;
try {
stream = new FileInputStream( file );
return createReader( stream, charset );
} catch( IOException e ) {
IOUtils.closeQuietly( stream );
throw e;
} catch( RuntimeException e ) {
IOUtils.closeQuietly( stream );
throw e;
}
}
public static BufferedReader createReader( InputStream stream, Encoding charset ) throws IOException {
JavaUtils.notNull( "stream", stream );
JavaUtils.notNull( "charset", charset );
try {
return new BufferedReader( new InputStreamReader( stream, charset.encoding() ) );
} catch( UnsupportedEncodingException e ) {
IOUtils.closeQuietly( stream );
throw new UnknownEncodingException( charset, e );
} catch( RuntimeException e ) {
IOUtils.closeQuietly( stream );
throw e;
}
}
public static BufferedWriter createWriter( File file, Encoding charset ) throws IOException {
JavaUtils.notNull( FILE, file );
JavaUtils.notNull( CHARSET, charset );
FileOutputStream stream = null;
try {
stream = new FileOutputStream( file );
return new BufferedWriter( new OutputStreamWriter( stream, charset.encoding() ) );
} catch( UnsupportedEncodingException e ) {
IOUtils.closeQuietly( stream );
throw new UnknownEncodingException( charset, e );
} catch( IOException e ) {
IOUtils.closeQuietly( stream );
throw e;
} catch( RuntimeException e ) {
IOUtils.closeQuietly( stream );
throw e;
}
}
The type Encoding is an interface which I implement using one or more enums:
public interface Encoding {
String encoding();
Charset charset();
}
Related
Im trying to "pack" several files (previously inside a jar archive) in another single non-jar file by using DataInputStream / DataOutputStream.
The idea was:
First int = number of entries
First UTF is the first entry name
Second Int is entry byte array length (entry size)
Then repeat for every entry.
The code:
public static void main(String[] args) throws Throwable {
test();
System.out.println("========================================================================================");
final DataInputStream dataInputStream = new DataInputStream(new FileInputStream(new File("C:\\Users\\Admin\\Desktop\\randomJarOut")));
for (int int1 = dataInputStream.readInt(), i = 0; i < int1; ++i) {
final String utf = dataInputStream.readUTF();
System.out.println("Entry name: " + utf);
final byte[] array = new byte[dataInputStream.readInt()];
for (int j = 0; j < array.length; ++j) {
array[j] = dataInputStream.readByte();
}
System.out.println("Entry bytes length: " + array.length);
}
}
Unpacking original & packing to new one:
private static void test() throws Throwable {
JarInputStream stream = new JarInputStream(new FileInputStream(new File("C:\\Users\\Admin\\Desktop\\randomJar.jar")));
JarInputStream stream1 = new JarInputStream(new FileInputStream(new File("C:\\Users\\Admin\\Desktop\\randomJar.jar")));
final byte[] buffer = new byte[2048];
final DataOutputStream outputStream = new DataOutputStream(new FileOutputStream(new File("C:\\Users\\Admin\\Desktop\\randomJarOut")));
int entryCount = 0;
for (ZipEntry entry; (entry = stream.getNextJarEntry()) != null; ) {
entryCount++;
}
outputStream.writeInt(entryCount);
for (JarEntry entry; (entry = stream1.getNextJarEntry()) != null; ) {
int entryRealSize = stream1.read(buffer);
if (!(entryRealSize == -1)) {
System.out.println("Writing: " + entry.getName() + " Length: " + entryRealSize);
outputStream.writeUTF(entry.getName());
outputStream.writeInt(entryRealSize);
for (int len = stream1.read(buffer); len != -1; len = stream1.read(buffer)) {
outputStream.write(buffer, 0, len);
}
}
}
outputStream.flush();
outputStream.close();
}
Apparently im able to unpack the first entry without any problems, the second one and others:
Entry name: META-INF/services/org.jd.gui.spi.ContainerFactory
Entry bytes length: 434
Exception in thread "main" java.io.UTFDataFormatException: malformed input around byte 279
at java.io.DataInputStream.readUTF(DataInputStream.java:656)
at java.io.DataInputStream.readUTF(DataInputStream.java:564)
at it.princekin.esercizio.Bootstrap.main(Bootstrap.java:29)
Disconnected from the target VM, address: '127.0.0.1:54384', transport: 'socket'
Process finished with exit code 1
Does anyone knows how to fix this? Why is this working for the first entry but not the others?
My take on this is that the jar file (which in fact is a zip file) has a Central Directory which is only read with the ZipFile (or JarFile) class.
The Central Directory contains some data about the entries such as the size.
I think the ZipInputStream will not read the Central Directory and thus the ZipEntry will not contain the size (returning -1 as it is unknown) whereas reading ZipEntry from ZipFile class will.
So if you first read the size of each entry using a ZipFile and store that in a map, you can easily get it when reading the data with the ZipInputStream.
This page includes some good examples as well.
So my version of your code would be:
import java.io.*;
import java.util.HashMap;
import java.util.Map;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
import java.util.zip.ZipInputStream;
public class JarRepacker {
public static void main(String[] args) throws Throwable {
JarRepacker repacker = new JarRepacker();
repacker.repackJarToMyFileFormat("commons-cli-1.3.1.jar", "randomJarOut.bin");
repacker.readMyFileFormat("randomJarOut.bin");
}
private void repackJarToMyFileFormat(String inputJar, String outputFile) throws Throwable {
int entryCount;
Map<String, Integer> sizeMap = new HashMap<>();
try (ZipFile zipFile = new ZipFile(inputJar)) {
entryCount = zipFile.size();
zipFile.entries().asIterator().forEachRemaining(e -> sizeMap.put(e.getName(), (int) e.getSize()));
}
try (final DataOutputStream outputStream = new DataOutputStream(new FileOutputStream(outputFile))) {
outputStream.writeInt(entryCount);
try (ZipInputStream stream = new ZipInputStream(new BufferedInputStream(new FileInputStream(inputJar)))) {
ZipEntry entry;
final byte[] buffer = new byte[2048];
while ((entry = stream.getNextEntry()) != null) {
final String name = entry.getName();
outputStream.writeUTF(name);
final Integer size = sizeMap.get(name);
outputStream.writeInt(size);
//System.out.println("Writing: " + name + " Size: " + size);
int len;
while ((len = stream.read(buffer)) > 0) {
outputStream.write(buffer, 0, len);
}
}
}
outputStream.flush();
}
}
private void readMyFileFormat(String fileToRead) throws IOException {
try (DataInputStream dataInputStream
= new DataInputStream(new BufferedInputStream(new FileInputStream(fileToRead)))) {
int entries = dataInputStream.readInt();
System.out.println("Entries in file: " + entries);
for (int i = 1; i <= entries; i++) {
final String name = dataInputStream.readUTF();
final int size = dataInputStream.readInt();
System.out.printf("[%3d] Reading: %s of size: %d%n", i, name, size);
final byte[] array = new byte[size];
for (int j = 0; j < array.length; ++j) {
array[j] = dataInputStream.readByte();
}
// Still need to do something with this array...
}
}
}
}
The problem, probably, lies in that you are mixing not reciprocal read/write methods:
The writer method writes with outputStream.writeInt(entryCount) and the main method reads with dataInputStream.readInt(). That is OK.
The writer method writes with outputStream.writeUTF(entry.getName()) and the main method reads with dataInputStream.readUTF(). That is OK.
The writer method writes with outputStream.writeInt(entryRealSize) and the main method reads with dataInputStream.readInt(). That is OK.
The writer method writes with outputStream.write(buffer, 0, len) and the main method reads with dataInputStream.readByte() several times. WRONG.
If you write an array of bytes with write(buffer, offset, len), you must read it with read(buffer, offset, len), because write(buffer, offset, len) writes exactly len physical bytes onto the output stream, while writeByte (the counterpart of readByte) writes a lot of metadata overhead about the object type, and then its state variables.
Bugs in the writer method
There is also a mayor bug in the writer method: It invokes up to three times stream1.read(buffer), but it just uses once the buffer contents. The result is that the real size of file is actually written onto the output stream metadata, but it is followed by just a small part of the data.
If you need to know the input file size before writing it in the output stream, you have two choices:
Either chose a large enough buffer size (like 204800) which will allow you to read the whole file in just one read and write it in just one write.
Or either separate read from write algorithms: First a method to read the whole file and store it in memory (a byte[], for example), and then another method to write the byte[] onto the output stream.
Full fixed solution
I've fixed your program, with specific, decoupled methods for each task. The process consists in parsing the input file to a memory model, write it to an intermediate file according to your custom definition, and then read it back.
public static void main(String[] args)
throws Throwable
{
File inputJarFile=new File(args[0]);
File intermediateFile=new File(args[1]);
List<FileData> fileDataEntries=parse(inputJarFile);
write(fileDataEntries, intermediateFile);
read(intermediateFile);
}
public static List<FileData> parse(File inputJarFile)
throws IOException
{
List<FileData> list=new ArrayList<>();
try (JarInputStream stream=new JarInputStream(new FileInputStream(inputJarFile)))
{
for (ZipEntry entry; (entry=stream.getNextJarEntry()) != null;)
{
byte[] data=readAllBytes(stream);
if (data.length > 0)
{
list.add(new FileData(entry.getName(), data));
}
stream.closeEntry();
}
}
return list;
}
public static void write(List<FileData> fileDataEntries, File output)
throws Throwable
{
try (DataOutputStream outputStream=new DataOutputStream(new FileOutputStream(output)))
{
int entryCount=fileDataEntries.size();
outputStream.writeInt(entryCount);
for (FileData fileData : fileDataEntries)
{
int entryRealSize=fileData.getData().length;
{
System.out.println("Writing: " + fileData.getName() + " Length: " + entryRealSize);
outputStream.writeUTF(fileData.getName());
outputStream.writeInt(entryRealSize);
outputStream.write(fileData.getData());
}
}
outputStream.flush();
}
}
public static void read(File intermediateFile)
throws IOException
{
try (DataInputStream dataInputStream=new DataInputStream(new FileInputStream(intermediateFile)))
{
for (int entryCount=dataInputStream.readInt(), i=0; i < entryCount; i++)
{
String utf=dataInputStream.readUTF();
int entrySize=dataInputStream.readInt();
System.out.println("Entry name: " + utf + " size: " + entrySize);
byte[] data=readFixedLengthBuffer(dataInputStream, entrySize);
System.out.println("Entry bytes length: " + data.length);
}
}
}
private static byte[] readAllBytes(InputStream input)
throws IOException
{
byte[] buffer=new byte[4096];
byte[] total=new byte[0];
int len;
do
{
len=input.read(buffer);
if (len > 0)
{
byte[] total0=total;
total=new byte[total0.length + len];
System.arraycopy(total0, 0, total, 0, total0.length);
System.arraycopy(buffer, 0, total, total0.length, len);
}
}
while (len >= 0);
return total;
}
private static byte[] readFixedLengthBuffer(InputStream input, int size)
throws IOException
{
byte[] buffer=new byte[size];
int pos=0;
int len;
do
{
len=input.read(buffer, pos, size - pos);
if (len > 0)
{
pos+=len;
}
}
while (pos < size);
return buffer;
}
private static class FileData
{
private final String name;
private final byte[] data;
public FileData(String name, byte[] data)
{
super();
this.name=name;
this.data=data;
}
public String getName()
{
return this.name;
}
public byte[] getData()
{
return this.data;
}
}
am trying to parse binary CDRs using JASN1
I have successfully generated Java classes using grammer file
not I have a CDR which I need to decode, but I can't get it to work, I don't understand what kind of inputs it requires
I have reached a point where I can parse CDR into lines like below
[1][[0]#01, [1]#26fd, [3]#4131002400, [8]#14040020236233, [9]#21436500000041, [10]#196105000045ffffffffffff, [13]#13900049999957, [14]#21436549999961, [15]#05, [16]#05, [17]#116102999954ffffffffffff, [22]#00a2, [23]#0001, [37]#0010, [38]#03, [40]#0324, [46]#06, [47]#05, [54]#00580720111220, [85]#04f4]
Java code
public class JASN1 {
public static void main(String[] args) throws IOException {
// TODO Auto-generated method stub
ByteArrayInputStream bais = new ByteArrayInputStream(readContentIntoByteArray(new File("sample.asn")));
ASN1InputStream ais = new ASN1InputStream(new FileInputStream(new File("sample.asn")));
while (ais.available() > 0) {
DERTaggedObject primitive = (DERTaggedObject) ais.readObject();
System.out.println(primitive.toASN1Object());
String encoded = toHexadecimal(new String(primitive.getEncoded()));
bais = new ByteArrayInputStream(encoded.getBytes());
MobileSampleMsg mobileSampleMsg = new MobileSampleMsg();
mobileSampleMsg.decode(bais, true);
System.out.println("MobileSampleMsg = " + mobileSampleMsg);
}
ais.close();
/*
* System.out.println(bais); MobileSampleMsg personnelRecord_decoded =
* new MobileSampleMsg(); personnelRecord_decoded.decode(bais, true);
*
* System.out.println("");
* System.out.println("PersonnelRecord.name.givenName = " +
* personnelRecord_decoded);
*/
}
private static byte[] readContentIntoByteArray(File file) {
FileInputStream fileInputStream = null;
byte[] bFile = new byte[(int) file.length()];
try {
// convert file into array of bytes
fileInputStream = new FileInputStream(file);
fileInputStream.read(bFile);
fileInputStream.close();
} catch (Exception e) {
e.printStackTrace();
}
return bFile;
}
public static String toHexadecimal(String text) throws UnsupportedEncodingException {
byte[] myBytes = text.getBytes("UTF-8");
return DatatypeConverter.printHexBinary(myBytes);
}
}
download samples from here
download grammer from here
I was able to encode/decode the file with the help of JASN1 and BouncyCastle, I used JASN1 to compile grammer file into java classes then used BouncyCastle to decode/encode new objects, below is code snippet used to show how I did that
public class BouncyCastle {
public static void main(String[] args) throws IOException {
DetailOutputRecord detailOutputRecord = new DetailOutputRecord();
MyRecord myRecord = new MyRecord();
myRecord.setOriginNodeType(new NodeType("xxxx".getBytes()));
myRecord.setTransactionAmount(new MoneyAmount("xxxx".getBytes()));
myRecord.setSubscriberNumber(new NumberString("xxxx".getBytes()));
ReverseByteArrayOutputStream ros = new ReverseByteArrayOutputStream(1000);
detailOutputRecord.setMyRecord(myRecord);
myRecord.encode(ros);
System.out.println(DatatypeConverter.printHexBinary(ros.getArray()));
System.out.println(print(ros.getArray()));
DERTaggedObject dermyRecord = new DERTaggedObject(false, 6, ASN1Primitive.fromByteArray(ros.getArray()));
File f = new File(String.valueOf("1_dermyRecord.ASN"));
FileOutputStream stream = new FileOutputStream(f);
try {
stream.write(dermyRecord.getEncoded());
} finally {
stream.close();
}
ros = new ReverseByteArrayOutputStream(1000);
detailOutputRecord.encode(ros);
DLSequence ddetailOutputRecord = new DLSequence(ASN1Primitive.fromByteArray(ros.getArray()));
stream = new FileOutputStream(new File("detailOutputRecord.ASN"));
try {
stream.write(ros.buffer);
} finally {
stream.close();
}
}
public static String print(byte[] bytes) {
StringBuilder sb = new StringBuilder();
sb.append("[ ");
for (byte b : bytes) {
// sb.append(String.format("0x%02X ", b));
sb.append(String.format("\\x%02X", b));
}
sb.append("]");
return sb.toString();
}
private static DERTaggedObject toDERObject(byte[] data) throws IOException {
ByteArrayInputStream inStream = new ByteArrayInputStream(data);
ASN1InputStream asnInputStream = new ASN1InputStream(inStream);
return (DERTaggedObject) asnInputStream.readObject();
}
}
Here is alternative way of processing given binary record files using org.bouncycastle.asn1 package only :
/*Sequence Tags */
static final int MSCRecordType = 0;
static final int RecordNumber = 1;
static final int RecordStatus = 3;
static final int CallingImsi = 8;
static final int CallingImei = 9;
void process() throws IOException, ClassNotFoundException {
String path = System.getProperty("user.dir") + "/telconetqual_sampledata/CDR_RGN0_20121120081859.asn";
File file = new File(path);
byte[] bFile = new byte[(int) file.length()];
FileInputStream fis = new FileInputStream(file);
fis.read(bFile);
ASN1InputStream in = new ASN1InputStream(bFile);
while (in.available() > 0) {
ASN1Primitive primitive = in.readObject();
DERTaggedObject derTaggedObject = (DERTaggedObject) primitive;
DLSequence sequence = (DLSequence)derTaggedObject.getObject();
for(int i =0; i < sequence.size(); i++){
DERTaggedObject seqElement = (DERTaggedObject)sequence.getObjectAt(i);
switch (seqElement.getTagNo()) {
case MSCRecordType:
DEROctetString recordTypeOctet =(DEROctetString)seqElement.getObject();
int recordType = Integer.valueOf(Hex.toHexString(recordTypeOctet.getOctets()));
break;
case CallingImsi:
DEROctetString CallingImsiOctet =(DEROctetString)seqElement.getObject();
String CallingImsi = Hex.toHexString(CallingImsiOctet.getOctets());
...
}
}
}
}
I want to split data based on character values which are two right parenthesis )) as start of substring and carriage return CR as the end of substring. The data comes in form of bytes Am stuck on how to split it. This is so far what I have come up with.
public class ByteDecoder {
public static void main(String[] args) throws IOException {
InputStream is = null;
DataInputStream dis = null;
try{
is = new FileInputStream("byte.log");
dis = new DataInputStream(is);
int count = is.available();
byte[] bs = new byte[count];
dis.read(bs);
for (byte b:bs)
{
char c = (char)b;
System.out.println(c);
//convert bytes to hex string
// String c = DatatypeConverter.printHexBinary( bs);
}
}catch(Exception e){
e.printStackTrace();
}finally{
if(is!=null)
is.close();
if(dis!=null)
dis.close();
}
}
}
CR (unlucky 13) as end marker of binary data might be a bit dangerous. More dangerous seems how the text and bytes became written: the text must be written as bytes in some encoding.
But considering that, one could wrap the FileInputStream in your own ByteLogInputStream, and there hold the reading state:
/**
* An InputStream converting bytes between ASCII "))" and CR to hexadecimal.
* Typically wrapped as:
* <pre>
* try (BufferedReader in = new BufferedReader(
* new InputStreamReader(
* new ByteLogInputStream(
* new FileInputStream(file), "UTF-8"))) {
* ...
* }
* </pre>
*/
public class ByteLogInputStream extends InputStream {
private enum State {
TEXT,
AFTER_RIGHT_PARENT,
BINARY
}
private final InputStream in;
private State state = State.TEXT;
private int nextHexDigit = 0;
public ByteLogInputStream(InputStream in) {
this.in = in;
}
#Override
public int read() throws IOException {
if (nextHexDigit != 0) {
int hex = nextHexDigit;
nextHexDigit = 0;
return hex;
}
int ch = in.read();
if (ch != -1) {
switch (state) {
case TEXT:
if (ch == ')') {
state = State.AFTER_RIGHT_PARENT;
}
break;
case AFTER_RIGHT_PARENT:
if (ch == ')') {
state = State.BINARY;
}
break;
case BINARY:
if (ch == '\r') {
state = State.TEXT;
} else {
String hex2 = String.format("%02X", ch);
ch = hex2.charAt(0);
nextHexDigit = hex2.charAt(1);
}
break;
}
}
return ch;
}
}
As one binary byte results in two hexadecimal digits, you need to buffer a nextHexDigit for the next digit.
I did not override available (to account for a possible nextHexDigit).
If you want to check whether \r\n follows, one should use a PushBackReader. I did use an InputStream, as you did not specify the encoding.
I searched for an example of how to compress a string in Java.
I have a function to compress then uncompress. The compress seems to work fine:
public static String encStage1(String str)
{
String format1 = "ISO-8859-1";
String format2 = "UTF-8";
if (str == null || str.length() == 0)
{
return str;
}
System.out.println("String length : " + str.length());
ByteArrayOutputStream out = new ByteArrayOutputStream();
String outStr = null;
try
{
GZIPOutputStream gzip = new GZIPOutputStream(out);
gzip.write(str.getBytes());
gzip.close();
outStr = out.toString(format2);
System.out.println("Output String lenght : " + outStr.length());
} catch (Exception e)
{
e.printStackTrace();
}
return outStr;
}
But the reverse is complaining about the string not being in GZIP format, even when I pass the return from encStage1 straight back into the decStage3:
public static String decStage3(String str)
{
if (str == null || str.length() == 0)
{
return str;
}
System.out.println("Input String length : " + str.length());
String outStr = "";
try
{
String format1 = "ISO-8859-1";
String format2 = "UTF-8";
GZIPInputStream gis = new GZIPInputStream(new ByteArrayInputStream(str.getBytes(format2)));
BufferedReader bf = new BufferedReader(new InputStreamReader(gis, format2));
String line;
while ((line = bf.readLine()) != null)
{
outStr += line;
}
System.out.println("Output String lenght : " + outStr.length());
} catch (Exception e)
{
e.printStackTrace();
}
return outStr;
}
I get this error when I call with a string return from encStage1:
public String encIDData(String idData)
{
String tst = "A simple test string";
System.out.println("Enc 0: " + tst);
String stg1 = encStage1(tst);
System.out.println("Enc 1: " + toHex(stg1));
String dec1 = decStage3(stg1);
System.out.println("unzip: " + toHex(dec1));
}
Output/Error:
Enc 0: A simple test string
String length : 20
Output String lenght : 40
Enc 1: 1fefbfbd0800000000000000735428efbfbdefbfbd2defbfbd495528492d2e51282e29efbfbdefbfbd4b07005aefbfbd21efbfbd14000000
Input String length : 40
java.io.IOException: Not in GZIP format
at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:137)
at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:58)
at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:68)
A small error is:
gzip.write(str.getBytes());
takes the default platform encoding, which on Windows will never be ISO-8859-1. Better:
gzip.write(str.getBytes(format1));
You could consider taking "Cp1252", Windows Latin-1 (for some European languages), instead of "ISO-8859-1", Latin-1. That adds comma like quotes and such.
The major error is converting the compressed bytes to a String. Java separates binary data (byte[], InputStream, OutputStream) from text (String, char, Reader, Writer) which internally is always kept in Unicode. A byte sequence does not need to be valid UTF-8. You might get away by converting the bytes as a single byte encoding (ISO-8859-1 for instance).
The best way would be
gzip.write(str.getBytes(StandardCharsets.UTF_8));
So you have full Unicode, every script may be combined.
And uncompressing to a ByteArrayOutputStream and new String(baos.toByteArray(), StandardCharsets.UTF_8).
Using BufferedReader on an InputStreamReader with UTF-8 is okay too, but a readLine throws away the newline characters
outStr += line + "\r\n"; // Or so.
Clean answer:
public static byte[] encStage1(String str) throws IOException
{
try (ByteArrayOutputStream out = new ByteArrayOutputStream())
{
try (GZIPOutputStream gzip = new GZIPOutputStream(out))
{
gzip.write(str.getBytes(StandardCharsets.UTF_8));
}
return out.toByteArray();
//return out.toString(StandardCharsets.ISO_8859_1);
// Some single byte encoding
}
}
public static String decStage3(byte[] str) throws IOException
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (GZIPInputStream gis = new GZIPInputStream(new ByteArrayInputStream(str)))
{
int b;
while ((b = gis.read()) != -1) {
baos.write((byte) b);
}
}
return new String(baos.toByteArray(), StandardCharset.UTF_8);
}
usage of toString/getBytes for encoding/decoding is a wrong way. try to use something like BASE64 encoding for this purpose (java.util.Base64 in jdk 1.8)
as a proof try this simple test:
import org.testng.annotations.Test;
import java.io.ByteArrayOutputStream;
import static org.testng.Assert.assertEquals;
public class SimpleTest {
#Test
public void test() throws Exception {
final String CS = "utf-8";
byte[] b0 = {(byte) 0xff};
ByteArrayOutputStream out = new ByteArrayOutputStream();
out.write(b0);
out.close();
byte[] b1 = out.toString(CS).getBytes(CS);
assertEquals(b0, b1);
}
}
When I open a websocket connection to my websocket server application from Java, the server sees two connections. The first one never sends any data and the second one sends all the proper headers, etc. Anyone know what the reason for this is?
Client side connection is:
var websocket = new WebSocket( "ws://192.168.1.19:3333/websession" );
On the server side, in a while loop I call "serverSocket.accept()" and this gets called twice. But one of them never sends any data (the in.read() simply times out eventually without returning anything).
JAVA SERVER CODE
import java.net.*;
import java.io.*;
import java.security.*;
public class WebListener {
public static void main(String[] args) throws Exception {
ServerSocket serverSocket = null;
boolean listening = true;
try {
serverSocket = new ServerSocket(4444);
} catch (IOException e) {
System.err.println("Could not listen on port: 4444.");
System.exit(-1);
}
while (listening) new ServerThread(serverSocket.accept()).start();
serverSocket.close();
}
}
class ServerThread extends Thread {
private Socket socket = null;
public ServerThread(Socket socket) {
super("ServerThread");
this.socket = socket;
}
public void run() {
try {
OutputStream outStream = null;
PrintWriter out = new PrintWriter( outStream = socket.getOutputStream(), true);
BufferedReader in = new BufferedReader( new InputStreamReader( socket.getInputStream()));
String inputLine, outputLine;
//Handle the headers first
doHeaders( out, in );
// ..elided..
out.close();
in.close();
socket.close();
} catch (Exception e) {
e.printStackTrace();
}
}
public void doHeaders(PrintWriter out, BufferedReader in) throws Exception {
String inputLine = null;
String key = null;
//Read the headers
while ( ( inputLine = in.readLine() ) != null ) {
//Get the key
if ( inputLine.startsWith( "Sec-WebSocket-Key" ) )
key = inputLine.substring( "Sec-WebSocket-Key: ".length() );
//They're done
if ( inputLine.equals( "" ) ) break;
}
//We need a key to continue
if ( key == null ) throw new Exception( "No Sec-WebSocket-Key was passed!" );
//Send our headers
out.println( "HTTP/1.1 101 Web Socket Protocol Handshake\r" );
out.println( "Upgrade: websocket\r" );
out.println( "Connection: Upgrade\r" );
out.println( "Sec-WebSocket-Accept: " + createOK( key ) + "\r" );
out.println( "\r" );
}
public String createOK(String key) throws NoSuchAlgorithmException, UnsupportedEncodingException, Exception {
String uid = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11";
String text = key + uid;
MessageDigest md = MessageDigest.getInstance( "SHA-1" );
byte[] sha1hash = new byte[40];
md.update( text.getBytes("iso-8859-1"), 0, text.length());
sha1hash = md.digest();
return new String( base64( sha1hash ) );
}
public byte[] base64(byte[] bytes) throws Exception {
ByteArrayOutputStream out_bytes = new ByteArrayOutputStream();
OutputStream out = new Base64.OutputStream(out_bytes); //Using http://iharder.net/base64
out.write(bytes);
out.close();
return out_bytes.toByteArray();
}
private String convertToHex(byte[] data) {
StringBuffer buf = new StringBuffer();
for (int i = 0; i < data.length; i++) {
int halfbyte = (data[i] >>> 4) & 0x0F;
int two_halfs = 0;
do {
if ((0 <= halfbyte) && (halfbyte <= 9))
buf.append((char) ('0' + halfbyte));
else
buf.append((char) ('a' + (halfbyte - 10)));
halfbyte = data[i] & 0x0F;
} while(two_halfs++ < 1);
}
return buf.toString();
}
}
This looks to be a bug with Firefox. In Chrome it only opens one connection, while the same page in Firefox 15 opens two connections.