We Keep Coding

Java SSL/TLS Handshake

im writing a small HTTP/S Server for an Embeded Device Framework.
Because i use plain Java Sockets i need to implement the TLS/SSL myself.
For the Client Handshake i wrote following Function:
if (secureIO) {
// Print Message if Client Handshake was correct.
((SSLSocket) clientIO)
.addHandshakeCompletedListener(new HandshakeCompletedListener() {
#Override
public void handshakeCompleted(HandshakeCompletedEvent eventIO) {
logIO.debug(GuardianLog.Type.SUCCESS,
"Handshake with Client "
+ clientIO.getInetAddress().getHostAddress() + " via Method "
+ eventIO + " was correct.");
// Unlock Stream Lock.
lockIO[0] = false;
}
});
// ((SSLSocket) clientIO).setNeedClientAuth(true);
// Session Creation Blocks everything...
((SSLSocket) clientIO).setEnableSessionCreation(true);
// Set Cipher and Protocols for Client.
((SSLSocket) clientIO)
.setEnabledProtocols(((SSLServerSocket) tcpIO).getEnabledProtocols());
((SSLSocket) clientIO)
.setEnabledCipherSuites(
((SSLServerSocket) tcpIO).getEnabledCipherSuites());
// Start Handshake to Client.
((SSLSocket) clientIO).startHandshake();
}
The Problem is, when i opening the Socket Port in the Browser, the Page loads forever.
So i tried to debug the SSLSocket with OPENSSL it seems that the Handshake gets not fully finished.
I thought some Stream gets confused by an Listener on my Server, but i disabled the Input/Output Stream entirely with the lock bool, until the Handshake Complete Listener gets executed.
Some outstanding Debug Traces from the Server have given me this theory.
javax.net.ssl|ALL|1C|Server-Socket|2022-07-08 21:51:57.696 CEST|SSLSessionImpl.java:250|Session initialized: Session(1657309917387|TLS_AES_256_GCM_SHA384)
javax.net.ssl|FINE|1C|Server-Socket|2022-07-08 21:51:57.696 CEST|SSLSocketOutputRecord.java:241|WRITE: TLS13 handshake, length = 50
javax.net.ssl|FINE|1C|Server-Socket|2022-07-08 21:51:57.697 CEST|SSLCipher.java:2013|Plaintext before ENCRYPTION (
0000: 04 00 00 2E 00 01 51 80 F9 1F 1E 31 01 01 00 20 ......Q....1...
0010: E5 5B CF E4 29 3B 0E 9F E4 12 D7 CD 8B 34 2C 22 .[..);.......4,"
0020: 0B 14 45 B3 E9 94 CC 62 64 C3 06 E4 4F 72 82 D3 ..E....bd...Or..
0030: 00 00 16 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0040: 00 00 00 ...
)
javax.net.ssl|FINE|1C|Server-Socket|2022-07-08 21:51:57.697 CEST|SSLSocketOutputRecord.java:255|Raw write (
0000: 17 03 03 00 53 6B D3 EE EF C5 DD D2 E3 3F DF 15 ....Sk.......?..
0010: 13 87 A5 9E BB AC 0A 1F 1F 6A 77 50 B1 4D 51 39 .........jwP.MQ9
0020: 4D 0C 00 78 D1 4A D8 78 79 9A 79 7E AD EB 20 80 M..x.J.xy.y... .
0030: 91 C3 A5 14 33 FF 13 01 9B D6 37 7E 3A 7B 9F 7D ....3.....7.:...
0040: 03 E6 59 90 FF 9B E3 63 87 18 FD 84 37 C7 21 CA ..Y....c....7.!.
0050: A7 AC ED 25 C3 05 73 A8 ...%..s.
)
Heres an example of my Read Function, the InputStream is setet after the Socket.accpet() Method:
private void read(InputStream streamIO, java.net.Socket clientIO) {
// todo: Test Performance between execute and submit.
poolIO.execute(new Runnable() {
#Override
public void run() {
try {
// clientIO.isConnected() && !tcpIO.isClosed() &&
while (!shutdownIO && clientIO.isConnected() && !tcpIO.isClosed()) {
// Byte Buffer ho contains read Data.
// #todo: add buffer max size, read until -1 (EOF).
byte[] bufferIO = new byte[streamIO.available()];
// Read Data into Buffer and store (EOF).
int codeIO = streamIO.read(bufferIO);
if (codeIO != 0 || bufferIO.length != 0) {
System.out.println(codeIO);
System.out.println(bufferIO.length);
}
// Check if bytes in the Buffer.
if (bufferIO.length != 0 && streamIO.available() == 0) {
if (binaryIO)
binary(new Socket(clientIO), bufferIO);
if (stringIO)
string(new Socket(clientIO), new String(bufferIO, charsetIO));
}
if (eofIO && (codeIO == -1)) {
break;
}
// Sleep for 100 Milliseconds (CPU Usage)
Thread.sleep(100);
}
streamIO.reset();
streamIO.close();
clientIO.close();
disconnect(new Socket(clientIO));
// Clear Buffer Stream.
// bufferIO = new byte[0];
} catch (IOException | InterruptedException errorIO) {
if (errorIO.getMessage().equalsIgnoreCase("Stream closed.")
|| errorIO.getMessage().equalsIgnoreCase("Connection reset")) {
logIO.debug(GuardianLog.Type.INFO,
"Client with IP " + clientIO.getInetAddress().getHostAddress()
+ " disconnected from Server with Port " + networkIO.getPort()
+ ".");
disconnect(new Socket(clientIO));
} else if (errorIO.getMessage().equalsIgnoreCase("sleep interrupted")) {
logIO.debug(
GuardianLog.Type.WARNING, "Socket Thread interrupted.", errorIO);
} else
logIO.append(
GuardianLog.Type.ERROR, "Socket throw an Error ", errorIO);
}
}
});
}

Extract .gz files in java

I'm trying to unzip some .gz files in java. After some researches i wrote this method:
public static void gunzipIt(String name){
byte[] buffer = new byte[1024];
try{
GZIPInputStream gzis = new GZIPInputStream(new FileInputStream("/var/www/html/grepobot/API/"+ name + ".txt.gz"));
FileOutputStream out = new FileOutputStream("/var/www/html/grepobot/API/"+ name + ".txt");
int len;
while ((len = gzis.read(buffer)) > 0) {
out.write(buffer, 0, len);
}
gzis.close();
out.close();
System.out.println("Extracted " + name);
} catch(IOException ex){
ex.printStackTrace();
}
}
when i try to execute it i get this error:
java.util.zip.ZipException: Not in GZIP format
how can i solve it? Thanks in advance for your help

Test a sample, correct, gzipped file to see whether the problem lies in your code or not.
There are many possible ways to build a (g)zip file. Your file may have been built differently from what Java's built-in support expects, and the fact that one uncompressor understands a compression variant is no guarantee that Java will also recognize that variant. Please verify exact file type with file and/or other uncompression utilities that can tell you which options were used when compressing it. You may also have a look at the file itself with a tool such as hexdump. This is the output of the following command:
$ hexdump -C lgpl-2.1.txt.gz | head
00000000 1f 8b 08 08 ed 4f a9 4b 00 03 6c 67 70 6c 2d 32 |.....O.K..lgpl-2|
00000010 2e 31 2e 74 78 74 00 a5 5d 6d 73 1b 37 92 fe 8e |.1.txt..]ms.7...|
00000020 ba 1f 81 d3 97 48 55 34 13 7b 77 73 97 78 2b 55 |.....HU4.{ws.x+U|
00000030 b4 44 d9 bc 95 25 2d 29 c5 eb ba ba aa 1b 92 20 |.D...%-)....... |
00000040 39 f1 70 86 99 17 29 bc 5f 7f fd 74 37 30 98 21 |9.p...)._..t70.!|
00000050 29 7b ef 52 9b da 58 c2 00 8d 46 bf 3c fd 02 d8 |){.R..X...F.<...|
00000060 da fe 3f ef 6f 1f ed cd 78 36 1b 4f ed fb f1 ed |..?.o...x6.O....|
00000070 78 3a ba b1 f7 8f ef 6e 26 97 96 fe 1d df ce c6 |x:.....n&.......|
00000080 e6 e0 13 f9 e7 57 57 56 69 91 db 37 c3 d7 03 7b |.....WWVi..7...{|
00000090 ed e6 65 93 94 7b fb fa a7 9f 7e 32 c6 5e 16 bb |..e..{....~2.^..|
In this case, I used standard gzip on this license text. The 1st few bytes are unique to GZipped files (although they do not specify variants) - if your file does not start with 1f 8b, Java will complain, regardless of remaining contents.
If the problem is due to the file, it is possible that other uncompression libraries available in Java may deal with the format correctly - for example, see Commons Compress

import com.horsefly.utils.GZIP;
import org.apache.commons.io.FileUtils;
....
String content = new String(new GZIP().decompresGzipToBytes(FileUtils.readFileToByteArray(fileName)), "UTF-8");
in case someone needs it.

Java serialization with empty and substrings

Had a look at the implementation and haven't been able to think of an explanation to this but maybe someone here will know.
public static void main(String[] args) throws Exception {
List<String> emptyStrings = new ArrayList<String>();
List<String> emptySubStrings = new ArrayList<String>();
for (int i = 0; i < 20000; i++) {
String actuallyEmpty = "";
String subStringedEmpty = " ";
subStringedEmpty = subStringedEmpty.substring(0, 0);
emptyStrings.add(actuallyEmpty);
emptySubStrings.add(subStringedEmpty);
}
System.out.println("Substring test");
// Write to files
long time = System.currentTimeMillis();
writeObjectToFile(emptyStrings, "empty.list");
System.out.println("Time taken to write empty list " + (System.currentTimeMillis() - time));
time = System.currentTimeMillis();
writeObjectToFile(emptySubStrings, "substring.list");
System.out.println("Time taken to write substring list " + (System.currentTimeMillis() - time));
//Read from files
time = System.currentTimeMillis();
List<String> readEmptyString = readObjectFromFile("empty.list");
System.out.println("Time taken to read empty list " + (System.currentTimeMillis() - time));
time = System.currentTimeMillis();
List<String> readEmptySubStrings = readObjectFromFile("substring.list");
System.out.println("Time taken to read substring list " + (System.currentTimeMillis() - time));
}
private static void writeObjectToFile(Object o, String file) throws Exception {
FileOutputStream out = new FileOutputStream(file);
ObjectOutputStream oout = new ObjectOutputStream(out);
oout.writeObject(o);
oout.flush();
oout.close();
}
private static <T> T readObjectFromFile(String file) throws Exception {
ObjectInputStream ois = null;
try {
ois = new ObjectInputStream(new FileInputStream(file));
return (T) ois.readObject();
} finally {
ois.close();
}
}
Ultimately these 2 lists contain 20,000 empty strings (one list contains "" empty strings and the other contains empty strings generated by substring(0,0)). But if you check the sizes of the serialized files generated (empty.list and substring.list) you will notice that the empty.list contains substantially more data.
I have noticed that the callers of remote EJB's which un-serialize these substring objects seem to have severe performance issues also.

The sizes of the lists are different because java uses a mechanism to store multiples references to the same object, like described:
References to other objects (except in transient or static fields)
cause those objects to be written also. Multiple references to a
single object are encoded using a reference sharing mechanism so that
graphs of objects can be restored to the same shape as when the
original was written.
see ObjectOutputStream
If you look the generated serialized file, you will see:
With 1 String empty inside:
empty.list:
ac ed 00 05 73 72 00 13 6a 61 76 61 2e 75 74 69
6c 2e 41 72 72 61 79 4c 69 73 74 78 81 d2 1d 99
c7 61 9d 03 00 01 49 00 04 73 69 7a 65 78 70 00
00 00 01 77 04 00 00 00 01 74 00 00 78
The string "" corresponds to the last three bytes (00 00 78)
substring.list
ac ed 00 05 73 72 00 13 6a 61 76 61 2e 75 74 69
6c 2e 41 72 72 61 79 4c 69 73 74 78 81 d2 1d 99
c7 61 9d 03 00 01 49 00 04 73 69 7a 65 78 70 00
00 00 01 77 04 00 00 00 01 74 00 00 78
Note that with one element the resulted file is the same.
But if we want to add more times the same object, we will be faced with other behavior.
Look the respective files with 2 times that string.
empty.list:
ac ed 00 05 73 72 00 13 6a 61 76 61 2e 75 74 69
6c 2e 41 72 72 61 79 4c 69 73 74 78 81 d2 1d 99
c7 61 9d 03 00 01 49 00 04 73 69 7a 65 78 70 00
00 00 02 77 04 00 00 00 02 74 00 00 71 00 7e 00
02 78
substring.list
ac ed 00 05 73 72 00 13 6a 61 76 61 2e 75 74 69
6c 2e 41 72 72 61 79 4c 69 73 74 78 81 d2 1d 99
c7 61 9d 03 00 01 49 00 04 73 69 7a 65 78 70 00
00 00 02 77 04 00 00 00 02 74 00 00 74 00 00 78
Note that substring continues "normal", two non related strings with different references. But empty has some extra bytes to handle the issue of same reference.
Six bytes from substring (00 00 74 00 00 78) versus eight bytes from emptylist (00 00 71 00 7e 00 02 78)
This goes wrong because every repeated string that you add, more extra bytes are added. So when you full your arrayList there will be a lot of extra bytes to make it possible to reconstruct as it's original way.
If you want to know why there is that sharing mechanism, I suggest you to take a look at this question:
What is the meaning of reference sharing in Serialization? How Enums are Serialized?

empty.list contains one String object and lots of references to it.
substring.list contains 2000 string objects, all of them are equal in content.
You could "fix" this by intern()ing the strings.
private void verify(String name, Supplier<String> stringSupplier) throws IOException, ClassNotFoundException {
List<String> inputStrings = new ArrayList<String>();
inputStrings.add(stringSupplier.get());
inputStrings.add(stringSupplier.get());
ByteArrayOutputStream boas = new ByteArrayOutputStream();
ObjectOutputStream emptyOut = new ObjectOutputStream(boas);
emptyOut.writeObject(inputStrings);
emptyOut.flush();
ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(boas.toByteArray()));
List<String> returnedStrings = (List<String>)ois.readObject();
if(returnedStrings.get(0) == returnedStrings.get(1)) {
System.out.println(name + " contains the same object");
} else {
System.out.println(name + " contains DIFFERENT objects");
}
}
#Test
public void test() throws IOException, ClassNotFoundException {
verify("empty string", new Supplier<String>() {
#Override
public String get() {
return "";
}
});
verify("sub string", new Supplier<String>() {
#Override
public String get() {
String data = " ";
return data.substring(0, 0);
}
});
verify("intern()ed substring", new Supplier<String>() {
#Override
public String get() {
String data = " ";
return data.substring(0, 0).intern();
}
});
}

Decode unknow charset text save by php

I have a some record in MySQL such as
Vận hành linh hoạt trong má»i Ä‘k giao thông
which in hex as
56 c3 a1 c2 ba c2 ad 6e 20 68 c3 83 c2 a0 6e 68 20 6c 69 6e 68 20 68
6f c3 a1 c2 ba c2 a1 74 20 74 72 6f 6e 67 20 6d c3 a1 c2 bb c2 8d 69
20 c3 84 e2 80 98 6b 20 67 69 61 6f 20 74 68 c3 83 c2 b4 6e 67 20
I dont know how PHP save it, but read it from Java MySQL Connector show some strange character. And I can make it show the origin text by
copy the text above --> Notepad++ - Encoding in ASCII --> Paste text
--> Encoding in UTF-8
the original text should be:
Vận hành linh hoạt trong mọi đk giao thông
I know the problem is PHP save incorrect text format, but is there a way to decode it correctly in Java?

Are you sure the hex is exactly correct? Here is what I did...
String MESS = "56 c3 a1 c2 ba c2 ad 6e 20 68 c3 83 c2 a0 6e 68 20 6c 69 6e 68 20 68 6f c3 a1 c2 ba c2 a1 74 20 74 72 6f 6e 67 20 6d c3 a1 c2 bb c2 8d 69 20 c3 84 e2 80 98 6b 20 67 69 61 6f 20 74 68 c3 83 c2 b4 6e 67 20";
String[] hexchars = MESS.split(" ");
byte[] buf = new byte[hexchars.length];
for (int i = 0; i < hexchars.length; i++) {
buf[i] = (byte) Integer.parseInt(hexchars[i], 16);
}
try {
String s1 = new String(buf, "UTF-8"); // First encode UTF-8
buf = s1.getBytes("cp1252"); // ...then translate to cp1252
s1 = new String(buf, "UTF-8"); // ...then back to UTF-8
System.out.println(s1);
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
And the printed result is:
Vận hành linh hoạt trong m�?i đk giao thông
Which is almost right. Except the decoding of mọi it is incorrect, which makes me suspect the hex that you provided may not be correct. If you are 100% sure it is correct, I can try a little more to decode it.
UPDATE:
Here are my further thoughts:
You need to find out what encoding MySQL itself (the database) is set to.
You need to find out what encoding PHP is set to
possibly in PHP.INI
possibly set in the HTML metadata for the page that populates the table.
You need to find out what if any encoding the PHP MySQL driver runs with
Only then will there be a possibility of setting the MySQL Connector/J to the right encoding, and then possibly applying a second conversion in Java.

Read Java in as Hex

I have tried to solve this but I keep coming up with stuff that is no help I'm sure this is easy (when you know how of course ;) )
What I would like to do is read in a file using a byte stream like below:
while((read = in.read()) != -1){
//code removed to save space
Integer.toHexString(read);
System.out.println(read);
}
When it prints out the Hex to the screen it will print out numbers fine e.g
31
13
12
0
but when it comes to a hex code that should be 01 31 it will print 0 131. I want to read it in to a variable like you would see in a hex editor i.e 00 11 21 31 no single numbers as i need to scan the whole file and look for patterns which I know how to do I'm just stuck on this :/
so in short i need a variabe to contain the two hex characters i.e int temp = 01 not int temp = 0 , I hope this all makes sense, I'm a little confused as it's 3am!
If anyone knows how to do this I would be most greatful, p.s thanks for the help in advance this site has saved me loads of research and have learnt a lot!
Many thanks.

This method :
public static void printHexStream(final InputStream inputStream, final int numberOfColumns) throws IOException{
long streamPtr=0;
while (inputStream.available() > 0) {
final long col = streamPtr++ % numberOfColumns;
System.out.printf("%02x ",inputStream.read());
if (col == (numberOfColumns-1)) {
System.out.printf("\n");
}
}
}
will output something like this :
40 32 38 00 5f 57 69 64 65 43
68 61 72 54 6f 4d 75 6c 74 69
42 79 74 65 40 33 32 00 5f 5f
69 6d 70 5f 5f 44 65 6c 65 74
65 46 69 6c 65 41 40 34 00 5f
53 65 74 46 69 6c 65 50 6f 69
6e 74 65 72 40 31 36 00 5f 5f
69 6d 70 5f 5f 47 65 74 54 65
6d 70 50 61 74 68 41 40 38 00
Is it what you are looking for?

I think what you're looking for is a formatter. Try:
Formatter formatter = new Formatter();
formatter.format("%02x", your_int);
System.out.println(formatter.toString());
Does that do what you're looking for? Your question wasn't all that clear (and I think maybe you deleted too much code from your snippet).

import org.apache.commons.io.IOUtils;
import org.apache.commons.codec.binary.Hex;
InputStream is = new FileInputStream(new File("c:/file.txt"));
String hexString = Hex.encodeHexString(IOUtils.toByteArray(is));
In java 7 you can read byte array directly from file as below :
import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.file.Path;
Path path = Paths.get("path/to/file");
byte[] data = Files.readAllBytes(path)

Hi everyone one posted, thanks for the reply but the way I eneded up doing it was:
hexIn = in.read();
s = Integer.toHexString(hexIn);
if(s.length() < 2){
s = "0" + Integer.toHexString(hexIn);
}
Just thought I would post they way I did it for anyone else in future, thank you soo much for your help though!