Java coder sends me some data (some headers and content of picture).
It look like this:
436f 6e74 656e 742d 4c65 6e67 7468 3a20
3138 3830 0d0a 0d0a ffd8 ffe0 0010 4a46
4946 0001 0101 0060 0060 0000 ffdb 0043
0010 0b0c 0e0c 0a10 0e0d 0e12 1110 1318
281a 1816 1618 3123 251d 283a 333d 3c39
3338 3740 485c 4e40 4457 4537 3850 6d51
...
I'm trying to parse it with $queryHex=pack("H*", $query); where $query = file_get_contents("php://input");
When I try to var_dump $queryHex I get
Co�#enB�Le�pth�18�
and than some not readable symbols that I suppose to be a content of picture.
Also I tried to set encoding (header('Content-Type: text/html; charset=utf-8');) and result remains.
What I'm doing wrong? How to get propper data?
Update:
According to his API he should send something like this:
POST /url_for_detect HTTP/1.1
Content-Type: image/jpeg; boundary=-
[DATA]
DATA is a content of picture.
But there is Content-length there. He said that it's not necessary. However I stil can't see content-type there or get DATA. Asked him to send me a query that he sends.
The bytes you posted contains.
Content-Length: 1880
FF D8 FF E0 00 10 4A 46 49 46 <-- the start of the JPEG file
As #awons suggested already. Ask the sender what he send to you.
edit
Small "quick'n'dirty" code to convert the bytes into binary.
String content = "436f 6e74 656e 742d 4c65 6e67 7468 3a20"
+ "3138 3830 0d0a 0d0a ffd8 ffe0 0010 4a46"
+ "4946 0001 0101 0060 0060 0000 ffdb 0043"
+ "0010 0b0c 0e0c 0a10 0e0d 0e12 1110 1318"
+ "281a 1816 1618 3123 251d 283a 333d 3c39"
+ "3338 3740 485c 4e40 4457 4537 3850 6d51";
StringBuilder sb = new StringBuilder(content.replaceAll(" ", ""));
sb.delete(0, 48); // remove "Content-Length: 1880"
try (FileOutputStream fos = new FileOutputStream("content.jpg")) {
while (sb.length() > 2) {
fos.write(Integer.parseInt(sb.substring(0, 2), 16));
sb.delete(0, 2);
}
}
Related
I have such a java class:
public class UnicodeTest {
public static void main(String[] args) {
String s = "中";
String s1 = "𩄀";
System.out.println(s.length());
System.out.println(s1.length());
System.out.println(s1.toCharArray().length);
}
}
then I use xxd to see the compiled class file:
0000000: cafe babe 0000 0032 0031 0700 0201 000b .......2.1......
0000010: 556e 6963 6f64 6554 6573 7407 0004 0100 UnicodeTest.....
0000020: 106a 6176 612f 6c61 6e67 2f4f 626a 6563 .java/lang/Objec
0000030: 7401 0006 3c69 6e69 743e 0100 0328 2956 t...<init>...()V
0000040: 0100 0443 6f64 650a 0003 0009 0c00 0500 ...Code.........
0000050: 0601 000f 4c69 6e65 4e75 6d62 6572 5461 ....LineNumberTa
0000060: 626c 6501 0012 4c6f 6361 6c56 6172 6961 ble...LocalVaria
0000070: 626c 6554 6162 6c65 0100 0474 6869 7301 bleTable...this.
0000080: 000d 4c55 6e69 636f 6465 5465 7374 3b01 ..LUnicodeTest;.
0000090: 0004 6d61 696e 0100 1628 5b4c 6a61 7661 ..main...([Ljava
00000a0: 2f6c 616e 672f 5374 7269 6e67 3b29 5608 /lang/String;)V.
00000b0: 0011 0100 03e4 b8ad 0800 1301 0006 eda1 ................
00000c0: a4ed b480 0900 1500 1707 0016 0100 106a ...............j
00000d0: 6176 612f 6c61 6e67 2f53 7973 7465 6d0c ava/lang/System.
00000e0: 0018 0019 0100 036f 7574 0100 154c 6a61 .......out...Lja
00000f0: 7661 2f69 6f2f 5072 696e 7453 7472 6561 va/io/PrintStrea
0000100: 6d3b 0a00 1b00 1d07 001c 0100 106a 6176 m;...........jav
0000110: 612f 6c61 6e67 2f53 7472 696e 670c 001e a/lang/String...
0000120: 001f 0100 066c 656e 6774 6801 0003 2829 .....length...()
0000130: 490a 0021 0023 0700 2201 0013 6a61 7661 I..!.#.."...java
0000140: 2f69 6f2f 5072 696e 7453 7472 6561 6d0c /io/PrintStream.
0000150: 0024 0025 0100 0770 7269 6e74 6c6e 0100 .$.%...println..
0000160: 0428 4929 560a 001b 0027 0c00 2800 2901 .(I)V....'..(.).
0000170: 000b 746f 4368 6172 4172 7261 7901 0004 ..toCharArray...
0000180: 2829 5b43 0100 0461 7267 7301 0013 5b4c ()[C...args...[L
0000190: 6a61 7661 2f6c 616e 672f 5374 7269 6e67 java/lang/String
00001a0: 3b01 0001 7301 0012 4c6a 6176 612f 6c61 ;...s...Ljava/la
00001b0: 6e67 2f53 7472 696e 673b 0100 0273 3101 ng/String;...s1.
00001c0: 000a 536f 7572 6365 4669 6c65 0100 1055 ..SourceFile...U
00001d0: 6e69 636f 6465 5465 7374 2e6a 6176 6100 nicodeTest.java.
00001e0: 2100 0100 0300 0000 0000 0200 0100 0500 !...............
00001f0: 0600 0100 0700 0000 2f00 0100 0100 0000 ......../.......
0000200: 052a b700 08b1 0000 0002 000a 0000 0006 .*..............
0000210: 0001 0000 0002 000b 0000 000c 0001 0000 ................
0000220: 0005 000c 000d 0000 0009 000e 000f 0001 ................
0000230: 0007 0000 0078 0002 0003 0000 0026 1210 .....x.......&..
0000240: 4c12 124d b200 142b b600 1ab6 0020 b200 L..M...+..... ..
0000250: 142c b600 1ab6 0020 b200 142c b600 26be .,..... ...,..&.
0000260: b600 20b1 0000 0002 000a 0000 001a 0006 .. .............
0000270: 0000 0005 0003 0006 0006 0007 0010 0008 ................
0000280: 001a 0009 0025 000a 000b 0000 0020 0003 .....%....... ..
0000290: 0000 0026 002a 002b 0000 0003 0023 002c ...&.*.+.....#.,
00002a0: 002d 0001 0006 0020 002e 002d 0002 0001 .-..... ...-....
00002b0: 002f 0000 0002 0030 ./.....0
I have found the Chinese character "中" in line 12 03e4 b8ad, unicode U+4E2D, which in UTF-8 is E4 B8 AD, but I can't find another character "𩄀", unicode U+29100, which I expected something like "04 F0 A9 84 80", why?
Let's use javap.
Compile first with
javac UnicodeTest.java
Then disassemble with
javap -v UnicodeTest.class (truncated to relevant part) :
Constant pool:
#1 = Methodref #9.#18 // java/lang/Object."<init>":()V
#2 = String #19 // 中
#3 = String #20 // 𩄀
#4 = Fieldref #21.#22 //
... truncated
#17 = Utf8 UnicodeTest.java
#18 = NameAndType #10:#11 // "<init>":()V
#19 = Utf8 中
#20 = Utf8 𩄀
Item #20 in constant pool is what you are looking for.
Now, you let's check JVM class file format.
Utf8 datastructure is CONSTANT_Utf8_info
CONSTANT_Utf8_info {
u1 tag;
u2 length;
u1 bytes[length];
}
tag is CONSTANT_Utf8 (01). length is 00 06, bytes are ed a1 a4 ed b4 80
According to unicode lookup, mentioned character should have codepoint 0x29100.
Now back to JVM spec.
Characters with code points above U+FFFF (so-called supplementary
characters) are represented by separately encoding the two surrogate
code units of their UTF-16 representation. Each of the surrogate code
units is represented by three bytes. This means supplementary
characters are represented by six bytes, u, v, w, x, y, and z :
I will not paste content here because it's too long, but you can look it up as Table 4.12. under CONSTANT_Utf8_info info (link above)
So that's why it is 6 bytes long.
Now let's take the formula
0x10000 + ((v & 0x0f) << 16) + ((w & 0x3f) << 10) +
((y & 0x0f) << 6) + (z & 0x3f)
By substituting v, w ,y and z output is 168192(10) which is 0x29100, which is expected code point.
The classic technique you can use to find that out is changing the value by something else and check the hex-dump for the difference. This was already used in the eighties when you "hacked" save games for games to increase e.g. attributes of your role playing character, etc.
I changed the character by a and it seems that the character can be found at offset 0xBE-0xC3 and has the value ED A1 A4 ED B4 80. I would have to look up the specifics of this to be able to explain why the value differs from the one you expected but Java's original support for Unicode was limited to two bytes (that's what the char type is defined. Unicode-characters with a 3+ bytes or more need to be encoded in a particular way in the Bytecode to tell the ClassLoader that it needs to be treated in a different way.
I am trying to read a file that is under MF. The EF.DIR file. I got the file's SFID, so I don't use the method SELEC_FILE first (since it's not necessary).
I might be having some problems with understanding the P2 parameter (the OFFSET). I read couple of explanations, but still don't get what OFFSET do they mean. But I tried all the numbers from 0-8 just in case, none worked.
CLA = 0x00
INS_READ = 0xB0
P1_READ = 0x9E (by the datasheed: bit(8) = 1, bit(7:6) = 00, bit(5:1): SFID)
P2 = 0x04 (I figured that the offset should be from bit0 to bit4 (the SFID)
Le = 0 (by the datasheet I have, this should mean that any size will be returned)
This is my code:
byte[] readBinary = { CLA, INS_READ, P1_READ, (byte) 0x04, (short) 0};
ResponseAPDU read = channel.transmit(new CommandAPDU(readBinary));
String responseReadToString =read.toString();
System.out.println("Response Read: " + responseReadToString + "\n" + "Response Read (HEX): " + responseReadHex );
The output I get in Console is:
Response Read: ResponseAPDU: 2 bytes, SW=6b00
Response Read (HEX): 6B00
Explanations of SW1-SW2 for 6B00:
Incorrect parameters P1-P2
I really don't know what is wrong and it's really hard to find support on SmartCards online, so hopefully someone who knows this better can help me out. I also tried with using SELECT_FILE first and the use READ_BINARY after it (without the SFID in P1 parametr ofcourse), but it responded with "No EF is set as current".
Any help guys?
The offset is the position/startpoint from where you start reading.
Example: Data = [0x00 0x01 0x02 0x03 0x04 0x05]
When you query a ReadBinary with offset=2 then returned data will be [0x02 0x03 0x4 0x05]
As you probably want to read the whole EF.DIR file offset shall be zero.
For reading EF.DIR you can either send
00 B0 9E 00 00
or
00 B1 2F 00 04 54 02 00 00 00
or
00 A4 02 0C 02 2F 00
00 B0 00 00 00
I am writing a PNG decoder and I am encountering some weirdness. Going through the PNG file format, I have managed to decode PNGs with Color Index + tRNS (alpha) and True Color + tRNS (alpha) correctly. I am currently not sure why I cannot decode a PNG with True Color with Alpha type PNG. I have verified that my inflate of IDAT chunk is correct. Here's what the chunks looks like:
Width: 256
Height: 256
Bit Depth: 8
Color Type: 6
Compression: 0
Filter: 0
Interlace: 0
Length: 25
Type: tEXt
Data: 53 6f 66 74 77 61 72 65 00 41 64 6f 62 65 20 49 6d 61 67 65 52 65 61 64 79
CRC: 71c9653c
Length: 20690
Type: IDAT
Data: 78 da ec 7d 09 9c 1c 57 99 df f7 7a a6 e7 3e 5a a3 fb ...
CRC: 21550259
The actual data is too long to be printed here. Here's my logic of decoding this, please correct me if I'm wrong:
Inflate all the bytes given in the IDAT chunk
Un-filter the inflated chunks. In this case, all filters are of type 0 and therefore, we simply discard the filter byte.
Since this is color type 6, a pixel is represented by RGBA channels with 1 byte each. This means we need to interpret 4 bytes at a time. The following code is used:
ByteBuffer image = BufferUtil.getDirectByteBuffer(data.length, ByteOrder.BIG_ENDIAN);
int i = 0;
while(i < data.length){
int color = ( (data[i] & 0xff) << 24) | ( (data[i+1] & 0xff) << 16) | ( (data[i+2] & 0xff) << 8) | (data[i+3] & 0xff);
image.putInt(color);
i += 4;
What's strange is that I get mostly RRGGBBAA = 0x00000000 data resulting in a clear image with little color.
The problem is you are neglecting to observe the filtering for each scanline.
From the image provided the decompressed data looks like
1 ffffffff 0 0 0 ...
2 0 0 0 0 0 0 ...
..
the first value in each line conforms to the filter method used [http://www.w3.org/TR/PNG/#9Filters]
the scanlines post processing will look like
ffffffff ffffffff ffffffff ...
ffffffff ffffffff ffffffff ...
...
here is some example code that handles methods 0, 1 and 2.
private static void processScanLine(byte filterValue, byte[] scanLine, byte[] previousScanLine) {
switch(filterValue){
case 0:break;
case 1:
for (int i =4;i<scanLine.length;i++){
scanLine[i] = (byte)(scanLine[i]+scanLine[i-4]);
}
break;
case 2:
for (int i =0;i<scanLine.length;i++){
scanLine[i] = (byte)(scanLine[i]+previousScanLine[i]);
}
break;
}
}
My code is attempting to decompress an input stream read from a gzipped file.
Here is the code snippet:
InputStream is = new GZIPInputStream(new ByteArrayInputStream(fcontents.getBytes()));
The file itself is fine:
$cat storefront3.gz | gunzip
180028796
80026920
180028796
180026921
8002790180
800001
1800002
1800007
800008
800009
The data read in prior to the top code snippet via FileInputStream sure looks like gzip stuff (note the original file was storefront3.tsv):
��[�Rstorefront3.tsvu���0k{)�?�/FBģ��Y'��Q�a���s~���}6���d�{2+���O���D�m~�O��
But get the following:
Caused by: java.io.IOException: Not in GZIP format
at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:141)
at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:56)
at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:65)
Here is a hex dump of the .gz file
23:40:44/storefronts:72 $od -cx storefront3.gz
0000000 037 213 \b \b 201 [ 347 R \0 003 s t o r e f
8b1f 0808 5b81 52e7 0300 7473 726f 6665
0000020 r o n t 3 . t s v \0 u 212 273 025 200 0
6f72 746e 2e33 7374 0076 8a75 15bb 3080
0000040 \f 003 k { 032 ) 200 ? 373 / F B ģ ** 302 131
030c 7b6b 291a 3f80 2ffb 4246 a3c4 cdc2
0000060 Y ' 261 200 Q 331 a 276 276 350 001 s ~ 222 262 175
2759 80b1 d951 be61 e8be 7301 927e dcb2
0000100 } 6 226 231 367 d 200 { 2 + 211 337 342 020 O 022
367d 9996 64f7 7b80 2b32 df89 10e2 f14f
0000120 022 343 035 246 D 211 m ~ 003 326 O 235 030 236 \0 \0
e312 a61d 8944 7e6d d603 9d4f 9e18 0000
0000140 \0
0000
UPDATE
I also tried to use FileInputStream. Following gives same error
GZIPInputStream strm = new GZIPInputStream(new FileInputStream(tmpFileName));
Since fcontents contains your gzipped data it should be a byte[] and not a String?
I recommend using IOUtils for reading the file into a byte array as reading it into a string will most likely corrupt your data.
I have been working with a client/server where the client is written in Java (Android) and the server is written in C++. I basically need to be able to pass it a dataset and have it interpret that data correctly.
I wrote a prototype in Python, as I know what its supposed to look like there. With struct.pack, it will pack the data and you can specify that it do it in network byte order (Big Endian). This is the prototype that I wrote (and this works with the server).
5 def _BuildPDUToTx(pduObj):
6 txData = struct.pack('!i i', pduObj.Src.Type, pduObj.Src.Len) #pduObj.Src.Type: 1
#pduObj.Src.Len: 16
7 txData += pduObj.Src.Value #pduObj.Src.Value: sourceid
8 txData += struct.pack('!i i', pduObj.Dst.Type, pduObj.Dst.Len) #pduObj.Dst.Type: 2
#pduObj.Dst.Len: 14
9 txData += pduObj.Dst.Value #pduObj.Dst.Value: destid
10 txData += struct.pack('!i i i i', pduObj.Metadata.Type, pduObj.Metadata.Len,\ #pduObj.Metadata.Type: 3
11 pduObj.Metadata.Status, pduObj.Metadata.Remaining) #pduObj.Metadata.Len: 16
#pduObj.Metadata.Status: 0
#pduObj.Metadata.Remaining: 0
12 txData += struct.pack('!i i', pduObj.Msg.Type, pduObj.Msg.Len) #pduObj.Msg.Type: 6
#pduObj.Msg.Len: 27
13 txData += pduObj.Msg.Value #pduObj.Msg.Value: This is the message
14 return(txData)
I ran it with some example code (the example values are the comments on the side), and I get the following if I setup a raw netcat listener on the server side (just to see what the data looks like).
0000000 0000 0100 0000 1000 [6f73 7275 6563 6469] *brackets have "sourceid"
0000010 0000 0200 0000 0e00 [6564 7473 6469] 0000 *brackets have "destid"
0000020 0300 0000 1000 0000 0000 0000 0000 0000
0000030 0600 0000 1b00 [6854 7369 6920 2073 6874 *brackets have "This is the message"
0000040 2065 656d 7373 6761 0065]
I have tried doing the same (and used the advice of Java equivalent of Python's struct.pack? and Using Java's ByteBuffer to replicate Python's struct.pack).
I used the following code:
client = new Socket(server, port);
ObjectOutputStream dos = new ObjectOutputStream(new BufferedOutputStream(client.getOutputStream()));
ByteBuffer buffer = ByteBuffer.allocate(1000);
dos.writeInt(Src.Type); // int value 1
dos.writeInt(Src.Len); // int value 16
dos.writeUTF(Src.Value); // String value "sourceid"
dos.writeInt(Dst.Type); // int value 2
dos.writeInt(Dst.Len); // int value 14
dos.writeUTF(Dst.Value); // String value "destid"
dos.writeInt(Metadata.Type); // int value 3
dos.writeInt(Metadata.Len); // int value 16
dos.writeInt(Metadata.Status); // int value 0
dos.writeInt(Metadata.Remaining); // int value 0
dos.writeInt(Msg.Type); // int value 6
dos.writeInt(Msg.Len); // int value 27
dos.writeUTF(Msg.Value); // String value "this is the message"
dos.flush();
InputStream is = client.getInputStream();
byte[] buf = new byte[Constants.ResponseSize];
is.read(buf, 0, Constants.ResponseSize);
client.close();
However, I get the following.
0000000 edac 0500 4f77 0000 0100 0000 1000 0800
0000010 [6f73 7275 6563 6469] 0000 0200 0000 0e00 *brackets have "sourceid"
0000020 0600 [6564 7473 6469] 0000 0300 0000 1000 *brackets have "destid"
0000030 0000 0000 0000 0000 0000 0600 0000 1b00
0000040 1300 [6854 7369 6920 2073 6874 2065 656d *brackets have "This is the message"
0000050 7373 6761 0065]
0000055
Java seems to be adding quite a bit of extra. Like the first two bytes dont go with anything I put in the stream. Thats followed with my value of 1 (32 bits) and 16 (the next 32 bits), but then it has 0800 before the "sourceid".
What the heck is Java doing?? And how can I get it to pack more like python?
Use DataOutputStream, not ObjectOutputStream.
ObjectOutputStream is used to ship Java objects from one JVM to another, and is primarily used by java object serialization framework.
DataOutputStream writes everything in Big Endian.
You may have to be careful, though, with writeUTF. By spec, it prepends the length of the byte sequence. This may, or may not, be what you want.