We Keep Coding

How to convert rtp packet payload bytes H264/90000 to any video data?

Studying the sip protocol, I got to the topic of the H264 codec. I began to receive data in the form of rtp packets. I managed to successfully get the following data from the package: Payload type (in my case 97), Timestamp, Sequence number and payload data (byte array). Next, I want to draw images encoded in this data. On the android platform, I use the android.media.MediaCodec class. I follow examples like MediaCodec failing on S7.
I create an instance of MediaCodec. Configuring with MediaFormat. Then I transfer the received bytes to inputBuffer and wait for updates via dequeueOutputBuffer. In my case, the dequeueOutputBuffer method always returns MediaCodec.INFO_TRY_AGAIN_LATER.
I was trying to process bytes before passing to MediaCodec. Defined nal_unit_type. I get 7, 8 and 28. I also defined startBit and endBit in the package. I tried to glue all packages starting with startBit and ending with endBit and transfer them to MediaCodec in a glued form. The result is the same - the dequeueOutputBuffer method always returns MediaCodec.INFO_TRY_AGAIN_LATER
Tell me what I missed.
The server sends the following information about the video to the SDP:
m=video 23542 RTP/AVP 97
b=TIAS:4096000
a=content:main
a=rtpmap:97 H264/90000
a=fmtp:97 profile-level-id=428028; max-fs=8192; packetization-mode=0; max-br=4096; max-fps=3000
a=sendrecv
Edit #1
For example, first received packet payload (array of unsigned byte):
27 42 00 28 95 a0 1e 00 89 f9 70 11 00 00 03 00
42 00 28 95 a0 1e 00 89 f9 70 11 00 00 03 00 01
00 28 95 a0 1e 00 89 f9 70 11 00 00 03 00 01 00
I would venture to suggest that this is a Single NAL Unit Packet. This packet has no padding.
By rfc3984/1.3 i got in first byte:
// 0 1 2 3 4 5 6 7
// +-----+---------+
// | type |
// +-----+---------+
val nal_unit_type = payload[0].toInt() and 0b0_0_0_1_1_1_1_1
nal_unit_type == 7 And I decided that this package contains Sequence Parameter Set data. Next, I want to get the decrypt SPS and get useful information from it (width and height, frame rate ...)
I got in second byte:
// 0 1 2 3 4 5 6 7
// +-+-+-----------+
// |s|e|
// +-+-+-----------+
val start_bit = payload[1].toInt() and 0b1_0_0_0_0_0_0_0 != 0
// +-+-+-----------+
val end_bit = payload[1].toInt() and 0b0_1_0_0_0_0_0_0 != 0
start_bit == false and end_bit == true
Starting from the third byte (payload[2]), I parse the SPS.
Edit #2
I was wrong when I decided that the second byte for nal_unit_type 7 or 8 is the FU header (with start and end bits). The second byte of the payload is already the first byte of the SPS. Thus, I managed to successfully decrypt the SPS and, for example, find out that the image size 1920/1080 is encrypted there (as was expected). But this has not yet helped me in any way to draw the resulting video stream to the android surface view.

SmartCard - Select MF succedd but no response

So I am trying to read my transportation card using what I have learned so far about smartcards.
My ATR is: 3B 6F 00 00 80 5A 0A 07 06 20 04 01 03 01 F4 1F 82 90 00
when I looked in the ATR parser it didn't give me much information.
when I chose the MF file like this: "00 A4 04 00"
I got the response: "90 00"
output: but no data.
How can I go on from here to read files on my card?
Note: [it would be nice if someone can give me a link to a book or guide about smart cards, cause I found nice one about EMV cards but it is not working on all smartcards]

https://www.eftlab.com/knowledge-base/171-atr-list-full/ shows that from your atr, there are similar cards with similar ATR data.
You can try selecting the dedicated file using the offsets below and see what happens;
0x0002
0x0003
0x2000
0x2001
0x2004
0x2010
0x2020
0x202a
0x202b
0x202c
0x202d
0x2030
0x2040
0x2050
0x2069
0x206a
0x20f0
0x2100
0x2101
0x2104
0x2110
0x2120
0x2140
0x2150
0x2169
0x21f0
0x2f10
0x3f04
0xfeff
hope you can continue from there.

Select first the MF by performing CLA INS P1 P2 Lc DATA
EX.
CLA 00
INS A4
P1 04 - to select by Name
P2 00 - Select first or only occurrence
Lc - Length of FID
Data - FID

How to get specific lines from the file ( between two sections)?

I'm trying to read specific lines in-between two sections using Java 8.
I need to get the information in between ~CURVE INFORMATION and ~PARAMETER INFORMATION
I was able to get it using by checking startsWith() or equals and start storing the lines in some stringbuilder or collection. But is there any method available to get some specific lines in-between some sections.
I was looking at below questions for reference.
How to read specific parts of a .txt file in JAVA
How to read specific parts of the text file using Java
Sample data from file:
~WELL INFORMATION
#MNEM.UNIT DATA TYPE INFORMATION
#---------- ------------ ------------------------------
STRT.FT 5560.0000: START DEPTH
STOP.FT 16769.5000: STOP DEPTH
STEP.FT 0.5000: STEP LENGTH
NULL. -999.2500: NULL VALUE
COMP. SHELL: COMPANY
~CURVE INFORMATION
#MNEM.UNIT API CODE CURVE DESCRIPTION
#---------- ------------ ------------------------------
DEPT.F :
SEWP.OHMM 99 000 00 00:
SEMP.OHMM 99 120 00 00:
SEDP.OHMM 99 120 00 00:
SESP.OHMM 99 220 01 00:
SGRC.GAPI 99 310 01 00:
SROP.FT/HR 99 000 00 00:
SBDC.G/C3 45 350 01 00:
SCOR.G/C3 99 365 01 00:
SPSF.DEC 99 890 03 00:
~PARAMETER INFORMATION
#MNEM.UNIT VALUE DESCRIPTION
#---------- ------------ ------------------------------
RMF .OHMM -: RMF
MFST.F -: RMF MEAS. TEMP.
RMC .OHMM -: RMC
MCST.F -: RMC MEAS. TEMP.
MFSS. -: SOURCE RMF.
MCSS. -: SOURCE RMC.
WITN. MILLER: WITNESSED BY
~OTHER INFORMATION

Using Java9 you can do it elegantly with streams
public static void main(String[] args) {
try (Stream<String> stream = Files.lines(Paths.get(args[0]))) {
System.out.println(stream.dropWhile(string -> !"~CURVE INFORMATION".equals(string)).takeWhile( string -> !"~PARAMETER INFORMATION".equals(string)).skip(1).collect(Collectors.joining("\n")));
} catch (IOException e) {
e.printStackTrace();
}
}
What makes it pleasing is the declarative nature of streams, your literally writing code that says drop elements until start mark then take elements until end mark and join them using "\n"! Java9 added takeWhile and dropWhile, I'm sure you can implement them or get their implementation from a library for java 8. Of course this is just another way to achieve the original goal.

Android BLE BluetoothAdapter.LeScanCallback scanRecord length ambiguity

I am using the example project from google (BluetoothLeGatt) to receive data from a BLE device and trying to read a specific byte within it's scanRecord obtained by the onLeScan method.
My problem is that there is missmatch between the data I am observing in the network and what I see on logs.
This is on Android 4.3 and using a Samsung Galaxy S4 to test it.
To verify that the scanRecord logs are correct on Android, I am using TI's Packet Sniffer to observe the byte stream being broadcasted by the device, and here it is:
That is 31 bytes of data being broadcasted by the device to the network, and there are no other working devices around.
02 01 1A 1A FF 4C 00 02 15 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 0C C6 64
On the other hand, Android logs claim that the data being received has the length of 62 bytes and it matches the data until the 29th[0-indexed] byte, having 0s for the rest of the data.
02-12 15:34:09.548: D/DEBUG(26801): len: 62
data:02011a1aff4c000215000000000000000000000000000000000000000cc60000000000000000000000000000000000000000000000000000000000000000
And this is the code piece I used in order to obtain the logs within the LeScanCallback method:
int len = scanRecord.length;
String scanHex = bytesToHex(scanRecord);
Log.d("DEBUG", "len: " + len + " data:" + scanHex);
The method used to convert byte array to hex representation:
private static String bytesToHex(byte[] bytes) {
char[] hexChars = new char[bytes.length * 2];
int v;
for ( int j = 0; j < bytes.length; j++ ) {
v = bytes[j] & 0xFF;
hexChars[j * 2] = hexArray[v >>> 4];
hexChars[j * 2 + 1] = hexArray[v & 0x0F];
}
return new String(hexChars);
}
I used a few other example projects including Dave Smith's example and RadiusNetworks' Android iBeacon Library and I ended up with the same results. I can't possibly understand why do I receive 62 bytes of data when "Packet Sniffer" shows (and I also know) that it should be 31 bytes. This would not be my main concern if I was able to read the data in the last byte correctly (I get 00 instead of 64 from Android's BluetoothAdapter). But that is not the case either.
I would appreciate any suggestions about what might potentially be the reason for this missmatch for both the data(last byte only) and the data size between what Android receives and what is actually on the network.

Your transmission is malformed, containing 31 bytes of payload data (PDU length of 37) when its internal length fields indicate it should in total contain only 30 bytes (PDU length 36).
Let's take a look at your data
02 01 1a
This is a length (2) of type codes - 01 and 1a, and good so far
1a ff 4c ...
Now we have a problem - the 1a is a length code for this field (manufacturer specific data), value of 26. Yet 27 bytes of data follow it in your case, instead of the proper 26 you have indicated you will provide.
Now, if you have a properly formed packet, you will still get a larger buffer padded with meaningless (likely uninitialized) values following the proper content, but you can simply ignore that by parsing the buffer in accordance with the field-length values and ignoring anything not accounted for in the proclaimed lengths.
But with your current malformed packet, the copying of packet data to the buffer stops at the proclaimed content size, and the unannounced extra byte never makes it into the buffer your program receives - so you see instead something random there, as with the rest of the unused length.
Probably, when you made up your all-zeroes "region UUID" (might want to rethink that) you simply typed an extra byte...

"Fix" String encoding in Java

I have a String created from a byte[] array, using UTF-8 encoding.
However, it should have been created using another encoding (Windows-1252).
Is there a way to convert this String back to the right encoding?
I know it's easy to do if you have access to the original byte array, but it my case it's too late because it's given by a closed source library.

As there seems to be some confusion on whether this is possible or not I think I'll need to provide an extensive example.
The question claims that the (initial) input is a byte[] that contains Windows-1252 encoded data. I'll call that byte[] ib (for "initial bytes").
For this example I'll choose the German word "Bär" (meaning bear) as the input:
byte[] ib = new byte[] { (byte) 0x42, (byte) 0xE4, (byte) 0x72 };
String correctString = new String(ib, "Windows-1252");
assert correctString.charAt(1) == '\u00E4'; //verify that the character was correctly decoded.
(If your JVM doesn't support that encoding, then you can use ISO-8859-1 instead, because those three letters (and most others) are at the same position in those two encodings).
The question goes on to state that some other code (that is outside of our influence) already converted that byte[] to a String using the UTF-8 encoding (I'll call that String is for "input String"). That String is the only input that is available to achieve our goal (if ib were available, it would be trivial):
String is = new String(ib, "UTF-8");
System.out.println(is);
This obviously produces the incorrect output "B�".
The goal would be to produce ib (or the correct decoding of that byte[]) with only is available.
Now some people claim that getting the UTF-8 encoded bytes from that is will return an array with the same values as the initial array:
byte[] utf8Again = is.getBytes("UTF-8");
But that returns the UTF-8 encoding of the two characters B and � and definitely returns the wrong result when re-interpreted as Windows-1252:
System.out.println(new String(utf8Again, "Windows-1252");
This line produces the output "B�", which is totally wrong (it is also the same output that would be the result if the initial array contained the non-word "Bür" instead).
So in this case you can't undo the operation, because some information was lost.
There are in fact cases where such mis-encodings can be undone. It's more likely to work, when all possible (or at least occuring) byte sequences are valid in that encoding. Since UTF-8 has several byte sequences that are simply not valid values, you will have problems.

I tried this and it worked for some reason
Code to repair encoding problem (it doesn't work perfectly, which we will see shortly):
final Charset fromCharset = Charset.forName("windows-1252");
final Charset toCharset = Charset.forName("UTF-8");
String fixed = new String(input.getBytes(fromCharset), toCharset);
System.out.println(input);
System.out.println(fixed);
The results are:
input: …Und ich beweg mich (aber heut nur langsam)
fixed: …Und ich beweg mich (aber heut nur langsam)
Here's another example:
input: Waun da wuan ned wa (feat. Wolfgang Kühn)
fixed: Waun da wuan ned wa (feat. Wolfgang Kühn)
Here's what is happening and why the trick above seems to work:
The original file was a UTF-8 encoded text file (comma delimited)
That file was imported with Excel BUT the user mistakenly entered Windows 1252 for the encoding (which was probably the default encoding on his or her computer)
The user thought the import was successful because all of the characters in the ASCII range looked okay.
Now, when we try to "reverse" the process, here is what happens:
// we start with this garbage, two characters we don't want!
String input = "ü";
final Charset cp1252 = Charset.forName("windows-1252");
final Charset utf8 = Charset.forName("UTF-8");
// lets convert it to bytes in windows-1252:
// this gives you 2 bytes: c3 bc
// "Ã" ==> c3
// "¼" ==> bc
bytes[] windows1252Bytes = input.getBytes(cp1252);
// but in utf-8, c3 bc is "ü"
String fixed = new String(windows1252Bytes, utf8);
System.out.println(input);
System.out.println(fixed);
The encoding fixing code above kind of works but fails for the following characters:
(Assuming the only characters used 1 byte characters from Windows 1252):
char utf-8 bytes | string decoded as cp1252 --> as cp1252 bytes
” e2 80 9d | â€� e2 80 3f
Á c3 81 | Ã� c3 3f
Í c3 8d | Ã� c3 3f
Ï c3 8f | Ã� c3 3f
Рc3 90 | � c3 3f
Ý c3 9d | Ã� c3 3f
It does work for some of the characters, e.g. these:
Þ c3 9e | Þ c3 9e Þ
ß c3 9f | ß c3 9f ß
à c3 a0 | à c3 a0 à
á c3 a1 | á c3 a1 á
â c3 a2 | â c3 a2 â
ã c3 a3 | ã c3 a3 ã
ä c3 a4 | ä c3 a4 ä
å c3 a5 | Ã¥ c3 a5 å
æ c3 a6 | æ c3 a6 æ
ç c3 a7 | ç c3 a7 ç
NOTE - I originally thought this was relevant to your question (and as I was working on the same thing myself I figured I'd share what I've learned), but it seems my problem was slightly different. Maybe this will help someone else.

What you want to do is impossible. Once you have a Java String, the information about the byte array is lost. You may have luck doing a "manual conversion". Create a list of all windows-1252 characters and their mapping to UTF-8. Then iterate over all characters in the string to convert them to the right encoding.
Edit:
As a commenter said this won't work. When you convert a Windows-1252 byte array as it if was UTF-8 you are bound to get encoding exceptions. (See here and here).

You can use this tutorial
The charset you need should be defined in rt.jar (according to this)