We Keep Coding

Java Serial read \u0002 how to remove?

I am using RFID reader ID-12LA and library for Java RxTx.
Loading data from reader but data: "\u000267009CB3541C"
How do I remove \u0002? Card ID is 67009CB3541C
System.out.print is 67009CB3541C
BufferedReader input = new BufferedReader(new InputStreamReader(port.getInputStream()));
port.addEventListener(event -> {
if (event.getEventType() == SerialPortEvent.DATA_AVAILABLE) {
try {
String inputLine = input.readLine();
inputLine.replace("\"\\u0002\"", "");
System.out.println("Read data: " + inputLine);
}
catch (IOException | URISyntaxException e) {
System.err.println(e.toString());
}
});
I need to get a String that represents the card code.
I need a card number reader and then allow access.

I don’t know the protocol used by that RFID reader, but it looks like it is not safe to use a java.io.Reader. If you read raw bytes into a String, you risk corrupting data when it is encoded using a charset.
It appears the device sends back a response byte (02 in this case), followed by ASCII bytes representing the card ID. So, avoid using InputStreamReader; instead, read the first byte, then read bytes until you encounter a newline and convert them to a String. (Do not omit the charset when converting—you do not want to rely on the system’s default charset!)
InputStream input = port.getInputStream();
int code = input.read();
if (code != 2) {
throw new IOException("Reader did not return expected code 2.");
}
ByteArrayOutputStream idBuffer = new ByteArrayOutputStream();
int b;
while ((b = input.read()) >= 0 && b != '\r' && b != '\n') {
idBuffer.write(b);
}
String cardID = idBuffer.toString(StandardCharsets.UTF_8);

Then you can indeed replace it as follows:
inputLine = inputLine.replace("\u0002", "");
Note the \u0002 syntax that represents one character.
or, if you're certain it is always the first character:
inputLine = inputLine.substring(1);

system.out redirection to jfx fextfield not showing scandinavian characters

I've been scratching my head for a few days now trying to figure out how to correctly encode scandinavian characters for use in a text field
These three characters: Æ Ø Å ( æ ø å) show up as ���.
meaning that the running this code
System.out.println("øst");
prints"�st"
. and i have no idea why.
Here's the code where i redirect the System.out to a printstream.
System.setProperty("user.language", "da");
OutputStream out = new OutputStream() {
#Override
public void write(int b) throws IOException {
appendConsole(new String(new byte[]{(byte)b}, "UTF-8"));
}
};
PrintStream ps;
try {
ps = new PrintStream(out, true, "UTF-8");
System.setOut(ps);
} catch (UnsupportedEncodingException ex) {
Logger.getLogger(GameController.class.getName()).log(Level.SEVERE, null, ex);
}
If anyone has a solution to this problem, it is greatly appreciated!

Try to not cast the int to a byte to a string, but to a char. Does that work?

When those characters are encoded as UTF-8, they each require two bytes. For example, Æ in UTF-8 is { (byte) 0xc3, (byte) 0x86 }. You cannot construct a String from just one of the bytes; both bytes are required to form a valid UTF-8 sequence.
You'll need to accumulate bytes until you have enough to form a full UTF-8 sequence, then create a String from that. ByteBuffer and CharsetDecoder are made for that:
// A UTF-8 sequence for a single character is at most 4 bytes long.
private final ByteBuffer buffer = ByteBuffer.allocate(4);
private final CharsetDecoder decoder = StandardCharsets.UTF_8.newDecoder();
#Override
public void write(int b)
throws IOException {
buffer.put((byte) b);
int size = buffer.position();
int first = buffer.get(0) & 0xff;
if (size == 4 ||
(first >= 0xe0 && size == 3) ||
(first >= 0xc0 && size == 2) ||
first < 0x80) {
buffer.flip();
appendConsole(decoder.decode(buffer).toString());
buffer.clear();
}
}

How to read binary file to binary characters java

I have a binary file that I need to read and save as characters or a string of 0's and 1's in the same order that they are in the binary file. I am currently able to read in the binary file, but am unable to obtain the 0's and 1's. Here is the code I am currently using:
public void read()
{
try
{
byte[] buffer = new byte[(int)infile.length()];
FileInputStream inputStream = new FileInputStream(infile);
int total = 0;
int nRead = 0;
while((nRead = inputStream.read(buffer)) != -1)
{
System.out.println(new String(buffer));
total += nRead;
}
inputStream.close();
System.out.println(total);
}
catch(FileNotFoundException ex)
{
System.out.println("File not found.");
}
catch(IOException ex)
{
System.out.println(ex);
}
}
and the output from running this with the binary file:
�, �¨Ã �¨ÊÃ
�!Cˇ¯åaÃ!Dˇ¸åÇÃ�"( ≠EÃ!J�H���û�������
����������������������������������������������������������������������������������������
156
Thanks for any help you can give.

Check out String to binary output in Java. Basically you need to take your String, convert it to a byte array, and print out each byte as a binary string.

Instead of converting the bytes directly into characters and then printing them, convert each byte into a binary string and print them out. In other words, replace
System.out.println(new String(buffer));
with
for (int i = 0; i<nRead; i++) {
String bin=Integer.toBinaryString(0xFF & buffer[i] | 0x100).substring(1);
System.out.println(bin);
}
Notice though that the bits of each byte are printed in big-endian order. There is no way to know if bits are actually stored in this order on disk.

with JBBP such operation will be very easy
public static final void main(final String ... args) throws Exception {
try (InputStream inStream = ClassLoader.getSystemClassLoader().getResourceAsStream("somefile.txt")) {
class Bits { #Bin(type = BinType.BIT_ARRAY) byte [] bits; }
for(final byte b : JBBPParser.prepare("bit [_] bits;",JBBPBitOrder.MSB0).parse(inStream).mapTo(Bits.class).bits)
System.out.print(b != 0 ? "1" : "0");
}
}
But it will not be working with huge files because parsed data will be cached in memory during operatio

Even though this response is in C, you can use the JNI to access it natively from a Java program.
Since they are in a binary format, you will not be able to read it. I would do it like this.
fstream fs;
int value; //Since you are reading bytes, change accordingly.
fs.open( fileName, is.in | is.binary );
fs.read((char *) &value, sizeof(int));
while(!fs.eof())
{
//Print or do something with value
fs.read((char *) &value, sizeof(long));
}

Java: Reading file in two parts - partly as String and partly as byte[]

I have a file which is split in two parts by "\n\n" - first part is not too long String and second is byte array, which can be quite long.
I am trying to read the file as follows:
byte[] result;
try (final FileInputStream fis = new FileInputStream(file)) {
final InputStreamReader isr = new InputStreamReader(fis);
final BufferedReader reader = new BufferedReader(isr);
String line;
// reading until \n\n
while (!(line = reader.readLine()).trim().isEmpty()){
// processing the line
}
// copying the rest of the byte array
result = IOUtils.toByteArray(reader);
reader.close();
}
Even though the resulting array is the size it should be, its contents are broken. If I try to use toByteArray directly on fis or isr, the contents of result are empty.
How can I read the rest of the file correctly and efficiently?
Thanks!

The reason your contents are broken is because the IOUtils.toByteArray(...) function reads your data as a string in the default character encoding, i.e. it converts the 8-bit binary values into text characters using whatever logic your default encoding prescribes. This usually leads to many of the binary values getting corrupted.
Depending on how exactly the charset is implemented, there is a slight chance that this might work:
result = IOUtils.toByteArray(reader, "ISO-8859-1");
ISO-8859-1 uses only a single byte per character. Not all character values are defined, but many implementations will pass them anyways. Maybe you're lucky with it.
But a much cleaner solution would be to instead read the String in the beginning as binary data first and then converting it to text via new String(bytes) rather than reading the binary data at the end as a String and then converting it back.
This might mean, though, that you need to implement your own version of a BufferedReader for performance purposes.
You can find the source code of the standard BufferedReader via the obvious Google search, which will (for example) lead you here:
http://www.docjar.com/html/api/java/io/BufferedReader.java.html
It's a bit long, but conceptually not too difficult to understand, so hopefully it will be useful as a reference.

Alternatively, you could read the file into byte array, find \n\n position and split the array into the line and bytes
byte[] a = Files.readAllBytes(Paths.get("file"));
String line = "";
byte[] result = a;
for (int i = 0; i < a.length - 1; i++) {
if (a[i] == '\n' && a[i + 1] == '\n') {
line = new String(a, 0, i);
int len = a.length - i - 1;
result = new byte[len];
System.arraycopy(a, i + 1, result, 0, len);
break;
}
}

Thanks for all the comments - the final implementation was done in this way:
try (final FileInputStream fis = new FileInputStream(file)) {
ByteBuffer buffer = ByteBuffer.allocate(64);
boolean wasLast = false;
String headerValue = null, headerKey = null;
byte[] result = null;
while (true) {
byte current = (byte) fis.read();
if (current == '\n') {
if (wasLast) {
// this is \n\n
break;
} else {
// just a new line in header
wasLast = true;
headerValue = new String(buffer.array(), 0, buffer.position()));
buffer.clear();
}
} else if (current == '\t') {
// headerKey\theaderValue\n
headerKey = new String(buffer.array(), 0, buffer.position());
buffer.clear();
} else {
buffer.put(current);
wasLast = false;
}
}
// reading the rest
result = IOUtils.toByteArray(fis);
}

How SHA-1 hashing works on Java on Android

I searched around for how to hash device identifiers and stumbled on the following code.
I don't really understand what it's doing.
Why do I need to urlEncode the device id ?
Why do I need to hash the bytes, couldn't I just do that on a String ?
Why do I need to convert it to a BigInteger ?
Why do I need to shift bits to get a String with the hashed id ?
Can anyone explain what's going on line by line? I hope this will help other people understand this snippet that's getting passed around in blogs and forums, too.
String hashedId = "";
String deviceId = urlEncode(Secure.getString(context.getContentResolver(), Secure.ANDROID_ID));
try {
MessageDigest digest = MessageDigest.getInstance("SHA-1");
byte bytes[] = digest.digest(deviceId.getBytes());
BigInteger b = new BigInteger(1, bytes);
hashedId = String.format("%0" + (bytes.length << 1) + "x", b);
} catch (NoSuchAlgorithmException e) {
//ignored
}
return hashedId;

Why do I need to urlEncode the device id ?
Why do I need to hash the bytes, couldn't I just do that on a String ?
Most hashing algorithms, including SHA-1, work on binary data as input (i.e. bytes). Strings themselves don't have a specific binary representation; it changes depending on the encoding.
The line of code they provide uses the default encoding, which is a bit fragile. I would prefer to see something like
byte bytes[] = digest.digest(deviceId.getBytes(Charset.forName("UTF-8")));
Why do I need to convert it to a BigInteger ?
This is being used for convenience to help with the conversion to a hexadecimal representation.
Why do I need to shift bits to get a String with the hashed id ?
The format String being used is %0Nx, which causes the string to be zero-padded to N characters. Since it takes two characters to represent a byte in hexadecimal, N is bytes*2, which is the result as bytes << 1.
I don't really understand why you wouldn't just include Guava for Android and use the Hashing builder:
String hash = Hashing.sha1().hashString(deviceId, Charsets.UTF_8).toString();
It's one line and doesn't throw checked exceptions.

About the bit-shifting: shifting left by one is equivalent to multiplying by 2. Each byte in the string is represented by 2 hex characters, so the resulting string will be twice as long as the number of bytes in the hash.
This will create a format string that looks something like %032x, which will print an integral value as a zero-padded 32-character string.

You need to hash the bytes, rather than the String, so that you're hashing the character data rather than the String object, which may have unpredictable internal state for a given sequence of characters.
It's converted to BigInteger so it can be consistently formatted with two hex digits per byte. (This is why the length is multiplied by two with the left shift.)
Basically, the answer to all of your questions is: so that you get reliable, repeatable results, even on different platforms.

You Can Use this Code also :
public class sha1Calculate {
public static void main(String[] args)throws Exception
{
File file = new File("D:\\Android Links.txt");
String outputTxt= "";
String hashcode = null;
try {
FileInputStream input = new FileInputStream(file);
ByteArrayOutputStream output = new ByteArrayOutputStream ();
byte [] buffer = new byte [65536];
int l;
while ((l = input.read (buffer)) > 0)
output.write (buffer, 0, l);
input.close ();
output.close ();
byte [] data = output.toByteArray ();
MessageDigest digest = MessageDigest.getInstance( "SHA-1" );
byte[] bytes = data;
digest.update(bytes, 0, bytes.length);
bytes = digest.digest();
StringBuilder sb = new StringBuilder();
for( byte b : bytes )
{
sb.append( String.format("%02X", b) );
}
System.out.println("Digest(in hex format):: " + sb.toString());
}catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (NoSuchAlgorithmException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}