We Keep Coding

How to consume messages from kafka topics using protocol buffers (protobuf) by spring cloud stream binder kafka streams dependency?

I have an application that consumes messages in a protobuf format and when I run it I am getting this error:
Exception in thread "NotificationProcessorService-process-applicationId-0300a3f8-6dab-4f3f-a631-8719178823ce-StreamThread-1" org.apache.kafka.streams.errors.StreamsException: Deserialization exception handler is set to fail upon a deserialization error. If you would rather have the streaming pipeline continue after a deserialization error, please set the default.deserialization.exception.handler appropriately.
at org.apache.kafka.streams.processor.internals.RecordDeserializer.deserialize(RecordDeserializer.java:80)
at org.apache.kafka.streams.processor.internals.RecordQueue.updateHead(RecordQueue.java:176)
at org.apache.kafka.streams.processor.internals.RecordQueue.addRawRecords(RecordQueue.java:112)
at org.apache.kafka.streams.processor.internals.PartitionGroup.addRawRecords(PartitionGroup.java:185)
at org.apache.kafka.streams.processor.internals.StreamTask.addRecords(StreamTask.java:865)
at org.apache.kafka.streams.processor.internals.TaskManager.addRecordsToTasks(TaskManager.java:938)
at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:640)
at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:551)
at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:510)
Caused by: org.apache.kafka.common.errors.SerializationException: Can't deserialize data [[0, 0, 0, 0, 5, 0, 10, 8, 57, 53, 52, 50, 56, 51, 51, 51, 16, -7, -12, -106, -97, -119, 47, 26, 6, 57, 56, 55, 56, 54, 55, 34, 4, 56, 54, 50, 50, 42, 6, 56, 57, 55, 51, 50, 57, 50, 5, 80, 82, 73, 77, 69, 58, 5, 56, 55, 57, 50, 51, 65, 31, -123, -21, 81, -72, 93, -108, 64, 72, 2, 82, 6, 67, 82, 69, 68, 73, 84, 89, 31, -123, -21, 81, -72, 93, -108, 64, 97, -26, -48, 34, -37, -7, 74, 64, 64, 105, -26, -48, 34, -37, -7, 74, 64, 64, 113, -26, -48, 34, -37, -7, 74, 64, 64, 122, 4, 77, 65, 73, 76]] from topic [pos-proto-topic]
Caused by: java.io.CharConversionException: Invalid UTF-32 character 0x4ff0a08 (above 0x0010ffff) at char #1, byte #7)
at com.fasterxml.jackson.core.io.UTF32Reader.reportInvalid(UTF32Reader.java:195)
at com.fasterxml.jackson.core.io.UTF32Reader.read(UTF32Reader.java:158)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._loadMore(ReaderBasedJsonParser.java:250)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._skipWSOrEnd(ReaderBasedJsonParser.java:2384)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:672)
at com.fasterxml.jackson.databind.ObjectReader._initForReading(ObjectReader.java:357)
at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2064)
at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1555)
at org.springframework.kafka.support.serializer.JsonDeserializer.deserialize(JsonDeserializer.java:517)
at org.apache.kafka.streams.processor.internals.SourceNode.deserializeValue(SourceNode.java:55)
at org.apache.kafka.streams.processor.internals.RecordDeserializer.deserialize(RecordDeserializer.java:66)
at org.apache.kafka.streams.processor.internals.RecordQueue.updateHead(RecordQueue.java:176)
at org.apache.kafka.streams.processor.internals.RecordQueue.addRawRecords(RecordQueue.java:112)
at org.apache.kafka.streams.processor.internals.PartitionGroup.addRawRecords(PartitionGroup.java:185)
at org.apache.kafka.streams.processor.internals.StreamTask.addRecords(StreamTask.java:865)
at org.apache.kafka.streams.processor.internals.TaskManager.addRecordsToTasks(TaskManager.java:938)
at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:640)
at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:551)
at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:510)
I think the application expects a Json message by default and I need to change some configuration saying "hey, I am waiting for protobuf messages here". I feel I searched by the whole internet and didn't find how to set it.
Here is my application.yaml file:
spring:
cloud:
stream:
bindings:
notification-input-channel:
destination: pos-proto-topic
notification-output-channel:
destination: notification-topic
kafka:
streams:
binder:
brokers: localhost:9092
configuration:
schema.registry.url: http://localhost:8081
bindings:
notification-output-channel:
producer:
valueSerde: io.confluent.kafka.streams.serdes.protobuf.KafkaProtobufSerde
I am also using the Hoxton.SR9 as spring-cloud.version. Does anyone know how to solve this?

You need to set:
spring.cloud.stream.kafka.streams.bindings.<channel-name>-in-0.consumer.valueSerde

How to send multiple array with array of objects send to android

I am using this code this format shows Invalid JSON Please tell me any code of android
Format:-->
{
"supplyType":"O",
"subSupplyType":"1",
"subSupplyDesc":"TESTDESCRIPTION",
"docType": "BIL",
"docNo": "18-86"
"itemList": [{
"productName": "Wheat",
"productDesc": "Wheat",
"hsnCode": 84,
"quantity": 10,
"qtyUnit": "BOX",
"cgstRate": 0,
"sgstRate": 0,
"igstRate": 12,
"cessRate": 0,
"cessAdvol": 0,
"taxableAmount":5609889
},
{
"productName": "Wheat",
"productDesc": "Wheat",
"hsnCode": 84,
"quantity": 10,
"qtyUnit": "BOX",
"cgstRate": 0,
"sgstRate": 0,
"igstRate": 12,
"cessRate": 0,
"cessAdvol": 0,
"taxableAmount":5609889
}]
}

Check this -
{
"supplyType":"O",
"subSupplyType":"1",
"subSupplyDesc":"TESTDESCRIPTION",
"docType": "BIL",
"docNo": "18-86",
"itemList": [
{
"productName": "Wheat",
"productDesc": "Wheat",
"hsnCode": 84,
"quantity": 10,
"qtyUnit": "BOX",
"cgstRate": 0,
"sgstRate": 0,
"igstRate": 12,
"cessRate": 0,
"cessAdvol": 0,
"taxableAmount":5609889
},
{
"productName": "Wheat",
"productDesc": "Wheat",
"hsnCode": 84,
"quantity": 10,
"qtyUnit": "BOX",
"cgstRate": 0,
"sgstRate": 0,
"igstRate": 12,
"cessRate": 0,
"cessAdvol": 0,
"taxableAmount":5609889
}
]
}

Why is org.mindrot.JBCrypt saying Bad salt length here?

Example is worth a thousand words, hopefully. In case it doesn't, here're a couple tests to hash the plainText hello world using a salt seed static seed to be used in the first test and static seed to be usedd in the second test.
Salt seed is used to generate a static salt that can be passed into BCrypt.hashpw(plainText, staticSalt) function. As you can see, the salt bytes, and salt byte strings are similar length but it is throwing an error. I know it's bad but I have my reasons for static salt, so please put your focus on the question.
org.mindrot.jbcrypt.BCrypt with JDK1.7 Test 1 - PlainText: "hello world", saltseed: "static seed to be used":
Salt bytes for "static seed to be used": [-30, -8, 86, -8, 6, -126, -64, -30, -82, -82, -104, -64, -8, -118, -64, 108, -82, -64, 14, -30, -82, -104]
Salt bytes string: 4vhW+AaCwOKurpjA+IrAbK7ADuKumA==, length: 32
complete salt: $2a$12$4vhW+AaCwOKurpjA+IrAbK7ADuKumA==
Exception in thread "main" java.lang.IllegalArgumentException: Bad salt length
at org.mindrot.jbcrypt.BCrypt.crypt_raw(BCrypt.java:619)
at org.mindrot.jbcrypt.BCrypt.hashpw(BCrypt.java:684)
org.springframework.security.crypto.bcrypt.BCrypt with JDK1.8 Test 1 - PlainText: "hello world", saltseed: "static seed to be used":
Salt bytes for "static seed to be used": [-30, -8, 86, -8, 6, -126, -64, -30, -82, -82, -104, -64, -8, -118, -64, 108, -82, -64, 14, -30, -82, -104]
Salt bytes string: 4vhW+AaCwOKurpjA+IrAbK7ADuKumA==, length: 32
complete salt: $2a$12$4vhW+AaCwOKurpjA+IrAbK7ADuKumA==
Plain text: hello world, Hash text: $2a$12$4vhWHrTxEMtyyv6wmpOtX.YYbTqHwHv/dxe
org.mindrot.jbcrypt.BCrypt with JDK1.7 Test 2 - PlainText: "hello world", saltseed: "static seed to be usedd":
Salt bytes for "static seed to be usedd": [85, 108, -73, 108, 111, -27, -32, 85, 19, 19, -4, -32, 108, -7, -32, -50, 19, -32, -125, 85, 19, -4]
Salt bytes string: VWy3bG/l4FUTE/zgbPngzhPgg1UT/A==, length: 32
complete salt: $2a$12$VWy3bG/l4FUTE/zgbPngzhPgg1UT/A==
Plain text: hello world, Hash text: $2a$12$VWy3bG/l4FUTE/zgbPngze9KDSXjF72NBMBNE6ZJk4StahyAhykgO
org.springframework.security.crypto.bcrypt.BCrypt with JDK1.8 Test 2 - PlainText: "hello world", saltseed: "static seed to be usedd":
Salt bytes for "static seed to be usedd": [85, 108, -73, 108, 111, -27, -32, 85, 19, 19, -4, -32, 108, -7, -32, -50, 19, -32, -125, 85, 19, -4]
Salt bytes string: VWy3bG/l4FUTE/zgbPngzhPgg1UT/A==, length: 32
complete salt: $2a$12$VWy3bG/l4FUTE/zgbPngzhPgg1UT/A==
Plain text: hello world, Hash text: $2a$12$VWy3bG/l4FUTE/zgbPngze9KDSXjF72NBMBNE6ZJk4StahyAhykgO
I've tried adding and deleting more letters and got successful hashes. I was kind of happy that the first String used in the JUnit test threw an error.
Thanks in advance.

I found the reason by looking at its implementation on a GitHub page...
This implementation of BCrypt supports base64 dot(.) character instead of the standard plus(+) character.
static private final byte index_64[] = {
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, 0, 1, 54, 55,
56, 57, 58, 59, 60, 61, 62, 63, -1, -1,
-1, -1, -1, -1, -1, 2, 3, 4, 5, 6,
7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27,
-1, -1, -1, -1, -1, -1, 28, 29, 30,
31, 32, 33, 34, 35, 36, 37, 38, 39, 40,
41, 42, 43, 44, 45, 46, 47, 48, 49, 50,
51, 52, 53, -1, -1, -1, -1, -1
};
"+" character has integer value of 43, thus returns a -1 from this array, and the function that is retrieving the salt bytes from salt breaks earlier leaving me with a 4 bytes salts. Even its implementation says it doesn't support standard base64 encoding string:
/**
* Decode a string encoded using bcrypt's base64 scheme to a
* byte array. Note that this is *not* compatible with
* the standard MIME-base64 encoding.
*/
static byte[] decode_base64(String s, int maxolen)
throws IllegalArgumentException {
Switching from . to + gives me a different hashed from Spring on salts containing +. Not that familiar with shift and & to make the changes so going to look for the Spring-Security's implementation instead.
EDIT: Just took a look at Spring's implementation. No wonder it didn't work... It is exactly the same except Spring continues to hash with a 4 bytes salt while jbcrypt throws an error, thus there's a bug in Spring whereas if you generate your own hash and it contains +, then the checkpw(plainText, hashedText) will return false because the generated salt part of the hashtedText is different from the user generated salt.

How do I print out string values stored in an Array?

I am using Java on NetBeans 8. I have searched google and other stack overflow questions but I cannot find anything that relates to my question. Also, I have consulted the oracle documentation and their example gave me the same error I am experiencing.
I would like the array string values below to print out to the output box using a line of code like:
System.out.println(pbArray[0][0] + pbArray[1][0]);
////////////////////////////////////////////////
But no matter where I try to input that line of code I cannot make it work. NetBeans tells me there is "an identifier expected."
package com.mnlottery.console;
public class ComMnlotteryConsole {
public static void main(String[] args) {
class powerball{
String[][] pbArray = {
{"August 16th, 2014","August 13th, 2014","August 9th, 2014"},
{"07, 08, 17, 48, 69, 09, $50,000,000","08, 37, 39, 40, 52, 24,"
+ " $40,000,000","03, 12, 31, 34, 51, 24, 90,000,000"}};
}
}
}
I feel like this should be fairly simple. Thanks in advance.

I think you wanted (no inner class)
public static void main(String[] args) {
String[][] pbArray = {
{ "August 16th, 2014", "August 13th, 2014", "August 9th, 2014" },
{ "07, 08, 17, 48, 69, 09, $50,000,000",
"08, 37, 39, 40, 52, 24," + " $40,000,000",
"03, 12, 31, 34, 51, 24, 90,000,000" } };
System.out.println(pbArray[0][0] + pbArray[1][0]);
}
But, you can print the array(s) in a few ways,
Arrays.deepToString(Object[])
A loop, and Arrays.toString(Object[])
like
String[][] pbArray = {
{ "August 16th, 2014", "August 13th, 2014", "August 9th, 2014" },
{ "07, 08, 17, 48, 69, 09, $50,000,000",
"08, 37, 39, 40, 52, 24," + " $40,000,000",
"03, 12, 31, 34, 51, 24, 90,000,000" } };
// 1.
System.out.println(Arrays.deepToString(pbArray));
// or 2.
for (String [] arr : pbArray) {
System.out.println(Arrays.toString(arr));
}

Change System.out.println(pbarray[0][0] + names[1][0]); to
System.out.println(pbArray[0][0] + pbArray[1][0]);
names and pbarray do not exist. pbArray is the correct identifier.

How to decode a Base64 string in Scala or Java?

I have a string encoded in Base64:
eJx9xEERACAIBMBKJyKDcTzR_hEsgOxjAcBQFVVNvi3qEsrRnWXwbhHOmzWnctPHPVkPu-4vBQ==
How can I decode it in Scala language?
I tried to use:
val bytes1 = new sun.misc.BASE64Decoder().decodeBuffer(compressed_code_string)
But when I compare the byte array with the correct one that I generated in Python language, there is an error. Here is the command I used in python:
import base64
base64.urlsafe_b64decode(compressed_code_string)
The Byte Array in Scala is:
(120, -100, 125, -60, 65, 17, 0, 32, 8, 4, -64, 74, 39, 34, -125, 113, 60, -47, -2, 17, 44, -128, -20, 99, 1, -64, 80, 21, 85, 77, -66, 45, -22, 18, -54, -47, -99, 101, -16, 110, 17, -50, -101, 53, -89, 114, -45, -57, 61, 89, 15, -69, -2, 47, 5)
And the one generated in python is:
(120, -100, 125, -60, 65, 17, 0, 32, 8, 4, -64, 74, 39, 34, -125, 113, 60, -47, -2, 17, 44, -128, -20, 99, 1, -64, 80, 21, 85, 77, -66, 45, -22, 18, -54, -47, -99, 101, -16, 110, 17, -50, -101, 53, -89, 114, -45, -57, 61, 89, 15, -69, -18, 47, 5)
Note that there is a single difference in the end of the array

In Scala, Encoding a String to Base64 and decoding back to the original String using Java APIs:
import java.util.Base64
import java.nio.charset.StandardCharsets
scala> val bytes = "foo".getBytes(StandardCharsets.UTF_8)
bytes: Array[Byte] = Array(102, 111, 111)
scala> val encoded = Base64.getEncoder().encodeToString(bytes)
encoded: String = Zm9v
scala> val decoded = Base64.getDecoder().decode(encoded)
decoded: Array[Byte] = Array(102, 111, 111)
scala> val str = new String(decoded, StandardCharsets.UTF_8)
str: String = foo

There is unfortunately not just one Base64 encoding. The - character doesn't have the same representation in all encodings. For example, in the MIME encoding, it's not used at all. In the encoding for URLs, it is a value of 62--and this is the one that Python is using. The default sun.misc decoder wants + for 62. If you change the - to +, you get the correct answer (i.e. the Python answer).
In Scala, you can convert the string s to MIME format like so:
s.map{ case '-' => '+'; case '_' => '/'; case c => c }
and then the Java MIME decoder will work.

Both Python and Java are correct in terms of the decoding. They are just using a different RFC for this purpose. Python library is using RFC 3548 and the used java library is using RFC 4648 and RFC 2045.
Changing the hyphen(-) into a plus(+) from your input string will make the both decoded byte data are similar.