We Keep Coding

[Hyperledger Fabric][Kotlin] Passing client request parameters as ByteArray to chaincode

I've been recently working with Hyperledger Fabric using kotlin as a development language. I have the local test network fully working and after some time i have pretty good understanding on how everything works. However, im still stuck passing arguments to the chaincode as Strings because every time i try to pass objects as ByteArrays it implodes somehow along the way before it reaches my chaincode method. Here is a sample of the client sending the data and the chaincode entrypoint:
Client
val tl = MyClass()
tl.machineKey = "machineHashKey"
val path = File("/var/cenas.txt")
tl.loadFileFromPath(path.toPath())
try {
contract!!.submitTransaction(
"testLimitSet", tl.toByteArray()
)
}catch (e: Exception){
printException(e)
}finally {
println("*** Transaction completed")
}
ChainCode:
#Transaction(intent = Transaction.TYPE.SUBMIT)
fun testLimitSet(ctx: Context, testLimit: ByteArray){
println("Here 1!")
val id = IdentityChecks.checkIdentity(ctx)
println("Here 2!")
val x509 = id.second
println("Here 3!")
}
MyClass simply contains 2 text fields and 1 byteArray.
I've already managed to serialize it and deserialize it locally.
The log on the chaincode machine is the following:
Thread[fabric-txinvoke:3,5,main] 16:39:28:613 INFO org.hyperledger.fabric.contract.ContractRouter processRequest Got routing:testLimitSet:pt.fraunhofer.newgen.chaincode.NewGenMainContract
Thread[fabric-txinvoke:3,5,main] 16:39:28:625 SEVERE org.hyperledger.fabric.Logger error A JSONArray text must start with '[' at 1 [character 2 line 1]org.json.JSONException: A JSONArray text must start with '[' at 1 [character 2 line 1]
at org.json.JSONTokener.syntaxError(JSONTokener.java:507)
at org.json.JSONArray.<init>(JSONArray.java:109)
at org.json.JSONArray.<init>(JSONArray.java:162)
at org.hyperledger.fabric.contract.execution.JSONTransactionSerializer.convert(JSONTransactionSerializer.java:267)
at org.hyperledger.fabric.contract.execution.JSONTransactionSerializer.fromBuffer(JSONTransactionSerializer.java:165)
at org.hyperledger.fabric.contract.execution.impl.ContractExecutionService.convertArgs(ContractExecutionService.java:99)
at org.hyperledger.fabric.contract.execution.impl.ContractExecutionService.executeRequest(ContractExecutionService.java:57)
at org.hyperledger.fabric.contract.ContractRouter.processRequest(ContractRouter.java:123)
at org.hyperledger.fabric.contract.ContractRouter.invoke(ContractRouter.java:134)
at org.hyperledger.fabric.shim.impl.ChaincodeInvocationTask.call(ChaincodeInvocationTask.java:106)
at org.hyperledger.fabric.shim.impl.InvocationTaskManager.lambda$newTask$17(InvocationTaskManager.java:265)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
I think im most likely using the ByteArray arguments improperly. But i haven't found any chaincode examples in java/kotlin receiving data as ByteArrays.
Thanks in advance

When using the contract api in your chaincode implementation it uses JSON as the serialisation mechanism and thus expects information sent from the client to be in stringified JSON format.
The client sdks don't do this for you as for example you can write chaincode without the contract api and define your own serialiation mechanism.
With the line
contract!!.submitTransaction(
"testLimitSet", tl.toByteArray()
)
you are sending an array of bytes as the string which won't mean anything to a JSON Parser. You need to convert tl.toByteArray() into a stringified JSON representation of that data, for example a stringified JSON representation of an array with bytes 01,02,03 might look like
'["\u0001","\u0002","\u0003"]'
However I am not really familiar with the java chaincode libraries so whether this will actually work I don't know given that the JSON representation allows for values larger that bytes and that may cause the contract api to deserialize the JSON into something that isn't type compatible with what you declare on your chaincode ie testLimit: ByteArray you may need to change the type or perhaps send the binary data in a different manner, for example encode it as a base64 string then send a string instead

I've solved the problem by creating a string representation of the ByteArray. It is not the solution i originally wanted, but it works well nevertheless. I still find it strange to the SDK to provide this method:
byte[] submitTransaction(String name, byte[]... args)
and im not able to use it because it doesn't reach the chaincode endpoint with the problem described above.
My solution uses this instead:
byte[] submitTransaction(String name, String[]... args)
Here is my current solution:
val tl = MyClass()
tl.machineKey = "machineHashKey"
val path = File("/var/cenas.txt")
tl.loadFileFromPath(path.toPath())
try {
val payload = tl.toByteArrayString()
contract!!.submitTransaction(
"testLimitSet", payload
)
}catch (e: Exception){
printException(e)
}finally {
println("*** Transaction completed")
}
Where:
fun toByteArrayString(): String {
return toHexString(this.toByteArray())
}
// String <- Hex(ByteArray)
fun toHexString(ba: ByteArray): String {
return BaseEncoding.base16().encode(ba)
}
// String -> Hex(ByteArray)
fun toByteArray(s: String): ByteArray {
return BaseEncoding.base16().decode(s)
}
Note that this is using
import com.google.common.io.BaseEncoding
but there are multiple implementations that can be used for this

Websphere MQ: reading from DLQ with JMS

I have to process messages in Dead Letter Queue (DLQ) using JMS API. The goal is to read body of the original messages and it's user properties. I realize that such approach to DLQ processing might be considered as bad design, but I have to deal with it anyway.
Once read with JMS, body of DLQ message contains body of the original one, prepended with DL header and a structure very similar to RFH2 header of the original message (so containing all the needed user properties).
The question is, how to parse these 2 structures in java?
Yet I only found a doc about how DLH could be constructed from raw data (https://www.ibm.com/support/knowledgecenter/SS8JB4/com.ibm.wbpm.main.doc/topics/esbprog_bindings_wmq5.html). But while DLH seems to be a fixed-lenght structure, RFH2 is definitely not - so the most tricky part of parsing is there.
Any idea would be appreciated.
UPDATE
Here is what I have found:
1) DLH was parsed from raw byte array without any problem, as simple as follows:
MQDLH rfh = new MQDLH(new DataInputStream(new ByteArrayInputStream(bytes)));
Once constructed, all the properties are available.
2) MQRFH2 could be created in a similar manner, if MQLONG values were written there as usual, in big endian. But for some reason, completely unclear to me, in this case all MQLONG are little endian.
So, to create MQRFH2 from raw bytes I have to reverse bytes for all MQLONGs. Not a problem for a fixed part (as described in https://www.ibm.com/support/knowledgecenter/SSFKSJ_7.5.0/com.ibm.mq.dev.doc/q032000_.htm), but a bit more complicated for variable part.
I haven't seen any confirmation in docs, but it seems that each folder in variable part is prepended with MQLONG (well, just 4-bytes integer) containing folder length. Once these values were converted from LE to BE as well, MQRFH2 seem to be working correctly.

I wouldn't process the DLQ with a JMS application. It will be so, so tricky and you will spend days or weeks trying to get it right. I would write a regular Java application to do it, far simpler.
i.e.
MQMessage rcvMsg = new MQMessage();
MQDLH dlh = new MQDLH(rcvMsg);
MQRFH2 rfh2 = new MQRFH2(rcvMsg);
byte[] bData = new byte[rcvMsg.getDataLength()];
rcvMsg.readFully(bData);
Updated on March 4, 2020.
I am normally not into banging my head against the wall but if you want to then here is the code that I would try:
ByteArrayInputStream bais = new ByteArrayInputStream(bytes);
DataInput di = new DataInputStream(bais);
MQDLH dlh = new MQDLH(di);
MQRFH2 rfh2 = new MQRFH2(di)
// Get all folders
String[] folderStrings = rfh2.getFolderStrings();
// or you can get individual name/values using
// get***FieldValue() methods of the MQRFH2 class.
/*
* At this point, the cursor for "di" is pointing
* to the beginning of the message payload and I
* would normal do:
*/
byte[] bData = new byte[mqMsg.getDataLength()];
mqMsg.readFully(bData);

Which charset should i use to decode this array of bytes in java?

I am currently working with Soap web services, and more precisely, recovering a file sent within.
It is working manually :
In SOAPUi, i do receive this (truncated for readability)
JVBERi0xLjQKJeLjz9MKMTIgMCBvY [...]
dL0luZm8gMTggMCBSL1NpemUgMTk+PgpzdGFydHhyZWYKNjk5OQolJUVPRgo=
I can paste this string within notepad++ and after clicking on MIME Tools > base 64 Decode, it become a proper PDF File as follows ( Truncated , just header is shown)
%PDF-1.4 %xE2xE3xCFxD3LF 12 0 obj <>stream
PDF File can be thus read without any problem.
Problem is now to recover this data using java.
I am receiving an array of byte (acopier variable in example below) and using the following code to store into a file.
I tried a couple of the numerous examples found on the net without any success.
Also tried to use UTF-8, ISO-8859-1 amongst others.
OutputStreamWriter osw = null;
try{
String filePath="c:\\temp\\";
filePath = filePath.concat("test.pdf");
FileOutputStream fos = new FileOutputStream(filePath,false);
osw = new OutputStreamWriter(fos,"UTF-8");
osw.write("\uFEFF");
osw.write(new String(acopier));
osw.close();
System.out.println("Success");
fos.close();
}
catch(Exception e)
{
System.out.println(e.getMessage());
osw.close();
}
Unfortunately, file can't be seen as a pdf file,
%PDF-1.4 %âãÏÓ 12 0 obj <>stream
When i tried to check what's within the array of bytes, console is showing me this : (truncated)
% P D F
- 1 . 4
% ? ? ? ?
1 2 0
I presume that windows or notepad++ or soapui is doing something in the background to guess what charset to use but i don't know with certitude which way to go.
Can please someone clarify me how to do it from scratch in java (meaning from the original array of bytes)?
Regards,
Pierre

Get the original (Base64) string data
Use your preferred Base64 decoder to turn it into bytes (plenty of them for Java)
Write bytes to file. As bytes, not as character data (i.e. no Writer class).
Since in your example you're trying to write binary data as character data (and using the String constructor), I assume you're quite new to Java?
Your mistake was converting base64 to binary data in notepad, then saving the result thinking that it would be valid binary data (which it almost most definitely isn't, and even if it did work, that's not the road you want to ).

Java equivalent of perl unpack function

I have a perl code (say client) which sends packed data as HTTP POST to another perl code running on apache mod_perl module (say server).
In client side, I have the pack function like this,
$postData = pack("N a*", length($metaData), $metaData);
From perl pack document, it seems,
N -> An unsigned long (32-bit) in "network" (big-endian) order.
a -> A string with arbitrary binary data, will be null padded.
Now the $postData will be sent to server using perl LWP User Agent.
In the server side perl, we used to unpack like this,
# first reading the metaData Length
my $buf;
$request->read($buf, 4); #$request is apache request handler
my $metaDataLength = unpack("N", $buf);
# now read the metaData itself
$request->read($buf, $metaDataLength);
Now I have to do this server side data parsing in java (moving away from perl for some reasons). I have searched google for this and it seems to be not a single line solution as in perl. Some suggested to write our own unpack function. I am using java 1.7 version.
Is there any simple solution available in java for the above server side data parsing ?
Edit: Thanks Elliot for 'ByteBuffer' idea. The following code works fine for me,
InputStream is = request.getInputStream(); //request is HTTPServletRequest
byte[] bArr = new byte[4]; //reading first 4 bytes to get metaDataLength
int bytesRead = is.read(bArr);
ByteBuffer buf = ByteBuffer.wrap(bArr);
int metaDataLength = buf.getInt(); //shows value matches with clientside perl code.

potentially JBBP can be such one
final int value = JBBPParser.prepare("int;").parse(theInputStream).findFieldForType(JBBPFieldInt.class).getAsInt();

coldfusion calculating HMAC256 of a getHTTPRequestData

I'm working with Shopify at the moment and using their webhook notifications so I can save stuff to our database.
Within their webhook headers, they provide a header of: X-Shopify-Hmac-Sha256
which is:
Each Webhook request includes a X-Shopify-Hmac-SHA256 header which is generated using the app's shared secret (looks like: '267bb1719a8e6ff75c4f2d709be0ca11'), along with the data sent in the request.
This is jolly wonderful; However, I'm really struggling to calculate the value of the X-Shopify-Hmac-Sha256.
I have a .cfm page which the webhook hits and I pass through the getHTTPRequestData to a function like thus:
<cfscript>
variables.stArgs = {};
variables.stArgs.stWebHookData = getHTTPRequestData();
application.stObj.stShopify.oShopifyWebHookBusiness.receiveWebHook(argumentCollection=variables.stArgs);
</cfscript>
I then stole some code from StackOverflow and Ben Nadel, but neither seem to end up giving me the value that I want. Currently I'm using Ben Nadels code like thus:
local.data = arguments.stWebHookData.toString();
local.macClass = createObject( "java", "javax.crypto.Mac" );
local.secretkeySpec = createObject( "java", "javax.crypto.spec.SecretKeySpec" ).init(toBinary(toBase64(arguments.key)),'HmacSHA256');
local.mac = local.macClass.getInstance('HmacSHA256');
local.mac.init(local.secretkeySpec );
local.hashedBytes = local.mac.doFinal(toBinary(toBase64(local.data)));
return lcase( binaryEncode( local.hashedBytes, "base64" ) );
(arguments.key is the shared secret)
Somewhere along the way, I am going wrong. Have I completely misunderstood what I am meant to be doing. This looks so easy in PHP.

So, getHTTPRequestData() returns a struct with a number of members. The one we're interested is content, which will be a byte array.
The MAC classes' doFinal() method expects an array of bytes (in our case the HTTP request's content) and returns an array of bytes (the HMac of the content)
The returned byte array needs to be base-64 encoded in order to compare it to the one Shopify puts in the webhook's headers. toBase64() will do that for us.
Putting it all together, you get this:
toBase64(local.mac.doFinal(getHTTPRequestData().content))