I have to process messages in Dead Letter Queue (DLQ) using JMS API. The goal is to read body of the original messages and it's user properties. I realize that such approach to DLQ processing might be considered as bad design, but I have to deal with it anyway.
Once read with JMS, body of DLQ message contains body of the original one, prepended with DL header and a structure very similar to RFH2 header of the original message (so containing all the needed user properties).
The question is, how to parse these 2 structures in java?
Yet I only found a doc about how DLH could be constructed from raw data (https://www.ibm.com/support/knowledgecenter/SS8JB4/com.ibm.wbpm.main.doc/topics/esbprog_bindings_wmq5.html). But while DLH seems to be a fixed-lenght structure, RFH2 is definitely not - so the most tricky part of parsing is there.
Any idea would be appreciated.
UPDATE
Here is what I have found:
1) DLH was parsed from raw byte array without any problem, as simple as follows:
MQDLH rfh = new MQDLH(new DataInputStream(new ByteArrayInputStream(bytes)));
Once constructed, all the properties are available.
2) MQRFH2 could be created in a similar manner, if MQLONG values were written there as usual, in big endian. But for some reason, completely unclear to me, in this case all MQLONG are little endian.
So, to create MQRFH2 from raw bytes I have to reverse bytes for all MQLONGs. Not a problem for a fixed part (as described in https://www.ibm.com/support/knowledgecenter/SSFKSJ_7.5.0/com.ibm.mq.dev.doc/q032000_.htm), but a bit more complicated for variable part.
I haven't seen any confirmation in docs, but it seems that each folder in variable part is prepended with MQLONG (well, just 4-bytes integer) containing folder length. Once these values were converted from LE to BE as well, MQRFH2 seem to be working correctly.
I wouldn't process the DLQ with a JMS application. It will be so, so tricky and you will spend days or weeks trying to get it right. I would write a regular Java application to do it, far simpler.
i.e.
MQMessage rcvMsg = new MQMessage();
MQDLH dlh = new MQDLH(rcvMsg);
MQRFH2 rfh2 = new MQRFH2(rcvMsg);
byte[] bData = new byte[rcvMsg.getDataLength()];
rcvMsg.readFully(bData);
Updated on March 4, 2020.
I am normally not into banging my head against the wall but if you want to then here is the code that I would try:
ByteArrayInputStream bais = new ByteArrayInputStream(bytes);
DataInput di = new DataInputStream(bais);
MQDLH dlh = new MQDLH(di);
MQRFH2 rfh2 = new MQRFH2(di)
// Get all folders
String[] folderStrings = rfh2.getFolderStrings();
// or you can get individual name/values using
// get***FieldValue() methods of the MQRFH2 class.
/*
* At this point, the cursor for "di" is pointing
* to the beginning of the message payload and I
* would normal do:
*/
byte[] bData = new byte[mqMsg.getDataLength()];
mqMsg.readFully(bData);
Related
It has been at least 5 applications in which I have attempted to display UTF8 encoded characters and every time, quite sporadically and rarely I see random characters being replaced by diamond question marks (see image for better details).
I enclose a page layout to demonstrate my issues. The layout is very basic, it is very simple poll I am creating. The "Съгласен съм" text is takes from a database, where it has just been inserted by a script, using copy-pasted constant. The text is displayed in TextViews.
Has anyone ever encountered such an issue? Please advise!
EDIT: Something I forgot to mention is that the amount and position of weird characters varies on diffferent Android Phone models.
Finally I got it all sorted out in all my applications. Actually the issues mlet down to 3 different reasons and I will list all of them below so that this findings of mine could help people in the future.
Reason 1: Incorrect encoding of user created file.
This actually was the problem with the application I posted about in the question. The problem was that the encoding of the insert script I used for introducing the values in the database was "UTF8 without BOM". I converted this encoding to "UTF8" using Notepad++ and reinserted the values in the database and the issue was resolved. Thanks to #user3249477 for pointing me to thinking in this direction. By the way "UTF8 without BOM" seems to be the default encoding Eclipse uses when creating URF8 files, so take care!
Reason 2: Incorrect encoding of generated file.
The problem of reason 1, pointed me to what to think for in some of the other cases I was facing. In one application of mine I am provided with raw data that I insert in my backend database using simple Java application. The problem there turned out to be that I was passing through intermediate format, files stored on the file system that ?I used to verify I interpretted the raw data correctly. I noticed that these files were also created "UTF8 without BOM". I used this code to write to these files:
BufferedOutputStream outputStream = new BufferedOutputStream(new FileOutputStream(outputFilePath));
writer = new BufferedWriter(new OutputStreamWriter(outputStream, STRING_ENCODING));
writer.append(string);
Which I changed to:
BufferedOutputStream outputStream = new BufferedOutputStream(new FileOutputStream(outputFilePath));
writer = new BufferedWriter(new OutputStreamWriter(outputStream, STRING_ENCODING));
// prepending a bom
writer.write('\ufeff');
writer.append(string);
Following the prescriptions from this answer. This line I add basically made all the intermediate files be encoded in "UTF8" with BOM and resolved my encoding issues.
Reason 3: Incorrect parsing of HTTP responses
The last issue I encountered in few of my applications was that I was not interpretting the UTF8 http responses correctly. I used to have the following code:
HttpResponse response = httpClient.execute(host, request, (HttpContext) null);
String responseBody = null;
responseBody = IOHelper.getInputStreamContents(responseStream);
Where IOHelper is an util I have written myself and reads stream contents to String. I replaced this code with the already provided method in the Android API:
HttpResponse response = httpClient.execute(host, request, (HttpContext) null);
String responseBody = null;
if (response.getEntity() != null) {
responseBody = EntityUtils.toString(response.getEntity(), HTTP.UTF_8);
}
And this fixed the encoding issues I was having with HTTP responses.
As conclusion I can say that one needs to take special care of BOM / without BOM strings when using UTF8 encoding in Android. I am very happy I learnt so many new things during this investigation.
I used already working code for save/load game for sending a player state via sockets. And I encountered a problem that game save is correct, but server is not receiving client's player state.
Here is the base code that is tested and working:
int retval = fc.showSaveDialog(givenComponent);
if (retval == JFileChooser.APPROVE_OPTION) {
File file = fc.getSelectedFile();
try {
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(file), "UTF-8"));
XStream xs = new XStream();
GameSave gs = new GameSave();
ArrayList<PlayerSerialize> listps = new ArrayList<PlayerSerialize>();
for (Player tempplayer : Players.players) {
PlayerSerialize ps = new PlayerSerialize();
ps.getPlayerData(tempplayer);
listps.add(ps);
}
gs.playersSerialize = listps;
gs.gamedate = Dateutils.gamedate;
String s = xs.toXML(gs);
bw.write(s);
bw.close();
} catch (IOException ex) {
Logger.getLogger(DialogMainField.class.getName()).log(Level.SEVERE, null, ex);
}
}
Here is the client side code that is not sending anything to server:
XStream xs = new XStream();
GameSave gs = new GameSave();
ArrayList<PlayerSerialize> listps = new ArrayList<PlayerSerialize>();
PlayerSerialize ps = new PlayerSerialize();
ps.getPlayerData(Players.players.get(1));
listps.add(ps);
gs.playersSerialize = listps;
gs.gamedate = Dateutils.gamedate;
String s = xs.toXML(gs);
out.println("clientplayertoserver");
out.println(s);
Here is the server side just in case:
if (strIn.contains("clientplayertoserver")) {
strIn = in.readLine();
XStream xs = new XStream();
GameSave gs = (GameSave) xs.fromXML(strIn);
Players.players.get(1).getPlayerSerializeData(gs.playersSerialize.get(0));
}
I need some kind of clue because I'm stuck investigating the problem. Are there any XStream limitations? Or the error is in the working with sockets? The same code is working in one place and is not working in another - I greatly thank in advance for any help with this weird situation.
Well, you are doing two different things here:
1) Saving the data to a file, which is ok.
2) Sending data via a socket. You seem to assume that all your data (the XStream serialized object) is actually in one line. This will usually not be the case. Even if you configure XStream to serialize all data without identation, you still cannot be sure you won't have linebreaks in the serialized data (your variables).
So solve your issue, you should separate your concerns here.
1st serialize / deserialize your objects to String and back (that seems to be working for you.
2nd send this data to a medium, like a file (which you already have) or to a server.
For sending string data to a server, you'll need some kind of protocol. Either you can reuse an existing protocol, like HTTP (POST request to a server), Web Service, Rest Call or whatever else your server is running.
If you want to implement your own protocol (as you have tried above), you must ensure that the server knows what to expect and how to treat it properly. Usually you should split your request in a header and a payload section or something like that.
Include in your header what you want to do (e.g save player state) and the meta information of that (e.g how many bytes payload you are sending).
After the header, send the payload.
The server must now read the header 1st (like everything until the first newline), parse the header to understand what is going on (e.g save player state, 543 bytes data) and act on it (read the data, transform it to a string, deserialize the XStream object and store it in a local database or whatever the server should do with that).
So and after all this information, please adapt your question. As you have seen you do not really have a question about XStream, but about how to send some data from client to a custom server.
How can I format the the following erlang term:
{ atom, "message" }
In jInterface to an external format that I may call in an erlang shell
erlang:binary_to_term( Binary )
Example:
Note that since the tuple will be sent over the net, I finish by converting to byte[].
OtpErlangObject[] msg = new OtpErlangObject[2];
msg[0] = new OtpErlangAtom( "atom" );
msg[1] = new OtpErlangString( "message" );
OtpErlangTuple reply = new OtpErlangTuple(msg);
OtpOutputStream stream = new OtpOutputStream(reply);
stream.toByteArray() // byte[] which I send over net
The binary received by Erlang is:
B = <<104,2,100,0,4,97,116,111,109,107,0,7,109,101,115,115,97,103,101>>
Then in an erlang shell converting the received term to binary gives a badarg.
binary_to_term( B ).
** exception error: bad argument
in function binary_to_term/1
called as binary_to_term(<<104,2,107,0,4,97,116,111,109,107,0,7,109,
101,115,115,97,103,101>>)
binary_to_term( <<131,104,2,107,0,4,97,116,111,109,107,0,7,109,101,115,115,97,103,101>> ).
{"atom","message"}
It seems that the message is missing the 131 tag required by term_to_binary. As is evident from the Java output, this tag is not being added by jinterface encode. If I simply add 131 to the beginning of the binary it decodes correctly.
Now why is Java not adding it?
I will still accept answers as I have not officially answered my question ( in a supported way ie. not hacking with 131 )
Ref:
http://www.erlang.org/doc/apps/erts/erl_ext_dist.html
I haven't tested this, but if you're encoding {atom, "message"}, shouldn't you be sending over a tuple, not 2 objects one after the other? Try creating a Tuple object and adding atom and message as elements.
I have an instance of MimeMessage which contains encrypted Parts.
The original content type is "multipart/encrypted; protocol="application/pgp-encrypted"; boundary="EncryptedBoundary12312345654654"
After decryption of each parts, I want the multipart header to change as:
"multipart/mixed; boundary="EncryptedBoundary12312345654654"
The boundary number is obviously dynamic, then I cannot just make
mime.setHeader("Content-Type", "multipart/mixed;" );
Do you have an idea about the best practice for that case?
I don't understand what you mean when you say you "want the multipart header to change".
Are you trying to decrypt the message "in place"? That's probably not going to work well.
You can create a new message using the decrypted contents of the original message.
If it's important to you that things like the "boundary" value remain the same,
you'll probably need to subclass MimeMultipart and use the ContentType class to
construct a new content type value.
I answer to publish the code of my solution:
// source is the encrypted MimeMessage
// MimeMessageWrapper is a wrapper which can copy a messgae but keep the message ID unchanged
boolean keepMessageId = true;
MimeMessageWrapper newMime = new MimeMessageWrapper(source, keepMessageId);
MimeMultipart mmp = new MimeMultipart("mixed");
List<MimePart> parts = MimeMultipartUtils.findPartsByMimeType(mime, "*");
for (MimePart part : parts) {
// Do some part processing
// Decrypt Adn verify individual parts
// End of processing
ContentType type = new ContentType(part.getContentType());
String encoding = part.getEncoding();
String name = type.getParameter("name");
part.setContent(new String(decPart.toByteArray()), type.toString());
// Add the part to the brand new MimeMultipart
mmp.addBodyPart((BodyPart) part);
}
// Set the original copy Message with the new modified content (decrypted parts)
mime.setContent(mmp);
mime.saveChanges();
In fact it seems there is no another way to alter the original message but create a copy was enough for me. The important point was just to create a new MimeMultipart object which will contains the decrypted parts and then given as the content to the MimeMessage(Wrapper). This will generate the new content type values "automagically".
For information, we did use a MimeMessageWrapper which is just a wrapper class that enable to keep the message ID unchanged (or not) to the copies. One possible implementation is on the Apache James project.
Another important point, finally in that solution, the underlying parts were changed but the boundary was adapted as well (it is not said EncryptedXXXX anymore) which is even cleaner for our case.
I've got marshaled CDR data all by itself in the form of a file (i.e., not packed in a GIOP message) which I need to unmarshal and display on the screen. I get to know what type the data is and have working code to do this successfully by the following:
ValueFactory myFactory = (ValueFactory)myConstructor.newInstance( objParam );
StreamableValue myObject = myFactory.init();
myObject._read( myCDRInputStream );
where init() calls the constructor of myObjectImpl(). and _read is the org.omg.CORBA.portable.Streamable _read(InputStream) method.
This works as long as the marshaled data is of the same endianness as the computer running my reader program, but I will need to be able to handle cases where the endianness of the data is different than the endianness of the computer running the reader. I know that endianness is in GIOP messages, which I don't have. Assuming I figure out that I need to change the endianness, how can I tell this to the stream reader?
Thanks!
If you access to the underlying ByteBuffer of your input stream, and then you can set the endianness. For example I use this to open matlab files myself
File file = new File("swiss_roll_data.matlab5");
FileChannel channel = new FileInputStream(file).getChannel();
ByteBuffer scan = channel.map(MapMode.READ_ONLY,0,channel.size());
scan.order(ByteOrder.BIG_ENDIAN);
However, I dont know if you corba framework is happy to read from a bytebuffer (corba is so 90ies). So maybe that does not work for you.