I am trying to make a call to a ws-security secured webservice from a server which unfortunately does not support this natively. The approach I have taken is to implement a .jsp which acts as reverse proxy to the actual end point URL, in the process adding the element with ws-security elements.
This seems to be working quite well and I am confident I've constructed the XML correctly with the correct namespaces etc. I've verified this by comparing the XML with XML produced by SOAP-UI.
The problem is in implementing the password digest generator. I don't get the same result as what SOAP-UI does using the same inputs for NOnce, xsd:dateTime and password, and the following code.
StringBuffer passwordDigestStr_ = new StringBuffer();
// First append the NOnce from the SOAP header
passwordDigestStr_.append(Base64.decode("PzlbwtWRpmFWjG0JRIRn7A=="));
// Then append the xsd:dateTime in UTC timezone
passwordDigestStr_.append("2012-06-09T18:41:03.640Z");
// Finally append the password/secret
passwordDigestStr_.append("password");
System.out.println("Generated password digest: " + new String(com.bea.xbean.util.Base64.encode(org.apache.commons.codec.digest.DigestUtils.sha(passwordDigestStr_.toString())), "UTF-8"));
I think the problem is with implementing the hashing of the first two elements as explained by http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0.pdf
Note that the nonce is hashed using the octet sequence of its decoded value while the timestamp is hashed using the octet sequence of its UTF8 encoding as specified in the contents of the element.
If anyone could help me solve this problem that would be great because it's beginning to drive me crazy! It would be ideal if you could provide source code.
I'll take a crack at it without SOAP-UI. The input to the hash function is supposed to be bytes, not a string. DigestUtils.sha() will allow you to use a string, but that string must be properly encoded. When you wrote the nonce, you were calling StringBuffer.append(Object) which ends up calling byte[].toString(). That gives you something like [B#3e25a5, definitely not what you want. By using bytes everywhere, you should avoid this problem. Note that the example below uses org.apache.commons.codec.binary.Base64, not the Base64 class you were using. It doesn't matter, that's just the one I had handy.
ByteBuffer buf = ByteBuffer.allocate(1000);
buf.put(Base64.decodeBase64("PzlbwtWRpmFWjG0JRIRn7A=="));
buf.put("2012-06-09T18:41:03.640Z".getBytes("UTF-8"));
buf.put("password".getBytes("UTF-8"));
byte[] toHash = new byte[buf.position()];
buf.rewind();
buf.get(toHash);
byte[] hash = DigestUtils.sha(toHash);
System.out.println("Generated password digest: " + Base64.encodeBase64String(hash));
Apologies for the delay in replying, especially considering your initial quick response. I have now been able to get this to work using the essence of your approach to avoid any character encoding issues. However, java.nio.ByteBuffer caused me issues so I modified the code to use basic byte[]s which I combined using System.arrayCopy(). The problem I faced with java.nio.ByteBuffer was that despite 'buf.position()' returning an appropriate number of bytes, all the bytes injected into byte[] toHash through buf.get(toHash) were 0s!
Thanks very much for your assistance.
Related
I am planning to do game data mining in LOL but stuck at parsing replay files. I find that the most popular replay recorder is LOL Replay which records games in .lrf files. They are saved as binary files. I try to print a lrf file to find some patterns in it. As far as I know, the file has two parts:
The initial part is meta data. It's human readable. At the end of it, it shows an encryption key(32bytes) and a client hash for this .lrf file.
The second part has several sections. Each section is in "RESTful URL+encryption+padding(possibly)" format. For example:
?S4GI____GET /observer-mode/rest/consumer/getGameDataChunk/EUW1/1390319411/1/token
?S4GH____?￥?G??,\??1?q??"Lq}?n??&??????l??(?^P???￥I?v??k>x??Z?￡??3Gug
......
??6GI____GET /observer-mode/rest/consumer/getGameDataChunk/EUW1/1390319411/2/token
Some are even unreadable characters.3
I have followed this link and this wiki. It seems like they use BlowFish ECB Algorithm plus PKCS5Padding to encrypt after using GZIP to compress contents. But I failed to decrypt contents using the 32 bytes encryptionkey in meta data. And I am not sure where I should start to read and where to stop because JVM keeps warning me that Given final block not properly padded.
So my question is:
Is there any one who is familiar with Blowfish Algorithm and PKCS5Padding? Which part of those binary files should I read to decrypt between two consecutive RESTful URL? Do I use the right key to decrypt? (the 32 bytes encryption key in the meta data)
Given the patterns around each RESRful URL, could anyone make a guess which algorithm exactly LOL uses to encrypt/decrypt contents? Is it Blowfish algorithm?
Any help would be appreciated. Thank you guys.
Edit #6.17:
Following Divis and avbor's answers, I tried the following Java snippet to decode chunks:
// Decode EncryptKey with GameId
byte[] gameIdBytes = ("502719605").getBytes();
SecretKeySpec gameIdKeySpec = new SecretKeySpec(gameIdBytes, "Blowfish");
Cipher gameIdCipher = Cipher.getInstance("Blowfish/ECB/PKCS5Padding");
gameIdCipher.init(Cipher.DECRYPT_MODE, gameIdKeySpec);
byte[] encryptKeyBytes = Base64.decode("Sf9c+zGDyyST9DtcHn2zToscfeuN4u3/");
byte[] encryptkeyDecryptedByGameId = gameIdCipher.doFinal(encryptKeyBytes);
// Initialize the chunk cipher
SecretKeySpec chunkSpec = new SecretKeySpec(encryptkeyDecryptedByGameId, "Blowfish");
Cipher chunkCipher = Cipher.getInstance("Blowfish/ECB/PKCS5Padding");
chunkCipher.init(Cipher.DECRYPT_MODE, chunkSpec);
byte[] chunkContent = getChunkContent();
byte[] chunkDecryptedBytes = chunkCipher.doFinal(chunkContent);
It works with no error when decoding encryptionkey with gameid. However it doesn't work in the last two lines. Currently I just hard coded getChunkContent() to return an byte array containing the bytes between two RESTful URLs. But Java either returns "Exception in thread "main" javax.crypto.IllegalBlockSizeException: Input length must be multiple of 8 when decrypting with padded cipher"
Or
returns "Exception in thread "main" javax.crypto.BadPaddingException: Given final block not properly padded".
I notice that the hex pattern between two RESTful URLs are as follows:
(hex for first URL e.g. /observer-mode/rest/consumer/getKeyFrame/EUW1/502719605/2/token) + 0a + (chunk contents) + 000000 + (hex for next URL)
My questions are:
Which part of chunks need to be included? Do I need to include "0a" right after the last URL? Do I need to include "000000" before the next URL?
Am I using the right padding algorithm (Blowfish/ECB/PKCS5Padding)?
My test lrf file could be downloaded on : https://www.dropbox.com/s/yl1havphnb3z86d/game1.lrf
EDIT # 6.18
Thanks to Divis! Using the snippet above, I successfully got some chunk info decrypted without error. Two things worth noting when you write your own getChunkContent():
The chunk content starts right after "hex for previous url 0a".
The chunk content ends as close as possible to "0000000 (hex for next url)" when its size reaches exactly a multiple of 8.
But I still got two questions to ask:
Here is an example of what I decode for the content between two .../getKeyframe/... RESTful urls.
39117e0cc2f7e4bb1f8b080000000000000bed7d0b5c15d5 ... 7f23a90000
I know Gzip compressed data starts with "1f8b08..." according to this RFC doc. Can I just discard "39117e0cc2f7e4bb" and start gzip decompress the proceeding content? (Actually I've already tried to start decoding from "1f8b08..", at least it could be decompressed without error)
After the gzip decompression, the result is still a long sequence of binary (with some readable strings like summoners names, champions names, etc.) When I look at the wiki, it seems like it is far from finish. What I expect is to read every item, rune, or movement in readable string. How exactly can I read those game events from it? Or we just need some patience to figure them out ourselves with the community?
Millions of thanks!
Repository dev contributor here, according to the wiki, the key is the base64 Blowfish ECB "encryption_key" (with game id as key for the blowfish).
Then, use this decrypted key to decode the content (blow fish ECB too). Then, gzip decode.
base64decode encryptionkey = decodedKey
blowfishECBdecode decodedKey with (string) gameId as key = decodedKey
blowfishECBdecode content with decodedKey as key = decodedContent
gzipdecode decodedContent = binary
I made a library to download and decode replay files : https://github.com/EloGank/lol-replay-downloader and the CLI command is also available : https://github.com/EloGank/lol-replay-downloader-cli
Hope it'll help :)
To my knowledge, you decrypt the chunks and keyframes using Blowfish. In order to get the key to decrypt said chunks and keyframes, you take the given encryption key, base64 encode it, and then use Blowfish on that using the game id as the key in order to get the actual encryption key for the chunks and keyframes.
I'm working with Shopify at the moment and using their webhook notifications so I can save stuff to our database.
Within their webhook headers, they provide a header of: X-Shopify-Hmac-Sha256
which is:
Each Webhook request includes a X-Shopify-Hmac-SHA256 header which is generated using the app's shared secret (looks like: '267bb1719a8e6ff75c4f2d709be0ca11'), along with the data sent in the request.
This is jolly wonderful; However, I'm really struggling to calculate the value of the X-Shopify-Hmac-Sha256.
I have a .cfm page which the webhook hits and I pass through the getHTTPRequestData to a function like thus:
<cfscript>
variables.stArgs = {};
variables.stArgs.stWebHookData = getHTTPRequestData();
application.stObj.stShopify.oShopifyWebHookBusiness.receiveWebHook(argumentCollection=variables.stArgs);
</cfscript>
I then stole some code from StackOverflow and Ben Nadel, but neither seem to end up giving me the value that I want. Currently I'm using Ben Nadels code like thus:
local.data = arguments.stWebHookData.toString();
local.macClass = createObject( "java", "javax.crypto.Mac" );
local.secretkeySpec = createObject( "java", "javax.crypto.spec.SecretKeySpec" ).init(toBinary(toBase64(arguments.key)),'HmacSHA256');
local.mac = local.macClass.getInstance('HmacSHA256');
local.mac.init(local.secretkeySpec );
local.hashedBytes = local.mac.doFinal(toBinary(toBase64(local.data)));
return lcase( binaryEncode( local.hashedBytes, "base64" ) );
(arguments.key is the shared secret)
Somewhere along the way, I am going wrong. Have I completely misunderstood what I am meant to be doing. This looks so easy in PHP.
So, getHTTPRequestData() returns a struct with a number of members. The one we're interested is content, which will be a byte array.
The MAC classes' doFinal() method expects an array of bytes (in our case the HTTP request's content) and returns an array of bytes (the HMac of the content)
The returned byte array needs to be base-64 encoded in order to compare it to the one Shopify puts in the webhook's headers. toBase64() will do that for us.
Putting it all together, you get this:
toBase64(local.mac.doFinal(getHTTPRequestData().content))
I am writing some web-services for a social networking website. These web-services would be utilized by android for making android-app. As the person who designed the website is no longer under contact, I looked at the whole website code which was written in java with spring framework. I am writing web services in php.
Now, when I tried to send a post request to a php page that would confirm if the given username & pass combination is correct or not and then return a session id. But i'm not being able to get the correct hashing method to get correct hash value that is saved in the database.
Because of this, everytime, I am getting rejected by the php code.
The encryption that I found on the website code is as follows:
public static final synchronized String encrypt(String plaintext, String algorithm, String encoding) throws Exception
{
MessageDigest msgDigest = null;
String hashValue = null;
try
{
msgDigest = MessageDigest.getInstance(algorithm);
msgDigest.update(plaintext.getBytes(encoding));
byte rawByte[] = msgDigest.digest();
hashValue = (new BASE64Encoder()).encode(rawByte);
}
catch (NoSuchAlgorithmException e)
{
System.out.println("No Such Algorithm Exists");
}
catch (UnsupportedEncodingException e)
{
System.out.println("The Encoding Is Not Supported");
}
return hashValue;
}
For example, if i am giving password as monkey123 as password, it is giving hash value encoded in base 64 as: hge2WiM7vlaTTS1qU404+Q==
Now, after struggling to do the same in php for hours, I realised I could do the above procedure in android itself. So, I wrote the following code:
MessageDigest pwdDigest=MessageDigest.getInstance("MD5");
pwdDigest.update(password.getBytes("UTF-16"));
byte rawbyte[]=pwdDigest.digest();
String passwordHash=Base64.encodeToString(rawbyte,Base64.DEFAULT);
URL url = new URL(loginURL);
HttpURLConnection Connection = (HttpURLConnection) url.openConnection();
Connection.setReadTimeout(10000);
Connection.setAllowUserInteraction(false);
Connection.setDoOutput(true);
//set the request to POST and send
Connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
DataOutputStream out = new DataOutputStream(Connection.getOutputStream());
out.writeBytes("username=" + URLEncoder.encode(username, "UTF-8"));
out.writeBytes("&password="+URLEncoder.encode(passwordHash,"UTF-8"));
out.flush();
out.close();
if(Connection.getResponseCode()==200){
String data="Connected";
return data;
} else
return Connection.getResponseCode()+": "+Connection.getResponseMessage();
I expected this would be successful because in both the cases, I am doing same process to encrypt the password, but amazingly, this is not giving the hash value as:
hge2WiM7vlaTTS1qU404+Q== but it's giving : nZlvVe7GSS2Zso1dOwJrIA==
I am really struggling to find out a reason why these two are not the same. Any help would be hugely appreciated.
I don't expect MD5 to differ between platforms. It's stable and well documented and part of the core libraries. If this were broken in some Android version, nothing would work on that phone.
The re-encoding into UTF-8 is harmless, because all Base64 characters fit into the lower ASCII range. Three characters of the base64 alphabet require URL encoding, but you would have seen the %-escapes if something went wrong there.
Base64 is less stable ground (lots and lots of different implementations, no single canonical one), but it's not exactly rocket science either. Again, I don't expect a faulty implementation to really make it out into the wild, but the Base64 step may be where the difference arises.
Personally, I suspect the error is introduced during the password.getBytes("UTF-16") call. A quick way to verify this hunch is to inspect the resulting byte array in a debugger on both platforms. According to java.lang.Charset, the "UTF-16" encoding will use big endian byte order, whereas your PHP code may be defaulting to little endian because it's running on x86 and no byte order mark is present (I don't know PHP well enough to tell if this behaviour is well defined). Try modifying the Java code to use password.getBytes("UTF-16LE") and see if that makes a difference.
Side note: MD5 is no longer considered secure for hashing passwords; you'll want to use something like scrypt or PBKDF2 with plenty of rounds and a random salt, but that's a topic unto itself.
I've been writing a Web Application recently that interacts with iPhones. The iPhone iphone will actually send information to the server in the form of a plist. So it's not uncommon to see something like...
<key>RandomData</key>
<data>UW31vrxbUTl07PaDRDEln3EWTLojFFmsm7YuRAscirI=</data>
Now I know this data is hashed/encrypted in some fashion. When I open up the plist with an editor (Property List Editor), it shows me a more "human readable" format. For example, the data above would be converted into something like...
<346df5da 3c5b5259 74ecf683 4431249f 711630ba 232c54ac 9bf2ee44 0r1c8ab2>
Any idea what the method of converting it is? Mainly I'm looking to get this into a Java String.
Thanks!
According to our friends at wikipedia, the <data> tag contains Base64 encoded data. So, use your favorite Java "Base64" class to decode (see also this question).
ps. technically, this is neither "hashed" nor "encrypted", simply "encoded". "Hashed" implies a one-way transformation where multiple input values can yield the same output value. "Encrypted" implies the need for a (usually secret) "key" to reverse the encryption. Base64 encoding is simply a way of representing arbitrary binary data using only printable characters.
After base64 decoding it you need to hex encode it. This is what PL Editor is showing you.
So...
<key>SomeData</key>
<data>UW31ejxbelle7PaeRAEen3EWMLojbFmsm7LuRAscirI=</data?
Can be represented with...
byte[] bytes = Base64.decode("UW31ejxbelle7PaeRAEen3EWMLojbFmsm7LuRAscirI=");
BigInteger bigInt = new BigInteger(bytes);
String hexString = bigInt.toString(16);
System.out.println(hexString);
To get...
<516df5aa 3c5b5259 74ecf683 4401259f 711630ba 236c59ac 9bb2ee44 0b1c8ab2>
I am working on encryption-decryption program.
Program gets an input from the user and encrypts it. Then it stores the encrypted data in ms access database table.
Later, the data is retrieved from the table , decrypted and given back to the user.
I am storing the data as text in the ms access. The encryption algorithm returns a byte array of size 16.
But when i retrieve the data from the database, i am getting a byte array of size 8 only.
Help me to get through this...
I think the problem is that you are using it as text while it isn't (it is binary data). The halving of the length sounds like a Unicode related issue (i.e. the 'text' is stored as wide with two bytes for character, but retrieved as one byte per character).
I have an app that stores encrypted credit card numbers using the MS Crypto interface. I got the code from the MS Knowledge Base, and the key thing is running ByteToString() and StringToByte() conversion in the proper places. I'm storing the actual data in a plain Jet text field and have had no problems whatsoever.
one possible solution is to encode the cipher text as String using Base64 encoding
you can use Appache Commons Library for that:
http://commons.apache.org/codec/apidocs/org/apache/commons/codec/binary/Base64.html
Edited:
i dont know why you want MS-ACCESS Specific solution ! the DMBS may change, the OS also may change.. you must to write general solution that can work in many cases..
here small example for using Base64 Encoder/Decoder:
import sun.misc.BASE64Decoder;
import sun.misc.BASE64Encoder;
import java.io.IOException ;
public class Decoder {
public static void main(String[] args) throws IOException{
byte[] cipherBytes = "stackoverflow".getBytes(); // say this the is encrypted bytes
String encodedBytes = new BASE64Encoder().encode(cipherBytes);
System.out.println("stored as: " + encodedBytes );
byte[] decodedBytes = new BASE64Decoder().decodeBuffer(encodedBytes);
System.out.println("extracted as: " + new String(decodedBytes) );
}
}
Note: this code using Internal Sun Classes (BASE64Encoder/Decoder) and its not recommended to use these classes in your program because it may change in the next version of JDK.
using BASE64 Encoder/Decoder in Appache Commons is better.
if you want the MS-ACCESS solution, try to store the ciphertext in LONGBINARY , see this:
How to specify blob type in MS Access?