I am trying to figure out how to generate a matching MD 5 hash value between SQL Server and ColdFusion. The root cause appears to be that the SQL Server field is an nvarchar datatype, which seems to mean I need to do something with the encoding of the string I would hash in ColdFusion or Java to make it match, but I am unable to figure it out. To be clear if this was a SQL Server varchar field, everything works.
Here's the code I'm trying:
<cfset stringToHash = "Hello world!">
<cfquery name="sqlserver" datasource="#mySqlServerDSN#">
SELECT RIGHT(
master.dbo.fn_varbintohexstr(
HashBytes(
'MD5',
CAST(<cfqueryparam value="#stringToHash#" cfsqltype="cf_sql_varchar"> AS nvarchar(max))
)
)
, 32) AS HASHED
</cfquery>
<cfoutput>
<pre>
CF UFT-8: #hash(stringToHash, 'MD5', 'UTF-8')#
CF UFT-16: #hash(stringToHash, 'MD5', 'UTF-16')#
SQL Server: #sqlserver.hashed#
</pre>
</cfoutput>
Produces
CF UTF-8: 86FB269D190D2C85F6E0468CECA42A20
CF UTF-16: 0C89A9720D83539E3723BB99C07D069F
SQL Server: f9a6119c6ec37ce652960382f8b59f2c
So I'm guessing I need to change the final argument I'm passing to hash() to be a different encoding, but I can't figure it out. I've also tagged this question as Java too, because I'm more than happy to take an answer in that language as well.
By default SQL Server uses the UTF-16 in little-endian byte order character set for nvarchar fields.
In ColdFusion you must use the 'UTF-16LE' character set.
<cfscript>
helloWorld = "Hello, World!";
utf8HashCF = lcase(hash(helloWorld, 'MD5', 'UTF-16LE'));
</cfscript>
<cfoutput>
#utf8HashCF# <br />
</cfoutput>
I'm curious why your sql server column is nvarchar; it's not necessary for hashes. nvarchar is for storing extended character sets, which you shouldn't be getting back from a hash function.
Regardless, I tried all of the hash algorithms available in CF9 and none of them generate the hash you're looking for.
Unless you need to keep the column set to nvarchar for some reason you haven't already explained, why not change it to varchar?
I don't think it's the CF hashing because if you compare the CF to Java they create the same hash. Both the CF & Java output "65a8e27d8879283831b664bd8b7f0ad4" on my box and it matched the SQL hash when I changed the cast to varchar(32).
In the past when I've needed to do any sort of hash creation and comparison, I created a service that returns a string so you don't have to worry about cross platform algorithm issues. You could also just have sql do it all for you, but then you have the business logic in the wrong layers but to each their own.
<cfscript>
helloWorld = "Hello, World!";
javaString = CreateObject( "java", "java.lang.String" ).Init(helloWorld);
javaHash = CreateObject( "java", "java.security.MessageDigest" ).getInstance("MD5");
javaHash.reset();
javaHash.update(javaString.getBytes("UTF-8"),0,javaString.length());
javaBigInt = CreateObject( "java", "java.math.BigInteger" ).Init(1,javaHash.digest());
utf8HashCF = lcase(hash(helloWorld, 'MD5', 'UTF-8'));
utf8HashJava = variables.javaBigInt.toString(16);
</cfscript>
<cfoutput>
#utf8HashCF# <br />
#utf8HashJava#
</cfoutput>
Related
There's a GUID of a printer device with this format in a nxlog generated log:
119d0d80-699d-4e81-8e4e-5477e22ac1b3
I'd like to get the device name by resolving its GUID via LDAP. However, the nearest field I encountered is objectGUID in ldapsearch response, which is apparently a Base64 encoded value:
objectGUID:: fuAW6fefLke30d46TDTUWA==
Are these two above GUIDs relative in any way? (In other words, Should I search the first GUID among objectGUID fields in AD after format conversion?
How may I get the object name by using LDAP query in java?
It wasn't possible to achieve that via getAttribute() method and below code:
//Create the initial directory context
LdapContext ctx = new InitialLdapContext(env,null);
//Bind directly using the string form of the GUID
String strGUID = "<GUID="+guid+">";
//Specify the attributes to return
String returnedAtts[]={"distinguishedName"};
Attributes attr = ctx.getAttributes(strGUID,returnedAtts);
//print out the retrieved attributes
if(attr!=null)
distinguishedName = attr.get("distinguishedName").get().toString();
System.out.println("distinguishedName: " + distinguishedName);
ctx.close();
and I got NameNotFoundException:
javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310021B, problem 2001 (NO_OBJECT), data 0, best match of:
''
Is the format of name (<GUID="+guid+">) correct? (I don't see any sign of the usage of this format in the repsponse output of ldapsearch)
The ObjectGuid as used in Microsoft Active Directory is a little strange compared to must GUIDs.
You have to contend with a few aspects including Endianness
I did find a reference to some Java Code.
I'm integrating an application to the AS400 using Java/JT400 driver. I'm having an issue when I extract data from a parameter file - the data retrieved seems to be encoded.
SELECT SUBSTR(F00001,1,20) FROM QS36F."FX.PARA" WHERE K00001 LIKE '16FFC%%%%%' FETCH FIRST 5 ROWS ONLY
Output
00001: C6C9D9C540C3D6D4D4C5D9C3C9C1D34040404040, - 1
00001: C6C9D9C5406040C3D6D4D4C5D9C3C9C1D3406040, - 2
How can I convert this to a readable format? Is there a function which I can use to decode this?
On the terminal connection to the AS400 the information is displayed correctly through the same SQL query.
I have no experience working with AS400 before this and could really use some help. This issue is only with the parameter files. The database tables work fine.
What you are seeing is EBCDIC output instead of ASCII. This is due to the CCSID not being specified in the database as mentioned in other answers. The ideal solution is to assign the CCSID to your field in the database. If you don't have the ability to do so and can't convince those responsible to do so, then the following solution should also work:
SELECT CAST(SUBSTR(F00001,1,20) AS CHAR(20) CCSID(37))
FROM QS36F."FX.PARA"
WHERE K00001 LIKE '16FFC%%%%%'
FETCH FIRST 5 ROWS ONLY
Replace the CCSID with whichever one you need. The CCSID definitions can be found here: https://www-01.ibm.com/software/globalization/ccsid/ccsid_registered.html
Since the file is in QS36F, I would guess that the file is a flat file and not externally defined ... so the data in the file would have to be manually interpreted if being accessed via SQL.
You could try casting the field, after you substring it, into a character format.
(I don't have a S/36 file handy, so I really can't try it)
It is hex of bytes of a text in EBCDIC, the AS/400 charset.
static String fromEbcdic(String hex) {
int m = hex.length();
if (m % 2 != 0) {
throw new IllegalArgumentException("Must be even length");
}
int n = m/2;
byte[] bytes = new byte[n];
for (int i = 0; i < n; ++i) {
int b = Integer.parseInt(hex.substring(i*2, i*2 + 2), 16);
bytes[i] = (byte) b;
}
return new String(bytes, Charset.forName("Cp500"));
}
passing "C6C9D9C540C3D6D4D4C5D9C3C9C1D34040404040".
Convert the file with Cp500 as charset:
Path path = Paths.get("...");
List<String> lines = Files.readAllLines(path, Charset.forName("Cp500"));
For line endings, which are on AS/400 the NEL char, U+0085, one can use regex:
content = content.replaceAll("\\R", "\r\n");
The regex \R will match exactly one line break, whether \r, \n, \r\n, \u0085.
A Big thank you for all the answers provided, they are all correct.
It is a flat parameter file in the AS400 and I have no control over changing anything in the system. So it has to be at runtime of the SQL query or once received.
I had absolutely no clue about what the code page was as I have no prior experience with AS400 and files in it. Hence all your answers have helped resolve and enlighten me on this. :)
So, the best answer is the last one. I have changed the SQL as follows and I get the desired result.
SELECT CAST(F00001 AS CHAR(20) CCSID 37) FROM QS36F."FX.PARA" WHERE K00001 LIKE '16FFC%%%%%' FETCH FIRST 5 ROWS ONLY
00001: FIRE COMMERCIAL , - 1
00001: FIRE - COMMERCIAL - , - 2
Thanks once again.
Dilanke
I have a SP that contain 3 input parameters and 4 outputs. inputs are NUMBER, the outputs one is a VARCHAR(descResponse) and the others are NUMBER.
In java, when I log the response:
logger.info("Out["+res.getCodResponse()+"]["+res.getDescResponse()+"]");
the print is:
Out[-1861][ORA-01861: literal does not match format string]
The map between java attributes with the SP parameters are the same type (number : Integer/Double, varchar: String).
the problem is when i use sql developer with the same parameters the response is ok. So i don't know if my WS is not working or the SP.
I hava distinct NLS settings on my sql devoloper than my DB so I change the NLS settings on my sql devoloper and i could detect the wrong date reference and fix it.
I want to write a little .jar which is used as a "translator" for SQL-Queries directed to a z/OS-DB2-Database.
My goal is that the application accepts SQL-Queries as Command Line Arguments manually or via shell script/cron, next to other parameters like IP, Port, User etc.
Is there a way to leave those arguments unaffected while passing them to the jar?
Example:
java -jar db2sql.jar SQL=={SELECT * FROM TABLE1 TAB1, TABLE2 TAB2 WHERE TAB1.XYZ = TAB2.ZYX AND TAB2.ABC LIKE 'blabla' AND TAB1.DATE >= '01.01.2015'} IP=={192.168.0.1} User=={Santa} Password=={CLAUS}
(please ignore that this statement is senseless, but i hope you get the trick)
My Problem is reading out that Command Line parameters, mostly special characters like * , " ' etc.
Questions:
Is there a list of all possible SQL-Parameters which must be escaped?
Is there a special character which can be used as delimiter that will never occur in an SQL-Query?
Is it possible to pass all kind of SQL Statments as ONE argument?
Is it possible to leave special characters unhandled, e.g. Argument "" = String "", and not .classpath etc. ?
Kind Regards
Although I wouldn't recommend what you're trying to do for several reasons, at least in a *NIX environment you could just use the standard way.
java -jar foo.jar -s "SELECT * FROM SOMETHING WHERE foo = 2" -u username -h hostname
You can use additional libraries to parse the parameters, but this way you would use -s to specify the SQL query, and wrap the param value in " to make it a single argument with automatic escape.
In your main method you can then get the full query with (simplified)
if(args[0].equals("-s"))
sqlString = args[1];
I have to find a user-defined String in a Document (using Java), which is stored in a database in a BLOB. When I search a String with special characters ("Umlaute", äöü etc.), it failes, meaning it does not return any positions at all. And I am not allowed to convert the document's content into UTF-8 (which would have fixed this problem but raised a new, even bigger one).
Some additional information:
The document's content is returned as String in "ISO-8859-1" (Latin1).
Here is an example, what a String could look like:
Die Erkenntnis, daà der Künstler Schutz braucht, ...
This is how it should look like:
Die Erkenntnis, daß der Künstler Schutz braucht, ...
If I am searching for Künstler it would fail to find it, because it looks for ü but only finds ü.
Is it possible to convert Künstler into Künstler so I can search for the wrong encoded version instead?
Note:
We are using the Hibernate Framework for Database access. The original Getter for the Document's Content returns a byte[]. The String is than returned by calling
new String(getContent(), "ISO-8859-1")
The problem here is, that I cannot change this to UTF-8, because it would then mess up the rest of our application which is based on a third party application that delivers data this way.
Okay, looks like I've found a way to mess up the encoding on purpose.
new String("Künstler".getBytes("UTF-8"), "ISO-8859-1")
By getting the Bytes of the String Künstler in UTF-8 and then creating a new String, telling Java that this is Latin1, it converts to Künstler. It's a hell of a hack but seems to work well.
Already answered by yourself.
An altoghether different approach:
If you can search the blob, you could search using
"SELECT .. FROM ... WHERE"
+ " ... LIKE '%" + key.replaceAll("\\P{Ascii}+", "%") + "%'"
This replaces non-ASCII sequences by the % wildcard: UTF-8 multibyte sequences are non-ASCII by design.