We Keep Coding

How to update User Defined Variable in JMeter in JSR223 Sampler?

I'm new to JMeter & Java and now writing Authorization script for testing API.
I had some troubles with updating variable with vars.put(key,value)
Here is my code example:
import java.util.Arrays;
import java.security.MessageDigest;
import java.util.Base64;
public class StringToByte {
public void main(String[] args) {
String str_salt = "${salt}";
byte[] b_salt = str_salt.getBytes();
String str_pass = "c3000Hub";
byte[] b_pass = str_pass.getBytes();
byte[] b_pass_hash = new byte[b_salt.length + b_pass.length];
MessageDigest md = MessageDigest.getInstance("SHA-256");
byte[] hash = md.digest(b_pass_hash);
String pass_hash = Base64.getEncoder().encodeToString(hash);
vars.put("passhash", pass_hash);
}
}
Variable in User Defined Variables just not updating and I've got no idea why?

You need to explicitly call this main() function in order to get it working, you declare it but I fail to see where it's being invoked
Change String str_salt = "${salt}"; to String str_salt = vars.get("salt");, as per JSR223 Sampler Documentation:
The JSR223 test elements have a feature (compilation) that can significantly increase performance. To benefit from this feature:
Use Script files instead of inlining them. This will make JMeter compile them if this feature is available on ScriptEngine and cache them.
Or Use Script Text and check Cache compiled script if available property.
When using this feature, ensure your script code does not use JMeter variables or JMeter function calls directly in script code as caching would only cache first replacement. Instead use script parameters.
Suggested code change (if you want to keep this class/method approach):
import org.apache.jmeter.threads.JMeterVariables
import java.security.MessageDigest
public class StringToByte {
public void main(JMeterVariables vars) {
String str_salt = vars.get("salt");
byte[] b_salt = str_salt.getBytes();
String str_pass = "c3000Hub";
byte[] b_pass = str_pass.getBytes();
byte[] b_pass_hash = new byte[b_salt.length + b_pass.length];
MessageDigest md = MessageDigest.getInstance("SHA-256");
byte[] hash = md.digest(b_pass_hash);
String pass_hash = Base64.getEncoder().encodeToString(hash);
vars.put("passhash", pass_hash);
}
}
new StringToByte().main(vars)
or you can just do something like:
import java.security.MessageDigest
String str_salt = vars.get("salt");
byte[] b_salt = str_salt.getBytes();
String str_pass = "c3000Hub";
byte[] b_pass = str_pass.getBytes();
byte[] b_pass_hash = new byte[b_salt.length + b_pass.length];
MessageDigest md = MessageDigest.getInstance("SHA-256");
byte[] hash = md.digest(b_pass_hash);
String pass_hash = Base64.getEncoder().encodeToString(hash);
vars.put("passhash", pass_hash);
More information on Groovy scripting in JMeter: Apache Groovy - Why and How You Should Use It

Correctly compare code_verifier with code_challenge in Java

I'm using passport-oauth2 (passportjs.org and https://github.com/jaredhanson/passport-oauth2/blob/master/lib/strategy.js) for OAuth2+PKCE integration in a nodejs application.
The backend it's authenticating against is written in Java.
The problem is that I can't seem to decode->hash the code_verifier to correctly match the code_challenge that comes from passport-oauth2.
I know that the Base64 encoding that comes from passport has been generated to be URL safe (no padding, no wrapping, replacements for + or /), so I'm using a Url Decoder:
Base64.getUrlDecoder().decode(...)
Then I'm using commons DigestUtils to generate a SHA256 of the decoded verifier and comparing it with the challenge. So the whole thing looks something like this:
java.util.Base64.Decoder decoder = java.util.Base64.getUrlDecoder();
String codeChallenge = // get the code challenge from my cache
byte[] decodedCodeChallenge = decoder.decode(codeChallenge);
byte[] decodedCodeVerifier = decoder.decode(codeVerifier);
if (!Arrays.equals(sha256(decodedCodeVerifier), decodedCodeChallenge)) {
return Response.status(400).entity(ERROR_INVALID_CHALLENGE_VERIFIER).build();
}
Example:
This code verifier: 5CFCAiZC0g0OA-jmBmmjTBZiyPCQsnq_2q5k9fD-aAY
should match this code challenge: Fw7s3XHRVb2m1nT7s646UrYiYLMJ54as0ZIU_injyqw once both have been Base64-url-decoded and the verifier has been SHA256 hashed, but it doesn't.
What am I doing wrong?

Just 5 minutes later I figured it out.
In passport-oauth2, the code verifier is Base64-url-encoded(random bytes):
verifier = base64url(crypto.pseudoRandomBytes(32))
See: https://github.com/jaredhanson/passport-oauth2/blob/master/lib/strategy.js#L236
The challenge is then Base64-url-encoded(sha256(verifier)), which expands to Base64-url-encoded(sha256(Base64-url-encoded(random bytes))):
challenge = base64url(crypto.createHash('sha256').update(verifier).digest());
See: https://github.com/jaredhanson/passport-oauth2/blob/master/lib/strategy.js#L242
So to do the verification, I don't need to decode anything. It was sha256-d in it's encoded state.
This worked in the end:
java.util.Base64.Encoder encoder = java.util.Base64.getUrlEncoder();
String codeChallenge = // get code challenge from my cache;
String encodedVerifier = new String(encoder.encode(sha256(codeVerifier))).split("=")[0]; // Remember to remove padding
if (!encodedVerifier.equals(codeChallenge)) {
return Response.status(400).entity(ERROR_INVALID_CHALLENGE_VERIFIER).build();
}

How to generate Signature in AWS from Java

When I invoke API endpoints from REST client, I got error by concerning with Signature.
Request:
Host: https://xxx.execute-api.ap-southeast-1.amazonaws.com/latest/api/name
Authorization: AWS4-HMAC-SHA256 Credential={AWSKEY}/20160314/ap-southeast-1/execute-api/aws4_request,SignedHeaders=host;range;x-amz-date,Signature={signature}
X-Amz-Date: 20160314T102915Z
Response:
{
"message": "The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details. The Canonical String for this request should have been 'xxx' "
}
From Java code, I followed AWS reference of how to generate Signature.
String secretKey = "{mysecretkey}";
String dateStamp = "20160314";
String regionName = "ap-southeast-1";
String serviceName = "execute-api";
byte[] signature = getSignatureKey(secretKey, dateStamp, regionName, serviceName);
System.out.println("Signature : " + Hex.encodeHexString(signature));
static byte[] HmacSHA256(String data, byte[] key) throws Exception {
String algorithm="HmacSHA256";
Mac mac = Mac.getInstance(algorithm);
mac.init(new SecretKeySpec(key, algorithm));
return mac.doFinal(data.getBytes("UTF8"));
}
static byte[] getSignatureKey(String key, String dateStamp, String regionName, String serviceName) throws Exception {
byte[] kSecret = ("AWS4" + key).getBytes("UTF8");
byte[] kDate = HmacSHA256(dateStamp, kSecret);
byte[] kRegion = HmacSHA256(regionName, kDate);
byte[] kService = HmacSHA256(serviceName, kRegion);
byte[] kSigning = HmacSHA256("aws4_request", kService);
return kSigning;
}
May I know what I was wrong while generating Signature?
Reference how to generate Signature : http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-java

You can use classes from aws-java-sdk-core: https://github.com/aws/aws-sdk-java/tree/master/aws-java-sdk-core
More specifically, Request, Aws4Signer and a few other ones:
//Instantiate the request
Request<Void> request = new DefaultRequest<Void>("es"); //Request to ElasticSearch
request.setHttpMethod(HttpMethodName.GET);
request.setEndpoint(URI.create("http://..."));
//Sign it...
AWS4Signer signer = new AWS4Signer();
signer.setRegionName("...");
signer.setServiceName(request.getServiceName());
signer.sign(request, new AwsCredentialsFromSystem());
//Execute it and get the response...
Response<String> rsp = new AmazonHttpClient(new ClientConfiguration())
.requestExecutionBuilder()
.executionContext(new ExecutionContext(true))
.request(request)
.errorResponseHandler(new SimpleAwsErrorHandler())
.execute(new SimpleResponseHandler<String>());
If you want a cleaner design, you can use the Decorator pattern to compose some elegant classes and hide the above mess. An example for that here: http://www.amihaiemil.com/2017/02/18/decorators-with-tunnels.html

From the code example above it looks like you are not creating a canonical request and including it in the string that gets signed as per http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
Instead of implementing this yourself have you looked at using a third-party library.
aws-v4-signer-java is a lightweight, zero-dependency library that makes it easy to generate AWS V4 signatures.
String contentSha256 = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855";
HttpRequest request = new HttpRequest("GET", new URI("https://examplebucket.s3.amazonaws.com?max-keys=2&prefix=J"));
String signature = Signer.builder()
.awsCredentials(new AwsCredentials(ACCESS_KEY, SECRET_KEY))
.header("Host", "examplebucket.s3.amazonaws.com")
.header("x-amz-date", "20130524T000000Z")
.header("x-amz-content-sha256", contentSha256)
.buildS3(request, contentSha256)
.getSignature();
Disclaimer: I'm the libraries author.

This is possible using 100% java libraries without additional dependencies, just use the query parameters generated here:
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.security.SignatureException;
import java.util.Formatter;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.util.Base64;
...
private static final String ACCESS_KEY = "...";
private static final String SECRET_KEY = "...";
private static final int expiresTime = 1 * 24 * 60 * 60;
private static final String HMAC_SHA1_ALGORITHM = "HmacSHA1";
public void sign(String protocol, String bucketName, String contentPath) throws Exception {
Calendar cal = Calendar.getInstance();
cal.add(Calendar.HOUR_OF_DAY, 24);
String host = bucketName + ".s3-us-west-2.amazonaws.com";
long expireTime = cal.getTimeInMillis() / 1000;
String signString = "GET\n" +
"\n" +
"\n" +
expireTime + "\n" +
"/" + bucketName + contentPath;
SecretKeySpec signingKey = new SecretKeySpec(SECRET_KEY.getBytes(), HMAC_SHA1_ALGORITHM);
Mac mac = Mac.getInstance(HMAC_SHA1_ALGORITHM);
mac.init(signingKey);
String signature = URLEncoder.encode(new String(Base64.getEncoder().encode(mac.doFinal(signString.getBytes()))));
System.out.println(signature);
String fullPayload = "?AWSAccessKeyId=" + ACCESS_KEY +
"&Expires=" + expireTime +
"&Signature=" + signature;
System.out.println(protocol + "://" + host + contentPath + fullPayload);
}
...

The signing process is lengthy and error-prone, here are some tips
Make sure your access key and secret is correct, try to use Postman to test the request at first, it's easy and fast, see Use Postman to Call a REST API
Make sure you use UTC time
The signing process uses both timestamp(YYYYMMDD'T'HHMMSS'Z') and datetime(YYYYMMDD), so double check your implementation for that
Use any online hash tool to verify your hash algorithm behaves as expected
Read the python implementation carefully, see Examples of the Complete Version 4 Signing Process (Python)
See my fully working java implementation on Github - A Java(SpringBoot) template for Java and AWS SageMaker DeepAR model endpoint invocation integration

You may investigate code samples that is shared by AWS web site. I used some of the util classes and a few java class I need. So you don't have to use all classes and other stuff. I left the link below.
AWS Java Samples in doc of Amazon

For me, in Java, the following code worked to generate a signed request to sent to web socket client via api gateway -
Request<Void> request = new DefaultRequest<Void>("execute-api"); //Request to API gateway
request.setHttpMethod(HttpMethodName.POST);
request.setEndpoint(URI.create(url));
String bodyContnt= "test data";
InputStream targetStream = new ByteArrayInputStream(bodyContnt.getBytes());
request.setContent(targetStream);
//Sign it...
AWS4Signer signer = new AWS4Signer();
signer.setRegionName("ap-south-1");
signer.setServiceName(request.getServiceName());
signer.sign(request, new Creds());
signer.setOverrideDate(new Date()); // needed as current ts is required
//Execute it and get the response...
Response<String> rsp = new AmazonHttpClient(new ClientConfiguration())
.requestExecutionBuilder()
.executionContext(new ExecutionContext(true))
.request(request)
.errorResponseHandler(new SimpleAwsErrorHandler(true))
.execute(new SimpleResponseHandler());

encode a groovy variable using base64

I have a string like this and am trying to encode this string using base64
{"htmlBrowserType":"Default","mimeType":"text/html","url":"https://github.comcast.com"}
String base64Config = {"htmlBrowserType\":\"Default\",\"mimeType\":\"text/html\",\"url\":\"https://github.comcast.com"}
Actually it is a groovy code
def encoded = base64Config.bytes.encodeBase64().toString()
While encoding using the tool am getting
eyJodG1sQnJvd3NlclR5cGUiOiJEZWZhdWx0IiwibWltZVR5cGUiOiJ0ZXh0L2h0bWwiLCJ1cmwiOiJodHRwczovL2dpdGh1Yi5jb21jYXN0LmNvbSJ9
But it is not working using java code while am decoding the string getting as the result of the above java code is not
{"htmlBrowserType":"Default","mimeType":"text/html","url":"https://github.comcast.com"}

Your base64Config is not a String, but Closure:
String base64Config = {"htmlBrowserType\":\"Default\",\"mimeType\":\"text/html\",\"url\":\"https://github.comcast.com"}
It should be:
String base64Config = "{\"htmlBrowserType\":\"Default\",\"mimeType\":\"text/html\",\"url\":\"https://github.comcast.com\"}"

Unable to convert bytes from charset 65535 in to Japanese (5035)

I have a character set conversion issue:
I am updating Japanese Kanji characters in DB2 in iSeries system with the following conversion method:
AS400 sys = new AS400("<host>","username","password");
CharConverter charConv = new CharConverter(5035, sys);
byte[] b = charConv.stringToByteArray(5035, sys, "試験");
AS400Text textConverter = new AS400Text(b.length, 65535,sys);
While retrieving, I use the following code to convert & display:
CharConverter charConv = new CharConverter(5035, sys);
byte[] bytes = charConv.stringToByteArray(5035, sys, dbRemarks);
String s = new String(bytes);
System.out.println("Remarks after conversion to AS400Text :"+s);
But, the system is displaying garbled characters while displaying. Can anybody help me to decode Japanese characters from binary storage?

Well I don't know anything about CharConverter or AS400Text, but code like this is almost always a mistake:
String s = new String(bytes);
That uses the platform default encoding to convert the binary data to text.
Usually storage and retrieval should go through opposite processes - so while you've started with a string and then converted it to bytes, and converted that to an AS400Text object when storing it, I'd expect you to start with an AS400Text object, convert that to a byte array, and then convert that to a String using CharConverter when fetching. The fact that you're calling stringToByteArray in both cases suggests there's something amiss.
(It would also help if you'd tell us what dbRemarks is, and how you've fetched it.)
I do note that having checked some documentation for AS400Text, I've seen this:
Due to recent changes in the behavior of the character conversion routines, this system object is no longer necessary, except when the AS400Text object is to be passed as a parameter on a Toolbox Proxy connection.
There's similar documentation for CharConverter. Are you sure you actually need to go through this at all? Have you tried just storing the string directly and retrieving it directly, without going through intermediate steps?

Thank you Jon Skeet!
Yes. I have committed a mistake, not encoding the string while declaration.
My issue is to get the data stored in DB2, convert it into Japanese and provide for editing in web page. I am getting dbRemarks from the result set. I have missed another thing in my post:
While inserting, I am converting to text like:
String text = (String) textConverter.toObject(b);
PreparedStatement prepareStatementUpdate = connection.prepareStatement(updateSql);
prepareStatementUpdate.setString(1, text);
int count = prepareStatementUpdate.executeUpdate();
I am able to retrieve and display clearly with this code:
String selectSQL = "SELECT remarks FROM empTable WHERE emp_id = ? AND dep_id=? AND join_date='2013-11-15' ";
prepareStatement = connection.prepareStatement(selectSQL);
prepareStatement.setInt(1, 1);
prepareStatement.setString(2, 1);
ResultSet resultSet = prepareStatement.executeQuery();
while ( resultSet.next() ) {
byte[] bytedata = resultSet.getBytes( "remarks" );
AS400Text textConverter2 = new AS400Text(bytedata.length, 5035,sys);
String javaText = (String) textConverter2.toObject(bytedata);
System.out.println("Remarks after conversion to AS400Text :"+javaText);
}
It is working fine with JDBC, but for working with JPA, I need to convert to string for editing in web page or store in table. So, I have tried this way, but could not succeed:
String remarks = resultSet.getString( "remarks" );
byte[] bytedata = remarks.getBytes();
AS400Text textConverter2 = new AS400Text(bytedata.length, 5035,sys);
String javaText = (String) textConverter2.toObject(bytedata);
System.out.println("Remarks after conversion to AS400Text :"+javaText);

Thanks a lot Jon and Buck Calabro !
With your clues, I have succeeded with the following approach:
String remarks = new String(resultSet.getBytes("remarks"),"SJIS");
byte[] byteData = remarks.getBytes("SJIS");
CharConverter charConv = new CharConverter(5035, sys);
String convertedStr = charConv.byteArrayToString(5035, sys, byteData);
I am able to convert from string. I am planning to implement the same with JPA, and started coding.