We Keep Coding

Get access to a key in Java HttpClient response

New to API's in Java. I'm trying to get a response from an API and print some of it.
My response in the code below prints the whole dictionary, but I want to print just part of it. In the current way, I have no idea how I can get access to the dictionary, as my response is a String and I couldn't find a relevant BodyHandlers method.
How can I do that? Thanks a lot.
This is my code:
HttpRequest urlAnalysisRequest = HttpRequest.newBuilder()
.uri(URI.create("https://www.virustotal.com/api/v3/analyses/....(I put the id here)"))
.header("Accept", "application/json")
.header("x-apikey", "....(I put api key here)")
.method("GET", HttpRequest.BodyPublishers.noBody())
.build();
HttpResponse<String> urlAnalysisResponse;
try {
urlAnalysisResponse = HttpClient.newHttpClient().send(urlAnalysisRequest, HttpResponse.BodyHandlers.ofString());
System.out.println(urlAnalysisResponse.body());
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
This is my response. I'm trying to get "stats":
{
"meta": {
"url_info": {
"url": "http://www.facebook.com/",
"id": "114fb86b9b4e868f8bac2249eb5c444b545f0240c3dadd23312a0bc1622b5488"
}
},
"data": {
"attributes": {
"date": 1641238171,
"status": "completed",
"stats": {
"harmless": 84,
"malicious": 0,
"suspicious": 0,
"undetected": 9,
"timeout": 0
},
..........

The response is written in a format known as JSON. You need a library to parse this. There are a few options; I strongly suggest you go with Jackson.
You can choose to refer to it with string paths, or, you can make a java class that 'matches' this output, e.g:
class VirusTotalResponse {
VTMeta meta;
VTData data;
}
class VTMeta {
VTUrlInfo urlInfo;
}
class VTUrlInfo {
String url;
String id;
}
and so on. With all those classes in place, turn them all into proper classes (use your IDE's various options, or use Project Lombok) and then just ask Jackson to turn that response into an instance of VirusTotalResponse and you'll have a nice shiny java object, you can then just:
int harmless = response.getData().getAttributes().getStats().getHarmless();

You can use ObjectMapper which is library for handling JSON data easily. You can use not only Class for mapping, but also Map.
I recommend this tutorial site.
https://www.baeldung.com/jackson-object-mapper-tutorial

PACT testing: correct way to write PactDslJsonBody

I have the following JSON format
{
"file": {
"version": "v1.4",
"release": "1.1"
},
"status": "ON",
"document": {
"status": "NOT_FOUND",
"release": "undefined"
}
}
and I would like to know how I can add the format into my PactDslJsonBody, something like?
DslPart result = new PactDslJsonBody()
.stringType("file.version", "v1.4")
.stringType("file.release", "1.1")
.stringType("status", "ON")
.stringType("document.status", "NOT_FOUND")
.stringType("document.release", "release")
.asBody();
Or is it possible to add a Java Pojo? I have the class ApplicationResponse:
public class ApplicationResponse {
private File file;
private String status;
private Document document;
//...
}
Something like ??
DslPart result = new PactDslJsonBody()
.object(ApplicationResponse)
.asBody();
What could be the best approach? could you please add an example

We attempted to do what you are trying to do using reflection to stub out our pojos. However, our classes carry many Lombok annotations & we couldn't get default values out from builder annotated fields. We gave up trying to use it. But a dev with more time could achieve this no doubt.
I am now actively creating Pacts for our projects and use both LambdaDsl and PactDslJsonBodyto build the interaction.
DslPart actualPactDsl = LambdaDsl.newJsonBody((body) -> {
body
.stringType("status", "ON")
.object("document", (doc) -> {
doc.stringType("status", "NOT_FOUND");
doc.stringType("release", "undefined");
})
.object("file", (file) -> {
file.stringType("version", "v1.4");
file.stringType("release", "1.1");
});
}).build();
or
String pactDslJson = new PactDslJsonBody()
.stringType("status", "ON")
.object("document")
.stringType("status", "NOT_FOUND")
.stringType("release", "undefined")
.closeObject()
.object("file")
.stringType("version", "v1.4")
.stringType("release", "1.1")
.closeObject()
.getBody().toString();
Both of these examples will produce the Json string from your example.
The examples that are a part of Pact-Jvm are really helpful to get your head around the different types of tests you can create.

Ninja framework endpoint throws 500 error when trying to map JSON to custom object

So I've got a Ninja endpoint here:
public Result processRecurring(Context context, RecurOrderJSON recurOrderJSON) {
String id = recurOrderJSON.id;
String event_type = recurOrderJSON.event_type;
String request_id = recurOrderJSON.request_id;
//Map data = recurOrderJSON.data;
//recurringRouter(event_type, data);
log.info("ID value");
log.info(id);
return JsonResponse.build()
.message("OK")
.toResult();
}
The class I am trying to map to:
public class RecurOrderJSON {
public String id;
public String event_type;
public String request_id;
// Maybe switch data type?
//public Map data;
}
And the route:
router.POST().route("/recurring").with(RecurringController::processRecurring);
I am just trying to send some simple JSON to a webhook and for some reason the object mapping doesn't seem to be working. I think maybe I am misunderstanding the documentation?
http://www.ninjaframework.org/documentation/working_with_json_jsonp.html
The example they give you is this:
If you send that JSON to your application via the HTTP body you only need to add the POJO class to the controller method and Ninja will parse the incoming JSON for you:
package controllers;
public class ApplicationController {
public Result parsePerson(Person person) {
String nameOfPerson = person.name; // will be John Johnson
...
}
}
As far as I can tell, I am doing this correctly? Am I understanding the documentation wrong? Here's an example JSON object - currently I am only trying to grab the top level strings, but I'll eventually want to grab data as well:
{
"id": "hook-XXXXX",
"event_type": "tx-pending",
"data": {
"button_id": "static",
"publisher_organization": "org-XXXXXXX",
"campaign_id": "camp-097714a40aaf8965",
"currency": "USD",
"order_currency": "USD",
"id": "tx-XXXXXXX",
"category": "new-user-order",
"modified_date": "2018-10-15T05:41:12.577Z",
"order_total": 9680,
"button_order_id": "btnorder-77c9e56fd990f127",
"publisher_customer_id": "XymEz8GO2M",
"rate_card_id": "ratecard-41480b2a6b1196a7",
"advertising_id": null,
"event_date": "2018-10-15T05:41:06Z",
"status": "pending",
"pub_ref": null,
"account_id": "acc-4b17f5a014d0de1a",
"btn_ref": "srctok-0adf9e958510b3f1",
"order_id": null,
"posting_rule_id": null,
"order_line_items": [
{
"identifier": "Antique Trading Card",
"description": "Includes Lifetime Warranty",
"amount": 9680,
"publisher_commission": 968,
"attributes": {},
"total": 9680,
"quantity": 1
}
],
"order_click_channel": "webview",
"order_purchase_date": null,
"validated_date": null,
"amount": 968,
"customer_order_id": null,
"created_date": "2018-10-15T05:41:12.577Z",
"commerce_organization": "org-XXXXXX"
},
"request_id": "attempt-XXXXXXX"
}
Currently I am just trying to get the string values, yet I am constantly getting a 500 error and no other indication in my logs of any error.
As far as I can tell, Ninja should just automatically map the JSON to my object, correct?

I successfully reproduced your issue, and then fixed it.
First, for easy way to try/test, I recommend (temporary) modifications:
package controllers;
import models.RecurOrderJSON;
import ninja.Context;
import ninja.Result;
public class RecurringController {
public Result processRecurring(Context context, RecurOrderJSON recurOrderJSON) {
log.info("recurOrderJSON => " + recurOrderJSON);
return ninja.Results.ok();
}
}
And then, update your model this way:
package models;
import java.util.Map;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
#JsonIgnoreProperties(ignoreUnknown = true)
public class RecurOrderJSON {
public String id;
public String event_type;
public String request_id;
public Map data;
#Override
public String toString() {
return "RecurOrderJSON [id=" + id + ", event_type=" + event_type + ", request_id=" + request_id + ", data="
+ data.toString() + "]";
}
}
You can notice:
The data type must stay raw (generic can't be used here)
the important #JsonIgnoreProperties(ignoreUnknown = true) annotation to avoid deserialize issue, if ever your source data does not perfectly match your model (be sure to use the recent version of annotation, in fasterxml sub-package, instead of the old one, in codehaus sub-package)
the toString() implementation only allowing quick check of OK/KO deserialization
Then you can easily test the system with wget, or curl:
curl -H 'Content-Type: application/json' -d "#/tmp/jsonINput.json" -X POST http://localhost:8080/recurring
Notice it is very important to specify the Content-type for good interpretation.
With the /tmp/jsonINput.json file containing exactly the json contents you specified in your question.
This way, everything is working like a charm, obtaining this output:
recurOrderJSON => RecurOrderJSON [id=hook-XXXXX, event_type=tx-pending, request_id=attempt-XXXXXXX, data={button_id=static, publisher_organization=org-XXXXXXX, campaign_id=camp-097714a40aaf8965, currency=USD, order_currency=USD, id=tx-XXXXXXX, category=new-user-order, modified_date=2018-10-15T05:41:12.577Z, order_total=9680, button_order_id=btnorder-77c9e56fd990f127, publisher_customer_id=XymEz8GO2M, rate_card_id=ratecard-41480b2a6b1196a7, advertising_id=null, event_date=2018-10-15T05:41:06Z, status=pending, pub_ref=null, account_id=acc-4b17f5a014d0de1a, btn_ref=srctok-0adf9e958510b3f1, order_id=null, posting_rule_id=null, order_line_items=[{identifier=Antique Trading Card, description=Includes Lifetime Warranty, amount=9680, publisher_commission=968, attributes={}, total=9680, quantity=1}], order_click_channel=webview, order_purchase_date=null, validated_date=null, amount=968, customer_order_id=null, created_date=2018-10-15T05:41:12.577Z, commerce_organization=org-XXXXXX}]

Given the specific input code with data field commented out
//public Map data;
and the posted input JSON that includes this field, the request should fail with 400 Bad Request.
The reason being that Ninja uses Jackson for JSON parsing and it will throw on unknown fields by default.
The quick workaround is to add #JsonIgnoreProperties annotation to RecurOrderJSON class.
e.g.
#JsonIgnoreProperties(ignoreUnknown = true)
public class RecurOrderJSON {
...
}
See: Ignoring new fields on JSON objects using Jackson
Now if the error was not 400 there isn't much information to go by as there doesn't seem to be anything else obviously wrong with the code.
Either post an SSCCE demonstrating the problem or attempt to debug by surfacing the error page with the following method:
Launch the application in debug mode with mvn package ninja:run
Access the end-point with a tool that allows to inspect the response in detail such as curl e.g.
Store request JSON in input.json
Run curl -v -o result.html -H 'Content-Type: application/json' --data '#input.json' http://localhost:8080/recurring
Open result.html to examine the response

Might it be that you are performing a bad request (hence the JSON is not found) but for some Ninja bug it returns error 500?
For example you can take a look here where is stated that parsing an empty JSON in a JSON request does leads to a misguiding error (500) while it is supposed to return 400 "Bad Request"

Context not needed in processRecurring and use Results.json() and return original
public Result processRecurring(RecurOrderJSON recurOrderJSON) {
String id = recurOrderJSON.id;
String event_type = recurOrderJSON.event_type;
String request_id = recurOrderJSON.request_id;
//Map data = recurOrderJSON.data;
//recurringRouter(event_type, data);
log.info("ID value");
log.info(id);
return Results.json().render(recurOrderJSON);
}
Make sure you get the namespace in your RecurOrderJSON
package models;
public class RecurOrderJSON {
public String id;
public String event_type;
public String request_id;
// Maybe switch data type?
//public Map data;
}
Good luck!

How to call an aws java lambda function from another AWS Java Lambda function when both are in same account, same region

I have a java aws lambda function or handler as AHandler that does some stuff e.g. It has been subscribed to SNS events, It parses that SNS event and log relevant data to the database.
I have another java aws lambda BHandler, Objective of this BHandler to receive a request from AHandler and provide a response back to AHandler. Because BHandler's objective is to provide a response with some json data. and that would be used by the AHandler.
May I see any clear example which tells how we can do such things ?
I saw this example call lambda function from a java class and Invoke lambda function from java
My question talks about that situation, when one aws java lambda function (or handler) calls to another aws java lambda function when both are in same region, same account,same vpc execution stuff, same rights. In that case aws java lambda function can directly call( or invoke) to another or still it has to provide aws key,region etc stuff (as in above links) ? A clear example/explanation would be very helpful.
EDIT
The AHandler who is calling another Lambda function (BHandler) , exist on same account have given complete AWSLambdaFullAccess with everything e.g.
“iam:PassRole",
"lambda:*",
Here is the code to call :
Note : Below code works when I call the same function with everything same from a normal java main function. But its not working like calling from on lambda function (like ALambdaHandler calling BLambdaHandler as a function call). Even its not returning any exception. Its just showing timeout, its got stuck at the code of: lambdaClient.invoke
String awsAccessKeyId = PropertyManager.getSetting("awsAccessKeyId");
String awsSecretAccessKey = PropertyManager.getSetting("awsSecretAccessKey");
String regionName = PropertyManager.getSetting("regionName");
String geoIPFunctionName = PropertyManager.getSetting("FunctionName");
Region region;
AWSCredentials credentials;
AWSLambdaClient lambdaClient;
credentials = new BasicAWSCredentials(awsAccessKeyId,
awsSecretAccessKey);
lambdaClient = (credentials == null) ? new AWSLambdaClient()
: new AWSLambdaClient(credentials);
region = Region.getRegion(Regions.fromName(regionName));
lambdaClient.setRegion(region);
String returnGeoIPDetails = null;
try {
InvokeRequest invokeRequest = new InvokeRequest();
invokeRequest.setFunctionName(FunctionName);
invokeRequest.setPayload(ipInput);
returnDetails = byteBufferToString(
lambdaClient.invoke(invokeRequest).getPayload(),
Charset.forName("UTF-8"),logger);
} catch (Exception e) {
logger.log(e.getMessage());
}
EDIT
I did everything as suggested by others and followed everything. At the end I reached to AWS support, and the problem was related to some VPC configurations stuff, and that got solved.If you have encountered similar stuff, then may be check security configs, VPC stuff.

We have achieved this by using com.amazonaws.services.lambda.model.InvokeRequest.
Here is code sample.
public class LambdaInvokerFromCode {
public void runWithoutPayload(String functionName) {
runWithPayload(functionName, null);
}
public void runWithPayload(String functionName, String payload) {
AWSLambdaAsyncClient client = new AWSLambdaAsyncClient();
client.withRegion(Regions.US_EAST_1);
InvokeRequest request = new InvokeRequest();
request.withFunctionName(functionName).withPayload(payload);
InvokeResult invoke = client.invoke(request);
System.out.println("Result invoking " + functionName + ": " + invoke);
}
public static void main(String[] args) {
String KeyName ="41159569322017486.json";
String status = "success";
String body = "{\"bucketName\":\""+DBUtils.S3BUCKET_BULKORDER+"\",\"keyName\":\""+KeyName+"\", \"status\":\""+status+"\"}";
System.out.println(body);
JSONObject inputjson = new JSONObject(body);
String bucketName = inputjson.getString("bucketName");
String keyName = inputjson.getString("keyName");
String Status = inputjson.getString("status");
String destinationKeyName = keyName+"_"+status;
LambdaInvokerFromCode obj = new LambdaInvokerFromCode();
obj.runWithPayload(DBUtils.FILE_RENAME_HANDLER_NAME,body);
}
}

Make sure the role which your Lambda function executes with has lambda:InvokeFunction permission.
Then use AWS SDK to invoke the 2rd function. (Doc: http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/lambda/AWSLambdaClient.html#invoke(com.amazonaws.services.lambda.model.InvokeRequest))
Edit: For such a scenario, consider using Step Functions.

We had similar problem and tried to gather various implementations to achieve this. Turns out it had nothing to do with the code.
Few basic rules:
Ensure proper policy and role for your lambda function, at minimum:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:::"
},
{
"Effect": "Allow",
"Action": [
"lambda:InvokeFunction"
],
"Resource": [
""
]
}
]
}
Have functions in same regions.
No VPC configurations needed. If your applications have VPC, make sure your lambda function has appropriate role policy (refer AWSLambdaVPCAccessExecutionRole)
Most important (primarily why it was failing for us), set right timeouts and heap sizes. Calling Lambda is going to wait until called one is finished. Simple math of 2x the called lambda values works. Also this was only with java lambda function calling another java lambda function. With node js lambda function calling another lambda function did not have this issue.
Following are some implementations that works for us:
Using service interface
import com.amazonaws.regions.Regions;
import com.amazonaws.services.lambda.AWSLambdaAsyncClientBuilder;
import com.amazonaws.services.lambda.invoke.LambdaInvokerFactory;
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
public class LambdaFunctionHandler implements RequestHandler {
#Override
public String handleRequest(Object input, Context context) {
context.getLogger().log("Input: " + input);
FineGrainedService fg = LambdaInvokerFactory.builder()
.lambdaClient(
AWSLambdaAsyncClientBuilder.standard()
.withRegion(Regions.US_EAST_2)
.build()
)
.build(FineGrainedService.class);
context.getLogger().log("Response back from FG" + fg.getClass());
String fgRespone = fg.callFineGrained("Call from Gateway");
context.getLogger().log("fgRespone: " + fgRespone);
// TODO: implement your handler
return "Hello from Gateway Lambda!";
}
}
import com.amazonaws.services.lambda.invoke.LambdaFunction;
public interface FineGrainedService {
#LambdaFunction(functionName="SimpleFineGrained")
String callFineGrained(String input);
}
Using invoker
import java.nio.ByteBuffer;
import com.amazonaws.services.lambda.AWSLambdaClient;
import com.amazonaws.services.lambda.model.InvokeRequest;
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
public class LambdaFunctionHandler implements RequestHandler {
#Override
public String handleRequest(Object input, Context context) {
context.getLogger().log("Input: " + input);
AWSLambdaClient lambdaClient = new AWSLambdaClient();
try {
InvokeRequest invokeRequest = new InvokeRequest();
invokeRequest.setFunctionName("SimpleFineGrained");
invokeRequest.setPayload("From gateway");
context.getLogger().log("Before Invoke");
ByteBuffer payload = lambdaClient.invoke(invokeRequest).getPayload();
context.getLogger().log("After Inoke");
context.getLogger().log(payload.toString());
context.getLogger().log("After Payload logger");
} catch (Exception e) {
// TODO: handle exception
}
// TODO: implement your handler
return "Hello from Lambda!";
}
}
AWSLambdaClient should be created from builder.

You can use LambdaClient to invoke Lambda asynchronously by passing InvocationType.EVENT parameter. Look at an example:
LambdaClient lambdaClient = LambdaClient.builder().build();
InvokeRequest invokeRequest = InvokeRequest.builder()
.functionName("functionName")
.invocationType(InvocationType.EVENT)
.payload(SdkBytes.fromUtf8String("payload"))
.build();
InvokeResponse response = lambdaClient.invoke(invokeRequest);

Escape JSON string in Java

I'm using Google's com.google.api.client.json.GenericJson and com.fasterxml.jackson.core.JsonGenerator. I would like to serialize JSON object and escape quotes and backslashes so that I can pass that string in Bash. And afterwards deserialize that string.
GenericJson.toString produces simple JSON, but \n etc. are not escaped:
{commands=ls -laF\ndu -h, id=0, timeout=0}
is there a simple way how to get something like this:
"{commands=\"ls -laF\\ndu -h\", id=0, timeout=0}"
I don't want to reinvent the wheel, so I'd like to use Jackson or an existing API, if possible.

No additional dependencies needed: You're looking for JsonStringEncoder#quoteAsString(String).
Click for JsonStringEncoder javadoc
Example:
import com.fasterxml.jackson.core.io.JsonStringEncoder;
JsonStringEncoder e = JsonStringEncoder.getInstance();
String commands = "ls -laF\\ndu -h";
String encCommands = new String(e.quoteAsString(commands));
String o = "{commands: \"" + encCommands + "\", id: 0, timeout: 0}"
Ref: http://fasterxml.github.io/jackson-core/javadoc/2.1.0/com/fasterxml/jackson/core/io/JsonStringEncoder.html

Using Gson for serialization proved to be quite easy and bulletproof. Afterwards Apache's commons-lang3 = 3.1 escapeEcmaScript is used. In 3.2 there's also escapeJson method.
import com.google.api.client.json.GenericJson;
import com.google.api.client.util.Key;
import com.google.gson.Gson;
import org.apache.commons.lang3.StringEscapeUtils;
public class MyJson extends GenericJson {
#Key("commands")
public String commands;
public String serialize() throws IOException {
Gson gson = new Gson();
String g = gson.toJson(this);
return StringEscapeUtils.escapeEcmaScript(g);
}
}
This produces escaped JSON:
{\"commands\":\"ls -laF\\ndu -h\"}
Deserialization is then quite simple:
protected MyJson deserialize(String str) throws IOException {
String json = StringEscapeUtils.unescapeEcmaScript(str);
JsonObjectParser parser = (new JacksonFactory()).createJsonObjectParser();
return parser.parseAndClose(new StringReader(json), MyJson.class);
}
The escapeEcmaScript method isn't complicated, it does following replacement:
{"'", "\\'"},
{"\"", "\\\""},
{"\\", "\\\\"},
{"/", "\\/"}
But at least is something I don't have to care about.