I'm trying to create a Java program, that will allow users to record and store API calls and responses in a NoSQL database. Basically, I want to take in the call name, the parameters, and the response and store that in the database. The intention here is to let users re-use the calls w/ the same parameters by accessing the data from the database, as opposed to making a live API call. Is there a way to listen to API calls on a specific URL using Java, and if so how can it be done?
I'm not sure if I completely understand the reason for doing it. But the answer to this question is tightly dependent on the technology you want to use for HTTP client, type of request/response and the NoSQL database and strategy you want to use.
As a suggestion, you can create two data classes, RequestDummy and ResponseDummy with all the required fields you need and use something like gson to serialize or deserialize them and store them as JSON file with the database to be more readable. Ex:
class RequestDummy {
String url;
String method; // (GET|HEAD|POST|PUT|PATCH|DELETE)
Map<String, String> headers;
String payload; // For (POST|PUT|PATCH) use Base64 to encode the binary to String
}
class ResponseDummy {
Map<String, String> headers;
int statusCode;
String body; // use Base64 to encode the binary to String
}
And use them during the HTTP call (briefly to just explain the idea):
// Initialize RequestDummy
RequestDummy req ...
HttpURLConnection con = (HttpURLConnection) new URL(req.url).openConnection();
// optional default is GET
con.setRequestMethod(req.method);
req.headers.entrySet().forEach(e -> con.setRequestProperty(e.getKey(), e.getValue()));
int responseCode = con.getResponseCode();
BufferedReader in = new BufferedReader(new InputStreamReader());
byte[] data = con.getInputStream().readAllBytes() // Java9
// Initialize ResponseDummy
RequestDummy req ...
in.close();
In 99% storing request and response is a bad design. As a suggestion, it's not a good idea to store HTTP request and response for any usage. They are just for communication, you must store the requested data which the HTTP request built based on, and the generated data from the HTTP response.
Related
I am having a hard time finding documentation on which is the correct return type.
For example, if I have a REST endpoint which lookups and returns a String should the endpoint have a return type of Future<String> or String? Further, what impact would this have on the event loop (i.e. would returning String over Future<String> cause more blocking)?
Thanks!
If you look at section (2) of the quick-start at https://vertx.io/get-started, you'll see chunk of code I've pasted below (I've added some numbered comments):
// Mount the handler for all incoming requests at every path and HTTP method
router
.route() // (1)
.handler(context -> { // (2)
// Get the address of the request
String address = context.request().connection().remoteAddress().toString();
// Get the query parameter "name"
MultiMap queryParams = context.queryParams();
String name = queryParams.contains("name") ? queryParams.get("name") : "unknown";
// Write a json response
context.json( // (3)
new JsonObject()
.put("name", name)
.put("address", address)
.put("message", "Hello " + name + " connected from " + address)
);
});
What this is doing is:
Registering a Handler (basically a callback) that will be invoked for every request that the router receives.
The handler will be called with a RoutingContext, which contains references to an HttpServerRequest object representing the current request, as well as an HttpServerResponse object representing the response. The latter allows you to control the response that is sent back to the client (ie headers, body, etc).
context.json() is a convenience method for writing a JSON formatted response payload - the body will be correctly formatted, the content-type header will be set, etc.
Fundamentally, what .json() is doing is:
final JsonObject myJson = ...;
final Buffer myJsonBuffer = Json.encodeToBuffer(myJson);
context.response()
.putHeader(HttpHeaders.CONTENT_TYPE, "application/json")
.write(myJsonBuffer);
Those last three lines are how the response is actually sent back to the client.
For a more detailed explanation, check out the Vert.x Web documentation regarding responses here.
Out of the box Vert.x allows any primitive/simple type, String, or
buffers to be sent as messages. However, it’s a convention and common
practice in Vert.x to send messages as JSON
I guess this is the documentation you are looking for. You can return Strings to the eventBus. Though Json is mostly used
new JsonObject().put("key", "stringValue");
It's better to return the String than the Future. The Future will need special Codec.
I have to send a query with some headers and graphQL variables as a POST call to a GraphQL API in java. I also have to send some headers and authentication parameters in the query. Right now, I am doing a manual call in POSTMAN, but I want to do this programmatically. Can you guys help me where to start off. My query and variables are as follows
query sampleQuery($param: SampleQueryParams!, $pagingParam: PagingParams) {
sampleLookup(params: $param, pagingParams: $pagingParam) {
ID
name1
name2
}
}
And my GraphQL variables are as follows :
{"param": {"id": 58763897}, "pagingParam": {"pageNumber": 0, "pageSize": 10 } }
I have no clue where to start at. Could you guys please help
Below is a typical graphql endpoint for a java backend
There are 2 basic flows here
1 an endpoint for http request that can handle the graghql query as a string and a map / json representation of the query's input variables
2 the graphql wiring for the backend that collates and returns the data
the backend would typicaly have an endpoint that looks like this (1)
public Map<String, Object> graphqlGET(#RequestParam("query") String query,
#RequestParam(value = "operationName", required = false) String operationName,
#RequestParam("variables") String variablesJson) throws IOException {...
note we have 3 inputs
a query string,
a string usually json for the querys variables
an optional "operationName"
once we have parsed these input parameters we would typically send them to the graphql implementation for the query
which could look like this (1)
private Map<String, Object> executeGraphqlQuery(String operationName,
String query, Map<String, Object> variables) {
ExecutionInput executionInput = ExecutionInput.newExecutionInput()
.query(query)
.variables(variables)
.operationName(operationName)
.build();
return graphql.execute(executionInput).toSpecification();
}
here the graphql object has all the wiring to return the data
So a solution is just to post the correctly formatted input parameters to the backend
I often use android and a http client that works with older android versions so a post request in kotlin could look like this as a very simple example
val client = HttpClients.createDefault()
val httpPost = HttpPost(url)
val postParameters = ArrayList<NameValuePair>()
postParameters.add(BasicNameValuePair("query", "query as string"))
postParameters.add(BasicNameValuePair("variables", "variables json string"))
httpPost.entity = UrlEncodedFormEntity(postParameters, Charset.defaultCharset())
val response = client.execute(httpPost)
val ret = EntityUtils.toString(response.getEntity())
please note the implementation for the http post with be dependent on the way the backend java implementation is set up
for the basic http client and post setup many good examples here
How to use parameters with HttpPost
possibly related
graphql allows for a introspection flow which publishes details on the query structure the implementation supports
more info here
https://graphql.org/learn/introspection/
[1] https://github.com/graphql-java/graphql-java-examples
I would recommend using graphql-java-codegen plugin for these purposes.
It provides a possibility to generate classes based on the schema which you can supply to any HTTP client.
For example, GraphQL server has following schema and we want to perform productById query:
type Query {
productById(id: ID!): Product
}
type Product {
id: ID!
title: String!
price: BigDecimal!
}
graphql-java-codegen will generate all classes required for you to perform a query:
// preparing request
ProductByIdQueryRequest request = new ProductByIdQueryRequest();
request.setId(productId);
// preparing response projection (which fields to expect in the response)
ProductResponseProjection responseProjection = new ProductResponseProjection()
.id()
.title()
.price();
// preparing a composite graphql request
GraphQLRequest graphQLRequest = new GraphQLRequest(request, responseProjection);
// performing a request with the constructed object
ProductByIdQueryResponse responseBody = restTemplate.exchange(URI.create("https://product-service:8080/graphql"),
HttpMethod.POST,
new HttpEntity<>(graphQLRequest.toHttpJsonBody()),
ProductByIdQueryResponse.class).getBody();
// Fetching a serialized object from response
Product product = responseBody.productById();
More examples can be found on GitHub: https://github.com/kobylynskyi/graphql-java-codegen#supported-plugins
I have some forms and databases that will fill up some POJO's with data. Using Json (gson) I"m going to upload the data to my web server. The server will send back a responses that will eventually be placed back in objects.
My question is, should I encode/decode my objects to json and then pass strings back and forth to my WebApi class? Or should I pass my WebApi class my object and have it pass back an appropriate response object (based on the method I call.)?
So at when I call my Api that has all the http conection stuff in it, it would look something like this.
myWebApi postSomeData( jsonData ); //
Should jsonData be a pojo that will be encoded into json string inside myWebApi or should I encode it into json and pass in a string.
Or in otherwords
String postSomeData( String jsonData){
web code here..
}
or
ResponsePojo postSomeData( PostData myData){
...
myMapper.MapFrom(myData); //converts pojo to json
...
webcode
}
At some point there will be images included in the data needed to be uploaded.
I would recommend using json for data transfer, it will make your services technology independent.
You can use flexjon to serialize java objects in a pretty direct way on the server side.
For images you will need to do post http POST anyway, this should be done separately from data submit since an error in a image upload should not result in data loss. Typically you'll associate the id of your data object in the uploaded image (or reversly), so you need to insert your object to db and wait for the response to insert the dependency id to your image.
Best Regards,
Zied Hamdi
http://1vu.fr
I want a java way to extract the parameters of a URL regardless the way these parameters are written in it, in the regular way like( https://www.facebook.com/Doly.mohamed.Smile9?ref=stream&hc_location=stream ) it's so easy because all i have to do is :
URL url = new URL("www.blabla....etc");
String query = url.getQuery();
try{
String [] params = query.split("&");
for(int i= 0 ; i < params.length; i++){
String [] split = params[i].split("=");
parameters.put(split[0], split[1]);
}
}catch(NullPointerException ex){}
so the parameters values would be :
key = ref value = stream , key = hc_location value = stream
but what shall i do if the URL has parameters written in another way or if the URL does't has it's parameters written in it like in the case of the doPost() way.
and is there is a way to get the extraPathInfo from a URL without using servlets?
You could do that easily with Apache's HTTP utils.
URIBuilder uriBuilder = new URIBuilder(uriString);
List<NameValuePair> urlParameters = uriBuilder.getQueryParams();
String uriWithoutParameters = uriBuilder.clearParameters().toString();
Now you could, for example, easily convert the GET request to a POST request, using other classes from the http utils API.
There is a difference between GET and POST urls
In GET url, parameters are part of URL and in POST they are part of Request-body.
So in POST, the URL may or may not contain the request params, and unless you don't have them in the URL its not possible to extract.
The POST request method is designed to request that a web server
accept the data enclosed in the request message's body for storage.1
It is often used when uploading a file or submitting a completed web
form.
So unless you have the POST request's body. Its difficult to extract the Parameter.
Typically you need HTTP request parameters on HTTP server side. Java HTTP server will parse the request and pass it as ServletRequest object to Servlet.service method. ServletRequest has methods to access the request parameters.
I'm currently using the OAuth-Signpost Java library to sign requests sent from a client to a server which implements OAuth authentication. When making GET requests (using HttpURLConnection) everything works fine: requests are signed, parameters are included and signatures match in destination. However, it doesn't seem to work with POST requests. I'm aware of the issues that may come up when signing POST using HttpURLConnection, so I moved to the Apache HttpComponents library for these requests. The parameters I send in the following example are plain strings and a XML-like string ('rxml'). My code goes as follows:
public Response exampleMethod(String user, String sp, String ep, String rn, String rxml){
//All these variables are proved to be correct (they work right in GET requests)
String uri = "...";
String consumerKey = "...";
String consumerSecret = "...";
String token = "...";
String secret = "...";
//create the parameters list
List<NameValuePair> params = new ArrayList<NameValuePair>();
params.add(new BasicNameValuePair("user", user));
params.add(new BasicNameValuePair("sp", sp));
params.add(new BasicNameValuePair("ep", ep));
params.add(new BasicNameValuePair("rn", rn));
params.add(new BasicNameValuePair("rxml", rxml));
// create a consumer object and configure it with the access
// token and token secret obtained from the service provider
OAuthConsumer consumer = new CommonsHttpOAuthConsumer(consumerKey, consumerSecret);
consumer.setTokenWithSecret(token, secret);
// create an HTTP request to a protected resource
HttpPost request = new HttpPost(uri);
// sign the request
consumer.sign(request);
// set the parameters into the request
request.setEntity(new UrlEncodedFormEntity(params));
// send the request
HttpClient httpClient = new DefaultHttpClient();
HttpResponse response = httpClient.execute(request);
//if request was unsuccessful
if(response.getStatusLine().getStatusCode()!=200){
return Response.status(response.getStatusLine().getStatusCode()).build();
}
//if successful, return the response body
HttpEntity resEntity = response.getEntity();
String responseBody = "";
if (resEntity != null) {
responseBody = EntityUtils.toString(resEntity);
}
EntityUtils.consume(resEntity);
httpClient.getConnectionManager().shutdown();
return Response.status(200).entity(responseBody).build();
}
When I send a POST request to the server I get an error telling that the signatures (the one I send and the one the server calculates by itself) don't match, so I guess it has to do with the base string they are signing and the way the POST signing works, since they're handling the same keys and secrets in both sides (checked).
I've read that a way to go through this is setting the parameters as part of the URL (as in a GET request). It wouldn't work for me though, since the XML parameter may exceed the URL length so it needs to be sent as a POST parameter.
I suppose I'm doing something wrong either signing the POST requests or handling the parameters, but I don't know what it is. Please, could you help me out?
P.S: I apologize if I lack context, error traces or additional information regarding this issue, but I'm newbie around here. So please don't hesitate to ask me for more information if you need it.
A bit of backstory/explanation
I've been having a similar problem for the past couple of days, and had almost given up. Until I heard that the guy at my company that was putting up the services I was communicating with, had configured them to read the OAuth information from the query string instead of header parameters.
So instead of reading it from the header parameter Authorization that Signpost puts into the request when you pass it on to be signed, for instance [Authorization: OAuth oauth_consumer_key="USER", oauth_nonce="4027096421883800497", oauth_signature="Vd%2BJEb0KnUhEv1E1g3nf4Vl3SSM%3D", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1363100774", oauth_version="1.0"], the services where trying to read the query string, for example http://myservice.mycompany.com?oauth_consumer_key=USER&oauth_nonce=4027096421883800497&oauth_signature=Vd%2BJEb0KnUhEv1E1g3nf4Vl3SSM%3D&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1363100774&oauth_version=1.0.
The problem with this is that when I tried to sign the url and then build a HttpPost request with it, the url got a basestring with the prefix GET instead of POST which gave another signature then the one the service computed. Signpost isn't doing anything wrong, its url signing method is just by default set to GET with no other possibility available out of the box. This is so because you should read header parameters when doing POST, not the query string (Going to egg the house of that "colleague" of mine), and Signpost adds these when signing request which you should do when doing POST.
The signingbasestring can be observed in the SigningBaseString class method generate in Signpost.
Solution
Now this is how I did it, but other ways may be possible or even better.
Get the signpost source code and add it to your project. Can get it here
Locate the OAuthConsumer class and change the signing method so that you can pass on information that the request should be POST. In my case I added a boolean like so public String sign(String url, boolean POST)
Now you need to change the sign method in the AbstractOAuthConsumer class, which CommonsHttpOAuthConsumer and DefaultOAuthConsumer extend. In my case I added the boolean variable to the method and the following if(POST) request.setMethod("POST"); right before the method calls sign(request);
Now the request is a Signpost specific object, HTTPRequest, so this will throw an error. You'll need to change it and add the method public void setMethod(String method);.
This will cause an error in the following classes HttpURLConnectionRequestAdapter, HttpRequestAdapter and UrlStringRequestAdapter. You'll need to add the method implementation to them all, but in different flavors. For the first you'll add
public void setMethod(String method){
try {
this.connection.setRequestMethod(method);
} catch (ProtocolException e) {
e.printStackTrace();
}
}
for the second you'll add
public void setMethod(String method){
try {
RequestWrapper wrapper = new RequestWrapper(this.request);
wrapper.setMethod(method);
request = wrapper;
} catch (org.apache.http.ProtocolException e) {
e.printStackTrace();
}
}
and for the last you'll add
public void setMethod(String method){
mMethod = method;
}
Be warned that I've only used and tried the first and last. But this at least gives you an idea about how to fix the problem, if you are having the same one as I.
Hope this helps in anyway.
-MrDresden
Signing POST requests using oauth-signpost and HttpURLConnection is doable, but it requires a bit of a hack:
The trick is to percent-encode the POST parameters, and add them to the OAuth library using method setAdditionalParameters().
See this article for an example.
This answer helped me.
but for PALINTEXT method, you do not need to have params and also url.
they don't change the signature. signature is constant and based on secrets.
but for SHA1 (and other methods) you can use the above answer.