LinkedIn rest API for people search has following format http://api.linkedin.com/people-search?search-param1=val1&search-param2=val2...
Since I intend to use this many time in my application, I am trying to create a Search object like this where we can set values for different search parameters and then a method called generateQueryUrl to generate the url in above format.
public class Search {
private String searchParam1;
private String searchParam2;
public void setSearchParam1(String val) { this.searchParam1 = val; }
public void setSearchParam2(String val) { this.searchParam2 = val; }
//Form the query url
public String generateQueryUrl(){
String url = "";
if(searchParam1 != null) {
url += "search-param1=" + searchParam1 + "&";
}
if(searchParam2 != null) {
url += "search-param2=" + searchParam2 + "&";
}
return url;
}
My question is, is there a better pattern/design to do this? If there are many parameters, checking for NULL and then appending corresponding parameter name, value seems to add redundant code to me.
Also please let me know if this approach is fine.
I think what you are designing is in fact a "Builder", in that you are building the URL that will be returned at the end. Think StringBuilder, or Apache's EqualsBuilder, HashCodeBuilder, etc.
For a more theoretical explanation, check http://sourcemaking.com/design_patterns/builder.
Now, as for your code, to properly build the URL, I would use "set" methods like you do, but inside them I would use Apache's HttpComponents (ex HttpClient) to properly append the parameters of the URL.
Think about it, if you insert a "&" in the value of one of the parameter, your query parameters will be messed up since you normally have to escape this character (and you don't seem to be doing it here). I prefer letting tested and known APIs (UriUtils) do that for me.
So, my class would look like so using Apache's HttpComponents:
public class SearchBuilder {
private URI baseUri;
private List<NameValuePair> parameters;
public SearchBuilder (URI baseUri) {
this.baseUri= baseUri;
this.parameters = new ArrayList<NameValuePair>();
}
public void addSearchParam1(String val) {
if(!StringUtils.isBlank(val)) {
parameters.add(new BasicNameValuePair("SearchParam1", val));
}
}
//Form the query url
public URI toURI(){
URI uri = URIUtils.createURI(baseUri.getScheme(), baseUri.getHost(), baseUri.getPort(), baseUri.getPath(), URLEncodedUtils.format(parameters, "UTF-8"), null);
return uri;
}
Edit: I added the "isBlank" check to ensure that parameters with null, empty of whitespace-only Strings will not be added to the query. Now, you can change that to check only for null, but I'm sure you got the idea.
You can maintain a Map with keys as the param names and values as the values. Your setter methods can put keys and values in the Map. Or, you can have one generic setter which accepts the name & the value.
In the generateQueryUrl method, you can just iterate over the Map.
You may also want to URLEncode the values if not being done at the later stage.
Related
Edit: I ended up solving the problem myself through more experimentation. The code seems quite verbose though so there is probably a better solution that doesn't involve typecasting strings to other things.
Answer posted below.
For my school work, I am supposed to create a GET mapping to receive a list of all entities of a specific type. This GET mapping should return simply all the entities if no parameter is provided, or otherwise it will apply something in the entity repository to use a JPQL query and the provided parameter which is used as an ordinal query parameter to filter the returned results.
"If no request parameter is provided, the existing functionality of returning all events
should be retained.
If more than one request parameter is specified, an error response should be
returned, indicating that at most one parameter can be provided.
If the ‘?status=XXX’ parameter is provided, with a status string value that does not
match the AEventStatus enumeration, an appropriate error response should be
returned."
I have tried to alter my GET mapping to have 3 optional #RequestParameter variables, but I found out that it is tedious logic-wise to check for the existence of multiple or no parameters, and then do something again based on the existence of which parameter is there.
Instead I tried this (I was in the middle of this and it is not complete):
#RequestMapping(
value="/aevents",
method = RequestMethod.GET)
public ResponseEntity<Object> getAllAEvents(HttpServletRequest request) {
if (request.getParameterMap().size() == 0) {
return ResponseEntity.status(HttpStatus.OK).body(repository.findAll());
}
if (request.getParameterMap().size() > 1) {
return new ResponseEntity<>("Can only handle one request paramter: title=, status= or minRegistrastions=", HttpStatus.BAD_REQUEST);
}
//incomplete from here
}
And I am now not sure if this is the correct approach or if I am simply overlooking something. I suppose I might be able to check for the names of the parameters that were provided in the request and then return a bad request response again if I find something that isn't a valid parameter. But I am not sure how to check the parameter map for the names of the parameters or if the parameter map even does what I think it does.
The parameters provided are supposed to be either an int, a value from an enum or a string.
Am I overlooking a simpler way to do this? i.e. a way to check the amount of parameters and the existence of parameters in a signature like:
#GetMapping("/aevents")
public List<AEvent> getAllAEvents(#RequestParam(required = false) String title,
#RequestParam(required = false) AEventStatus status,
#RequestParam(required = false) int minRegistrations) {
//Do something here
}
Or is my current approach feasible, and if it is, how do I continue on it?
Yes, you would likely do it your way, though:
You can inject the map of parameters directly in Spring.
Throw a ResponseStatusException (available since Spring 5) instead of fumbling around with the ResponseEntity.
#GetMapping("/aevents")
public List<AEvent> getAllAEvents( #RequestParam Map<String, String> allRequestParams){
if(allRequestParams.size() >1){
throw new ResponseStatusException(
HttpStatus.BAD_REQUEST,"too many params");
}
// do something
return list;
}
Answer to my own question after solving it:
What is the parameter map?
The parameterMap is actually a map of string keys and string arrays. To get the value for a parameter name (the key) you can get values of this key and then access it like an array.
However, using the parameterMap was not necessary. Instead it was better to just use the built-in Spring way of doing it which is by using the #RequestParam annotation with simply a #RequestParam Map<String, String> params in the method body.
Credits to https://stackoverflow.com/users/2255293/marco-behler for giving me an idea as well as providing a better way to throw exceptions.
#GetMapping("/aevents")
public List<AEvent> getAllAEvents(#RequestParam Map<String, String> params) {
if (params.size() == 0) { //Default case, no params
return repository.findAll();
}
if (params.size() > 1) { //Refuse to handle more than one provided param
throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "Can only handle one request parameter: title=, status= or minRegistrations=");
}
if (params.containsKey("title")) {
String value = params.get("title");
return repository.findByQuery("AEvent_find_by_title", ("%" + value + "%"));
}
if (params.containsKey("status")) {
String stringValue = params.get("status").toUpperCase();
for (AEventStatus e : AEventStatus.values()) {
if (e.name().equals(stringValue)) {
AEventStatus value = AEventStatus.valueOf(stringValue);
return repository.findByQuery("AEvent_find_by_status", value);
}
}
throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "status=" + stringValue + " is not a valid AEvent status value.");
}
if (params.containsKey("minRegistrations")) {
int value;
try {
value = Integer.parseInt(params.get("minRegistrations"));
} catch (NumberFormatException e) {
throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "Provided request parameter was not a valid number.");
}
return repository.findByQuery("AEvent_find_by_minRegistrations", value);
}
throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "Invalid query parameters.");
}
I'm trying to make sure my Jersey request parameters are sanitized.
When processing a Jersey GET request, do I need to filter non String types?
For example, if the parameter submitted is an integer are both option 1 (getIntData) and option 2 (getStringData) hacker safe? What about a JSON PUT request, is my ESAPI implementation enough, or do I need to validate each data parameter after it is mapped? Could it be validated before it is mapped?
Jersey Rest Example Class:
public class RestExample {
//Option 1 Submit data as an Integer
//Jersey throws an internal server error if the type is not Integer
//Is that a valid way to validate the data?
//Integer Data, not filtered
#Path("/data/int/{data}/")
#GET
#Produces(MediaType.TEXT_HTML)
public Response getIntData(#PathParam("data") Integer data){
return Response.ok("You entered:" + data).build();
}
//Option 2 Submit data as a String, then validate it and cast it to an Integer
//String Data, filtered
#Path("/data/string/{data}/")
#GET
#Produces(MediaType.TEXT_HTML)
public Response getStringData(#PathParam("data") String data) {
data = ESAPI.encoder().canonicalize(data);
if (ESAPI.validator().isValidInteger("data", data, 0, 999999, false))
{
int intData = Integer.parseInt(data);
return Response.ok("You entered:" + intData).build();
}
return Response.status(404).entity("404 Not Found").build();
}
//JSON data, HTML encoded
#Path("/post/{requestid}")
#POST
#Consumes({MediaType.APPLICATION_FORM_URLENCODED, MediaType.APPLICATION_JSON})
#Produces(MediaType.TEXT_HTML)
public Response postData(String json) {
json = ESAPI.encoder().canonicalize(json);
json = ESAPI.encoder().encodeForHTML(json);
//Is there a way to iterate through each JSON KeyValue and filter here?
ObjectMapper mapper = new ObjectMapper();
DataMap dm = new DataMap();
try {
dm = mapper.readValue(json, DataMap.class);
} catch (Exception e) {
e.printStackTrace();
}
//Do we need to validate each DataMap object value and is there a dynamic way to do it?
if (ESAPI.validator().isValidInput("strData", dm.strData, "HTTPParameterValue", 25, false, true))
{
//Is Integer validation needed or will the thrown exception be good enough?
return Response.ok("You entered:" + dm.strData + " and " + dm.intData).build();
}
return Response.status(404).entity("404 Not Found").build();
}
}
Data Map Class:
public class DataMap {
public DataMap(){}
String strData;
Integer intData;
}
The short answer is yes, though by "filter" I interpret it as "validate," because no amount of "filtering" will EVER provide you with SAFE data. You can still run into integer overflows in Java, and while those may not have immediate security concerns, they could still put parts of your application in an unplanned for state, and hacking is all about perturbing the system in ways you can control.
You packed waaaaay too many questions into one "question," but here we go:
First off, the lines
json = ESAPI.encoder().canonicalize(json);
json = ESAPI.encoder().encodeForHTML(json);
Aren't doing what you think they're doing. If your JSON is coming in as a raw String right here, these two calls are going to be applying mass rules across the entire string, when you really need to handle these with more surgical precision, which you seem to at least be subconsciously aware of in the next question.
//Is there a way to iterate through each JSON KeyValue and filter
here?
Partial duplicate of this question.
While you're in the loop discussed here, you can perform any data transformations you want, but what you should really be considering is using the JSONObject class referenced in that first link. Then you'll have JSON parsed into an object where you'll have better access to JSON key/value pairs.
//Do we need to validate each DataMap object value and is there a
dynamic way to do it?
Yes, we validate everything that comes from a user. All users are assumed to be trained hackers, and smarter than you. However if you handled filtering before you do your data mapping transformation, you don't need to do it a second time. Doing it dynamically?
Something like:
JSONObject json = new JSONObject(s);
Iterator iterator = json.keys();
while( iterator.hasNext() ){
String data = iterator.next();
//filter and or business logic
}
^^That syntax is skipping typechecks but it should get you where you need to go.
/Is Integer validation needed or will the thrown exception be good
enough?
I don't see where you're throwing an exception with these lines of code:
if (ESAPI.validator().isValidInput("strData", dm.strData, "HTTPParameterValue", 25, false, true))
{
//Is Integer validation needed or will the thrown exception be good enough?
return Response.ok("You entered:" + dm.strData + " and " + dm.intData).build();
}
Firstly, in java we have autoboxing which means this:
int foo = 555555;
String bar = "";
//the code
foo + bar;
Will be cast to a string in any instance. The compiler will promote the int to an Integer and then silently call the Integer.toString() method. Also, in your Response.ok( String ); call, THIS is where you're going to want to encodeForHTML or whatever the output context may be. Encoding methods are ALWAYS For outputting data to user, whereas canonicalize you want to call when receiving data. Finally, in this segment of code we also have an error where you're assuming that you're dealing with an HTTPParameter. NOT at this point in the code. You'll validate http Parameters in instances where you're calling request.getParameter("id"): where id isn't a large blob of data like an entire JSON response or an entire XML response. At this point you should be validating for things like "SafeString"
Usually there are parsing libraries in Java that can at least get you to the level of Java objects, but on the validation side you're always going to be running through every item and punting whatever might be malicious.
As a final note, while coding, keep these principles in mind your code will be cleaner and your thought process much more focused:
user input is NEVER safe. (Yes, even if you've run it through an XSS filter.)
Use validate and canonicalize methods whenever RECEIVING data, and encode methods whenever transferring data to a different context, where context is defined as "Html field. Http attribute. Javascript input, etc...)
Instead of using the method isValidInput() I'd suggest using getValidInput() because it will call canonicalize for you, making you have to provide one less call.
Encode ANY time your data is going to be passed to another dynamic language, like SQL, groovy, Perl, or javascript.
I'm building something of a relay server, using JAX-RS. I need to be able to extract any query parameters from a GET request and then re-wrap them into another request, to pass along to another server. I'm unfamiliar with MultivaluedMap, but I just figured out what is happening. The UriInfo class's getQueryParameters method returns a MultivaluedMap<String, String>. What bit me, unexpectedly, is that the values of each parameter are List values, even though they purport to be String values (by the way I read the JavaDoc).
In other words, if I have a key-value pair of foo=bar, in the URL query string, when I extract the parameter, it comes out as foo=[bar]. This totally throws me for a loop (a 500 Server Error, actually) when I try to re-wrap the parameter and send it along to the other server.
Is there another way to handle unpacking the query string from a Request, and then re-packing it for another Request? I'm including some code to illustrate my issue:
#GET
#Path("parameters")
public Response showParameters(#Context UriInfo uriInfo) {
MultivaluedMap<String, String> parameters = uriInfo.getQueryParameters();
StringBuffer sb = new StringBuffer();
sb.append("<h4>Parameters:</h4>");
if (parameters != null) {
sb.append("<ul>");
Iterator it = parameters.keySet().iterator();
while (it.hasNext()) {
String key = (String)it.next();
sb.append("<li>")
.append(key)
.append(": ")
.append(parameters.get(key))
.append("</li>");
}
sb.append("</ul>");
}
else {
sb.append("<p>None</p>");
}
return Response.ok(sb.toString()).build();
}
So, in summary, what gets printed out from the code above, if the request has query parameters, is something like this:
Parameters:
key1: [value1]
key2: [value2]
key3: [value3]
Is there another way to unpack/re-pack, and maybe avoid this whole issue altogether? Thanks.
Complete answer
#Jack deserves credit for pointing me in the right direction, and I am marking his answer as correct, but here is what I got working.
Client client = ClientBuilder.newClient();
// Assume instance variable providing URI (without query string).
WebTarget target = client.target(theRequestUri);
// Instance variable uriInfo.
MultivaluedMap<String, String> parameters = uriInfo.getQueryParameters();
if (parameters != null && parameters.size() > 0) {
Iterator it = parameters.keySet().iterator();
String key = null;
StringTokenizer st = null;
while (it.hasNext()) {
key = (String)it.next();
// RESTEasy API is quirky, here. It wraps the values in square
// brackets; moreover, each value is separated by a comma and
// padded by a space as well. ([one, two, three, etc])
st = new StringTokenizer(parameters.get(key).toString(), "[,]");
while (st.hasMoreTokens()) {
target = target.queryParam(key, st.nextToken().trim());
}
}
}
// Instance method: getContentType.
Response response = target.request().accept(getContentType()).get();
Its because of MultivaluedMap interface.
public interface MultivaluedMap<K, V> extends Map<K, List<V>> {
It returns the values as List.
Instead of parameters.get(key) try parameters.getFirst(key)
Note that this will drop the other values for the same parameter. It is possible to send multiple values for same parameter while making a rest call such as blahblah:8080?foo=bar1&foo=bar2. With getFirst() call you will get bar1 value only. If you are sure you will not get multiple calls, you can go with getFirst(key) approach
--- UPDATED ---
Based on your comments, it seems you need to iterate over the multivalued map and call queryParam on the WebTarget instance. I understand you are looking for a library/straight forward way to do this. I did not try RESTEasy. But code I believe should be simple enough.
multiValuesMap?.each { key, values->
webTargetInstance = webTargetInstance.queryParam(key, values as Object[])
}
The user can set a number of values (which are optional) for a search query that will be passed on to a REST api.
So I create a POJO to hold all the set values like
class Search{
private String minPrice;
private String maxPrice;
private String category;
// etc.pp.
}
When I construct the URL for the api call, i have to check wether that value is set all the time like
if (search.getMinPrice() != null){
url.append("&minprice=" + search.getMinPrice());
}
Is there a more convenient/ elegant way to do this
with pure Java, just a way to "do sth if sth is not null"
or even a library/tool that lets you construct Urls from objects?
Something like:
String url = "http://base.com/some/path?" + (maxPrice==null ? "" : "maxPrice="+maxPrice);
I have query string like that:
ObjectGUId=1abcde&ObjectType=2&ObjectTitle=maximumoflife&Content=racroi&TimeStamp=2012-11-05T17:20:06.056
And I have Java Object:
LogObject{
private String ObjectGUId;
private String ObjectType;
private String ObjectTitle;
private String Content;
private String TimeStamp;
}
So i want to parse this query string to this java Object.
I've searched and read many question but not gotten correct answer yet.
Show me what can solve this problem.
Inspired by #bruno.braga, here's a way using Apache http-components. You leverage all the parsing corner cases:
List<NameValuePair> params =
URLEncodedUtils.parse("http://example.com/?" + queryString, Charset.forName("UTF-8"));
That'll give you a List of NameValuePair objects that should be easy to work with.
If you do not really need to push the querystring into your own class (you might want that though), instead of parsing it manually, you could use the URLDecoder, as #Sonrobby has commented:
String qString = "ObjectGUId=1abcde&ObjectType=2&ObjectTitle=maximumoflife";
Uri uri = Uri.parse(URLDecoder.decode("http://dummy/?" + qString, "UTF-8"));
if (uri != null) {
for(String key: uri.getQueryParameterNames()) {
System.out.println("key=[" + key + "], value=[" + uri.getQueryParameter(key) + "]");
}
}
The "dummy" looks dirty but it is required if what you only have is the querystring values (qString). If you have the complete URL, just pass it directly to the URLDecoder, and you are done.
Etiquette
You really should be much more specific about what you have tried and why it didn't work.
A proper code sample of your LogObject would really be very helpful here.
Ideally, you would provide a SSCCE so others could easily test your problem themselves.
Answer
You can extract the name:value pairs like this:
String toParse = "ObjectGUId=1abcde&ObjectType=2&ObjectTitle=maximumoflife&Content=racroi&TimeStamp=2012-11-05T17:20:06.056";
String[] fields = toParse.split("&");
String[] kv;
HashMap<String, String> things = new HashMap<String, String>();
for (int i = 0; i < fields.length; ++i)
{
t = fields[i].split("=");
if (2 == kv.length)
{
things.put(kv[0], kv[1]);
}
}
I have chosen to put them into a HashMap, but you could just as easily look at the name part (kv[0]) and choose to do something with it. For example:
if kv[0].equals("ObjectGUId")
{
logObject.setGUId(kv[1]); // example mutator/setter method
}
else if //...
However, all your fields in LogObject are private and you haven't shown us any methods, so I hope you have some way of setting them from outside... bear in mind you will need to store the pairs in a data structure of some kind (as I have done with a HashMap) if you intend to intialise a LogObject with all the fields rather than setting the fields after a constructor call.
Speaking of SSCCEs, I made one for this answer.