I am new to Java and Vertx and I have a query string with the following format:
GET /examples/1/data?date_1[gt]=2021-09-28&date_1[lt]=2021-10-28
Here I have this date_1 parameter which is within a certain range. I have been using HttpServerRequest class to extract simple parameters like integers but not sure how to proceed with these kind of range parameters.
With the simple parameters, I can do something like:
String param = request.getParam(paramName);
paramAsInteger = Integer.valueOf(paramAsString);
However, confused as to how to deal with the gt and lt options and the fact that we have same parameter twice.
You say that you have difficulties parsing out these tokens. Here's how you can handle this.
The first thing to understand is that the parameter name is NOT "date1"
There are actually two parameters here
2.1. "date_1[gt]" with a value of "2021-09-28"
2.2. "date_1[lt]" with a value of "2021-10-28"
This is because in the URI parameter definition everything before the "=" sign is the parameter name and everything after is the parameter value.
You can just do
String dateAsString = request.getParam("date1[gt]");
paramAsInteger = toDate(dateAsString)
To implement the toDate() function read this simple article how to convert a string object into a data object using a standard library
(link)
Vert.x will treat these parameters as two separate ones. So RoutingContext#queryParam("date_1[gt]") will only give you the value for [gt]. If you want the value for [lt] you need to get that separately.
That being said, you can move this tedious logic into an extra handler and store the values in the RoutingContext. Something like this might be easier:
private void extractDates(RoutingContext ctx) {
var startDate = ctx.queryParam("date_1[gt]");
var endDate = ctx.queryParam("date_1[lt]");
var parsedStartDate = DateTimeFormatter.ISO_LOCAL_DATE.parse(startDate.get(0));
var parsedEndDate = DateTimeFormatter.ISO_LOCAL_DATE.parse(endDate.get(0));
// things we put in the context here can be retrieved by later handlers
ctx.put("startDate", parsedStartDate);
ctx.put("endDate", parsedEndDate);
ctx.next();
}
Then, in your actual handler you can access the two dates as follows:
router.get("/date")
.handler(this::extractDates)
.handler(ctx -> {
var responseBody = ctx.get("startDate") + " - " + ctx.get("endDate");
ctx.end(responseBody);
});
This allows you to keep your actual business logic concise.
Related
I have the below code as my restful service operation.
#GET
#UnitOfWork
#Timed(name = "get-requests")
#Path("/{referenceId}")
public Response get(#Auth #ApiParam(access = "internal") UserPrincipal user,
#ApiParam(name = "id", value = "reference ID", required = true)
#PathParam("referenceId") String id) {
return Response.ok(id).build();
}
However, I noticed if I pass in m1234;5678, I get only m1234 returned. I tried #Path("/{referenceId:.*}"), but it doesn't work.
I also tried use #Encode at the top of the method to make sure the url is not decoded and then try to replace %3B with ";" in the code. But it seems not working also.
Please note that I cannot use Spring framework. Thanks.
The ; denotes a matrix parameter. Use #MatrixParam to get its value.
See also the answers to this question: URL matrix parameters vs. request parameters
Edit: The key of the matrix parameter would be 5678, the value would be null.
There is a way to get achieve what you want by using PathSegment as the type of the parameter instead of String:
#PathParam("referenceId) PathSegment id
In the body of the method, you can use
String idValue = id.getPath();
to get m1234;5678.
This question probably is easy. I am trying to read a field of a IBM Maximo application and use this value in the method getList(). The value I want to use was not saved in the database yet.
Here is some pseudocode:
#Override
public MboSetRemote getList() throws MXException, RemoteException {
MboSetRemote result = super.getList();
//Here is where i dont know how to do it
Date field = getFieldValue(FieldName)
//Here is where i want to use the value
String string = "....field..."
result.setWhere(string);
return result;
}
Thanks everyone,
Regards
I think the easiest and safest means to achieve your end of using the field value in your where clause is to use a bind variable, like this:
#Override
public MboSetRemote getList() throws MXException, RemoteException {
MboSetRemote result = super.getList();
//Here is where i want to use the value
String string = "....:fieldName...";
result.setWhere(string);
return result;
}
Notice the colon on the front of :fieldName in string. When Maximo sees this, it will look (not case-sensitive) on the current record / Mbo for an attribute named fieldName and replace :fieldName with the value in the attribute -- wrapped in quotes or whatever, as applicable to the attribute's type (ALN, UPPER, DATE, etc).
This approach is better than the approach you presented because it will employ Maximo's framework to prevent SQL injection attacks and etc.
That said, the way to get the field value would be as follows:
Date fieldValue = getMboValue("FieldName").getDate();
Further, I strongly suggest you get yourself a copy of Maximo's JavaDocs. You can do that here.
I want to convert an object of another format into a protobuf, knowing the protobuf's Descriptors. It's easy to do for regular fields or even a nested field. But, I'm running into a problem for repeated fields.
message Foo {
optional MsgA a = 1;
repeated MsgB b = 2;
}
For "MsgA a", the code bld.getFieldBuilder(field) works:
Foo.Builder bld = Foo.newBuilder();
Descriptors.Descriptor msgDesc = Foo.getDescriptor();
List<Descriptors.FieldDescriptor> fields = msgDesc.getFields();
for (Descriptors.FieldDescriptor field : fields) {
Message.Builder subBld = bld.getFieldBuilder(field);
// set foreign value xyz using subBld
// subBld.setFleld(subfield1, xyz);
}
But for "MsgB b", the same code throws "UnsupportedOperationException: getFieldBuilder() called on a non-Message type."
I understand the repeated field is a list, I may set each one separately. But, how do I get a builder first? Is there a clean and easy way to do the similar?
Thanks for any input.
You don't get a builder for the repeated field itself - you call Builder.addRepeatedField(field, value) etc. To get a builder for the type of the repeated field, you can use:
Builder builder = bld.newBuilderForField(field)
If you want to modify an existing value, you can use Builder.getRepeatedFieldBuilder(field, index).
To create an instance to start with, you can use Builder.newBuilderForField:
Message.Builder subBld = bld.newBuilderForField(field);
// Now modify subBld, then...
bld.addRepeatedField(field, subBld.build());
I'm trying to make sure my Jersey request parameters are sanitized.
When processing a Jersey GET request, do I need to filter non String types?
For example, if the parameter submitted is an integer are both option 1 (getIntData) and option 2 (getStringData) hacker safe? What about a JSON PUT request, is my ESAPI implementation enough, or do I need to validate each data parameter after it is mapped? Could it be validated before it is mapped?
Jersey Rest Example Class:
public class RestExample {
//Option 1 Submit data as an Integer
//Jersey throws an internal server error if the type is not Integer
//Is that a valid way to validate the data?
//Integer Data, not filtered
#Path("/data/int/{data}/")
#GET
#Produces(MediaType.TEXT_HTML)
public Response getIntData(#PathParam("data") Integer data){
return Response.ok("You entered:" + data).build();
}
//Option 2 Submit data as a String, then validate it and cast it to an Integer
//String Data, filtered
#Path("/data/string/{data}/")
#GET
#Produces(MediaType.TEXT_HTML)
public Response getStringData(#PathParam("data") String data) {
data = ESAPI.encoder().canonicalize(data);
if (ESAPI.validator().isValidInteger("data", data, 0, 999999, false))
{
int intData = Integer.parseInt(data);
return Response.ok("You entered:" + intData).build();
}
return Response.status(404).entity("404 Not Found").build();
}
//JSON data, HTML encoded
#Path("/post/{requestid}")
#POST
#Consumes({MediaType.APPLICATION_FORM_URLENCODED, MediaType.APPLICATION_JSON})
#Produces(MediaType.TEXT_HTML)
public Response postData(String json) {
json = ESAPI.encoder().canonicalize(json);
json = ESAPI.encoder().encodeForHTML(json);
//Is there a way to iterate through each JSON KeyValue and filter here?
ObjectMapper mapper = new ObjectMapper();
DataMap dm = new DataMap();
try {
dm = mapper.readValue(json, DataMap.class);
} catch (Exception e) {
e.printStackTrace();
}
//Do we need to validate each DataMap object value and is there a dynamic way to do it?
if (ESAPI.validator().isValidInput("strData", dm.strData, "HTTPParameterValue", 25, false, true))
{
//Is Integer validation needed or will the thrown exception be good enough?
return Response.ok("You entered:" + dm.strData + " and " + dm.intData).build();
}
return Response.status(404).entity("404 Not Found").build();
}
}
Data Map Class:
public class DataMap {
public DataMap(){}
String strData;
Integer intData;
}
The short answer is yes, though by "filter" I interpret it as "validate," because no amount of "filtering" will EVER provide you with SAFE data. You can still run into integer overflows in Java, and while those may not have immediate security concerns, they could still put parts of your application in an unplanned for state, and hacking is all about perturbing the system in ways you can control.
You packed waaaaay too many questions into one "question," but here we go:
First off, the lines
json = ESAPI.encoder().canonicalize(json);
json = ESAPI.encoder().encodeForHTML(json);
Aren't doing what you think they're doing. If your JSON is coming in as a raw String right here, these two calls are going to be applying mass rules across the entire string, when you really need to handle these with more surgical precision, which you seem to at least be subconsciously aware of in the next question.
//Is there a way to iterate through each JSON KeyValue and filter
here?
Partial duplicate of this question.
While you're in the loop discussed here, you can perform any data transformations you want, but what you should really be considering is using the JSONObject class referenced in that first link. Then you'll have JSON parsed into an object where you'll have better access to JSON key/value pairs.
//Do we need to validate each DataMap object value and is there a
dynamic way to do it?
Yes, we validate everything that comes from a user. All users are assumed to be trained hackers, and smarter than you. However if you handled filtering before you do your data mapping transformation, you don't need to do it a second time. Doing it dynamically?
Something like:
JSONObject json = new JSONObject(s);
Iterator iterator = json.keys();
while( iterator.hasNext() ){
String data = iterator.next();
//filter and or business logic
}
^^That syntax is skipping typechecks but it should get you where you need to go.
/Is Integer validation needed or will the thrown exception be good
enough?
I don't see where you're throwing an exception with these lines of code:
if (ESAPI.validator().isValidInput("strData", dm.strData, "HTTPParameterValue", 25, false, true))
{
//Is Integer validation needed or will the thrown exception be good enough?
return Response.ok("You entered:" + dm.strData + " and " + dm.intData).build();
}
Firstly, in java we have autoboxing which means this:
int foo = 555555;
String bar = "";
//the code
foo + bar;
Will be cast to a string in any instance. The compiler will promote the int to an Integer and then silently call the Integer.toString() method. Also, in your Response.ok( String ); call, THIS is where you're going to want to encodeForHTML or whatever the output context may be. Encoding methods are ALWAYS For outputting data to user, whereas canonicalize you want to call when receiving data. Finally, in this segment of code we also have an error where you're assuming that you're dealing with an HTTPParameter. NOT at this point in the code. You'll validate http Parameters in instances where you're calling request.getParameter("id"): where id isn't a large blob of data like an entire JSON response or an entire XML response. At this point you should be validating for things like "SafeString"
Usually there are parsing libraries in Java that can at least get you to the level of Java objects, but on the validation side you're always going to be running through every item and punting whatever might be malicious.
As a final note, while coding, keep these principles in mind your code will be cleaner and your thought process much more focused:
user input is NEVER safe. (Yes, even if you've run it through an XSS filter.)
Use validate and canonicalize methods whenever RECEIVING data, and encode methods whenever transferring data to a different context, where context is defined as "Html field. Http attribute. Javascript input, etc...)
Instead of using the method isValidInput() I'd suggest using getValidInput() because it will call canonicalize for you, making you have to provide one less call.
Encode ANY time your data is going to be passed to another dynamic language, like SQL, groovy, Perl, or javascript.
I am trying to write a Spring REST Controller getting an array of strings as input parameter of a HTTP GET request.
The problem arises when in the GET request, in some of the strings of the array, I use special characters like commas ,, blank spaces or forward slash /, no matter if I URL encode the query part of the URL HTTP GET request.
That means that the string "1/4 cup ricotta, yogurt" (edit which needs to be considered as a unique ingredient contained as a string element of the input array) in either this format:
http://127.0.0.1:8080/[...]/parseThis?[...]&ingredients=1/4 cup ricotta, yogurt
This format (please note the blank spaces encoded as + plus, rather than the hex code):
http://127.0.0.1:8080/[...]/parseThis?[...]&ingredients=1%2F4+cup+ricotta%2C+yogurt
Or this format (please note the blank space encoded as hex code %20):
http://127.0.0.1:8080/[...]/parseThis?[...]&ingredients=1%2F4%20cup%20ricotta%2C%20yogurt
is not rendered properly.
The system does not recognize the input string as one single element of the array.
In the 2nd and 3rd case the system splits the input string on the comma and returns an array of 2 elements rather than 1 element. I am expecting 1 element here.
The relevant code for the controller is:
#RequestMapping(
value = "/parseThis",
params = {
"language",
"ingredients"
}, method = RequestMethod.GET, headers = HttpHeaders.ACCEPT + "=" + MediaType.APPLICATION_JSON_VALUE)
#ResponseBody
public HttpEntity<CustomOutputObject> parseThis(
#RequestParam String language,
#RequestParam String[] ingredients){
try {
CustomOutputObject responseFullData = parsingService.parseThis(ingredients, language);
return new ResponseEntity<>(responseFullData, HttpStatus.OK);
} catch (Exception e) {
// TODO
}
}
I need to perform HTTP GET request against this Spring controller, that's a requirement (so no HTTP POST can be used here).
Edit 1:
If I add HttpServletRequest request to the signature of the method in the controller, then I add a log statement like log.debug("The query string is: '" + request.getQueryString() + "'"); then I am seeing in the log a line like The query string is: '&language=en&ingredients=1%2F4+cup+ricotta%2C+yogurt' (So still URL encoded).
Edit 2:
On the other hand if I add WebRequest request to the signature of the method, the the log as log.debug("The query string is: '" + request.getParameter("ingredients") + "'"); then I am getting a string in the log as The query string is: '1/4 cup ricotta, yogurt' (So URL decoded).
I am using Apache Tomcat as a server.
Is there any filter or something I need to add/review to the Spring/webapp configuration files?
Edit 3:
The main problem is in the interpretation of a comma:
#ResponseBody
#RequestMapping(value="test", method=RequestMethod.GET)
public String renderTest(#RequestParam("test") String[] test) {
return test.length + ": " + Arrays.toString(test);
// /app/test?test=foo,bar => 2: [foo, bar]
// /app/test?test=foo,bar&test=baz => 2: [foo,bar, baz]
}
Can this behavior be prevented?
The path of a request parameter to your method argument goes through parameter value extraction and then parameter value conversion. Now what happens is:
Extraction:
The parameter is extracted as a single String value. This is probably to allow simple attributes to be passed as simple string values for later value conversion.
Conversion:
Spring uses ConversionService for the value conversion. In its default setup StringToArrayConverter is used, which unfortunately handles the string as comma delimited list.
What to do:
You are pretty much screwed with the way Spring handles single valued request parameters. So I would do the binding manually:
// Method annotations
public HttpEntity<CustomOutputObject> handlerMethod(WebRequest request) {
String[] ingredients = request.getParameterValues("ingredients");
// Do other stuff
}
You can also check what Spring guys have to say about this.. and the related SO question.
Well, you could register a custom conversion service (from this SO answer), but that seems like a lot of work. :) If it were me, I would ignore the declaration the #RequestParam in the method signature and parse the value using the incoming request object.
May I suggest you try the following format:
ingredients=egg&ingredients=milk&ingredients=butter
Appending &ingredients to the end will handle the case where the array only has a single value.
ingredients=egg&ingredients=milk&ingredients=butter&ingredients
ingredients=milk,skimmed&ingredients
The extra entry would need to be removed from the array, using a List<String> would make this easier.
Alternatively if you are trying to implement a REST controller to pipe straight into a database with spring-data-jpa, you should take a look at spring-data-rest. Here is an example.
You basically annotate your repository with #RepositoryRestResource and spring does the rest :)
A solution from here
public String get(WebRequest req) {
String[] ingredients = req.getParameterValues("ingredients");
for(String ingredient:ingredients ) {
System.out.println(ingredient);
}
...
}
This works for the case when you have a single ingredient containing commas