I have rest end point which has set of attributes. The user can select attributes need by them. As per the user selection i need to generate the reports.
I use the restTemplate to get the data from end point and populate my response object.
Is there a way I can generate the response object dynamically.
Like if user select
A
B
C
D
restTemplate.exchange(uri, HttpMethod.GET, null, response.class);
In this case my response object should look like
#JsonIgnoreProperties(ignoreUnknown = true)
public class Response {
#JsonProperty("A")
public String A;
#JsonProperty("B")
public String B;
#JsonProperty("C")
public Integer c;
#JsonProperty("D")
public String D;
}
Currently I am statically define the response class, can we dynamically define the response class as per the attributes required by user. please let me know how it can be done.
Map<String, Object> map = new HashMap<>();
map.put("A", "hello");
map.put("B", 2);
String s = objectMapper.writer().writeValueAsString(map);
System.out.println("s = " + s);
Output is
s = {"A":"hello","B":2}
One possible solution could be the creation of a dynamic filter.
Jackson has a built-in filter mechanism which works on simple property filtering.
The default implementation is pretty basic. It allows filtering on simple properties.
The idea would be to dynamically create the string array of properties to filter (include and exclude) with the input request and build the response accordingly.
If you want a more sofisticated filter you could try an addon library I just pushed on github.
Even if you dont need this kind of advanced filtering i think the Usage part of my readme can give you already some hints about the filtering approach.
https://github.com/Antibrumm/jackson-antpathfilter
Related
Context:
I want to write an endpoint that will return a Collection of users based on their usernames. How should those username be passed to the REST endpoint - note that I can (potentially) have a lot of usernames (say > 5000)?
Solution #1:
Use a GET endpoint, concatenate the usernames on client side and pass them as a single request parameter. Split the request parameter on server side to get the list of usernames.
#RestController
public class UserController {
#GetMapping
// able to deserialize `filename1,filename2` to List out of the box
public Collection<User> getUser(#RequestParam List<String> usernames) {
return userService.getUsersByUsername(usernames);
}
}
Solution #2:
Use a POST endpoint and pass the list of usernames as request body. Although cleaner form a coding perspective, I end up using a POST to fetch data.
#RestController
public class UserController {
#PostMapping
public Collection<User> getUser(#RequestBody List<String> usernames) {
return userService.getUsersByUsername(usernames);
}
}
Questions:
Which of the two solutions would be the better approach?
Do you have a better approach to pass the list of usernames into the endpoint?
Edits:
I've updated the signature of the first solution based on suggestions from answers. Spring is able to deserialize filename1,filename2 to List out of the box for #RequestParam.
POST looks like a cleaner approach in this case because -
Sending a huge string in a URL is not a good idea and there is scope for error
You need to write additional code (logic) to create the string on frontend and split it on backend.
Sending a huge string in a URL is not scalable as there are limits on the length of URL.
Get approach might result into an issue since URL length is limited and then you have to limit your query parameters.
Though its not a post request but in your case i think post is the only way out.
I would agree with all the answers given above. I would like to specify one more point , if you are going with post request you might have to increase the payload capacity a server can receive , the default post capacity(The maximum size in bytes) of spring boot is 2mb (based on your server). While testing your code might work fine with 1000-2000 usernames but make sure to change the property to accept more bytes in request.
GET is not limited, yet the browser is. Your server client does not seem to be the browser, so I would say GET is the way to go.
P.S GET can receive a body (not so great, but POST is not also the best match).
You don need to concatenated the string and add extra computation on server server, GET can receive a list of separate strings.
UPDATE with example:
#RestController
public class MyController {
#GetMapping(value = "/test")
public List<String> getTestParams(#RequestParam List<String> params) {
return params;
}
}
The test with 3000 params
#RunWith(SpringRunner.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class TestMyController {
#Autowired
private TestRestTemplate restTemplate;
#Test
public void testRequestWithParamsList() {
List<String> params = new ArrayList<>();
for (int i = 0; i < 3000; i++) {
params.add(String.valueOf(i));
}
List<String> result = restTemplate.getForObject(buildUrl(params),
List.class);
assertEquals(params, result);
}
private String buildUrl(List<?> params) {
return "/test?params=" + getUrlParameter(params);
}
private String getUrlParameter(List<?> params) {
return params.stream()
.map(Object::toString)
.collect(Collectors.joining(","));
}
}
If you are using tomcat you must specify also the max http header property in application.properties
server.max-http-header-size=30000
I have a form with following sections ( following example is for understanding purpose)
GeneralInformation - it 's an object with Cityname (String) and population(int)
Location Information: It's an object with locationCode (int) and Nearest HospitalName(String)
Companies: It's an object with company details. There is list of companies dynamically added
with Company as object. Basically List
Hospitals: it's like List
// generalInfo - populated from form
//locationInfo - populated from form
//companiesArr[] // this is dynamicallypopulated (each row each object) companies array
// hospitalsArr[] // // this is dynamicallypopulated (each row each object) Hospitals array
//Angular code starts..
controller('addGeneralController', function($scope, close,Service) {
$scope.companiesArr = [];
$scope.comapnyName='';
$scope.companyType='';
$scope.hospitalsArr = [];
$scope.hospitalName='';
$scope.locationCode='';
$scope.generalInfo = {};
$scope.locationInfo = {};
$scope.companies = {};
$scope.hospitals = {};
$scope.dataInfo = {};//this is to carry entire objects and arrays
//Following method calls after populating data from form and submit.
//companiesArr,hospitalsArr are populated from front end and passing as submission parameters
$scope.saveGeneral = functio(generalInfo,locationInfo,companiesArr,hospitalsArr){
$scope.companies = companiesArr;
$scope.hospitals = hospitalsArr;
//Create an empty array
//$scope.dataInfo = [];
$scope.dataInfo.push({'generalInfo' : generalInfo, 'locationInfo' : locationInfo,'companies' : $scope.companies,'hospitals' : $scope.hospitals});
$http.post("/addGeneralData",$scope.dataInfo);
});
//Angular code ends..
It's not reaching to the following Spring MVC method:
#RequestMapping(value = "/addGeneralData", method = RequestMethod.POST)
public #ResponseBody String addGeneralData(#RequestBody List<Data> dataInfo){
// not reaching here.With simple parametrs it's reaching here, so no other mapping issue apart from this complex data
// Data - is an object with generalInfo as object,locationInfo as object, //companies List ,hospitals List as it's attributes.
Data data = dataInfo.get(0);
GeneralInfo generalInfo = data.getgeneralInfo();
LocationInfo locationInfo = data.getLocationInfo();
List<Company> companies = data.getCompanies();
List<Hospital> hospitals = data.getHospitals();
}
Basically I want to know how can I transfer this complex data from angular controller to Spring MVC controller?
Please share the request sent from the browser to comment more
It certainly looks like you are sending DataInfo Object but receiving
List dataInfo in your controller. there is a mismatch.
Change the Signature of the handler method
to public #ResponseBody String addGeneralData(#RequestBody DataInfo dataInfo)
Thanks for your response.It's worked when I changed to array instead of list. I have changed all lists inside the Data objects also to array. In addition to that make sure that all data passing from input is as per the type mentioned in the concrete object. For example any data mentioned as int , make sure it's passing int only. If it is complex form and before input validation we are integrating front end with backend, make sure all data we passed exactly as the type mentioned in the mapping object.Is it good practice to use array as parameter in MVC controller?
Do you have any exception ?
It is very likely that you get Serialization Exception due to List interface passed as parameter to controller. Spring just can not initialize new instance of List. Try to use array instead of List. For example
#RequestMapping(value = "/addGeneralData", method = RequestMethod.POST)
public #ResponseBody String addGeneralData(#RequestBody Data[] dataInfo){
Data data = dataInfo[0];
GeneralInfo generalInfo = data.getgeneralInfo();
LocationInfo locationInfo = data.getLocationInfo();
Company[] companies = data.getCompanies();
Hospital[] hospitals = data.getHospitals();
}
Be sure that you use concrete implementations, not interfaces in your Data object.
Hope it helps
Hi I'm trying to see the content of a map with the following code:
#RequestMapping(value = "/demos", method = RequestMethod.GET,headers="Accept=application/json")
public #ResponseBody String getPartnerIdClusters(Model model) {
Map<Integer, List<Cluster>> partnerIdClusterMap = partnerService.getPartnerIdClusterMap(true, true);
return "partnerIdClusterMap: " + partnerIdClusterMap;
}
Which gave me the following output in the browser:
partnerIdClusterMap: {2=nl.irp.globetrotting.model.Cluster#7499394a}
After that I had tried this code:
String line = "test: /n";
for (Object entry : partnerIdClusterMap.values().toArray()) {
line += entry;
line += "/n";
}
return "partnerIdClusterMap " + line;
But that didn't work either because it has given me the following output:
partnerIdClusterMap test: /nnl.irp.globetrotting.model.Cluster#63769066/n
I already know that it is the Cluster with id 3. Here is a screenshot of it:
link: http://i.imgur.com/pKLu6gf.png
Here is how the getPartnerIdClusterMap() method looks like (in case you want to know):
#Override
public Map<Integer, List<Cluster>> getPartnerIdClusterMap(boolean minorOnly, boolean foreignCountriesOnly) {
BooleanBuilder predicate = new BooleanBuilder();
if (minorOnly) {
predicate.and(qCluster.type.eq(ClusterType.MINOR));
}
if (foreignCountriesOnly) {
predicate.and(qPartner.country.code2.ne("nl"));
}
return from(qCluster)
.innerJoin(qCluster.partner, qPartner)
.where(predicate)
.where(qPartner.country.code2.ne("nl"))
.map(qPartner.id, GroupBy.list(qCluster));
}
This is what I gladly want to know:
So I gladly want to see the Clusterwith all the values from the row.
Spring MVC should be able to convert maps to JSON, with the help of a message converter. Using Spring Boot, just returning maps works for me - a MappingJackson2HttpMessageConverter is automatically configured. As I recall, before using Spring Boot, I just had to configure an ObjectMapper, and then returning the map was working. So, in summary, I think that
Using Spring Boot, returning a map should work.
If not using Spring Boot, an ObjectMapper (and maybe also a MappingJackson2HttpMessageConverter) might be needed to be configured.
If the returned map holds things that the ObjectMapper can't convert by default, it might need you to supply some converting customization. Help material on Jackson (now called fasterxml) ObjectMapper will have more details.
partnerIdClusterMap: {2=nl.irp.globetrotting.model.Cluster#7499394a}
Basically you are printing the values that you specified. In this case you are printing the integer and then the list object populated with Cluster Objects.
Consider to iterate partnerIdClusterMap in your weblogic and extract the values with the methods provided by it or override the toString method of the object to get a full line with details.
I'm trying to make sure my Jersey request parameters are sanitized.
When processing a Jersey GET request, do I need to filter non String types?
For example, if the parameter submitted is an integer are both option 1 (getIntData) and option 2 (getStringData) hacker safe? What about a JSON PUT request, is my ESAPI implementation enough, or do I need to validate each data parameter after it is mapped? Could it be validated before it is mapped?
Jersey Rest Example Class:
public class RestExample {
//Option 1 Submit data as an Integer
//Jersey throws an internal server error if the type is not Integer
//Is that a valid way to validate the data?
//Integer Data, not filtered
#Path("/data/int/{data}/")
#GET
#Produces(MediaType.TEXT_HTML)
public Response getIntData(#PathParam("data") Integer data){
return Response.ok("You entered:" + data).build();
}
//Option 2 Submit data as a String, then validate it and cast it to an Integer
//String Data, filtered
#Path("/data/string/{data}/")
#GET
#Produces(MediaType.TEXT_HTML)
public Response getStringData(#PathParam("data") String data) {
data = ESAPI.encoder().canonicalize(data);
if (ESAPI.validator().isValidInteger("data", data, 0, 999999, false))
{
int intData = Integer.parseInt(data);
return Response.ok("You entered:" + intData).build();
}
return Response.status(404).entity("404 Not Found").build();
}
//JSON data, HTML encoded
#Path("/post/{requestid}")
#POST
#Consumes({MediaType.APPLICATION_FORM_URLENCODED, MediaType.APPLICATION_JSON})
#Produces(MediaType.TEXT_HTML)
public Response postData(String json) {
json = ESAPI.encoder().canonicalize(json);
json = ESAPI.encoder().encodeForHTML(json);
//Is there a way to iterate through each JSON KeyValue and filter here?
ObjectMapper mapper = new ObjectMapper();
DataMap dm = new DataMap();
try {
dm = mapper.readValue(json, DataMap.class);
} catch (Exception e) {
e.printStackTrace();
}
//Do we need to validate each DataMap object value and is there a dynamic way to do it?
if (ESAPI.validator().isValidInput("strData", dm.strData, "HTTPParameterValue", 25, false, true))
{
//Is Integer validation needed or will the thrown exception be good enough?
return Response.ok("You entered:" + dm.strData + " and " + dm.intData).build();
}
return Response.status(404).entity("404 Not Found").build();
}
}
Data Map Class:
public class DataMap {
public DataMap(){}
String strData;
Integer intData;
}
The short answer is yes, though by "filter" I interpret it as "validate," because no amount of "filtering" will EVER provide you with SAFE data. You can still run into integer overflows in Java, and while those may not have immediate security concerns, they could still put parts of your application in an unplanned for state, and hacking is all about perturbing the system in ways you can control.
You packed waaaaay too many questions into one "question," but here we go:
First off, the lines
json = ESAPI.encoder().canonicalize(json);
json = ESAPI.encoder().encodeForHTML(json);
Aren't doing what you think they're doing. If your JSON is coming in as a raw String right here, these two calls are going to be applying mass rules across the entire string, when you really need to handle these with more surgical precision, which you seem to at least be subconsciously aware of in the next question.
//Is there a way to iterate through each JSON KeyValue and filter
here?
Partial duplicate of this question.
While you're in the loop discussed here, you can perform any data transformations you want, but what you should really be considering is using the JSONObject class referenced in that first link. Then you'll have JSON parsed into an object where you'll have better access to JSON key/value pairs.
//Do we need to validate each DataMap object value and is there a
dynamic way to do it?
Yes, we validate everything that comes from a user. All users are assumed to be trained hackers, and smarter than you. However if you handled filtering before you do your data mapping transformation, you don't need to do it a second time. Doing it dynamically?
Something like:
JSONObject json = new JSONObject(s);
Iterator iterator = json.keys();
while( iterator.hasNext() ){
String data = iterator.next();
//filter and or business logic
}
^^That syntax is skipping typechecks but it should get you where you need to go.
/Is Integer validation needed or will the thrown exception be good
enough?
I don't see where you're throwing an exception with these lines of code:
if (ESAPI.validator().isValidInput("strData", dm.strData, "HTTPParameterValue", 25, false, true))
{
//Is Integer validation needed or will the thrown exception be good enough?
return Response.ok("You entered:" + dm.strData + " and " + dm.intData).build();
}
Firstly, in java we have autoboxing which means this:
int foo = 555555;
String bar = "";
//the code
foo + bar;
Will be cast to a string in any instance. The compiler will promote the int to an Integer and then silently call the Integer.toString() method. Also, in your Response.ok( String ); call, THIS is where you're going to want to encodeForHTML or whatever the output context may be. Encoding methods are ALWAYS For outputting data to user, whereas canonicalize you want to call when receiving data. Finally, in this segment of code we also have an error where you're assuming that you're dealing with an HTTPParameter. NOT at this point in the code. You'll validate http Parameters in instances where you're calling request.getParameter("id"): where id isn't a large blob of data like an entire JSON response or an entire XML response. At this point you should be validating for things like "SafeString"
Usually there are parsing libraries in Java that can at least get you to the level of Java objects, but on the validation side you're always going to be running through every item and punting whatever might be malicious.
As a final note, while coding, keep these principles in mind your code will be cleaner and your thought process much more focused:
user input is NEVER safe. (Yes, even if you've run it through an XSS filter.)
Use validate and canonicalize methods whenever RECEIVING data, and encode methods whenever transferring data to a different context, where context is defined as "Html field. Http attribute. Javascript input, etc...)
Instead of using the method isValidInput() I'd suggest using getValidInput() because it will call canonicalize for you, making you have to provide one less call.
Encode ANY time your data is going to be passed to another dynamic language, like SQL, groovy, Perl, or javascript.
var catids = new Array();
I have a catids array where i store the checked checkbox values like the below one.
cat = $("input[name=catChkBox]:checked").map(function () {
return $(this).data('name');
}).get().join(",");
the cat variable forms something like this 1,2,3..
I want to send this "cat" to a java method and print those values.
I pass the values to java through a dwr call like this
DataHandler.getTasks( categories, {callback:function(data){
}, errorHandler:function(){
},async:false
});
I have configured dwr for pojo. should I configure anything for parameters?
I tried the below code but I didn't get anything.
public List<Facade> getTasks(String myIds){
String[] ids = catids .split(",");
System.out.println("-------size of cat id------------" + myIds.length);
for (int i=0; i<myIds.length;i++)
System.out.println(myIds[i]);
//finally it will return a pojo which i l be receiving it in data of dwr call.
-------size of cat id------------ is 1
myIds[i] prints nothing
I need it as an integer back.
What mistake am I doing ?
I will do it in this way.
JavaScript creates json object like this {"categoryIds": [1,2,3,4,5]}
Java converter convert json to java POJO object using for example Gson or Jackson library.
After convert you can work with java POJO object which have list of categories.
If you use this solution your code will be more clear and you will be able to share more objects between JavaScript and Java using the same clear solution.
Example (pseudo code)
CategorList class
public class CategoryList {
private ArrayList<Category> categoryList;
// getters and setters
}
Converter
public class CategoryListConverter {
public CategoryList convert(String json) {
Gson g = new Gson();
CategoryList cl = g.fromJson(json, CategoryList.class);
return cl;
}
}
I tried the code it workd fine
getTasks("1,2,3");
check what the value of categoriesIds is sent to getTask
Send this as a form parameter from webpage. Then get this from HttpServletRequest request object in java.
request.getParameter('categoryId');