I am trying to read the authorization header for an HTTP request (because I need to add something to it), but I always get null for the header value. Other headers work fine.
public void testAuth() throws MalformedURLException, IOException{
URLConnection request = new URL("http://google.com").openConnection();
request.setRequestProperty("Authorization", "MyHeader");
request.setRequestProperty("Stackoverflow", "anotherHeader");
// works fine
assertEquals("anotherHeader", request.getRequestProperty("Stackoverflow"));
// Auth header returns null
assertEquals("MyHeader", request.getRequestProperty("Authorization"));
}
Am I doing something wrong? Is this a "security" feature? Is there a way to make this work with URLConnection, or do I need to use another HTTP client library?
Apparently, it's a security "feature". The URLConnection is actually an instance of sun.net.www.protocol.http.HttpURLConnection. It defines getRequestProperty as:
public String getRequestProperty (String key) {
// don't return headers containing security sensitive information
if (key != null) {
for (int i=0; i < EXCLUDE_HEADERS.length; i++) {
if (key.equalsIgnoreCase(EXCLUDE_HEADERS[i])) {
return null;
}
}
}
return requests.findValue(key);
}
The EXCLUDE_HEADERS array is defined as:
// the following http request headers should NOT have their values
// returned for security reasons.
private static final String[] EXCLUDE_HEADERS = {
"Proxy-Authorization",
"Authorization"
};
I am not happy about the extra dependencies, but following the suggestion to switch to Commons Http solved the immediate problem for me.
I'd still like to know what the problem was with my original code.
As Devon's answer correctly states: it's not a bug, it's a "security" feature 😉
But you don't have to switch to a different library: it is always possible to access the underlying MessageHeader-collection via reflection and extract the "Authorization"-header value.
After some headscratch i've managed to come up with a working snippet here.
Have you tried using URLConnection.addRequestProperty()?
This is how I use to add HTTP Request Headers.
Related
During debugging in IntelliJ, I'm getting the SOAPFaultException after evaluate the expression (or adding Watching in debug console) in the RequestContext object. More specifically, the error says:
javax.xml.ws.soap.SOAPFaultException: Server did not recognize the value of HTTP Header SOAPAction: .
Why this is happening?
This error occurs due to the Map custom implementation of the class RequestContext.java made in the jaxws-rt (version 2.1.4) library.
The custom map implementation has a Fallback mechanism that when the variable "mapView.fallbackMap" is null the header "soapAction" is clear in the fallback for-loop. Check the snipped bellow:
/**
* Fill a {#link Packet} with values of this {#link RequestContext}.
*/
public void fill(Packet packet) {
if(mapView.fallbackMap==null) {
if (endpointAddress != null)
packet.endpointAddress = endpointAddress;
packet.contentNegotiation = contentNegotiation;
if (soapAction != null) {
packet.soapAction = soapAction;
}
if(!others.isEmpty()) {
packet.invocationProperties.putAll(others);
//if it is not standard property it deafults to Scope.HANDLER
packet.getHandlerScopePropertyNames(false).addAll(others.keySet());
}
} else {
Set<String> handlerScopePropertyNames = new HashSet<String>();
// fallback mode, simply copy map in a slow way
for (Entry<String,Object> entry : mapView.fallbackMap.entrySet()) {
String key = entry.getKey();
if(packet.supports(key))
packet.put(key,entry.getValue());
else
packet.invocationProperties.put(key,entry.getValue());
//if it is not standard property it deafults to Scope.HANDLER
if(!super.supports(key)) {
handlerScopePropertyNames.add(key);
}
}
if(!handlerScopePropertyNames.isEmpty())
packet.getHandlerScopePropertyNames(false).addAll(handlerScopePropertyNames);
}
}
Additionaly, the variable "mapView.fallbackMap" is filled every time a method from Map not supported by the custom implementation is made, for example, the "entrySet" or "size" method. Therefore, everytime we call a "Watch" or "Evaluate" operation to debug the RequestContext in IntelliJ IDEA the fallback is filled broken the SOAP call.
So the solution, does not use to debug the RequestContext before SOAP call, or make sure to remove the following property before to execute the request:
((BindingProvider) service).getRequestContext().remove("javax.xml.ws.soap.http.soapaction.uri")
Context:
I want to write an endpoint that will return a Collection of users based on their usernames. How should those username be passed to the REST endpoint - note that I can (potentially) have a lot of usernames (say > 5000)?
Solution #1:
Use a GET endpoint, concatenate the usernames on client side and pass them as a single request parameter. Split the request parameter on server side to get the list of usernames.
#RestController
public class UserController {
#GetMapping
// able to deserialize `filename1,filename2` to List out of the box
public Collection<User> getUser(#RequestParam List<String> usernames) {
return userService.getUsersByUsername(usernames);
}
}
Solution #2:
Use a POST endpoint and pass the list of usernames as request body. Although cleaner form a coding perspective, I end up using a POST to fetch data.
#RestController
public class UserController {
#PostMapping
public Collection<User> getUser(#RequestBody List<String> usernames) {
return userService.getUsersByUsername(usernames);
}
}
Questions:
Which of the two solutions would be the better approach?
Do you have a better approach to pass the list of usernames into the endpoint?
Edits:
I've updated the signature of the first solution based on suggestions from answers. Spring is able to deserialize filename1,filename2 to List out of the box for #RequestParam.
POST looks like a cleaner approach in this case because -
Sending a huge string in a URL is not a good idea and there is scope for error
You need to write additional code (logic) to create the string on frontend and split it on backend.
Sending a huge string in a URL is not scalable as there are limits on the length of URL.
Get approach might result into an issue since URL length is limited and then you have to limit your query parameters.
Though its not a post request but in your case i think post is the only way out.
I would agree with all the answers given above. I would like to specify one more point , if you are going with post request you might have to increase the payload capacity a server can receive , the default post capacity(The maximum size in bytes) of spring boot is 2mb (based on your server). While testing your code might work fine with 1000-2000 usernames but make sure to change the property to accept more bytes in request.
GET is not limited, yet the browser is. Your server client does not seem to be the browser, so I would say GET is the way to go.
P.S GET can receive a body (not so great, but POST is not also the best match).
You don need to concatenated the string and add extra computation on server server, GET can receive a list of separate strings.
UPDATE with example:
#RestController
public class MyController {
#GetMapping(value = "/test")
public List<String> getTestParams(#RequestParam List<String> params) {
return params;
}
}
The test with 3000 params
#RunWith(SpringRunner.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class TestMyController {
#Autowired
private TestRestTemplate restTemplate;
#Test
public void testRequestWithParamsList() {
List<String> params = new ArrayList<>();
for (int i = 0; i < 3000; i++) {
params.add(String.valueOf(i));
}
List<String> result = restTemplate.getForObject(buildUrl(params),
List.class);
assertEquals(params, result);
}
private String buildUrl(List<?> params) {
return "/test?params=" + getUrlParameter(params);
}
private String getUrlParameter(List<?> params) {
return params.stream()
.map(Object::toString)
.collect(Collectors.joining(","));
}
}
If you are using tomcat you must specify also the max http header property in application.properties
server.max-http-header-size=30000
I have the following Java REST method I implemented using Jersey:
#POST
#Path("copy")
public List<Integer> copyCompanionTextRule(#QueryParam("ruleid") List<Integer> ruleIdList,
#QueryParam("workgroupid") List<WorkgroupId> workgroupIds,
#Context HttpHeaders hh)
throws ETMSException
{
List<Integer> insertedItems = new ArrayList<Integer>();
if ( null != ruleIdList ){
for(Integer ruleId : ruleIdList) {
insertedItems.addAll(copyCompanionTextRule(ruleId, workgroupIds));
}
}
return insertedItems;
}
It receives a list of integer and a list of objects of type WorkgroupId as well as the context for some extra processing I'll do later.
I'm working the client with Sencha EXTJS 4.2 and my request is being performed this way:
Ext.Ajax.request({
url: '/sysadmin/companiontextrules/copy',
method: 'POST',
showException: true,
scope: this,
params: {
ruleid: Ext.encode(ruleIdsArray),
workgroupid: toWorkgroups
},
callback: function(options, success, response) {
me.setLoading(false);
if (!success) {
return;
}
this.destroy();
}
});
The ruleIdsArray is just an array of integers: [1274,1292,1745].
The toWorkgroups is an array of objects which has a model that is related to the WorkgroupId entity.
As you can see, both lists are being processed as query parameters and I'm using the "params" config in the Ajax request; however, this is not working.
Seems like the ruleId array is empty, when it tries to iterate the rulesIdList is empty so the method POST works but it is returning always an empty list.
I know I cannot use them in the form "url?ruleid=a&workgroupid=b". When I tried it just by curiosity, I got a QueryParamException and NumberFormatException saying that the rule array is being considered as string.
When I use the "Ext.encode" for both params I receive a message in browser console that the Maximum callstack size exceeded.
This is what I got from Chrome Console:
I've tried almost everything, but maybe some more eyes can help me in this, I'd really appreciate comments or any kind of help.
Thanks in advance.
Looks like your parameters are going in the POST body instead of as query parameters.
url?ruleid=a&workgroupid=b is getting a NumberFormatException because ruleId is supposed to be Integer.
url?ruleid=1&workgroupid=b or url?ruleid=1&ruleid=2&workgroupid=b should work
I'm trying to make sure my Jersey request parameters are sanitized.
When processing a Jersey GET request, do I need to filter non String types?
For example, if the parameter submitted is an integer are both option 1 (getIntData) and option 2 (getStringData) hacker safe? What about a JSON PUT request, is my ESAPI implementation enough, or do I need to validate each data parameter after it is mapped? Could it be validated before it is mapped?
Jersey Rest Example Class:
public class RestExample {
//Option 1 Submit data as an Integer
//Jersey throws an internal server error if the type is not Integer
//Is that a valid way to validate the data?
//Integer Data, not filtered
#Path("/data/int/{data}/")
#GET
#Produces(MediaType.TEXT_HTML)
public Response getIntData(#PathParam("data") Integer data){
return Response.ok("You entered:" + data).build();
}
//Option 2 Submit data as a String, then validate it and cast it to an Integer
//String Data, filtered
#Path("/data/string/{data}/")
#GET
#Produces(MediaType.TEXT_HTML)
public Response getStringData(#PathParam("data") String data) {
data = ESAPI.encoder().canonicalize(data);
if (ESAPI.validator().isValidInteger("data", data, 0, 999999, false))
{
int intData = Integer.parseInt(data);
return Response.ok("You entered:" + intData).build();
}
return Response.status(404).entity("404 Not Found").build();
}
//JSON data, HTML encoded
#Path("/post/{requestid}")
#POST
#Consumes({MediaType.APPLICATION_FORM_URLENCODED, MediaType.APPLICATION_JSON})
#Produces(MediaType.TEXT_HTML)
public Response postData(String json) {
json = ESAPI.encoder().canonicalize(json);
json = ESAPI.encoder().encodeForHTML(json);
//Is there a way to iterate through each JSON KeyValue and filter here?
ObjectMapper mapper = new ObjectMapper();
DataMap dm = new DataMap();
try {
dm = mapper.readValue(json, DataMap.class);
} catch (Exception e) {
e.printStackTrace();
}
//Do we need to validate each DataMap object value and is there a dynamic way to do it?
if (ESAPI.validator().isValidInput("strData", dm.strData, "HTTPParameterValue", 25, false, true))
{
//Is Integer validation needed or will the thrown exception be good enough?
return Response.ok("You entered:" + dm.strData + " and " + dm.intData).build();
}
return Response.status(404).entity("404 Not Found").build();
}
}
Data Map Class:
public class DataMap {
public DataMap(){}
String strData;
Integer intData;
}
The short answer is yes, though by "filter" I interpret it as "validate," because no amount of "filtering" will EVER provide you with SAFE data. You can still run into integer overflows in Java, and while those may not have immediate security concerns, they could still put parts of your application in an unplanned for state, and hacking is all about perturbing the system in ways you can control.
You packed waaaaay too many questions into one "question," but here we go:
First off, the lines
json = ESAPI.encoder().canonicalize(json);
json = ESAPI.encoder().encodeForHTML(json);
Aren't doing what you think they're doing. If your JSON is coming in as a raw String right here, these two calls are going to be applying mass rules across the entire string, when you really need to handle these with more surgical precision, which you seem to at least be subconsciously aware of in the next question.
//Is there a way to iterate through each JSON KeyValue and filter
here?
Partial duplicate of this question.
While you're in the loop discussed here, you can perform any data transformations you want, but what you should really be considering is using the JSONObject class referenced in that first link. Then you'll have JSON parsed into an object where you'll have better access to JSON key/value pairs.
//Do we need to validate each DataMap object value and is there a
dynamic way to do it?
Yes, we validate everything that comes from a user. All users are assumed to be trained hackers, and smarter than you. However if you handled filtering before you do your data mapping transformation, you don't need to do it a second time. Doing it dynamically?
Something like:
JSONObject json = new JSONObject(s);
Iterator iterator = json.keys();
while( iterator.hasNext() ){
String data = iterator.next();
//filter and or business logic
}
^^That syntax is skipping typechecks but it should get you where you need to go.
/Is Integer validation needed or will the thrown exception be good
enough?
I don't see where you're throwing an exception with these lines of code:
if (ESAPI.validator().isValidInput("strData", dm.strData, "HTTPParameterValue", 25, false, true))
{
//Is Integer validation needed or will the thrown exception be good enough?
return Response.ok("You entered:" + dm.strData + " and " + dm.intData).build();
}
Firstly, in java we have autoboxing which means this:
int foo = 555555;
String bar = "";
//the code
foo + bar;
Will be cast to a string in any instance. The compiler will promote the int to an Integer and then silently call the Integer.toString() method. Also, in your Response.ok( String ); call, THIS is where you're going to want to encodeForHTML or whatever the output context may be. Encoding methods are ALWAYS For outputting data to user, whereas canonicalize you want to call when receiving data. Finally, in this segment of code we also have an error where you're assuming that you're dealing with an HTTPParameter. NOT at this point in the code. You'll validate http Parameters in instances where you're calling request.getParameter("id"): where id isn't a large blob of data like an entire JSON response or an entire XML response. At this point you should be validating for things like "SafeString"
Usually there are parsing libraries in Java that can at least get you to the level of Java objects, but on the validation side you're always going to be running through every item and punting whatever might be malicious.
As a final note, while coding, keep these principles in mind your code will be cleaner and your thought process much more focused:
user input is NEVER safe. (Yes, even if you've run it through an XSS filter.)
Use validate and canonicalize methods whenever RECEIVING data, and encode methods whenever transferring data to a different context, where context is defined as "Html field. Http attribute. Javascript input, etc...)
Instead of using the method isValidInput() I'd suggest using getValidInput() because it will call canonicalize for you, making you have to provide one less call.
Encode ANY time your data is going to be passed to another dynamic language, like SQL, groovy, Perl, or javascript.
I have a custom SOAP message handler for incoming messages that will run different code based on which operation is being called. My first try to get the operation name looked something liket this:
public boolean handleMessage(SOAPMessageContext context)
{
String op = context.get(MessageContext.WSDL_OPERATION);
...
This failed because the property MessageContext.WSDL_OPERATION appears to never be set. I then tried using this:
public boolean handleMessage(SOAPMessageContext context)
{
Map<?, ?> headers = (Map<?, ?>)context.get(MessageContext.HTTP_REQUEST_HEADERS);
ArrayList<String> SOAPAction = ((ArrayList<String>) headers.get("SOAPAction"));
String opName = SOAPAction.get(0);
//opName will be formatted like "urn#myOperation", so the prefix must be removed
opName = ((opName.replace("\"","").split("#"))[1]);
This works, but I'm concerned there could be situations where the header property "SOAPAction" isn't set (or doesn't even exist), or does not have the value that I'm expecting it to. I'm also a little concerned because I don't know if this is an "official" way to get the operation name - I figured it out by looking at the contents of context in the debugger.
Is there any better way to get the operation name when handling incoming SOAP messages?
You could call body.getElementName().getLocalName() to retrieve the name of SOAP body element of the message payload. It's a little bit verbose and manual but it works. You could have the following in your handler
if ((boolean) context.get(MessageContext.MESSAGE_INBOUND_PROPERTY){ //for requests only
SOAPEnvelope msg = context.getMessage().getSOAPPart().getEnvelope(); //get the SOAP Message envelope
SOAPBody body = msg.getBody();
String operationName = body.getChildNodes().item(1).getLocalName();
}
The result of the above code is guaranteed to carry the name of the operation as specified in your WSDL
EDIT: This solution is based solely on the condition that the web service is implemented as document/literal-wrapped or RPC/literal
I'm very late to this party but I tried to do this over the past week. The accepted answer doesn't actually work for every JAX-WS implementation (at least not that I tried).
I have been trying to make this work on standalone Metro in my development environment but also using Axis2 bundled with WebSphere 7 in a real environment.
I found the following works on Metro:
String operationName = body.getChildNodes().item(0).getLocalName();
and the following works on Axis2:
String operationName = body.getChildNodes().item(1).getLocalName();
What is happening is that Axis2 inserts a Node of type Text into the Body as the first child but Metro doesn't. This text node returns a null local name. My solution was to do the following:
NodeList nodes = body.getChildNodes();
// -- Loop over the nodes in the body.
for (int i=0; i<nodes.getLength(); i++) {
Node item = nodes.item(i);
// -- The first node of type SOAPBodyElement will be
// -- what we're after.
if (item instanceof SOAPBodyElement) {
return item.getLocalName();
}
}
As described in the comments we're actually looking for the first node of type SOAPBodyElement. Hopefully that will help out anyone else looking at this in the future.
The SOAPMessageContext contains this information and can be retrieved super easily like this:
public boolean handleMessage(SOAPMessageContext msgContext) {
QName svcn = (QName) smc.get(SOAPMessageContext.WSDL_SERVICE);
QName opn = (QName) smc.get(SOAPMessageContext.WSDL_OPERATION);
System.out.prinln("WSDL Service="+ svcn.getLocalPart());
System.out.prinln("WSDL Operation="+ opn.getLocalPart());
return true;
}
in case if someone searches for "elegant" way to get needed properties use
for(Map.Entry e : soapMessageContext.entrySet()){
log.info("entry:"+ e.getKey() + " = " + e.getValue());
}
then decide what info you need and get it!
soapMessageContext.get(YOUR_DESIRED_KEY);