We Keep Coding

Encoding a URL Query Parameter so it can have a '+'

Apparently, in the move from Spring Boot 1 to Spring Boot 2 (Spring 5), the encoding behavior of URL parameters for RestTemplates changed. It seems unusually difficult to get a general query parameter on rest templates passed so that characters that have special meanings such as "+" get properly escaped. It seems that, since "+" is a valid character, it doesn't get escaped, even though its meaning gets altered (see here). This seems bizarre, counter-intuitive, and against every other convention on every other platform. More importantly, I can't figure out how to easily get around it. If I encode the string first, it gets double-encoded, because the "%"s get re-encoded. Anyway, this seems like it should be something very simple that the framework does, but I'm not figuring it out.
Here is my code that worked in Spring Boot 1:
String url = "https://base/url/here";
UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(url);
for (Map.Entry<String, String> entry : query.entrySet()) {
builder.queryParam(entry.getKey(), entry.getValue());
}
HttpEntity<TheResponse> resp = myRestTemplate.exchange(builder.toUriString(), ...);
However, now it won't encode the "+" character, so the other end is interpreting it as a space. What is the correct way to build this URL in Java Spring Boot 2?
Note - I also tried this, but it actually DOUBLE-encodes everything:
try {
for (Map.Entry<String, String> entry : query.entrySet()) {
builder.queryParam(entry.getKey(), URLEncoder.encode(entry.getValue(),"UTF-8" ));
}
} catch(Exception e) {
System.out.println("Encoding error");
}
In the first one, if I put in "q" => "abc+1#efx.com", then, exactly in the URL, I get "abc+1#efx.com" (i.e., not encoded at all). However, in the second one, if I put in "abc+1#efx.com", then I get "abc%252B1%2540efx.com", which is DOUBLE-encoded.
I could hand-write an encoding method, but this seems (a) like overkill, and (b) doing encoding yourself is where security problems and weird bugs tend to creep in. But it seems insane to me that you can't just add a query parameter in Spring Boot 2. That seems like a basic task. What am I missing?

Found what I believe to be a decent solution. It turns out that a large part of the problem is actually the "exchange" function, which takes a string for a URL, but then re-encodes that URL for reasons I cannot fathom. However, the exchange function can be sent a java.net.URI instead. In this case, it does not try to interpolate anything, as it is already a URI. I then use java.net.URLEncoder.encode() to encode the pieces. I still have no idea why this isn't standard in Spring, but this should work.
private String mapToQueryString(Map<String, String> query) {
List<String> entries = new LinkedList<String>();
for (Map.Entry<String, String> entry : query.entrySet()) {
try {
entries.add(URLEncoder.encode(entry.getKey(), "UTF-8") + "=" + URLEncoder.encode(entry.getValue(), "UTF-8"));
} catch(Exception e) {
log.error("Unable to encode string for URL: " + entry.getKey() + " / " + entry.getValue(), e);
}
}
return String.join("&", entries);
}
/* Later in the code */
String endpoint = "https://baseurl.example.com/blah";
String finalUrl = query.isEmpty() ? endpoint : endpoint + "?" + mapToQueryString(query);
URI uri;
try {
uri = new URI(finalUrl);
} catch(URISyntaxException e) {
log.error("Bad URL // " + finalUrl, e);
return null;
}
}
/* ... */
HttpEntity<TheResponse> resp = myRestTemplate.exchange(uri, ...)

How to Pass URL as Path Variable

I am trying to pass a URL as a path variable, but when I input the URL as a parameter and access the route, it returns an error. I want to be able to see the address after it passes into the route as a parameter.
#RequestMapping("/addAddress/{address}")
public String addAddress(#PathVariable("address") String address) {
System.out.println("Address: "+address)
return address;
}
For example, if I put into the URL:
localhost:8080/addAddress/http://samplewebsite.com
I should see
http://samplewebsite.com
printed out in the back end.

The forward slashes are your issue.
You have a few choices.
Use 2 path variables. This still works with a double forward slash. This works with URL: "localhost:8080/addAddress/http://samplewebsite.com"
#GetMapping("/addAddress/{schema}/{address}")
public String addAddress(#PathVariable("schema") String schema, #PathVariable("address") String address) {
System.out.println("Address: "+ schema + "//" + address);
return schema + "//" + address;
}
Use a query (request) param, your url would then be URL: "localhost:8080/addAddress/?address=http://samplewebsite.com"
#GetMapping("/addAddress2")
public String addAddress2(#RequestParam("address") String address) {
System.out.println("Address: "+address);
return address;
}
Encode the slashes in URL:
localhost:8080/addAddress/http:%2F%2Fsamplewebsite.com"
and configure Tomcat or Jetty, whatever you use to allow encoded slashes. Here is an example in Tomcat

You can do the following;
#RequestMapping("/addAddress/**")
public String addAddress(HttpServletRequest request) {
String fullUrl = request.getRequestURL().toString();
String url = fullUrl.split("/addAddress/")[1];
System.out.println(url);
return url;
}
with #PathVariable that is not doable due to the / char breaking the behaviour you are looking for, unless you encode/decode, but I feel like this is a simpler way to go for both user of the endpoint, and for the backend.
Also this will not fetch the request query part, e.g. ?input=user,
to do that you can add this logic

You can use SafeUrl to parse your URL in your path.
Some documentation:
https://www.urlencoder.io/java/
It let you pass parameters safe by the url and you can decode where you need.

Invalid Identity Server Tokens

I have created a system with 4 main applications:
the identity server (C#)
the web app (angular, TS, JS)
the server side application (Java)
the API (C#)
The web app authenticates fine, the tokens work like a treat and allow access to the API, but the server-side app doesn't. The tokens seem to be generated correctly but when they are added to the request headers and dispatched they just return a 401 error.
If I generate the token through postman they work correctly but when I generate them through the java application they don't work. Also, they seem to be a different length.
The java application uses the client credentials flow to get token and is implemented like this:
String strBody;
try {
strBody = "client_id=" + id + "&client_secret=" + URLEncoder.encode(secret, "UTF-8") + "&grant_type=client_credentials&scope=" + scope;
} catch (UnsupportedEncodingException e) {
Logger.severe("Failed to get encode client secret with error: " + e.getMessage());
((CompletableFuture)accessToken).completeExceptionally(new Exception("Failed to get encode client secret with error: " + e.getMessage()));
return null;
}
RequestBody body = RequestBody.create(MediaType.parse("application/x-www-form-urlencoded; charset=utf-8"), strBody);
Request request = new Request.Builder().url(url).post(body).build();
String strResponse = executeRequest(request);
Logger.info("got token response: "+strResponse);
TokenResponse response = (new Gson()).fromJson(strResponse, TokenResponse.class);
if (response.getError() != null && response.getError().length() > 0) {
Logger.severe("Failed to get access token with error: " + response.getError());
((CompletableFuture)accessToken).completeExceptionally(new Exception("Failed to get access token with error: " + response.getError()));
} else {
Timers.runIn((timer, thing) -> clientCreditentalFlow(), response.getExpiresIn() - 10, TimeUnit.SECONDS);
Logger.info("token: "+accessToken);
((CompletableFuture)accessToken).complete(response.getAccessToken());
}
When I send the exact same details from post man, it returns a working token, although of a different length.
The web application uses the implicit work flow as opposes to the java app that uses the client credentials flow.

Android Programming - Post Google Form/Spreedsheet (REQUIRED FIELD)

So I have successfully post data onto a Google Spreadsheet using the Google Form source. Everything works perfect UNTIL I make the field (in the Google Form) "required." When I do that, the Android Emulator still responds as if the information sent was properly saved. But on the Google spreadsheet it isn't there.
Am I missing something?
This is my AsyncTask:
new BackgroundTask().execute(
"https://docs.google.com/forms/d/10QStmb9Nr-hcfv889FMSNTZdA_hNUErxeK7vISzkx0E/formResponse",
student.FirstName, "entry_2030274183=",
student.LastName, "entry_1558758483=",
student.Age, "entry_1871336861=",
student.Gender, "entry.2013677542=",
student.Grade, "entry_1921311866=");
This is my Background.
protected String doInBackground(String... params) {
HttpRequest reg = new HttpRequest();
String URL = params[0];
String FirstName = params[1];
String FirstNameEntry = params[2];
String LastName = params[3];
String LastNameEntry = params[4];
String Age = params[5];
String AgeEntry = params[6];
String Gender = params[7];
String GenderEntry = params[8];
String Grade = params[9];
String GradeEntry = params[10];
#SuppressWarnings("deprecation")
String data =
FirstNameEntry + URLEncoder.encode(FirstName) + "&" +
LastNameEntry + URLEncoder.encode(LastName) + "&" +
AgeEntry + URLEncoder.encode(Gender) + "&" +
GenderEntry + URLEncoder.encode(Age) + "&" +
GradeEntry + URLEncoder.encode(Grade);
String response = reg.sendPost(URL, data);
return response;
}
Do I need to put something in the entries if it is a required field?
If you want to look at the HttpRequest class go here (Not My Code):
Secure HTTP Post in Android
Much Appreciated

The only way I can immediately think of is by processing the response and then making your app behave accordingly.
For instance - I tried one test form and if the request send had some required field empty, then the HTTPResponse contains "Looks like you have a question or two that still need attention".
Another way would be to validate if the save was actually successful by searching for the text you gave in the "Confirmation Page".
In both cases, you should be able to differentiate between a successful post and a failed one.

request.isSecure() returns true when connection is http

I am trying to check if my url is http/https and then make an AJAX call to populate multiple dropdowns based on the value selected in the 1st dropdown.
var baseurl="<%=request.isSecure()%>";
if (baseurl=='true') {
var url =
"https://<%=request.getServerName()%><%=request.getContextPath()%>/JsonLookup?z=" + zone;
}
// if the protocol is 'http'
else
{
alert("inside protocol else");
var url = "http://<%=request.getServerName()%><%=request.getContextPath()%>/JsonLookup?z=" + zone;
alert("inside http:"+url);
}
I have problem when the protocol is http, the dropdowns do not change. The reason is, base url (isSecure()) is always returning true even if it is http.
Can anyone help me solve this issue, what exactly am I supposed to do in order to make request.isSecure() return false for http?

Can't you use references without host name?
/path/to/resource
instead of
http://myserver.com/path/to/resource
EDIT:
Here's how your example will look like:
var url = "/<%=request.getContextPath()%>/JsonLookup?z=" + zone;
alert("My root relative URL: "+url); }