I have a vendor that I wish to exchange data with. They want me to take the username and password that they gave me and use it on an Authorization header for a get request.
And now my dirty little secret. I've never created an Authorization header before.
So I do a bunch of research and figure out the following code.
public static final String AUTH_SEPARATOR = ":";
private static final String AUTH_TYPE = "Basic ";
public static final String HEADER_AUTHORIZATION = "Authorization";
public static void addAuthHeader(Map<String, String> headers, String user, String password) {
String secretKey = user
+ AUTH_SEPARATOR
+ password;
byte[] tokenBytes = secretKey.getBytes();
String token64 = Base64.getEncoder().encodeToString(tokenBytes);
String auth = AUTH_TYPE + token64;
headers.put(HEADER_AUTHORIZATION, auth);
}
I run it and I get no response back. Hmmm. So I go to open a support request with them and I want to create an example, so I open postman and use the APIs they gave me for postman. First, I run the one for the API I'm replicating and it works. Hmm.
So then I modify that API and use my username and password instead of the one included in the example and it works fine. Crikey!
So I bang around a bit and notice that the Base64 string in the auth created by postman is slightly different at the end than the one I created.
So, back to the research and all the code I find looks a lot like mine, although I had to update it some because of version differences. The string is still different and now I'm asking for help. Surely someone has solved this problem.
String from postman "Basic THVKZ...FvTg=="
String from code above "Basic THVKZ...FvTiA="
How did I do something wrong and end up with only a three byte difference?
Tg== is base64 for N.
TiA= is base64 for N (as in, N, then a space).
Sooo, it sounds like postman is sticking a space up there and you aren't. Hopefully, if you add a space at the very end of whatever you are base64 encoding, you should get the exact same string as postman is giving you, and, hopefully, it'll all just work out at that point :)
Postman using UTF-8 for basic auth encoding, check from https://github.com/postmanlabs/postman-app-support/issues/4070
change your code like this
secretKey.getBytes(StandardCharsets.UTF_8);
https://www.base64encode.org/ for test
The problem is caused by padding. I still don't understand exactly why, but the string I'm encoding is 49 bytes long, which is not evenly divisible by 3, which means that padding comes into play.
When I changed my code to the following:
String token64 = Base64.getEncoder().withoutPadding().encodeToString(tokenBytes);
and then ran it, I got the same string minus the two == at the end that base64 uses as a pad character. Sending that to the server got the answer I was looking for.
Why do they call it software when it's so damned hard?
Esteemed developer, are they not running on OAuth2? If so, kindly make use of the code blocks below and kill it
public HttpHeaders getHeaders() {
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Collections.singletonList(MediaType.APPLICATION_JSON));
headers.add("Authorization", "Bearer " + token you are either generating or getting from Eureka discovery service);
return headers;
}
AuthResponseObjectType getAuthorization() {
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Collections.singletonList(MediaType.APPLICATION_JSON));
String auth = basicAuthUsername + ":" + basicAuthPassword;
byte[] encodedAuth = org.apache.commons.net.util.Base64.encodeBase64(auth.getBytes(StandardCharsets.US_ASCII));
headers.add("Authorization", "Basic " + new String(encodedAuth));
MultiValueMap<String, String> map = new LinkedMultiValueMap<>();
map.add("grant_type", grantType);
map.add("username", username);
map.add("password", password);
map.add("scope", scope);
HttpEntity<MultiValueMap<String, String>> request = new HttpEntity<>(map, headers);
return restTemplate.exchange(authUrl, HttpMethod.POST, request, AuthResponseObjectType.class).getBody();
}
Let me know if you are facing any challenges then we can do it together
Related
I'm using passport-oauth2 (passportjs.org and https://github.com/jaredhanson/passport-oauth2/blob/master/lib/strategy.js) for OAuth2+PKCE integration in a nodejs application.
The backend it's authenticating against is written in Java.
The problem is that I can't seem to decode->hash the code_verifier to correctly match the code_challenge that comes from passport-oauth2.
I know that the Base64 encoding that comes from passport has been generated to be URL safe (no padding, no wrapping, replacements for + or /), so I'm using a Url Decoder:
Base64.getUrlDecoder().decode(...)
Then I'm using commons DigestUtils to generate a SHA256 of the decoded verifier and comparing it with the challenge. So the whole thing looks something like this:
java.util.Base64.Decoder decoder = java.util.Base64.getUrlDecoder();
String codeChallenge = // get the code challenge from my cache
byte[] decodedCodeChallenge = decoder.decode(codeChallenge);
byte[] decodedCodeVerifier = decoder.decode(codeVerifier);
if (!Arrays.equals(sha256(decodedCodeVerifier), decodedCodeChallenge)) {
return Response.status(400).entity(ERROR_INVALID_CHALLENGE_VERIFIER).build();
}
Example:
This code verifier: 5CFCAiZC0g0OA-jmBmmjTBZiyPCQsnq_2q5k9fD-aAY
should match this code challenge: Fw7s3XHRVb2m1nT7s646UrYiYLMJ54as0ZIU_injyqw once both have been Base64-url-decoded and the verifier has been SHA256 hashed, but it doesn't.
What am I doing wrong?
Just 5 minutes later I figured it out.
In passport-oauth2, the code verifier is Base64-url-encoded(random bytes):
verifier = base64url(crypto.pseudoRandomBytes(32))
See: https://github.com/jaredhanson/passport-oauth2/blob/master/lib/strategy.js#L236
The challenge is then Base64-url-encoded(sha256(verifier)), which expands to Base64-url-encoded(sha256(Base64-url-encoded(random bytes))):
challenge = base64url(crypto.createHash('sha256').update(verifier).digest());
See: https://github.com/jaredhanson/passport-oauth2/blob/master/lib/strategy.js#L242
So to do the verification, I don't need to decode anything. It was sha256-d in it's encoded state.
This worked in the end:
java.util.Base64.Encoder encoder = java.util.Base64.getUrlEncoder();
String codeChallenge = // get code challenge from my cache;
String encodedVerifier = new String(encoder.encode(sha256(codeVerifier))).split("=")[0]; // Remember to remove padding
if (!encodedVerifier.equals(codeChallenge)) {
return Response.status(400).entity(ERROR_INVALID_CHALLENGE_VERIFIER).build();
}
I'm working on this project in which I'm using a Google-App-Engine backend connected to an Android app via Google-Cloud-Endpoints. For Google-Cloud-Datastore access I'm using Objectify and everything works fine.
Now I decided to add the functionality to upload images to Google-Cloud-Storage but I couldn't find a clear explanation on how to do this using the Google-Cloud-Endpoints setup.
I found the following explanation how to use Google-Cloud-Storage with Google-App-Engine:
https://cloud.google.com/appengine/docs/java/googlecloudstorageclient/app-engine-cloud-storage-sample
but instead of adding it to the Endpoints Api the article writes an additional servlet.
Furthermore I found this example of upload/download for Android:
github.com /thorrism/GoogleCloudExample
Sadly this is using the Google Cloud Storage API for direct access to the Google-Cloud-Storage and you need to add a P12-file to the asset folder, which seems unsecure.
My Google-App-Engine code looks like that:
#Api(
name = "example",
version = "v1",
scopes = { Constants.EMAIL_SCOPE },
clientIds = { Constants.WEB_CLIENT_ID, Constants.ANDROID_CLIENT_ID, Constants.API_EXPLORER_CLIENT_ID },
audiences = {Constants.ANDROID_AUDIENCE},
description = "API for the Example Backend application."
)
public class ExampleApi{
#ApiMethod(name = "doSomething", path = "dosomething", httpMethod = HttpMethod.POST)
public String doSomething(#Named("text") String text){
TestEntity test = new TestEntity(text);
ofy().save().entity(test).now();
return test;
}
After I uploaded it I generated the Endpoints Client Library and imported it into my android project.
Then I'm calling Endpoints from Android like explained here:
https://cloud.google.com/appengine/docs/java/endpoints/calling-from-android#creating_the_service_object
public static com.appspot.******.example.Example buildServiceHandler(Context context, String email) {
GoogleAccountCredential credential = GoogleAccountCredential.usingAudience(
context, AppConstants.AUDIENCE);
credential.setSelectedAccountName(email);
com.appspot.******.example.Example.Builder builder = new com.appspot.******.example.Example.Builder(
AppConstants.HTTP_TRANSPORT,
AppConstants.JSON_FACTORY, null);
builder.setApplicationName("example-server");
return builder.build();
}
sApiServiceHandler = buildServiceHandlerWithAuth(context,email);
And each Api-Method I call like this:
com.appspot.******.example.Example.DoSomething doSomething = sApiServiceHandler.doSomething(someString);
doSomething.execute();
All of this works fine, but only for storing/receiving Datastore Entities. How would I go about uploading/downloading files to Google Cloud Storage using the Google Cloud Endpoints setup?
Is it somehow possible to send a POST with my image data via Endpoints to the UploadServlet using the already build ServiceHandler ?
Is it possible to call a servlet from an Endpoints Method? How am I supposed to send the Post to the Servlet and how would I go about the authentication?
Any help or advice would be greatly appreciated!
There are different ways to do this, but the most recommended way is to use Signed URLs, so that your Android app can upload the file securely to Google Cloud Storage directly, without going through your Endpoints backend. The basic process is:
1) Create an Endpoints method that creates a new signed URL and returns it to the Android client. Signing the URL on the server still requires a P12 key but is stored on App Engine, not on the client, so is secure. Try to use a short expiration for the URL, for example no more than 5 minutes.
2) Have the Android client upload the file directly to the signed URL, as you would doing a normal HTTP PUT to the Cloud Storage XML API to upload a file (resumable uploads with the JSON API are also supported, but not covered here).
Your Endpoints method might look like this:
#ApiMethod(name = "getUploadUrl", path = "getuploadurl", httpMethod = HttpMethod.GET)
public MyApiResponse getUploadUrl(#Named("fileName") String fileName
#Named("contentType" String contentType)
{
String stringToSign
= "PUT\n" + contentType
+ "\n" + EXPIRATION_TIMESTAMP_EPOCH_SECONDS + "\n"
+ YOUR_GCS_BUCKET + "/" + fileName;
// Load P12 key
FileInputStream fileInputStream = new FileInputStream(PATH_TO_P12_KEY);
KeyStore keyStore = KeyStore.getInstance("PKCS12");
keyStore.load(fileInputStream, password);
PrivateKey key = keyStore.getKey(privatekey", YOUR_P12_KEY_PASSWORD);
// Get signature
Signature signer = Signature.getInstance("SHA256withRSA");
signer.initSign(key);
signer.update(stringToSign.getBytes("UTF-8"));
byte[] rawSignature = signer.sign();
String signature = new String(Base64.encodeBase64(rawSignature, false), "UTF-8");
// Construct signed url
String url
= "http://storage.googleapis.com/" + YOUR_GCS_BUCKET + fileName
+ "?GoogleAccessId=" + P12_KEY_SERVICE_ACCOUNT_CLIENT_ID
+ "&Expires=" + EXPIRATION_TIMESTAMP_EPOCH_SECONDS
+ "&Signature=" + URLEncoder.encode(signature, "UTF-8");
// Endpoints doesn't let you return 'String' directly
MyApiResponse response = new MyApiResponse();
response.setString(url);
return response;
}
On the Android side, you might use the method like this:
// Get the upload URL from the API
getUploadUrl = sApiServiceHandler.getUploadUrl(fileName, contentType);
MyApiResponse response = getUploadUrl.execute();
String uploadUrl = response.getString();
// Open connection to GCS
URL url = new URL(uploadUrl);
HttpURLConnection httpConnection = (HttpURLConnection) url.openConnection();
httpConnection.setDoOutput(true);
httpConnection.setRequestMethod("PUT");
httpConnection.setRequestProperty("Content-Type", contentType);
// Write file data
OutputStreamWriter out = new OutputStreamWriter(httpConnection.getOutputStream());
out.write(fileData);
out.flush();
// Get response, check status code etc.
InputStreamReader in = new InputStreamReader(httpConnection.getInputStream());
// ...
(Disclaimer: I'm just typing code freely into a text editor but not actually testing it, but it should be enough to give you a general idea.)
The goal is to send an email with inline image. Everything is working well, except the image is not appearing in the email.
My approach is based on this Jersey-example of Mailgun's User Guide.
public static ClientResponse SendInlineImage() {
Client client = Client.create();
client.addFilter(new HTTPBasicAuthFilter("api",
"YOUR_API_KEY"));
WebResource webResource =
client.resource("https://api.mailgun.net/v3/YOUR_DOMAIN_NAME" +
"/messages");
FormDataMultiPart form = new FormDataMultiPart();
form.field("from", "Excited User <YOU#YOUR_DOMAIN_NAME>");
form.field("to", "baz#example.com");
form.field("subject", "Hello");
form.field("text", "Testing some Mailgun awesomness!");
form.field("html", "<html>Inline image here: <img src=\"cid:test.jpg\"></html>");
File jpgFile = new File("files/test.jpg");
form.bodyPart(new FileDataBodyPart("inline",jpgFile,
MediaType.APPLICATION_OCTET_STREAM_TYPE));
return webResource.type(MediaType.MULTIPART_FORM_DATA_TYPE).
post(ClientResponse.class, form);
}
However, I need to use Spring's RestTemplate.
This is what I've got so far:
RestTemplate template = new RestTemplate();
MultiValueMap<String, Object> map = new LinkedMultiValueMap<>();
// ... put all strings in map (from, to, subject, html)
HttpHeaders headers = new HttpHeaders();
// ... put auth credentials on header, and content type multipart/form-data
template.exchange(MAILGUN_API_BASE_URL + "/messages", HttpMethod.POST,
new HttpEntity<>(map, headers), String.class);
The remaining part is to put the *.png file into the map. Not sure how to do that. Have tried reading all its bytes via ServletContextResource#getInputStream, but without success: Image is not appearing in the resulting e-mail.
Am I missing something here?
This turned out to be a case where everything was set up correctly, but only a small detail prevented it from working.
map.add("inline", new ServletContextResource(this.servletContext,
"/resources/images/email-banner.png"));
For Mailgun you need to use the map-key "inline". Also, the ServletContextResource has a method getFilename(), which is used to resolve against the image tag. Thus, the image tag should have the following content id:
<img src="cid:email-banner.png"/>
Here is my sample url:
url.com/data?format=json&pro={%22merchanturl%22:%22http://url.com/logo.png%22,%22price%22:599,%22productDesc%22:%22Apple%2032GBBlack%22,%22prodID%22:%22913393%22,%22merchant%22:%224536%22,%22prourl%22:%22http://url.com/data%22,%22name%22:%22Apple%2032GB%20%2D%20Black%22,%22productUrl%22:%22http://www.url.com/image.jpg%22,%22myprice%22:550,%22mercname%22:%22hello%22,%22mybool%22:false}
I have an android app. I need to post this url to server. So that server responds back with a token. I am doing the httppost through app. But I am not getting any response/exception. If I copy the same url and paste it in browser, that works very well. I hope I am doing mistake with the encoding part. Can anyone point out my issue?
Here is my encoding method:
private String encodeString(String input) {
String output = new String(input.trim().replace(" ", "%20")
.replace("&", "%26").replace(",", "%2c").replace("(", "%28")
.replace(")", "%29").replace("!", "%21").replace("=", "%3D")
.replace("<", "%3C").replace(">", "%3E").replace("#", "%23")
.replace("$", "%24").replace("'", "%27").replace("*", "%2A")
.replace("-", "%2D").replace(".", "%2E").replace("/", "%2F")
.replace(":", "%3A").replace(";", "%3B").replace("?", "%3F")
.replace("#", "%40").replace("[", "%5B").replace("\\", "%5C")
.replace("]", "%5D").replace("_", "%5F").replace("`", "%60")
.replace("{", "%7B").replace("|", "%7C").replace("}", "%7D")
.replace("\"", "%22"));
return output;
}
Update:
The reason why I am doing like this is, I need to send the data as in this format. The parameters part of the url is a json data. If I encode the complete url, that is not working.
Try using URLEncoder, encode only the part after ?
String query = URLEncoder.encode(queryPart, "utf-8");
String url = "http://server.com/search?q=" + query;
Although a self-written encoding isn't bad, I recommend using built-in Java methods that have been proven to be working.
TextUtils contains a method htmlEncode(String s) just for this.
http://developer.android.com/reference/android/text/TextUtils.html#htmlEncode%28java.lang.String%29
This seems like an extremely easy problem but alas I cannot figure it out nor find a solution anywhere else. I'm concatenating a string that has a % within and for some reason it adds the number 25 after the %. Anyone know of a solution to this easy problem?
String buttonCheck = "%26" + DATABASE.getValue("buttonCheck") + "%26";
Comes out to
"%2526value%2526"
EDIT: It has become apparent that the issue is actually within URL encoding and I will add more relevant data to the issue.
I am developing an Android App that parses HTML from a site and allows the user to interact with it through the Android UI. I am having an issue with encoding % into a parameter for a form.
public class CLASS extends Activity
{
DefaultHttpClient client = new DefaultHttpClient;
String url = "http://www.url.com"
HttpRequestBase method = new HttpPost(url);
List<NameValuePair> nvps = new ArrayList<NameValuePair>();
nvps.add("buttoncheck", "%26" + DATABASE.getValue("buttonCheck") + "%26");
//DATABASE is simply a class that handles a HashMap
HttpPost methodPost = (HttpPost) method;
methodPost.setEntity(new UrlEncodedFormEntity(nvps, HTTP.UTF_8));
execute(method);
}
Rather than sending the form value of
buttoncheck=%26value%26
I get
buttoncheck=%2526value%2526
It won't come out as "%2526value%2526" in buttonCheck. I strongly suspect you're looking at a value later on - for example, after URI encoding. Work out what's doing that encoding, and what you actually want to be encoded.
Just to be clear, the 25 is completely distinct from the 26. You'll see the same thing if you get rid of the 26 completely, with
String buttonCheck = "%" + DATABASE.getValue("buttonCheck") + "%";
At that point I suspect you'll get
%25value%25
Basically something is just encoding the % as %25. For example, this would do it:
import java.net.*;
public class Test {
public static void main(String[] args) throws Exception {
String input = "%value%";
String encoded = URLEncoder.encode(input, "utf-8");
System.out.println(encoded); // Prints %25value%25
}
}
apparently the string went through "percent-encoding" when it becomes part of a URI.
If that's the case, you should not do percent-encoding so early. instead
String buttonCheck = "&" + DATABASE.getValue("buttonCheck") + "&";
which will end up in the URI as
"%26value%26"
Based on the answer to this question, I would guess that the literal % is being encoded to %25 in your string. Which explains the added 25. Without seeing relevant code, we won't be able to know why it gets there in the first place.