We Keep Coding

How to proxy Google API Client requests via WireMock to be able to Record and Play the API responses for unit testing

I have a class GoogleApiClientWrapper which basically handles interactions with Google for authentication and listing files and drives using Google Drive Java SDK version: v3-rev110-1.23.0.
I am using the following dependencies:
google-api-client:1.28.0
google-http-client:1.28.0
google-http-client-apache-2.0.0.jar
com.github.tomakehurst:wiremock-jre8:2.32.0
I am trying to unit test my class. So I thought about using WireMock Record & Playback feature to record the requests to Google APIs by routing the requests via WireMock to be able to record the responses for a given request.
So the idea is to create an instance of HttpTransport which is configured to proxy all requests to localhost on port 8443 which is the port WireMock is running on and inject this HttpTransport to the GoogleRefreshTokenRequest constructor or the Drive.Builder. Then WireMock is configured to proxy requests on localhost to the google api url. So ultimately the requests will be sent to google api the first time then after I am able to record the responses WireMock will be able to respond from the saved mappings.
Here it the implementation of GoogleRefreshTokenRequest in GoogleApiClientWrapper is as follows:
GoogleTokenResponse getGoogleTokenResponse(HttpTransport httpTransport,
JsonFactory jsonFactory,
String refreshToken,
String clientId,
String clientSecret,
List<String> exportScopes) throws IOException {
return new GoogleRefreshTokenRequest(httpTransport, jsonFactory, refreshToken, clientId, clientSecret)
.setScopes(exportScopes)
.execute();
}
Here is the test class with WireMock code to proxy the request and record the mappings.
public class GoogleApiWrapperTest {
#ClassRule
public static WireMockClassRule wireMockRule = new WireMockClassRule(WireMockConfiguration.wireMockConfig().httpsPort(8443));
#Test
public void testGetGoogleTokenResponse() throws IOException, GeneralSecurityException {
wireMockRule.startRecording("https://oauth2.googleapis.com/token");
GoogleApiClientWrapper clientWrapper= new GoogleApiClientWrapper();
HttpHost proxy = new HttpHost("127.0.0.1", 8443);
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
HttpClient client = HttpClients.custom()
.setRoutePlanner(routePlanner)
.build();
HttpTransport httpTransport = new ApacheHttpTransport(client);
String clientId = "my-client-id";
String clientSecret = "my-client-secret";
String tokenString = "sometoken";
String refreshToken = tokenString.split("\\|")[0];
GoogleTokenResponse actual = clientWrapper.getGoogleTokenResponse(httpTransport,JacksonFactory.getDefaultInstance(),refreshToken,clientId,clientSecret,Arrays.asList(DriveScopes.DRIVE_READONLY));
wireMockRule.stopRecording().getStubMappings();
}
}
The following code will be removed from the test class once mapping created:
wireMockRule.startRecording("https://oauth2.googleapis.com/token");
wireMockRule.stopRecording().getStubMappings();
When I run it I get the following exception:
org.apache.http.client.ClientProtocolException
Caused by: org.apache.http.ProtocolException: The server failed to respond with a valid HTTP response
I also tried to create the HttpTransport as follows based on this answer
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("127.0.0.0", 8443));
HttpTransport httpTransport = new NetHttpTransport.Builder().setProxy(proxy).build();
How to solve this issue or is there a missing configuration for the HttpTransport or for the WireMockClassRule? Or is there an easier way to be able to store the google api responses via WireMock?
I already used Mockito to mock the responses when testing other classes that use the GoogleApiClientWrapper class but in this unit test I want to try to test the behaviour.

List azure ad using msgraph-sdk-java

I am trying to bring current user from azure active directory as follows:
final ClientSecretCredential clientSecretCredential = new ClientSecretCredentialBuilder()
.clientId("client_id")
.clientSecret("secret")
.tenantId("tenat_id")
.build();
final TokenCredentialAuthProvider tokenCredentialAuthProvider = new TokenCredentialAuthProvider(Arrays.asList("https://graph.microsoft.com/User.Read.All"), clientSecretCredential);
final GraphServiceClient graphClient =
GraphServiceClient
.builder()
.authenticationProvider(tokenCredentialAuthProvider)
.buildClient();
final User me = graphClient.me().buildRequest().get();
and I am getting the following error:
java.io.IOException: java.util.concurrent.ExecutionException:
com.microsoft.aad.msal4j.MsalServiceException: AADSTS70011: The
provided request must include a 'scope' input parameter. The provided
value for the input parameter 'scope' is not valid. The scope openid
profile offline_access https://graph.microsoft.com/User.Read.All is
not valid. Trace ID: b035aaec-9c8a-4728-a237-6d63738adb00 Correlation
ID: de389e03-9d91-4f4b-aacf-449aa8fac460 Timestamp: 2021-04-05
21:44:43Z
any idea that I may be missing?????

You are using the client credential flow here.
The value passed for the scope parameter in this request should be the
resource identifier (application ID URI) of the resource you want,
affixed with the .default suffix.
The scope should be https://graph.microsoft.com/.default in your issue.

Hangout Chat API authentication fails with Default Service account

I'm testing the new Google Chat bot that should post messages asynchronously, thus I must use GoogleCredentials to authenticate requests.
It works perfectly fine when I use GoogleCredentials.getApplicationDefault() while working locally, but when I deploy the bot to Cloud Run it stops.
idiomatic secure way (does not work)
The HangoutsChat is created the following way:
var credentials = GoogleCredentials.getApplicationDefault()
.createScoped(CHAT_BOT_SCOPE);
var credentialsAdapter = new HttpCredentialsAdapter(credentials);
var chat = new HangoutsChat.Builder(
GoogleNetHttpTransport.newTrustedTransport(),
JacksonFactory.getDefaultInstance(),
credentialsAdapter)
.setApplicationName(BOT_NAME)
.build();
So in order to work locally, I set GOOGLE_APPLICATION_CREDENTIALS and point it to the service account key like this: export GOOGLE_APPLICATION_CREDENTIALS=./credentials/adc.json. While in the cloud environment I expect the same service account to just work, but if not set explicitly from the file, the credentials throw insufficientPermissions error with PERMISSION_DENIED.
workaround (works fine)
As a workaround, I'm creating credentials from the service account stored in Secret Manager like this:
var serviceAccount = Secrets.chatServiceAccount();
var credentials = GoogleCredentials.fromStream(streamFrom(serviceAccount))
.createScoped(CHAT_BOT_SCOPE);
var credentialsAdapter = new HttpCredentialsAdapter(credentials);
var chat = new HangoutsChat.Builder(
GoogleNetHttpTransport.newTrustedTransport(),
JacksonFactory.getDefaultInstance(),
credentialsAdapter)
.setApplicationName(BOT_NAME)
.build();
return chat;
private static InputStream streamFrom(String data) {
return new ByteArrayInputStream(data.getBytes(Charset.defaultCharset()));
}
The question is: is anyone able to use default credentials for HangoutsChat API? Is it even possible or I am missing something here?

HttpComponentMessageSender NTLM

Today I was trying to configure the HttpComponentsMessageSender which uses Apache's org.apache.httpcomponents.httpclient library. Before I used the standard Java classes (I think java.net.HttpUrlMessageSender or something along those lines).
My HTTP methods need to authenticate using NTLM and the software is running under Linux so I cannot use the default Windows mechanism to do authentication, but instead I must configure authentication inside the Java application.
Originally I was just using a custom implementation of java.net.Authenticator which I used to set the username and password
for the NTLM access. But when I switched to HttpComponentsMessageSender this approach did not work anymore. When I tried to setup a custom HttpClient configuration with the HttpComponentsMessageSender I ran into various issues which I thought I would document here.
I am going to post my own answer but if anyone has a better solution, please feel free to add your solution.

I solved this adding a custom build HttpClient object which I built with the HttpClientBuilder. However adding this caused the following exception to appear:
org.apache.http.ProtocolException: Content-Length header already present
The solution which I found in this answer was to add a HttpRequestInterceptor.
Using the setConnectionTimeout on the HttpComponentsMessageSender did not work anymore with my custom HttpClient object. Instead I had to inject a RequestConfig object into the HttpClientBuilder.
The NTCredentials object required the domain name as a parameters. Earlier when I was using the Authenticator interface from java.net it was enough to just supply the username and password.
This is the code I am using currently:
HttpComponentsMessageSender messageSender = new HttpComponentsMessageSender ();
AuthScope authscope;
NTCredentials credentials;
CredentialsProvider credentialsProvider;
Registry<AuthSchemeProvider> registry;
RequestConfig requestConfig;
authscope = new AuthScope (HOST_IP, HOST_PORT);
credentials = new NTCredentials ("user", "pass", null, "domain");
credentialsProvider = new BasicCredentialsProvider ();
credentialsProvider.setCredentials (authscope, credentials);
registry = RegistryBuilder.<AuthSchemeProvider>create ()
.register(AuthSchemes.NTLM, new NTLMSchemeFactory ())
.build ();
HttpRequestInterceptor interceptor
= (request, context) -> request.removeHeaders(HTTP.CONTENT_LEN);
requestConfig = RequestConfig.custom ()
.setConnectTimeout (3000)
.build ();
HttpClient httpClient
= HttpClientBuilder.create ()
.setDefaultRequestConfig (requestConfig)
.setDefaultAuthSchemeRegistry (registry)
.setDefaultCredentialsProvider (credentialsProvider)
.addInterceptorFirst (interceptor)
.build ();
messageSender.setHttpClient (httpClient);

google oauth fails fetching redirect url

I'm using Google OAuth2 client-side to authorize a web-app (Liferay Portlet) to use the Calendar Service.
On my Development Server, the whole flow completes successfully:
I start creating a GoogleAuthorizationCodeRequestUrl
Using com.google.api.client.extensions.jetty.auth.oauth2.LocalServerReceiver I create a new redirect URI and set it to wait for Google Response.
a new window/tab opens in user's browser, to googleAuthorizationCodeRequestUrl
user Logs in (if not already logged)
User authorizes the requested scopes
the tab closes automatically, the jetty's Callback URI is fetched from Google,
The flow continues with token exchange etc
But when Deploying on some other remote server (identical environment) the flow gets stuck in step 6. Google seems to be unable to find the redirect_uri. My browsers are landing on an error page, informing that they couldn't establish a connection to the server at localhost:[random port generated from jetty]
Checking the logs, I can see that in both cases (dev/remote server), the redirect_uri created by jetty is localhost:[5digits]/Callback. (not affected by the dns or ip on the remote server) Is this normal ? Did I miss something on the configuration? Maybe jetty was supposed to create another redirect URI, that I should additionally add from Google Dev Console (obviously I cant set to localhost..)
Is it possible that a firewall or proxy setting is blocking the redirect_url?
Any other ideas what I did wrong?
EDIT: posting some code for the URLs creation
GoogleAuthorizationCodeFlow flow = new GoogleAuthorizationCodeFlow.Builder(httpTransport, jsonFactory, secrets, scopes)
.setDataStoreFactory(dataStore).build();
Credential credents = flow.loadCredential(usermail);
String redirect_url = null;
// Checking if the given user is not authorized
if (credents == null) {
// Creating a local receiver
LocalServerReceiver receiver = new LocalServerReceiver();
try {
// Getting the redirect URI
String redirectUri = receiver.getRedirectUri();
// Creating a new authorization URL
AuthorizationCodeRequestUrl authorizationUrl = flow.newAuthorizationUrl();
// Setting the redirect URI
authorizationUrl.setRedirectUri(redirectUri);
// Building the authorization URL
String url = authorizationUrl.build();
// Logging a short message
log.info("Creating the authorization URL : " + url);
//This url will be fetched right after, as a button callback (target:_blank)
//by using :FacesContext.getCurrentInstance().getExternalContext().redirect(googleAuthUrl);
googleAuthUrl = url;
// Receiving authorization code
String code = receiver.waitForCode();
// Exchanging it for an access token
TokenResponse response = flow.newTokenRequest(code).setRedirectUri(redirectUri).execute();
// Storing the credentials for later access
credents = flow.createAndStoreCredential(response, id);
} finally {
// Releasing resources
receiver.stop();
}
}
// Setting up the calendar service client
client = new com.google.api.services.calendar.Calendar.Builder(httpTransport, jsonFactory, credents).setApplicationName(APPLICATION_NAME)
.build();

Instead of creating an instance of LocalServerReceiver by this:
// Creating a local receiver
LocalServerReceiver receiver = new LocalServerReceiver();
You should do this by using a LocalServerReceiver.Builder.
According to documentation non-parameter constructor is: Constructor that starts the server on "localhost" selects an unused port.
So you can use builder, set proper host name (remote server) and build LocalServerReceiver instance. (or you can use LocalServerReceiver(host, port) constructor)
This should set redirect_uri to proper address.