We Keep Coding

Receiving 401 when accessing authenticated Google Cloud Function

I am trying to invoke an authenticated HTTP-based cloud function from another cloud function. Let's call them CF1 and CF2 respectively, for the sake of brevity; thus I wish to invoke CF2 from CF1.
Following the example given by the Google Documentation: Authenticating for Invocation, I created a new service account for CF2, and then attached it to CF1 with the roles/cloudfunctions.admin . I downloaded a service key for local testing with Functions Framework, setting it as the Application Default Credentials(ADC); thus CF2 on my local machine connects to CF1 on GCP, authenticating as CF2's service account via ADC.
I have deployed CF1 on Cloud Functions successfully, and was testing whether CF2 on my local machine could reach to CF1 when I was surprised to receive a HTTP 401.
For reference, here is the code in question, which is almost identical to the samples provided by the Google Documentation:
String serviceUrl = "<cf1-url>";
GoogleCredentials credentials = GoogleCredentials.getApplicationDefault();
if (!(credentials instanceof IdTokenProvider)) {
throw new IllegalArgumentException("Credentials are not an instance of IdTokenProvider.");
}
IdTokenCredentials tokenCredential =
IdTokenCredentials.newBuilder()
.setIdTokenProvider((IdTokenProvider) credentials)
.setTargetAudience(serviceUrl)
.build();
GenericUrl genericUrl = new GenericUrl(serviceUrl);
HttpCredentialsAdapter adapter = new HttpCredentialsAdapter(tokenCredential);
HttpTransport transport = new NetHttpTransport();
com.google.api.client.http.HttpRequest request = transport.createRequestFactory(adapter).buildGetRequest(genericUrl);
com.google.api.client.http.HttpResponse response = request.execute();
I tried referring to:
Google Cloud Platform - cloud functions API - 401 Unauthorized
Cloud Function Permissions (403 when invoking from another cloud function)
Google Cloud Function Authorization Failing
but I was not able to find a solution to my problem from those questions.
Further testing revealed that the identity token generated via the client SDK:
tokenCredential.getIdToken().getTokenValue() is different from the GCloud CLI command gcloud auth print-identity-token. I could use the identity token generated by GCloud CLI to directly invoke CF1 (e.g. via Postman/cURL and authenticated as CF2's service account) but not the identity token printed by the client SDK. This was a surprise as I am using CF 2's service account keys as the ADC, and also authorized it for gcloud access via gcloud auth activate-service-account.
It seems to me that there is no issue with the permissions of the service accounts and cloud functions, as I can directly invoke CF1; thus it would appear to be an issue with the code. However, I am unable to determine the cause of the 401 error.

The target audience, your serviceURL, must be the raw url, this one provided by the Cloud Functions service.
If you add your parameters (query or path) it won't work.

Keycloak: Granted scope not saved for federated users

Problem:
I have a Keycloak realm with a mixed user base.
Some users are native (native users), some are provided trough a custom User Storage SPI implementation (federated users).
When I perform an OIDC login, Keycloak asks to approve 4 scopes (test-client, profile, email, roles).
The request is performed with scope openid. Keycloak transforms it to these four scopes.
test-client is the client name. There is no actual client scope with this name showing in the admin UI.
Using a native user everything works fine. I approve and the login completes successfully.
Having a look at the user consents confirms that everything is ok.
Using a federated user the login fails with the following message.
[invalid_scope] Client no longer has requested consent from user
Looking at the users consent we see that the test-client scope is not being added.
Trying to exchange the code for a token, produces a fitting error in the Keycloak log
11:06:31,964 WARN [org.keycloak.events] (default task-10) type=CODE_TO_TOKEN_ERROR, realmId=torment, clientId=test-client, userId=f:ff4c66e5-2a6f-465c-8418-200648a49973:dfb_user, ipAddress=127.0.0.1, error=not_allowed, grant_type=authorization_code, code_id=418dfa66-b4c8-4481-b46d-ceac97e65b39, client_auth_method=client-secret
All further login tries will ask to approve only the missing test-client scope, but it will never be added to the consent.
Question:
Why does this happen?
How do I make it work?

It seems like you are not passing scopes list as query parameters in the URL constructed to initiate Keycloak authentication. Please check your authentication URL.
Set<ClientScopeModel> clientScopes = TokenManager.getRequestedClientScopes(scopeParam, client);
if (!TokenManager.verifyConsentStillAvailable(session, user, client, clientScopes)) {
event.error(Errors.NOT_ALLOWED);
throw new CorsErrorResponseException(cors, OAuthErrorException.INVALID_SCOPE, "Client no longer has requested consent from user", Response.Status.BAD_REQUEST);
}
Your error is coming from here.
If you take a look at TokenManager.getRequestedClientScopes (line number 523), it checks whether user still has granted consents to all requested client scopes.
So, I think you can try to add all the scopes in request URL. For example, http://keycloak_url/auth/realms/.......?....scope=openid%20test_client%20profile.....
I hope it helps.
Additionally, you can enable this setting on Keycloak Client (Scopes tab).

Retrieve Firebase User using token, from Google Cloud application running locally

I'm working on a Java API that functions as an endpoint API, and on production
it runs on the Google Cloud Platform. API methods are called by passing a Firebase token as part of the URL, and the token is used to create a User that's available inside the API method:
#ApiMethod(path = "myPath/{tokenId}/doSomething", httpMethod = "get")
public ResponseMessage ReturnSomething(#Named("tokenId") String tokenId, User user) throws UnauthorizedException, BadRequestException, InternalServerErrorException, FirebaseAuthException
{
if (user == null)
...
In production, when the URL is called from an Angular application on Firebase that passes the token in the URL, user is correctly created. I don't fully understand how the User is created from the token, I only know that it somehow happens "automatically" as part of Firebase integration with Google Cloud.
I want to debug the API locally by using Debug As > App Engine from inside Eclipse. When I do this however, and call the API from my local Angular application running using Firebase serve, the token is correctly passed to my locally running API, however user is always null.
#ApiMethod(path = "myPath/{tokenId}/doSomething", httpMethod = "get")
public ResponseMessage ReturnSomething(#Named("tokenId") String tokenId, User user) throws UnauthorizedException, BadRequestException, InternalServerErrorException, FirebaseAuthException
{
if (user == null)
// this is always null
I suspect this is a problem with my locally running Java API correctly authenticating to Firebase. I've looked at this guide, which suggests that the GOOGLE_APPLICATION_CREDENTIALS property on Windows should be set to the path of the JSON key of the App Engine default service account, which is the normal way to ensure that local access is granted to Google Cloud (and presumably Firebase) resources.
I've added this explicitly (I'd already run gcloud auth application-default login anyway, using the command line) however it's still not working. I just get null for the user and there's no indication of what's going on. I don't want to programatically authenticate as that means altering the API code to authenticate differently during debugging. How do I retrieve a User when debugging locally as App Engine?
UPDATE
I've realised that although the tokenId in the URL is present, I'm getting the following error when the API is called:
WARNING: Authentication failed: com.google.api.auth.UnauthenticatedException: No auth token is contained in the HTTP request
The tokenId value in the code below is a valid value, so I'm not sure why I'm getting this message:
#ApiMethod(path = "myPath/{tokenId}/doSomething", httpMethod = "get")
public ResponseMessage ReturnSomething(#Named("tokenId") String tokenId, User user)

I discovered that this was actually a problem with the Auth0 library that's being used in Angular to support authenticated HTTP requests to the Java API. The Auth0 library is used to inject the auth token into the Bearer of the request header whenever an Angular http.get is called from the Angular application. Creation of the User depends on this property being present in the HTTP header, with its value set to the value of the auth token.
I fixed this by altering the config for this library. I needed to temporarily whitelist localhost for the port (8080) that the API runs on, to allow Auth0 to inject the token into the HTTP header whenever there is a request to localhost:8080
const jwtConf: JwtModuleOptions = {
config: {
tokenGetter: getToken,
whitelistedDomains: ['localhost:8080']
}
};

"Parameter client_assertion_type is missing" in keycloak

I am trying out get the access token from the super user so that I can the same to create new users in key cloak, I have deployed keycloak in wildfly and when I try to do the get call, I am getting Invalid user credentials as response,
How to know the actual credentials?
And when I try to update the password from the console, I getting the error message like below.
Since I am new to this and din't find enough information from internet also, any kind of help will be appreciated .
Updated:
Now i am getting new error description as Parameter client_assertion_type is missing like below. What should be client_assertion_type here ?

This keycloak help page describes the most likely reason for the second error:
Q: When logging in, I get an error: *Parameter client_assertion_type is missing [invalid_client].
A: This error means your client is configured with Signed JWT token credentials, which means you have to use the --keystore parameter when logging in.
Alternatively you can disable using JWT tokens for the client in Keycloak.

For your information, the client_assertion_type would probably be urn:ietf:params:oauth:client-assertion-type:jwt-bearer. But then you'd get another error because the client_assertion is missing.
If ccp-portal is a confidential client using client authentication with signed JWT then the Keycloak doc states that
During authentication, the client generates a JWT token and signs it
with its private key and sends it to Keycloak in the particular
backchannel request (for example, code-to-token request) in the
client_assertion parameter.
I guess it's not possible to generate a JWT with PostMan.
This is meant for backchannel client-keycloak communication, not for
user authentication.
Solutions
You can use the admin-cli as client_id instead of your ccp-portal client. The admin-cli should be in the list of clients configured for your ccp realm. You can see that from the Keycloak interface.
Another option is allow direct access grants in ccp-portal client config.
Finally you could use ccp-portal client in your application configured with one of the Keycloak client adapters, instead of POSTMan.
As subrob sugrobych mentionned, parameters should be passed as form-data.

first of all, when you are posting data to keycloak over a rest client, you need to input parameters as form paramaters, and not as query parameters. This is why you are getting this strange error of not providing parameter grant_type, when you obviously are providing it. Same is valid for accessing keycloak api via code.
Next thing you need to think about are roles for your superuser. You can assign realm roles and client roles. There is a client named 'realm-management' which contains roles which would normally count as "system roles". You will need to use them. When you are getting HTTP code 403, it means, that probably your user is missing a role from this client.

Upload videos to Youtube from my web server in Java

My goal is to upload videos that are uploaded to my web server to Youtube on my own channel, not the users' Youtube account (my web server is acting as a proxy).
I found the sample code for uploading video to Youtube here with the credential acquired this way. The problem that I have with this sample is that it writes to disk the credential, and it opens an http server. Since my web server can potentially have a lot of users uploading their videos concurrently, the credential file location has to be dynamic, and multiple binding to the same http port is not possible. Further more, after searching through other writing about uploading to Youtube, I think this approach is for users uploading to their Youtube account.
Could you share your experiences/code sample/solutions for my scenario? In short I am just trying to automate the process of me opening up Youtube dashboard, and uploading videos to a channel in my Youtube.

In general, starting at API V3, Google prefers OAuth2 over other mechanism, and uploading a video (or any other action that modifies user data) requires OAuth2.
Fortunately, there is a special kind of token called refresh token to the rescue. Refresh token does not expire like normal access token, and is used to generate normal access token when needed. So, I divided my application into 2 parts:
The 1st part is for generating refresh token, which is a Java desktop app, meant to be run by a user on a computer. See here for sample code from Google.
The 2nd part is is part of my web application, which uses a given refresh token to create a credential object.
Here is my implementation in Scala, which you can adapt to Java version easily:
For generating a refresh token, you should set the accessType to offline for the authorization flow. Note: if a token already exists on your system, it won't try to get new token, even if it does not have refresh token, so you also have to set approval prompt to force:
def authorize(dataStoreName: String, clientId: String, clientSecret: String): Credential = {
val builder = new GoogleAuthorizationCodeFlow.Builder(
HTTP_TRANSPORT,
JSON_FACTORY,
clientId,
clientSecret,
Seq(YouTubeScopes.YOUTUBE_UPLOAD)
)
val CREDENTIAL_DIRECTORY = s"${System.getProperty("user.home")}/.oauth-credentials"
val fileDataStoreFactory = new FileDataStoreFactory(new java.io.File(CREDENTIAL_DIRECTORY))
val dataStore: DataStore[StoredCredential] = fileDataStoreFactory.getDataStore(dataStoreName)
builder.setCredentialDataStore(dataStore).setAccessType("offline").setApprovalPrompt("force")
val flow = builder.build()
val localReceiver = new LocalServerReceiver.Builder().setPort(8000).build()
new AuthorizationCodeInstalledApp(flow, localReceiver).authorize("user")
}
val credential = authorize(dataStore, clientId, clientSecret)
val refreshToken = credential.getRefreshToken
For using the refresh token on the server, you can build a credential from a refresh token:
def getCredential = new GoogleCredential.Builder()
.setJsonFactory(JSON_FACTORY)
.setTransport(HTTP_TRANSPORT)
.setClientSecrets(clientId, clientSecret)
.build()
.setRefreshToken(refreshToken)

I have have bypassed the whole AuthorizationCodeInstalledApp authorize() method and created a new subclass which bypasses the jetty server implementation process.
The methods are as follows
getAuthorizationFromStorage : Get access token from stored credentials.
getAuthorizationFromGoogle : Get the authentication with the credentials from Google creates the url that will lead the user to the authentication page and creating a custom defined name-value pair in the state parameter. The value should be encoded with base64 encoder so we can receive the same code redirected from google after authentication.
saveAuthorizationFromGoogle : Save the credentials that we get from google.
Create the GoogleAuthorizationCodeFlow object from the credentialDatastorfrom the response received from the google after authentication.
Hit google to get the permanent refresh-token that can be used to get the accesstoken of the user any time .
Store the tokens like accesstoken and refreshtoken in the filename as userid
Checkout the code Implementation here