We Keep Coding

getting "401 Unauthorized" while trying to authenticated with service account

So here is the problem: Ive recently made this post.
Solution I've mentionned worked for one token and one API but when I tried to handle two APIs with two token (gmail and Sheets API) it failed.
So what I'm trying to do now is make the two work so I told myself "Hey let's create a service account". Even if I don't really understand the differences between both methods. Service account seems to prevent from having a consent screen (Am I right?).
I've crawled the web for answers but all of them seems to fail.
I've refreshed token, used GoogleCredential instead of Credential, created new key etc... one thing though I didn't tried is to use Gsuite account I'm using a basic account.
So now I'm at the point where I've created a new p12 file and instantly I get the 401 error. I will share my code for a better understanding.
my mail class
public class mailService {
private static final String APPLICATION_NAME = "AHS";
private static final JsonFactory JSON_FACTORY = JacksonFactory.getDefaultInstance();
private static final String TOKENS_DIRECTORY_PATH = "tokens";
//I've added sheet scope as it is activated in my project
private static final Collection<String> SCOPES = Arrays.asList(GmailScopes.GMAIL_SEND, GmailScopes.GMAIL_LABELS, SheetsScopes.SPREADSHEETS);
private static Credential getCredentials(final NetHttpTransport HTTP_TRANSPORT) throws IOException, GeneralSecurityException {
File sa = new File("WEB-INF/mykeyfile.p12");
Credential credential = new GoogleCredential.Builder()
.setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId(
"myapp#appspot.gserviceaccount.com")
.setServiceAccountScopes(SCOPES)
.setServiceAccountPrivateKeyFromP12File(sa)
.setServiceAccountUser("myemailadress#gmail.com")
.build();
//credential.refreshToken();
return (credential);
}
public static Gmail getService() throws GeneralSecurityException, IOException {
final NetHttpTransport HTTP_TRANSPORT = GoogleNetHttpTransport.newTrustedTransport();
/*Gmail service = new Gmail.Builder(HTTP_TRANSPORT, JSON_FACTORY, getCredentials(HTTP_TRANSPORT))
.setApplicationName(APPLICATION_NAME)
.build();*/
Gmail service = new Gmail.Builder(HTTP_TRANSPORT, JSON_FACTORY, null)
.setHttpRequestInitializer(getCredentials(HTTP_TRANSPORT)).setApplicationName(APPLICATION_NAME).build();
return (service);
}
...
So to give you a better understanding, I'm creating a web app using Angular and Google app engine. I wan't to use Gmail API to send mail from my account, also I'm using sheets API to read/write from/to a spreadsheet. Just to be clear I have a secret file for the google-sign-in (for the user of the web app) but there this is server side code and I don't wan't user to see a consent screen.
I'm also asking myself if I need to use gcloud in order to activate service account.
I'm running (for the moment) my server locally using Eclipse and google app engine plugin.
if you need other code or precisions for better understanding of the problem let me know

Gmail dosent support service accounts unless its a gsuite account and you set up domain wide deligation.
If you check the documentation you will only see information about using Oauth2 not server account this is because Google only documents things that are supported not those that aren't.
Sheets does support service accounts just remembered that you need to pre-authorization on the service a account. That is done via sharing the sheet with the service account like you would any other user using the service accounts email address.

Access google photos API via Java

I am very new to google API and I am having troubles with it. I red documentation Google photos API for Java, then I created OAuth credentials in google API console and downloaded it (credentials.json file).
After that I tried to access google photos. Here is code from documentation:
// Set up the Photos Library Client that interacts with the API
PhotosLibrarySettings settings =
PhotosLibrarySettings.newBuilder()
.setCredentialsProvider(
FixedCredentialsProvider.create(/* Add credentials here. */))
.build();
try (PhotosLibraryClient photosLibraryClient =
PhotosLibraryClient.initialize(settings)) {
// Create a new Album with at title
Album createdAlbum = photosLibraryClient.createAlbum("My Album");
// Get some properties from the album, such as its ID and product URL
String id = album.getId();
String url = album.getProductUrl();
} catch (ApiException e) {
// Error during album creation
}
But I don't understand how to create Credentials object to pass it to the FixedCredentialsProvider.create() method
Could you please provide me with some explanation/links about it?

You can create a UserCredentials Object and pass it
UserCredentials.newBuilder()
.setClientId("your client id")
.setClientSecret("your client secret")
.setAccessToken("Access Token")
.build()
Go through this answer
https://stackoverflow.com/a/54533855/6153171
Or check out this complete project on github
https://github.com/erickogi/AndroidGooglePhotosApi

The FixedCredentialsProvider.create(..) call takes in a com.google.auth.Credentials object. For the Google Photos Library API, this should be a UserCredentials object, that you can create UserCredentials.Builder that is part of the Google OAuth library. There you set the refresh token, client ID, client secret, etc. to initialise the credentials. Getting a refresh token requires your app to complete the
GoogleAuthorizationCodeFlow that prompts the user for authorization and approval.
You can check out the sample implementation on GitHub, but this is the relevant code:
GoogleAuthorizationCodeFlow flow =
new GoogleAuthorizationCodeFlow.Builder(
GoogleNetHttpTransport.newTrustedTransport(),
JSON_FACTORY,
clientSecrets,
selectedScopes)
.setDataStoreFactory(new FileDataStoreFactory(DATA_STORE_DIR))
.setAccessType("offline")
.build();
LocalServerReceiver receiver =
new LocalServerReceiver.Builder().setPort(LOCAL_RECEIVER_PORT).build();
Credential credential = new AuthorizationCodeInstalledApp(flow, receiver).authorize("user");
return UserCredentials.newBuilder()
.setClientId(clientId)
.setClientSecret(clientSecret)
.setRefreshToken(credential.getRefreshToken())
.build();
There are a few moving parts involved, but the Google Photos Library API client library works with the Google Authentication library to handle OAuth authentication.

You need to understand about OAuth 2 first:
https://www.digitalocean.com/community/tutorials/an-introduction-to-oauth-2
Then you can look this answer https://stackoverflow.com/a/54533855/6153171
P/s:
1. Grant Type: Authorization Code
Google developer console : credential is web client.
GoogleApiClient -> addScope, requestServerCode -> grand permission -> get account -> getServerAuthenCode -> get AccessToken (https://stackoverflow.com/a/54533855/6153171)
-> I tested and it works well. You can follow this way
Grant Type: Implicit
Google developer console : credential is android or ios app.
GoogleApiClient -> addScope, requestTokenID -> grand permission -> get account -> getTokenID -> get AccessToken. I didn't try successfully to grant authorization for google photo api. But with firebase authentication, we can use this way because firebase support class util for us.

Google authentication from server

I am need to upload video to youtube to specific youtube account. So I need to authenticate I am using java google lib. Code looks like this:
Credential credential = new GoogleCredential.Builder()
.setTransport(new ApacheHttpTransport())
.setJsonFactory(new JacksonFactory())
.setServiceAccountId("xxx#xx-app.iam.gserviceaccount.com")
.setClientSecrets("xx#gmail.com", "xx")
.setServiceAccountPrivateKeyFromP12File(new File("xx.p12"))
.setServiceAccountScopes(scopes)
.setServiceAccountUser("xx#gmail.com")
.build();
youtube = new YouTube.Builder(credential.getTransport(), credential.getJsonFactory(), credential).setApplicationName(
"tellews-app").build();
Video returnedVideo = videoInsert.execute();
YouTube.Videos.Insert videoInsert = youtube.videos()
.insert("snippet,statistics,status", videoObjectDefiningMetadata, mediaContent);
Video returnedVideo = videoInsert.execute();
And getting error:
IOException: 401 Unauthorized
{
"error" : "unauthorized_client",
"error_description" : "Client is unauthorized to retrieve access tokens using this method."
}
Maybe someone sees what I am doing wrong

Simple.
Do not use a Service Account - that's not what they're there for.
You need to obtain an Access Token for the target YouTube account. The simples way to do that is to get yourself a Refresh Token for that account from the Oauth Playground, and use it to fetch an Access Token whenever you need one. The steps to do this are enumerated at How do I authorise an app (web or installed) without user intervention? (canonical ?) . In the comments there is a link to a YouTube video which also explains the steps.

Status code 403 when trying to access DoubleClick Bid Manager Partner specific private buckets from google cloud java project

Hi all,
The current situation is as follows:
I currently have a google cloud project. The account that I log into the google cloud project with can also log into a DoubleClick bid Manager account. My aim is to use the DoubleClick Bid Manager api to retrieve certain buckets stored by DBM and save them in my separate google cloud project.
So far i can access the public buckets (gdbm-public) and pull and download the data, however when I try to access the partner specific buckets the same way, i.e. (gdbm-201032-201843) I get a status code 403.
Upon Reading the documentation here, I have discovered that I need to add a google group to the DBM partner information On DBM itself. However when i try to add a google group and save the changes i get an error saying the changes cannot be saved.
This is where i authenticate:
return new GoogleCredential.Builder().setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId("<service_account_i_cant_show_here>")
.setServiceAccountScopes(Collections.singleton(StorageScopes.DEVSTORAGE_READ_ONLY))
.setServiceAccountPrivateKeyFromP12File(new File("secret-privatekey.p12"))
.build();
I then try to access the bucket like this:
String bucketName = "gdbm-201032-201843";
GoogleCredential credentials = getCredentials();
Storage storage = new Storage(HTTP_TRANSPORT, JSON_FACTORY, credentials);
Storage.Objects.List listObjects = storage.objects().list(bucketName);
Objects objects;
do {
objects = listObjects.execute();
for (StorageObject object : objects.getItems()) {
System.out.println(object);
}
listObjects.setPageToken(objects.getNextPageToken());
} while (null != objects.getNextPageToken());
More specifically, listObjects.execute() is where the 403 is thrown.
The areas I am trying to edit are Log Read Google Group and Log Management Google Group in the partner itself.
Any help greatly appreciated, thanks!

I think i have a solution, I used a different means of authentication as found here.
Using the linked class i entered in my google cloud project client credentials and the user that i log into both the google cloud project, and DBM.
I also changed the scope to "https://www.googleapis.com/auth/cloud-platform" however i am not 100% sure this had an effect on the outcome.
Code example:
// Scopes for the OAuth token
private static final Set<String> SCOPES =
ImmutableSet.of("https://www.googleapis.com/auth/cloud-platform");
public static Credential getUserCredential() throws Exception {
GoogleClientSecrets clientSecrets = GoogleClientSecrets.load(
JSON_FACTORY,
new InputStreamReader(SecurityUtilities.class.
getResourceAsStream("/client_secret.json")));
dataStoreFactory = new FileDataStoreFactory(DATA_STORE_DIR);
// set up authorization code flow.
GoogleAuthorizationCodeFlow flow = new GoogleAuthorizationCodeFlow.Builder(
HTTP_TRANSPORT, JSON_FACTORY, clientSecrets, SCOPES)
.setDataStoreFactory(dataStoreFactory)
.build();
// authorize and get credentials.
return new AuthorizationCodeInstalledApp(flow, new LocalServerReceiver())
.authorize("<Personal user account>");
So instead of using a service account i used a personal account.

Upload videos to Youtube from my web server in Java

My goal is to upload videos that are uploaded to my web server to Youtube on my own channel, not the users' Youtube account (my web server is acting as a proxy).
I found the sample code for uploading video to Youtube here with the credential acquired this way. The problem that I have with this sample is that it writes to disk the credential, and it opens an http server. Since my web server can potentially have a lot of users uploading their videos concurrently, the credential file location has to be dynamic, and multiple binding to the same http port is not possible. Further more, after searching through other writing about uploading to Youtube, I think this approach is for users uploading to their Youtube account.
Could you share your experiences/code sample/solutions for my scenario? In short I am just trying to automate the process of me opening up Youtube dashboard, and uploading videos to a channel in my Youtube.

In general, starting at API V3, Google prefers OAuth2 over other mechanism, and uploading a video (or any other action that modifies user data) requires OAuth2.
Fortunately, there is a special kind of token called refresh token to the rescue. Refresh token does not expire like normal access token, and is used to generate normal access token when needed. So, I divided my application into 2 parts:
The 1st part is for generating refresh token, which is a Java desktop app, meant to be run by a user on a computer. See here for sample code from Google.
The 2nd part is is part of my web application, which uses a given refresh token to create a credential object.
Here is my implementation in Scala, which you can adapt to Java version easily:
For generating a refresh token, you should set the accessType to offline for the authorization flow. Note: if a token already exists on your system, it won't try to get new token, even if it does not have refresh token, so you also have to set approval prompt to force:
def authorize(dataStoreName: String, clientId: String, clientSecret: String): Credential = {
val builder = new GoogleAuthorizationCodeFlow.Builder(
HTTP_TRANSPORT,
JSON_FACTORY,
clientId,
clientSecret,
Seq(YouTubeScopes.YOUTUBE_UPLOAD)
)
val CREDENTIAL_DIRECTORY = s"${System.getProperty("user.home")}/.oauth-credentials"
val fileDataStoreFactory = new FileDataStoreFactory(new java.io.File(CREDENTIAL_DIRECTORY))
val dataStore: DataStore[StoredCredential] = fileDataStoreFactory.getDataStore(dataStoreName)
builder.setCredentialDataStore(dataStore).setAccessType("offline").setApprovalPrompt("force")
val flow = builder.build()
val localReceiver = new LocalServerReceiver.Builder().setPort(8000).build()
new AuthorizationCodeInstalledApp(flow, localReceiver).authorize("user")
}
val credential = authorize(dataStore, clientId, clientSecret)
val refreshToken = credential.getRefreshToken
For using the refresh token on the server, you can build a credential from a refresh token:
def getCredential = new GoogleCredential.Builder()
.setJsonFactory(JSON_FACTORY)
.setTransport(HTTP_TRANSPORT)
.setClientSecrets(clientId, clientSecret)
.build()
.setRefreshToken(refreshToken)

I have have bypassed the whole AuthorizationCodeInstalledApp authorize() method and created a new subclass which bypasses the jetty server implementation process.
The methods are as follows
getAuthorizationFromStorage : Get access token from stored credentials.
getAuthorizationFromGoogle : Get the authentication with the credentials from Google creates the url that will lead the user to the authentication page and creating a custom defined name-value pair in the state parameter. The value should be encoded with base64 encoder so we can receive the same code redirected from google after authentication.
saveAuthorizationFromGoogle : Save the credentials that we get from google.
Create the GoogleAuthorizationCodeFlow object from the credentialDatastorfrom the response received from the google after authentication.
Hit google to get the permanent refresh-token that can be used to get the accesstoken of the user any time .
Store the tokens like accesstoken and refreshtoken in the filename as userid
Checkout the code Implementation here