We Keep Coding

Using GMAIL API without OAuth for my own account

I'm trying to create a very simple script to automate some stuff ON MY OWN GMAIL ACCOUNT
All the tutorials on google to use GMAIL API focus on accessing someones else data so it right goes through an OAuth access request in the browser, but i'm trying to read my own mails and i'm using an ubuntu server so i have no access to graphical browser.
How can I do that?
please do not answer to say "is impossible, google requires it bla bla bla security" I've done something similar in the past I just don't have the previous code...
I believe if I create a service account and grant some permission to this account i would be able to do this, but i cant find which permission i need to grant or what step i'm missing currently i get
public static Gmail createGmailServiceClient(final List<String> scopes, final String jsonCredentials) throws GeneralSecurityException, IOException {
final NetHttpTransport HTTP_TRANSPORT = GoogleNetHttpTransport.newTrustedTransport();
return new Gmail.Builder(HTTP_TRANSPORT, JSON_FACTORY, GoogleCredential.fromStream(new ByteArrayInputStream(jsonCredentials.getBytes())).createScoped(scopes)).build();
}
Gmail gmail = SimpleGmail.createGmailServiceClient(Arrays.asList(GmailScopes.GMAIL_READONLY), json);
ListMessagesResponse mails = gmail.users().messages().list("me").execute();
System.out.println(mails);
but when doing this i get
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Precondition check failed.",
"reason" : "failedPrecondition"
} ],
"message" : "Precondition check failed.",
"status" : "FAILED_PRECONDITION"
}

Precondition check failed.
Means that you are trying to use a service account but did not declare which user you would like to delegate to. What its trying to do is access the service accounts gmail account, google wont let you do that. This is normally done by using the setServiceAccountUser method.
However if i understand you correctly you are using a standard gmail account. To use service accounts with gmail you need to have a google workspace domain account. Then the admin of your domain will be able to set up domain wide delegation for you. It will only work with domain emails.
You can not use a service account with a standard google gmail account.
Here is the section of the documentation that relats to service accounts and delegation Managing Delegates
Google Workspace organizations can use the Delegates resource to manage the delegates of accounts in their organization. This requires use of a service account that has been delegated domain-wide authority.
The solution to your problem is to use oauth2.
I recommend following the official tutorial.
private static Credential getCredentials(final NetHttpTransport HTTP_TRANSPORT) throws IOException {
// Load client secrets.
InputStream in = GmailQuickstart.class.getResourceAsStream(CREDENTIALS_FILE_PATH);
if (in == null) {
throw new FileNotFoundException("Resource not found: " + CREDENTIALS_FILE_PATH);
}
GoogleClientSecrets clientSecrets = GoogleClientSecrets.load(JSON_FACTORY, new InputStreamReader(in));
// Build flow and trigger user authorization request.
GoogleAuthorizationCodeFlow flow = new GoogleAuthorizationCodeFlow.Builder(
HTTP_TRANSPORT, JSON_FACTORY, clientSecrets, SCOPES)
.setDataStoreFactory(new FileDataStoreFactory(new java.io.File(TOKENS_DIRECTORY_PATH)))
.setAccessType("offline")
.build();
LocalServerReceiver receiver = new LocalServerReceiver.Builder().setPort(8888).build();
Credential credential = new AuthorizationCodeInstalledApp(flow, receiver).authorize("user");
//returns an authorized Credential object.
return credential;
}

Using a Service Account you need to declare impersonation, are you doing that in your code? There's a similar question that was answered here that corresponds to the same error you are getting.
Basically, as instructed you need to make sure impersonation is done as:
private static GoogleCredential _createCredentialUsingServerToken(final HttpTransport httpTransport,
final JsonFactory jsonFactory) throws IOException,
GeneralSecurityException {
// Use the client ID when making the OAuth 2.0 access token request (see Google's OAuth 2.0 Service Account documentation).
String serviceAccountClientID = "327116756300-thcjqf1mvrn0geefnu6ef3pe2sm61i2q.apps.googleusercontent.com";
// Use the email address when granting the service account access to supported Google APIs
String serviceAccountUserEmail = "xxxx#developer.gserviceaccount.com";
GoogleCredential credential = new GoogleCredential.Builder()
.setTransport(httpTransport)
.setJsonFactory(jsonFactory)
.setServiceAccountId(serviceAccountUserEmail) // requesting the token
.setServiceAccountPrivateKeyFromP12File(new File(SERVER_P12_SECRET_PATH))
.setServiceAccountScopes(SCOPES) // see https://developers.google.com/gmail/api/auth/scopes
.setServiceAccountUser("user#example.com")
.build();
credential.refreshToken();
return credential;}

getting "401 Unauthorized" while trying to authenticated with service account

So here is the problem: Ive recently made this post.
Solution I've mentionned worked for one token and one API but when I tried to handle two APIs with two token (gmail and Sheets API) it failed.
So what I'm trying to do now is make the two work so I told myself "Hey let's create a service account". Even if I don't really understand the differences between both methods. Service account seems to prevent from having a consent screen (Am I right?).
I've crawled the web for answers but all of them seems to fail.
I've refreshed token, used GoogleCredential instead of Credential, created new key etc... one thing though I didn't tried is to use Gsuite account I'm using a basic account.
So now I'm at the point where I've created a new p12 file and instantly I get the 401 error. I will share my code for a better understanding.
my mail class
public class mailService {
private static final String APPLICATION_NAME = "AHS";
private static final JsonFactory JSON_FACTORY = JacksonFactory.getDefaultInstance();
private static final String TOKENS_DIRECTORY_PATH = "tokens";
//I've added sheet scope as it is activated in my project
private static final Collection<String> SCOPES = Arrays.asList(GmailScopes.GMAIL_SEND, GmailScopes.GMAIL_LABELS, SheetsScopes.SPREADSHEETS);
private static Credential getCredentials(final NetHttpTransport HTTP_TRANSPORT) throws IOException, GeneralSecurityException {
File sa = new File("WEB-INF/mykeyfile.p12");
Credential credential = new GoogleCredential.Builder()
.setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId(
"myapp#appspot.gserviceaccount.com")
.setServiceAccountScopes(SCOPES)
.setServiceAccountPrivateKeyFromP12File(sa)
.setServiceAccountUser("myemailadress#gmail.com")
.build();
//credential.refreshToken();
return (credential);
}
public static Gmail getService() throws GeneralSecurityException, IOException {
final NetHttpTransport HTTP_TRANSPORT = GoogleNetHttpTransport.newTrustedTransport();
/*Gmail service = new Gmail.Builder(HTTP_TRANSPORT, JSON_FACTORY, getCredentials(HTTP_TRANSPORT))
.setApplicationName(APPLICATION_NAME)
.build();*/
Gmail service = new Gmail.Builder(HTTP_TRANSPORT, JSON_FACTORY, null)
.setHttpRequestInitializer(getCredentials(HTTP_TRANSPORT)).setApplicationName(APPLICATION_NAME).build();
return (service);
}
...
So to give you a better understanding, I'm creating a web app using Angular and Google app engine. I wan't to use Gmail API to send mail from my account, also I'm using sheets API to read/write from/to a spreadsheet. Just to be clear I have a secret file for the google-sign-in (for the user of the web app) but there this is server side code and I don't wan't user to see a consent screen.
I'm also asking myself if I need to use gcloud in order to activate service account.
I'm running (for the moment) my server locally using Eclipse and google app engine plugin.
if you need other code or precisions for better understanding of the problem let me know

Gmail dosent support service accounts unless its a gsuite account and you set up domain wide deligation.
If you check the documentation you will only see information about using Oauth2 not server account this is because Google only documents things that are supported not those that aren't.
Sheets does support service accounts just remembered that you need to pre-authorization on the service a account. That is done via sharing the sheet with the service account like you would any other user using the service accounts email address.

How do I search for a file/open/create a googlesheet in another users folder using a service account?

Here is the scenario. I have an account where I keep test result spreadsheets. I have a service account that I use to programatically (with java and google drive apiv3) to add/update those spreadsheets. I need a way to create a spreadsheet and to search for a spreadsheet in that users account and folder from the service account. I have the following code which gets a list of files, however it's the files inside the service accounts folder, not the user accounts folder. Any help?
private static boolean findSpreadsheetFile(String sSpreadsheetName)
throws GeneralSecurityException, IOException, URISyntaxException
{
String pageToken = null;
// Build a new authorized API client service.
Drive driveService = getDriveService();
// Print the names and IDs
do
{
FileList result = driveService.files().list()
.setPageToken(pageToken)
.setFields("nextPageToken, files(id, name)")
.execute();
for (com.google.api.services.drive.model.File file : result.getFiles())
{
System.out.printf("Found file: %s (%s)\n", file.getName(), file.getId());
}
pageToken = result.getNextPageToken();
}
while (pageToken != null);
return true;
}
public static Drive getDriveService() throws GeneralSecurityException, IOException, URISyntaxException
{
Credential credential = authorizeDrive();
return new Drive.Builder(HTTP_TRANSPORT, JSON_FACTORY, credential).setApplicationName(APPLICATION_NAME).build();
}
private static GoogleCredential authorizeDrive() throws GeneralSecurityException, IOException, URISyntaxException
{
JacksonFactory JSON_FACTORY = JacksonFactory.getDefaultInstance();
HttpTransport httpTransport = GoogleNetHttpTransport.newTrustedTransport();
URL fileUrl = ParseTestResultsEmails.class.getResource(sP12Filename);
GoogleCredential credential = new GoogleCredential.Builder().setTransport(httpTransport)
.setJsonFactory(JSON_FACTORY).setServiceAccountId(sServiceAccountID)
.setServiceAccountPrivateKeyFromP12File(new File(fileUrl.toURI()))
.setServiceAccountScopes(DRIVE_SCOPES).build();
return credential;
}

Based from this documentation, domain administrators can use service accounts with OAuth 2.0 to delegate authority this way.
In enterprise applications you may want to programmatically access users data without any manual authorization on their part. In G Suite domains, the domain administrator can grant to third party applications domain-wide access to its users' data — this is referred as domain-wide delegation of authority.
Warning: Service accounts should only be used for performing delegation where the effective identity is that of an individual user in a domain. Using the service account as a common owner to create many shared documents can have severe performance implications. Additionally, service accounts may not acquire additional storage quota, nor do they act as members of a domain.
You may see the Using OAuth 2.0 for Server to Server Applications documentation for more information.

Seems as if there is no way to do what I wanted in Google Drive using the API. What I ended up doing, was creating the sheet as the "Service Account", then transferring ownership to the user I wanted, then could share it to the groups that needed access. That worked great!

Status code 403 when trying to access DoubleClick Bid Manager Partner specific private buckets from google cloud java project

Hi all,
The current situation is as follows:
I currently have a google cloud project. The account that I log into the google cloud project with can also log into a DoubleClick bid Manager account. My aim is to use the DoubleClick Bid Manager api to retrieve certain buckets stored by DBM and save them in my separate google cloud project.
So far i can access the public buckets (gdbm-public) and pull and download the data, however when I try to access the partner specific buckets the same way, i.e. (gdbm-201032-201843) I get a status code 403.
Upon Reading the documentation here, I have discovered that I need to add a google group to the DBM partner information On DBM itself. However when i try to add a google group and save the changes i get an error saying the changes cannot be saved.
This is where i authenticate:
return new GoogleCredential.Builder().setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId("<service_account_i_cant_show_here>")
.setServiceAccountScopes(Collections.singleton(StorageScopes.DEVSTORAGE_READ_ONLY))
.setServiceAccountPrivateKeyFromP12File(new File("secret-privatekey.p12"))
.build();
I then try to access the bucket like this:
String bucketName = "gdbm-201032-201843";
GoogleCredential credentials = getCredentials();
Storage storage = new Storage(HTTP_TRANSPORT, JSON_FACTORY, credentials);
Storage.Objects.List listObjects = storage.objects().list(bucketName);
Objects objects;
do {
objects = listObjects.execute();
for (StorageObject object : objects.getItems()) {
System.out.println(object);
}
listObjects.setPageToken(objects.getNextPageToken());
} while (null != objects.getNextPageToken());
More specifically, listObjects.execute() is where the 403 is thrown.
The areas I am trying to edit are Log Read Google Group and Log Management Google Group in the partner itself.
Any help greatly appreciated, thanks!

I think i have a solution, I used a different means of authentication as found here.
Using the linked class i entered in my google cloud project client credentials and the user that i log into both the google cloud project, and DBM.
I also changed the scope to "https://www.googleapis.com/auth/cloud-platform" however i am not 100% sure this had an effect on the outcome.
Code example:
// Scopes for the OAuth token
private static final Set<String> SCOPES =
ImmutableSet.of("https://www.googleapis.com/auth/cloud-platform");
public static Credential getUserCredential() throws Exception {
GoogleClientSecrets clientSecrets = GoogleClientSecrets.load(
JSON_FACTORY,
new InputStreamReader(SecurityUtilities.class.
getResourceAsStream("/client_secret.json")));
dataStoreFactory = new FileDataStoreFactory(DATA_STORE_DIR);
// set up authorization code flow.
GoogleAuthorizationCodeFlow flow = new GoogleAuthorizationCodeFlow.Builder(
HTTP_TRANSPORT, JSON_FACTORY, clientSecrets, SCOPES)
.setDataStoreFactory(dataStoreFactory)
.build();
// authorize and get credentials.
return new AuthorizationCodeInstalledApp(flow, new LocalServerReceiver())
.authorize("<Personal user account>");
So instead of using a service account i used a personal account.

Google Admin SDK User access API using Java

I am trying to fetch and update the Users of my domain using Google Admin API.
private static final JsonFactory JSON_FACTORY = JacksonFactory.getDefaultInstance();
private static final List<String> SCOPES = Arrays.asList(
"https://www.googleapis.com/auth/admin.directory.user",
"https://www.googleapis.com/auth/admin.directory.user.readonly");
public static void main(String[] args) {
try {
HttpTransport httpTransport = new NetHttpTransport();
GoogleCredential credential =
new GoogleCredential.Builder()
.setTransport(httpTransport)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId(
"xxxxx-yyyyy#developer.gserviceaccount.com")
.setServiceAccountUser("superadmin#mydomain.com")
.setServiceAccountScopes(SCOPES)
.setServiceAccountPrivateKeyFromP12File(
new File("C:\\privatekey.p12")).build();
Directory admin =
new Directory.Builder(httpTransport, JSON_FACTORY, credential)
.setApplicationName("User Sync Service")
.setHttpRequestInitializer(credential).build();
Directory.Users.List list = admin.users().list();
Users users = list.execute();
System.out.println("************");
} catch (Exception e) {
e.printStackTrace();
}
}
From my Google Console
API Access is enabled from my Security panel
ServiceAccountUser is Super Admin.
But I am still getting this error
com.google.api.client.auth.oauth2.TokenResponseException: 400 Bad Request
{
"error" : "access_denied"
}
at com.google.api.client.auth.oauth2.TokenResponseException.from(TokenResponseException.java:105)
at com.google.api.client.auth.oauth2.TokenRequest.executeUnparsed(TokenRequest.java:287)
at com.google.api.client.auth.oauth2.TokenRequest.execute(TokenRequest.java:307)
at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.executeRefreshToken(GoogleCredential.java:269)
at com.google.api.client.auth.oauth2.Credential.refreshToken(Credential.java:489)
at com.google.api.client.auth.oauth2.Credential.intercept(Credential.java:217)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:858)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:410)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:343)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:460)
Update:
Screenshot from ManageOath

It looks like the documentation is somewhat ambiguous about customer and domain. One or the other must be specified. You can set the customer attribute when using the list() function. It should be set to the customer's ID (a unique, random looking string) or, if you're authenticating as an admin already in the Google Apps instance, you can just specify exactly customer=my_customer. Alternatively, you can specify domain=example.com where example.com is a primary or secondary domain in the Google Apps instance. When specifiying a domain, only users who have a primary (home) address in that domain will be returned in the results. When specifying customer, all users in the Google Apps instance will be returned.
You can confirm this using the Google API Explorer. Leaving customer blank always results in an error. However, putting my_customer for customer attribute should fix it.

You need to specify either the domain or the customer parameter:
Directory.Users.List list = admin.users().list()
.setDomain("<target_domain>");
// or
Directory.Users.List list = admin.users().list()
.setCustomer("<target_customer_id>");
I filed a bug to update the docs to make it clear that at least one of those two parameters is required.

Proceed to https://admin.google.com/yourdomain/ManageOauthClients and check that those scopes are added for your xxxxxxxxxxxxxxxxxx.apps.googleusercontent.com service account id.