Firestore import/export per API - java

I'm searching for a way to call the firestore import/export functionality programmatically from java code.
What i found so far is that the nice firestore client library does not yet support the import/export calls. But the more low level rest/grpc api already supports them. Using the java library i tried the following:
Firestore firestoreApi = new Firestore
.Builder(UrlFetchTransport.getDefaultInstance(), new GsonFactory(), null)
.setApplicationName(SystemProperty.applicationId.get())
.build();
GoogleFirestoreAdminV1beta2ImportDocumentsRequest importRequest = new GoogleFirestoreAdminV1beta2ImportDocumentsRequest();
importRequest.setInputUriPrefix(String.format("gs://{}/{}/", BUCKET, image));
GoogleLongrunningOperation operation = firestoreApi
.projects()
.databases()
.importDocuments("projects/" + SystemProperty.applicationId.get() + "/databases/(default)", importRequest)
.execute();
Which sadly ends with missing permissions when run in app engine:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 401
{
"code": 401,
"errors": [
{
"domain": "global",
"location": "Authorization",
"locationType": "header",
"message": "Login Required.",
"reason": "required"
}
],
"message": "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
"status": "UNAUTHENTICATED"
I cannot get the official way to login to work, because the firestore builder does not have a method to accept a instance of AppEngineCredentials.
I already checked the python client library which also seems not support these methods (yet).
Does anyone have a idea how i can either login with the old rest api or get a client library which supports these methods (some language which runs on app engine please :) )
Thanks for reading!
Carsten

You can adapt this Cloud Datastore example for Cloud Firestore. See how they get an access token here:
import com.google.appengine.api.appidentity.AppIdentityService;
import com.google.appengine.api.appidentity.AppIdentityServiceFactory;
// Get an access token to authorize export request
ArrayList<String> scopes = new ArrayList<String>();
scopes.add("https://www.googleapis.com/auth/datastore");
final AppIdentityService appIdentity = AppIdentityServiceFactory.getAppIdentityService();
final AppIdentityService.GetAccessTokenResult accessToken =
AppIdentityServiceFactory.getAppIdentityService().getAccessToken(scopes);
connection.addRequestProperty("Authorization", "Bearer " + accessToken.getAccessToken());

Related

Bad request (invalid value) when trying to retrieve in-app purchase info using the Java Google Play Developer API v3 with a valid purchase token

When using the Java Google Play Developer API (version 3), and requesting purchase information for a valid purchase token, I get the below exception. The API call returns a 400 Bad Request response with the following message.
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Invalid Value",
"reason" : "invalid"
} ],
"message" : "Invalid Value"
}
Unfortunately there is no further information. I do not know what value is invalid.
First, we have contacted Google support about this. Unfortunately we have been going back and forth with Google support for almost 2 months and they still haven't even indicated they correctly understand what our problem is. Every time they reply, they give us suggestions that have literally nothing to do with the Google Play Developer API, even after thoroughly explaining the problem multiple times. This is the most frustrating support experience I've had in 20 years. Google support is clearly not going to be of any help here.
Second, we have successfully linked our Google Play account and our back-end. All necessary configuration is complete and working. On our back-end we are getting notifications from Google Play whenever an Android in-app purchase is made, and these notifications include a purchase token. This took some substantial effort, but I am confident this is now all correct, our service account is configured correctly, and all permissions and such are good.
I am able to successfully call the API that returns a list of available in-app-purchase products. I get a full and correct listing of the products we have configured in Google Play for users to purchase in our app. In order to make this call, we must provide credentials, and I assume the server is validating these credentials when we make this call. Since I can successfully make this call, I am assuming the "Invalid Value" is not related to our credentials. We are using a Google-generated JSON file created for a specific service account (not using a p12 file).
I have searched StackOverflow, and the rest of the internet looking for information on this issue. I have found many pages indicating the same or similar problem, but using an older version of the Google Play Developer API, which apparently did things differently. The solutions offered do not seem to apply to the v3 API. I have been searching for many weeks and trying various solutions I've come across all with no success.
I have written a test app that I run on my local PC which works when I am requesting a listing of in-app products, but does not work when requesting the purchase details for a given purchase token. I have tried with many different purchase tokens. I read that the test tokens are not actually valid for use with this particular API, so I actually released to production a build of our app that I am able to make a real, actual purchase with. I have done this, been charged by Google, and our back-end has received real and valid purchase tokens from these purchases.
private static void testGooglePlayDevAPI_Purchase() throws Throwable {
final String productIdSku = "[the_product_sku]";
final String purchaseToken = "[a_valid_purchase_token]";
final String packageName = "[our_app_id]";
final File GOOGLE_PLAY_DEV_API_CREDENTIALS =
new File("path_to_file/google-api-credentials.json");
final FileInputStream credentials =
new FileInputStream(GOOGLE_PLAY_DEV_API_CREDENTIALS);
final JsonFactory JSON_FACTORY = JacksonFactory.getDefaultInstance();
final HttpTransport TRANSPORT =
GoogleNetHttpTransport.newTrustedTransport();
String scope = AndroidPublisherScopes.ANDROIDPUBLISHER;
GoogleCredential credential = GoogleCredential
.fromStream(credentials, TRANSPORT, JSON_FACTORY)
.createScoped(Arrays.asList(scope));
AndroidPublisher publisher = new AndroidPublisher
.Builder(TRANSPORT, JSON_FACTORY, credential)
.setApplicationName(packageName)
.build();
try {
AndroidPublisher.Purchases purchases = publisher.purchases();
AndroidPublisher.Purchases.Products products = purchases.products();
AndroidPublisher.Purchases.Products.Get get =
products.get(packageName, productIdSku, purchaseToken);
ProductPurchase product = get.execute();
System.out.println(String.format(
"Product purchase data found [developerPayload=%s]",
product.getDeveloperPayload()));
} catch (Throwable t) {
System.out.println(ExceptionSupport.getExceptionDetails(t));
}
}
I expect the above code to give me a response with valid product purchase data (a valid ProductPurchase instance that I can interrogate and process). Instead, all I ever get here is a 400 Bad Request response indicating that SOME VALUE (I don't know which) is invalid. The actual response is:
com.google.api.client.googleapis.json.GoogleJsonResponseException:
400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Invalid Value",
"reason" : "invalid"
} ],
"message" : "Invalid Value"
}
UPDATE:
I've provided my own answer to this question below.
I'll provide my own answer here because we finally found a work-around. It turns out that something in the Java Google Play Developer API (v3) client library is not working correctly. I don't know what it is, but the documentation for that client library is terrible and Google provides NO examples of using it. However, it turns out that we can successfully use the HTTPS/JSON/OAuth2 API directly without going through the Java client. We have implemented our server-side solution using this approach and caching the OAuth2 tokens in memcache until they expire.
I double-checked and we were emailing back-and-forth with Google support for 2 months!!! They were of no assistance at all. I find this to be remarkable (in a bad way).

Facebook OAuth error: Error validating verification code

I'm trying to get the access token for Facebook login, but got an error instead. All the answers that I found tell me about the wrong redirect_uri format.
This is the error I got:
{
"error": {
"message": "Error validating verification code. Please make sure your redirect_uri is identical to the one you used in the OAuth dialog request",
"type": "OAuthException",
"code": 100,
"fbtrace_id": "Ht12b5BKgRK"
}
}
And here is the redirect URI registered in Facebook App.
Steps that I do:
https://graph.facebook.com/oauth/authorize?client_id=APP_ID&redirect_uri=http://127.0.0.1:8080/MyProject/Mapping&scope=user_posts&response_type=code
This link redirect me to http://127.0.0.1:8080/MyProject/Mapping?code=GENERATED_CODE
Then in the code i generate next URI:
https://graph.facebook.com/oauth/access_token?client_id=549422435210997&redirect_uri=http://127.0.0.1:8080/MyProject/Mapping&client_secret=CLIENT_SECRET&code=GENERATED_CODE
Requesting this URI gives me error instead of access_token
I have tryied also redirect_uri=http://127.0.0.1:8080/MyProject/Mapping/. Still no results.
Also, I have tied the same steps using restFB library for Java. And got the same error.
ScopeBuilder scopeBuilder = new ScopeBuilder();
scopeBuilder.addPermission(UserDataPermissions.USER_POSTS);
FacebookClient client = new DefaultFacebookClient(Version.LATEST);
String loginDialogUrlString = client.getLoginDialogUrl(APP_ID, "http://127.0.0.1:8080/MyProject/Mapping", scopeBuilder);
System.out.println(loginDialogUrlString);
System.out.println();
AccessToken appAccessToken = client.obtainAppAccessToken(APP_ID, APP_SECRET);
System.out.println(appAccessToken.getAccessToken());
System.out.println(appAccessToken.getTokenType());
//On this step i got the same error
AccessToken userAccessToken = client.obtainUserAccessToken(APP_ID, APP_SECRET, "http://127.0.0.1:8080/MyProject/Mapping/", appAccessToken.getAccessToken());
System.out.println(userAccessToken.getAccessToken());
The one thing that bit me when I did this is to make sure that the redirect_uri is URL encoded. In Java my code did something like:
import java.net.URLEncoder;
URLEncoder.encode("http://127.0.0.1:8080/MyProject/Mapping", "UTF-8")
for my redirect URL. Don't encode the entire URL, just the portion that is your redirect_uri.

Youtube Video id by using Service account - got error 403 response

I’m developing Java code to retrieve Video Id for the corresponding asset id by using service account authentication.I have followed below mentioned steps.
Step 1: A Java program is written to retrieve the access token from Google authentication using service account.
the code is for getting access_token:
String EmailId = "XXXXXXXXXXX#developer.gserviceaccount.com";
//passing Scope
#SuppressWarnings({ "unchecked", "rawtypes" })
List<String>scops = new <String>ArrayList();
scops.add("https://www.googleapis.com/auth/youtubepartner");
final HttpTransport TRANSPORT = new NetHttpTransport();
final JsonFactory JSON_FACTORY = new JacksonFactory();
// Create a listener for automatic refresh OAuthAccessToken
List<CredentialRefreshListener> list = new ArrayList<CredentialRefreshListener>();
list.add(new CredentialRefreshListener() {
#Override
public void onTokenResponse(Credential credential,
TokenResponse tokenResponse) throws IOException {
System.out.println(tokenResponse.toPrettyString());
}
public void onTokenErrorResponse(Credential credential,
TokenErrorResponse tokenErrorResponse)
throws IOException {
System.err.println("Error: "
+ tokenErrorResponse.toPrettyString());
}
});
GoogleCredential credential = new GoogleCredential.Builder()
.setTransport(TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId(EmailId)
.setServiceAccountScopes(scops)
.setServiceAccountPrivateKeyFromP12File(new File("test.p12"))
.setRefreshListeners(list)
.build();
credential.refreshToken();
Step 2: The access token that is obtained in step 1 is passed as an argument in the below mentioned URL to get Youtube video id
Sample url is:
https://www.googleapis.com/youtube/partner/v1/claimSearch?assetId=xxxxxxxxx&onBehalfOfContentOwner=xxxxxxxx&status=active&access_token=ya29.-gCmzBHciDghrj2EDtBn1Vx0MV38pNLZTvqfwOyG0hNJCj75nsCBA5zaxmP1sr7UqI7ZrYI3AIZstA
I’m getting 403 error code in step 2.
{
"error": {
"errors": [
{
"domain": "usageLimits",
"reason": "accessNotConfigured",
"message": "Access Not Configured. The API is not enabled for your project, or there is a per-IP or per-Referer restriction configured on your API key and the request does not match these restrictions. Please use the Google Developers Console to update your configuration.",
"extendedHelp": "https://console.developers.google.com"
}
],
"code": 403,
"message": "Access Not Configured. The API is not enabled for your project, or there is a per-IP or per-Referer restriction configured on your API key and the request does not match these restrictions. Please use the Google Developers Console to update your configuration."
}
}
already enabled “GOOGLE COULD JSON API” and “YOUTUBE DATA API”.
Could any one please help me to resolve this issue?
I know from working on a previous project that you need to enable the API in the Google console. I would try that first.
https://code.google.com/apis/console
You try to use Youtube's Content ID API endpoint /youtube/partner/v1/claimSearch.
This endpoint is not part of the Youtube Data API and you can only access it if you participate in the "Youtube Partner Program". Otherwise it is not even listed in the Developer's Console.
Further information here: https://developers.google.com/youtube/partner/
Note: The YouTube Content ID API is intended for use by YouTube content partners and is not accessible to all developers or to all YouTube users. If you do not see the YouTube Content ID API as one of the services listed in the Google Developers Console, see www.youtube.com/partner to learn more about the YouTube Partner Program.

Get the Post Of User On Google Plus

I got a problem on getting the post of user on google plus. I only have the access token and refresh token of this user ( not the ID !), how could i get the post from google plus of this user ?, i do searching a lot, but none of them meets my circumstance. When the user provide his google plus Id on the front end side, the access token and refresh token were generated and stored in database, i, on the backend side, will use these tokens to retrieve the post of this user.
p/s: I use this library https://code.google.com/p/google-plus-java-api/ because it looks so simple, i don't want to use so complicated library just for getting the post, thanks you guy a lot
here is my code
import com.googlecode.googleplus.GooglePlusFactory;
import com.googlecode.googleplus.Plus;
import com.googlecode.googleplus.model.activity.ActivityCollection;
import com.googlecode.googleplus.model.activity.ActivityFeed;
public void getInfo(String accessToken, String refreshToken) {
GooglePlusFactory factory = new GooglePlusFactory(clientId, clientSecret);
Plus plus = factory.getApi(accessToken, refreshToken,null);
if (plus != null) {
ActivityFeed af = plus.getActivityOperations().list("me",ActivityCollection.PUBLIC);
String firstpost = af.getItems().get(0).getObject().getContent();
}
, i got 403 error when call this method, the error output was :
403 Forbidden
{
"code" : 403,
"errors" : [ {
"domain" : "usageLimits",
"message" : "Daily Limit Exceeded. Please sign up",
"reason" : "dailyLimitExceededUnreg",
"extendedHelp" : "https://code.google.com/apis/console"
} ],
"message" : "Daily Limit Exceeded. Please sign up"
}
Actually, i think the reason is not because daily limit exceeded, because i got that message at the first time i run, i also turn the google+ api, google+ domain api on google api console. I suppose the error come from using the "me" in the "list" method to retrieve the post. I still dont' have any idea how to fix this problem
Updated #1#:
I have change my code a lot, so here is exactly what i stuck at, i already get a new accessToken, and (when it is still not expired) , i run this code below:
GoogleCredential credential = new GoogleCredential.Builder()
.setJsonFactory(JSON_FACTORY)
.setTransport(TRANSPORT)
.setClientSecrets(CLIENT_ID, CLIENT_SECRET).build().setAccessToken(newAccessToken);
// Create a new authorized API client.
Plus service = new Plus.Builder(TRANSPORT, JSON_FACTORY, credential)
.setApplicationName(APPLICATION_NAME)
.build();
// Get a list of people that this user has shared with this app.
ActivityFeed feed = service.activities().list("me", "public").execute()
then i got this error:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
{
"code" : 403,
"errors" : [ {
"domain" : "global",
"message" : "Insufficient Permission",
"reason" : "insufficientPermissions"
} ],
"message" : "Insufficient Permission"
}
Using "me" as the ID for the list call should work correctly, and your code generally appears correct.
The error message suggests that your client is unregistered. I would check to make sure of the following:
Make sure you have created a Client ID and Secret (and don't post them here!) for your project and you're passing them to the GooglePlusFactory correctly.
Make sure this project has the Google+ API turned on.
Make sure that the access token you're providing for the user corresponds to them authenticating to this same project
I solved this problem, everything is fine except 1 thing, that's is the access token and refresh token was created by some scopes but not the scopes Plus.me, that why's i can't request a new access token by using this refresh token, and then i got the insufficient permission error 403. So when i get the new access and refresh token by using the approriated scope, i can query with keyword "me". Hope you guy don't stuck like me, it takes me nearly 2 days to figure it out

Google Drive API through Google App Engine

I'm trying to use the Google Drive API through the App Identity interface provided with Google App Engine. This basically allows my web application to communicate with Google's APIs from server to server.
I don't need my users to login, I simply need to display my own Google Drive documents.
However, after I set all the appropriate values and scopes, and enable all the right Google Drive knobs on the console page, I still get this for a simple GET request to https://www.googleapis.com/drive/v2/files:
{ "error": { "errors": [ { "domain": "usageLimits", "reason": "dailyLimitExceededUnreg", "message": "Daily Limit for Unauthenticated Use Exceeded. Continued use requires signup.", "extendedHelp": "https://code.google.com/apis/console" } ], "code": 403, "message": "Daily Limit for Unauthenticated Use Exceeded. Continued use requires signup." }}
What's wrong? What am I missing? Here's the code that actually does the request - funny thing is that it works great if I use other APIs such as the URL shortener API:
var scopes = new java.util.ArrayList();
scopes.add("https://www.googleapis.com/auth/drive");
var appIdentity = AppIdentityServiceFactory.getAppIdentityService();
var accessToken = appIdentity.getAccessToken(scopes);
var url = new URL("https://www.googleapis.com/drive/v2/files");
var connection = url.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("GET");
connection.addRequestProperty("Content-Type", "application/json");
connection.addRequestProperty("Authorization", "OAuth " + accessToken.getAccessToken());
EDIT
If I simply change the API to use the urlshortner API for example, it works:
var url = new URL("https://www.googleapis.com/urlshortener/v1/url/history");
And output:
{ "kind": "urlshortener#urlHistory", "totalItems": 0, "itemsPerPage": 30}
So there must be something not working with Google Drive and App Identity?
EDIT 2
I've found some help from the correct answer here: https://stackoverflow.com/a/12526286/50394
But it's talking about setting Client API scopes on Google Apps, and I'm not using Google Apps, I'm simply using Google App Engine's domain foo.appspot.com
The 403 error you are getting means that there was no Authorization header in your GET. The logic is that without an Authorization header, you are anonymous (you are legion blah blah :-)). The Drive quota for anonymous use is zero, hence the message. URL shortener has a higher quota for anonymous so it works.
I suggest you change the URL to point to an http server of your own, and check what headers you are actually sending.
AFAICT you should be using Bearer in the Authorization header.
Probably what's happening is, Drive API doesn't recognize the service account (because of the wrong header?) and thus taking it as an anonymous request since no key parameter wasn't provided either (see common query params).
Try this:
connection.addRequestProperty("Authorization", "Bearer " + accessToken.getAccessToken());
Or you could try adding the token as access_token query param.
I think you should at least setup an API console entry with Drive API enabled at https://code.google.com/apis/console
Once you create this you'll get an ID you can use in your GoogleCredential object. From the GoogleCredential object you can get the access token which you can than add to your request.
What I read here (Google drive via service accounts) was that you use a slightly different style that uses an API KEY that you retrieve from the Developer Console.
The pertinent parts for me were to generate a "Key for Server Applications", then use this technique, which I hadn't read anywhere else!
HttpTransport httpTransport = new NetHttpTransport();
JsonFactory jsonFactory = new JacksonFactory();
AppIdentityCredential credential =
new AppIdentityCredential.Builder(DriveScopes.DRIVE).build();
// API_KEY is from the Google Console as a server API key
GoogleClientRequestInitializer keyInitializer =
new CommonGoogleClientRequestInitializer(API_KEY);
Drive service = new Drive.Builder(httpTransport, jsonFactory, null)
.setHttpRequestInitializer(credential)
.setGoogleClientRequestInitializer(keyInitializer)
.build();
This answer claims that:
Service Accounts are not supported by the Drive SDK due to its
security model.
If that's still true, one workaround is to perform a regular OAuth dance once with a regular Google Account, and persist the access and refresh token in the datastore.

Categories