How the google app engine blobstore service working with file upload of 32 MB without any TimeOut Exception. Also it will not affect app engine's request-response interval of one minute.What kind of process is going behind it.
When using the blobstore to upload data you upload the data into a special system handler that will load the data save it in the blobstore and when finished redirect to the user handler while providing a reference to to the location in the blobstore of the uploaded data.
Related
I'm trying to implement App Engine app that should upload files from user computer to Team Drive folder.
Currently I'm trying to do it with .jsp and servlets but it seems that it couldn't be done that way.
I'm using Service Account and Drive API Client Library for Java - https://developers.google.com/api-client-library/java/apis/drive/v3
It seems that when uploading files in such way you should use java.io.File which is impossible on App Engine, because file isn't stored on instance but it is uploaded from user computer.
Is it any way to upload file directly to Google Drive using App Engine?
Additional note:
I'm using Drive API v3.
In Drive API v2 there was file.insert() method available which lets you sending InputStream as input parameter. I think such method could help me with upload but it is not included in API v3.
https://developers.google.com/drive/api/v2/reference/files/insert
I think you're looking for protected Create(File content, com.google.api.client.http.AbstractInputStreamContent mediaContent) from https://developers.google.com/resources/api-libraries/documentation/drive/v3/java/latest/
The BlobStore API is marked as 'superseded' also limited to Limits to 32 MB.
The Google Cloud Storage is a vendor lock-in.
Is there a way to upload blobs with a 3rd part lib
In Google App Engine (not flexible / managed-vms) for example JClouds
And how would one bypass the 60 Seconds request limit that causes DeadlineExceededException?
To enhance the question;
Security is an issue, it would be preferably to run every request trough the application, so also blob uploads. Which makes the 60 seconds an issue.
The seperate uploadUrl is an option, but i do not wish to use BlobStore or Cloud Storage, but is there a generic way to handle things like this in GAE?
32MB is not a limitation of the BlobStore, but rather request playloads that go to your GAE app. You can upload larger files to both Cloud Storage and the BlobStore by creating a temporary URL for the user to submit the file to, which does not go through your ap, but rather goes directly to the storage service. You can find documentation about that for blobstore here. I don't personally use Cloud Storage, so I don't the a documentation link handy.
You can certainly use any other service in a similar way, but I'm afraid I can't explain how other than to say "consult their documentation". I know that's not a great answer to your question, but maybe insight into how it works with Google's products will help you understand how to use a 3rd party as well.
As for the 60 second request limit: since your upload requests cannot go through your server anyway, this is a non-issue. The 60 second limit only applies to requests made directly to your app.
I'd like to upload image to S3 via CloudFront.
If you see the document about CloudFront, you can find that cloud front offers put method for uploading to cloudFront
There could be someone to ask me why i use the cloud front for uploading to S3
If you search out about that, you can find the solution
What i wanna ask is whether there is method in SDK for uploading to cloud front or not
As you know , there is method "putObejct" for uploading directly to S3 but i can't find for uploading cloud front ...
please help me..
Data can be sent through Amazon CloudFront to the back-end "origin". This is used for using a POST on web forms, to send information back to web servers. It can also be used to POST data to Amazon S3.
If you would rather use an SDK to upload data to Amazon S3, there is no benefit in sending it "via CloudFront". Instead, use the Amazon S3 APIs to upload the data directly to S3.
So, bottom line:
If you're uploading from a web page that was initially served via CloudFront, send it through CloudFront to S3
If you're calling an API, call S3 directly
If the bucket's region is far away from the uploading computer you can upload faster by enabling S3 Accelerate which uploads directly through the Amazon server located closest to you and then continues sending the file from there to the bucket's actual region at an optimal route.
Have a look here.
I design an mobile app which can upload photo on GAE.
These photos can not be changed, only displayed on other mobiles. Through my application, the picture will be sent only on GAE, and downloaded to the mobile clients. That's all!
I only use GAE for store photos, and datastore for store photo's data string (Like position, age...)
Do you think I have to go on the Cloud Storage (Because I would be locked after a while?), or I can stay on the Blobstore ?
I fear that in the future the Blobstore was deprecated..
Thanks !
Use GCS because:
Google is moving away from the blobstore
GCS offers more functionality like acl, folders, and more.
You can use filenames and a blobstore like serving url for images
You can create serving url's for non images
GCS is cheap and has a free default bucket
Blobstore vs. Cloud Storage is an ongoing debate. check out these resources:
Google Blobstore versus Google cloud storage
If you're storing an image Blob in App Engine, should you put it in the Blobstore or Google Cloud Storage?
Comparing Blobstore and Google Cloud Storage
Relative to Google itself, there no good reasons to use blobstore to store files because Cloud Storage is made to do this.
I think Google will deprecate the Blobstore soon...
Since 2 days Googling, I got to upload a file in Google Cloud Storage using java. Now I am facing troubles to download the same file from Google Cloud Storage using java.
I tried with BlobstoreService to upload a file. Can any body give me the suggestions to download from the GCS?
If you want to read a file on Google Cloud Storage from an App Engine application, you need to use the Google Cloud Storage Java Client Library, or you can read it using Blobstore API after you get the blobstore key using the function createGsBlobKey.
Using Google Cloud Storage Java Client Library in order to read/write files is fairly simple. Check out this page for more info:
https://developers.google.com/appengine/docs/java/googlecloudstorageclient/getstarted