I'm trying to remove a file after a broken upload using
final FileService fileService = FileServiceFactory.getFileService();
fileService.delete(file);
But I get:
java.lang.UnsupportedOperationException: File \/blobstore\/writable:AD8BvukH[...]qau-Bb7AD does not have a finalized name
When I try to finalize the file with
FileWriteChannel writeChannel = fileService.openWriteChannel(file, true);
writeChannel.closeFinally();
then openWriteChannel() fails with
com.google.appengine.api.files.FinalizationException
[...]
Caused by: com.google.apphosting.api.ApiProxy$ApplicationException: ApplicationError: 101:
What does ApplicationError 101 mean?
How can I properly delete the file?
It looks like others have reported this problem and, although it was addressed, there could still be a problem with broken files.
Sep 11, 2013 at 1:14 am
We have now fixed this issue from reoccurring in future. However,
there are some blobs created in the past that still give errors. We
are working on a fix for these blobs.
John Lowry On behalf of the App Engine team
http://grokbase.com/t/gg/google-appengine/138xrawqw0/broken-blobstore-files-what-to-do
UnsupportedOperationException
For the first error, the documentation states:
java.lang.UnsupportedOperationException - if a file's type is not supported by delete or file does not have a finalized name.
It could be that the file is already finalized, and you can't delete it for some other reason.
ApplicationError: 101
I think the second error refers to a not found exception.
FinalizationError: ApplicationError: 101 Blobkey not found.
This may clarify the issue for you.
You only use finalize if you create a file and write to it. But you
cannot write to a file, after it has been finalized. To update a file
in the blobstore, you always have to create a new one. And when you
read a file, you do not have to finalize it. To read a file you have
to use a blobreader. See:
https://developers.google.com/appengine/docs/python/blobstore/blobreaderclass
via https://stackoverflow.com/a/12855653/1085891
Fixing the Broken Upload
You could resume the upload.
If the transfer is interrupted, you can resume the transfer from where it left off using the --db_filename=... argument.
via How to finish a broken data upload to the production Google App Engine server?
Additional Solutions / Information:
Cannot delete entity with broken id from datastore
Handle Form Failure when uploading to Appengine Blobstore
Issue 4744: Java dev server fails at deleting blobs.
Related
When does the content of the file I am retrieving actually begin downloading from S3?
AmazonS3Client.getObject()
S3Object.getContent()
S3ObjectInputStream.read()
From what I can tell, it's the first one, but I haven't found the answer explicitly in the docs
It depends on how you define "begin downloading."
Technically, it's getObject(). Otherwise it wouldn't throw the exceptions it throws, some of which necessarily require that the service has been contacted and the download initiated (or failed or denied).
Be extremely careful when using this method; the returned Amazon S3 object contains a direct stream of data from the HTTP connection.
https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/AmazonS3Client.html#getObject-com.amazonaws.services.s3.model.GetObjectRequest-
The download must have already started, for the stream to be available. Bytes are waiting... but you are sitting on a stream that you haven't read from, beyond the response headers... so the download is stalled from proceeding further until you start reading.
I'm using the Java Servlet 3.0 to upload files, using the #MultipartConfig annotation and request.getParts() to obtain the files.
When a file is uploaded, a TMP file is created in the Web Application work directory (tomcat/work/Catalina/localhost/webappname). For example:
upload_7c59101b_9f97_4e3f_9fa5_e484056d26fa_00000209.tmp
The application copies the file to another directory on the server - I'm doing this using the part.write() method but it's also working by obtaining the input stream and writing the bytes. Either way works fine.
I need to remove the TMP files after the upload, but I'm having trouble doing so. The part.delete() method doesn't do anything. I've also tried accessing the files in the directory using javax.servlet.context.tempdir and iterating over them to delete, but when calling a delete method, it always returns false. Using the Files.delete(path) method from Files.nio returns an exception which claims the file is in use by another program (i.e. locked) and therefore cannot be deleted. The server is running Windows Server 2012 R2.
Does anyone have any other solutions to remove these TMP files? It's worth pointing out that I've tried using a HttpRequestListener too, but still cannot delete the files.
Many thanks
You should (must!) not manipulate the files directly, you should use the getInputStream() method of the particular Part to get the content of the uploaded file. The servlet container (Tomcat in your case) will - or at least should - take care of the temporary files.
Along with InputStream.close(), use Part.delete() to remove the stored temporary file under work directory. Please refer the javadoc: Part.delete().
I agree with Jozef Chocholacek answer, simple solution CLOSE the input.
We were using MultiPart messages with files upload.
Since we were not closing the inputStream the files were stored there for a loooong time. They were deleted only on server restart.
After slightly changing the implementation with always closing the input part at the end.
Use try-> catch-> finally and put closing in finally part which will be
called always even when the call of method fails.
The server is not storing .tmp files anymore.
We have a Google Appengine application that
uses the Blobservice being deprecated. Google
sent us an electronic mail saying that the
service will be deleted July 28th. It also
says May 20th "no new applicaitons
will have access to the Files API."
Our application (on June Second) received
an error message from the lines in our
application:
final FileService fileservice = FileServiceFactory.getFileService();
file = fileservice.createNewBlobFile("text/xml");
com.google.apphosting.api.ApiProxy$CallNotFoundException: The API package 'file' or call 'Create()' was not found.
...
at com.google.appengine.api.files.FileServiceImpl.makeSyncCall(FileServiceImpl.java:590)
at com.google.appengine.api.files.FileServiceImpl.create(FileServiceImpl.java:512)
at com.google.appengine.api.files.FileServiceImpl.createNewBlobFile(FileServiceImpl.java:111)
at com.google.appengine.api.files.FileServiceImpl.createNewBlobFile(FileServiceImpl.java:93)
Has Google removed this capability earlier than I was expecting or
have we misunderstood what
the Exceptions are trying to tell us? I thought we had more time
to do the conversion.
(We have recently uploaded to a project created some time ago. It
appears the capability is working on uthreee.appspot.com.)
I had the same problem with a couple of applications of mine, which I solved opening a ticket to the support.
My application has been erroneously marked as "new application" (even if is very old) and because of that the Files API has been shutted down.
You need to open a support ticket providing the application ID, the eng. team will proceed to re-enable the Files API
When I am trying to process a file in Camel and it fails I try to move the file to an error directory but get the following error message:
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot delete file:....
I don't know if Windows is locking the file, but I have closed any streams that may cause this to occur on my side. If this is Windows what could I do to release this lock?
The route is a simple from()... process().. to() and there are some headers set after the process() It is the to() where this fails and the file does not move into the error directory as expected.
Thanks.
Maybe it has something to do with this:
Cannot move/delete file after processing on Windows
There is a potential issue on Windows platform with Camel 1.5.x. That
it cannot move or delete the file after processing. You should get an
exception thrown.
The workaround is to convert the body to a String after consuming
using convertBodyTo: eg
from("file://inbox").convertBodyTo(String.class).to("file://outbox");.
There should be a fix in Camel 1.6.0 to remedy this, but we would like
to get feedback on this issue. So if you are on Windows please give it
a go.
It seems there was a rogue stream that needed changing. I found this out using Process Explorer as it identified the same file twice. One lock was removed and another was not and from this I found an open stream that wasn't handled in the catch block.
I have a GAE application (Java) and I have to populate my datastore from an external file. Using localhost it's working fine. The problem is while deploying it I always get
Error: Server Error
The server encountered an error and could not complete your request.
If the problem persists, please report your problem and mention this
error message and the query that caused it.
This is my path file "war\WEB-INF\test.data"
Question: Is there is any change between local and distant access?
any help?
We need to understand what you are executing to populate the datastore from the local file. Here are some points:
Assuming that you have uploaded the file in WEB-INF folder with name test.data, your file path should be `WEB-INF/test.data'
Are you running the code to load data via some url e.g. http://yorappid.appspot.com/loaddata or something like that? If yes, chances are that your code is taking much longer to process than the 60 seconds hard limit that AppEngine places on completing HTTP Requests. So that could be the problem where you request is not getting complete.
I suggest that if point 2 above is the case, please move your code to a Cron Job. They have a limit of 10 minutes and it might sufficient to load up your data. I am not sure about the amount of data that you plan to load but 10 minutes would be more than enough to load up a sufficiently large amount of data.
Hope this helps.