I am writing a web server with Play framework 2.6 in Java. I want to upload a file to WebServer through a multipart form and do some validations, then upload the file s3. The default implementation in play saves the file to a temporary file in the file system but I do no want to do that, I want to upload the file straight to AWS S3.
I looked into this tutorial, which explains how to save file the permanently in file system instead of using temporary file. To my knowledge I have to make a custom Accumulator or a Sink that saves the incoming ByteString(s) to a byte array? but I cannot find how to do so, can someone point me in the correct direction?
thanks
Related
I am trying to run a query in Google Big Query and export the data to Google cloud storage using GZIP compression.
JobConfigurationExtract jobExtractConfig = new JobConfigurationExtract().setSourceTable(tableReference).set("results.csv", "CSV")
.setDestinationUri("gs://dev-app-uploads/results.zip")
.setCompression("GZIP");
By using this config i am able to generate a results.zip file successfully in cloud storage in the configured bucket dev-app-uploads. But the file inside the zip is generated without a .csv extension. When i extract the zip file, i am getting a "results" file and when i manually add the extention .csv and open the file, the contents are there.
But my necessity is to generate the file with .csv extension and zip it and place it in cloud storage.
Please let me know if this is possible or any other better options to upload data from big query using compression.
Instead of
gs://dev-app-uploads/results.zip
use below
gs://dev-app-uploads/results.csv.zip
I am working on a Java based web application.This application consumes REST API for its data via a custom made rest client which is a sub component of the application.Now I am stuck in a situation where I have to write code for downloading a large file.The rest client calls the api , obtains data stream from the Response and writes the file at a particular location at the server.
Should I read the file and write it to the output stream? Or is there any more efficient way to achieve this functionality?
Thanks in advance
Finally I am using the same approach of reading the large file and writing it to the output stream.The rest client sub component reads the file data from the rest api and writes it at a temporary location on the server.
After that I read that file and write it to the output stream.
Is it possible to upload a file using some web-service directly to HDFS space. I tried to write file in to local system and moved it to HDFS.
WebHDFS provides REST APIs to support all the filesystem operations.
Direct uploading is not possible though.
It has to follow 2 steps
Create File in the hdfs location http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=CREATE
Write to that file - by specifying your local file path tat u want to upload in the Header http://<DATANODE>:<PORT>/webhdfs/v1/<PATH>?op=CREATE
Refer APIs here WebHDFS apis
In my project a compressed file is displayed on a php in bytes.
I am trying to find a way to read the php page, decompress the file using GZIP, and write it out to the assets folder.
The file that I am reading in has to be placed in the data/data/package/database file.
I have a class that reads a file from the assets folder and places the file into data/data/package/database.
Is it possible to write to the assets folder during runtime? If not is there a better way to do this?
if the PHP script is running on a server, your Android app will need to make an HTTP request to retrieve the content. of course then it has to store it somewhere. the SQLite database is a good option, but you could also store content in files on the SD card.
there are a couple ways to do the HTTP connection part of it, and they're written up on the Android Dev Blog. Myself, I prefer the HttpURLConnection.
I have been messing around with GWT uploads lately. I want to have the ability to upload an XML file from the client, and get the contents of that file (display the contents in a TextArea).
From what I have found on the net, it seems I would have to upload the file to the server, and then get the contents of the file. I do not particularly like the idea of allowing file uploads to the server (even if they are only XML). Is there anyway to pull the contents of a file that the client specifies without sending it to the server?
Thanks
Recent (decent?) browsers implement the "HTML5" File API that's quite easy to use in GWT using JSNI.
See also: https://developer.mozilla.org/en/Using_files_from_web_applications
Because of security restrictions you cannot access the file on the client side alone. It has to be sent to the server for processing.