I can not upload large video to server android?
i try to use this method
private void UploadLargeFile(String[] args) throws Exception {
CloseableHttpAsyncClient httpclient = HttpAsyncClients.createDefault();
try {
httpclient.start();
File upload = new File(args[0]);
File download = new File(args[1]);
ZeroCopyPost httpost = null;
try {
httpost = new ZeroCopyPost(URLTOUploadFile", upload,
ContentType.create("video/mp4"));
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
ZeroCopyConsumer<File> consumer = null;
consumer = new ZeroCopyConsumer<File>(download) {
#Override
protected File process(final HttpResponse response,
final File file, final ContentType contentType)
throws Exception {
if (response.getStatusLine().getStatusCode() != HttpStatus.SC_OK) {
throw new ClientProtocolException("Upload failed: "
+ response.getStatusLine());
}
return file;
}
};
Future<File> future = httpclient.execute(httpost, consumer, null);
File result;
result = future.get();
System.out.println("Response file length: " + result.length());
System.out.println("Shutting down");
} finally {
httpclient.close();
}
System.out.println("Done");
}
in the first line of CloseableHttpAsyncClient i got NoSuchFieldFound error
can any one help me?
Related
I use jdk11 HttpClient send msg which from file,and Send once per line。but about 40 times . 400 statusCode is appearing. i don't know why? thanks for your suggestions
Below is my source：
SendMsgHttp.java
//ignorance other source
public void sendData(String rawHex,String versionType){
String json_data = key_protocolType+versionType+ key_upLinkData+rawHex+key_reqTimestamp;
System.out.println("send data："+json_data);
HttpRequest request = httpRequestBuilder
.timeout(Duration.ofSeconds(10))
.header("Content-Type", "application/json")
.header("Authorization","Basic Y21kdDpDbUR0MTIzJA==")
.POST(HttpRequest.BodyPublishers.ofString(json_data))
.build();
try {
HttpResponse response= client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.statusCode());
atomicNum.addAndGet(1);
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
ReadFileAndSendThread.java
public class ReadFileAndSendThread implements Runnable{
private SendMsgHttp sendMsgHttp ;
private File file;
private CountDownLatch latch;
public ReadFileAndSendThread( File file,SendMsgHttp sendMsgHttp,CountDownLatch latch){
this.file = file;
this.sendMsgHttp = sendMsgHttp;
this.latch = latch;
}
#Override
public void run() {
try {
readSingleHandle();
latch.countDown();
} catch (IOException e) {
System.err.println("go on");
e.printStackTrace();
}
}
public void readSingleHandle() throws IOException {
InputStreamReader isr=new InputStreamReader(new FileInputStream(file), "UTF-8");
BufferedReader br = new BufferedReader(isr);
StandardOpenOption.READ);
String tempString;
while ((tempString = br.readLine()) != null) {
try {
String[] strData = tempString.split("\\|");
String rawData = strData[3].trim();
String versionType = strData[2].trim();
if(null !=rawData&&!rawData.equals("")
&& null !=versionType&&!versionType.equals("")){
sendMsgHttp.sendData(rawData,versionType);
Thread.sleep(1000);
}
}catch (Exception e){
e.printStackTrace();
}
}
br.close();
}
}
main SendMain.java
public class SendMain {
public static void main(String[] args) {
ExecutorService threadPool = new ThreadPoolExecutor(1, 25,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue());
String path = new File("").getAbsolutePath()+"/export";//"/root/xzg/send/exportdata";
File[] fs = new File(path).listFiles();
final CountDownLatch latch = new CountDownLatch(fs.length);
for(File file:fs){
threadPool.execute(new ReadFileAndSendThread(file,
new SendMsgHttp("http://localhost:9997/xxx")
,latch));
}
threadPool.shutdown();
try {
latch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("count nums："+SendMsgHttp.getSendCount());
}
}
and the file content like this
At last i run it. the result like below
//ok
response.statusCode:200
response.statusCode:200
response.statusCode:200
response.statusCode:200
response.statusCode:400
response.statusCode:400
response.statusCode:400
//always 400 ??? this wrong is happend
According to the result i find the first 47 data always were successful, and the rest failed.And Running many times, the results are the same
and i try sending the data which return "response.statusCode:400" and it's ok(response.statusCode:200)
what's going on? need u help . Thanks again.
I wrote a programm that can up-/download documents to sharepoint and check them in/out. It is used for data integration purposes and works quite well.
It was implemented using SOAP, but unfortunately the Server is configured to only be able to handle files with a size lesser than 50MB via SOAP.
The server configuration is fixed, so I have to work around that.
I added some code and I am able to up/download bigger files now, but If I want to check them in via SOAP I get the same error.
Now I wonder If it is possible to checkin/out files using the httpclient.
My code so far...
public class HttpClient {
private static final Logger LOGGER = LogManager.getLogger(HttpClient.class.getName());
HttpClient() {
}
public static void download(final String source, final File resultingFile) {
CloseableHttpClient client = WinHttpClients.createSystem();
HttpGet httpRequest = new HttpGet(source);
CloseableHttpResponse httpResponse = null;
try {
httpResponse = client.execute(httpRequest);
HttpEntity entity = httpResponse.getEntity();
if(httpResponse.getStatusLine() != null && httpResponse.getStatusLine().getStatusCode() != HttpStatus.SC_OK) {
LOGGER.warn(httpResponse.getStatusLine());
}else {
LOGGER.debug(httpResponse.getStatusLine());
FileUtils.touch(resultingFile);
InputStream is = entity.getContent();
File outFile = new File(resultingFile.getAbsolutePath());
FileOutputStream fos = new FileOutputStream(outFile);
int inByte;
while ((inByte = is.read()) != -1) {
fos.write(inByte);
}
is.close();
fos.close();
client.close();
}
} catch (ClientProtocolException e) {
LOGGER.warn(e);
} catch (UnsupportedOperationException e) {
LOGGER.warn(e);
} catch (IOException e) {
LOGGER.warn(e);
}
}
public static void upload(final File source, final String destination) {
CloseableHttpClient httpclient = WinHttpClients.createSystem();
HttpPut httpRequest = new HttpPut(destination);
httpRequest.setEntity(new FileEntity(new File(source.getPath())));
CloseableHttpResponse httpResponse = null;
try {
httpResponse = httpclient.execute(httpRequest);
EntityUtils.consume(httpResponse.getEntity());
if (httpResponse.getStatusLine() != null && httpResponse.getStatusLine().getStatusCode() == HttpStatus.SC_CREATED) {
LOGGER.debug(httpResponse.getStatusLine());
LOGGER.info("Upload of " + source.getName() + " via HTTP-Client succeeded.");
} else if (httpResponse.getStatusLine() != null && httpResponse.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
LOGGER.debug(httpResponse.getStatusLine());
}else {
LOGGER.warn("Uploading " + source.getName() + " failed.");
LOGGER.warn(httpResponse.getStatusLine().getStatusCode() + ": " + httpResponse.getStatusLine().getReasonPhrase());
}
} catch (IOException e) {
LOGGER.warn(e);
LOGGER.warn(e.getMessage());
}
return;
}
}
I've implemented a REST webservice for uploading a file to my server:
#Path("/upload")
#POST
#Produces(MediaType.APPLICATION_JSON)
#Consumes(MediaType.APPLICATION_XML)
public javax.ws.rs.core.Response uploadNewAdvJson(#FormDataParam("file") InputStream is) {
boolean res = true;
OutputStream out = null;
try {
File directory = new File("myFolder");
if (!directory.exists()) {
directory.mkdirs();
}
out = new FileOutputStream(new File("myFolder" + File.separator + "myFile.png"));
int read = 0;
byte[] bytes = new byte[1024];
while ((read = is.read(bytes)) != -1) {
out.write(bytes, 0, read);
}
out.flush();
out.close();
} catch (IOException e) {
e.printStackTrace();
res = false;
if (out != null) {
try {
out.close();
out.flush();
} catch (IOException e1) {
// do nothing
}
}
}
return new Response();
}
(where Response is my JAXB response Object).
I'm testing this service with this client:
public class Test {
public static void main(String[] args) {
final Client client = ClientBuilder.newBuilder().register(MultiPartFeature.class).build();
final FileDataBodyPart filePart = new FileDataBodyPart("file", new File("pathToImage/imgToUpload.png");
FormDataMultiPart formDataMultiPart = new FormDataMultiPart();
final FormDataMultiPart multipart = (FormDataMultiPart) formDataMultiPart.field("foo", "bar").bodyPart(filePart);
final WebTarget target = client.target("http://localhost:8080/myServer/rest/uploadNewAdv");
final Response response = target.request().post(Entity.entity(multipart, multipart.getMediaType()));
try {
formDataMultiPart.close();
multipart.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
But it doesn't work as i expect. In fact, a myFile.png is created and saved, but has different size than imageToUpload.png and i can't open it as an image (looks like a corrupted file).
What's wrong?
How to upload files from Android to Google App Engine Blobstore using Mobile Backend Starter or Google Cloud Endpoints?
Sharing my expirience with Mobile Backend Starter.
To obtain uploading and downloading urls you need to add this two methods to CloudBackend.java class to make urls accessible from the Activity:
public String getUploadBlobURL(String bucketName, String path, String accessMode) {
String url = null;
try {
url = getMBSEndpoint().blobEndpoint()
.getUploadUrl(bucketName, path, accessMode).execute()
.getShortLivedUrl();
} catch (IOException e) {
e.printStackTrace();
}
return url;
}
public String getDownloadBlobURL(String bucketName, String path) {
String url = null;
try {
url = getMBSEndpoint().blobEndpoint()
.getDownloadUrl(bucketName, path).execute()
.getShortLivedUrl();
} catch (IOException e) {
e.printStackTrace();
}
return url;
}
Then you can use urls to stream bytes to Google Cloud Storage with the helping of standard client libraries.
Below I'll give you examples how to use them.
For uploading file to the Google Cloud Storage you may use something similar to this:
Activity
File fileUp = new File(Environment.getExternalStorageDirectory(), fileName);
new AsyncBlobUploader(this, mProcessingFragment.getCloudBackend()).execute(fileUp);
AsyncTask
public class AsyncBlobUploader extends AsyncTask<File, Void, String> {
private Context context;
private ProgressDialog pd;
private CloudBackend cb;
public AsyncBlobUploader(Context context, CloudBackend cb) {
this.context = context;
this.cb = cb;
}
#Override
protected void onPreExecute() {
super.onPreExecute();
pd = ProgressDialog.show(context, null,
"Loading... Please wait...");
pd.setProgressStyle(ProgressDialog.STYLE_SPINNER);
pd.setIndeterminate(true);
pd.setCancelable(true);
pd.show();
}
protected String doInBackground(File... files) {
File file = files[0];
String uploadUrl = cb.getUploadBlobURL(bucketName, file.getName(),"PUBLIC_READ_FOR_APP_USERS");
String url = uploadUrl.split("&Signature")[0]; // url without Signature
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(url);
FileBody filebody = new FileBody(file,ContentType.create(getMimeType(file
.toString())), file.getName());
MultipartEntityBuilder multipartEntity = MultipartEntityBuilder.create();
multipartEntity.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
multipartEntity.addPart("file", filebody);
httppost.setEntity(multipartEntity.build());
System.out.println( "executing request " + httppost.getRequestLine( ) );
try {
HttpResponse response = httpclient.execute( httppost );
Log.i("response", response.getStatusLine().toString());
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
httpclient.getConnectionManager( ).shutdown( );
return (String) uploadUrl;
}
protected void onPostExecute(String result) {
pd.dismiss();
Log.d("BlobUrl", result);
}
public static String getMimeType(String url) {
String type = null;
String extension = MimeTypeMap.getFileExtensionFromUrl(url);
if (extension != null) {
MimeTypeMap mime = MimeTypeMap.getSingleton();
type = mime.getMimeTypeFromExtension(extension);
}
return type;
}
}
MultipartEntityBuilder class is not included into android standard libraries so you need to download httpclient and include into your project.
Pay attention to this line String url = uploadUrl.split("&Signature")[0]; where I am cutting off url signature. (With url signature I am getting 503 Internal Server Error but without it everything works as expected. I do not why this happens.)
For downloading you can use this snippet:
Activity
File fileDown = new File(Environment.getExternalStorageDirectory(),
fileName); //file to create
new AsyncBlobDownloader(imageView, mProcessingFragment.getCloudBackend())
.execute(fileDown);
AsyncTask
public class AsyncBlobDownloader extends AsyncTask<File, Integer, File> {
private ImageView imageView;
private ProgressDialog pd;
private CloudBackend cb;
public AsyncBlobDownloader(ImageView imageView, CloudBackend cb) {
this.imageView = imageView;
this.cb = cb;
}
#Override
protected void onPreExecute() {
super.onPreExecute();
pd = ProgressDialog.show(imageView.getContext(), null,
"Loading... Please wait...");
pd.setProgressStyle(ProgressDialog.STYLE_SPINNER);
pd.setCancelable(true);
pd.show();
}
protected File doInBackground(File... files) {
File file = files[0];
String downloadUrl = cb.getDownloadBlobURL(bucketName,
file.getName());
InputStream input = null;
OutputStream output = null;
HttpURLConnection connection = null;
try {
URL url = new URL(downloadUrl);
connection = (HttpURLConnection) url.openConnection();
connection.connect();
if (connection.getResponseCode() != HttpURLConnection.HTTP_OK) {
Log.i("Response",
"Server returned HTTP " + connection.getResponseCode()
+ " " + connection.getResponseMessage());
}
int fileLength = connection.getContentLength();
input = connection.getInputStream();
output = new FileOutputStream(file);
byte data[] = new byte[4096];
int count;
while ((count = input.read(data)) != -1) {
if (isCancelled()) {
input.close();
return null;
}
output.write(data, 0, count);
}
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
if (output != null)
output.close();
if (input != null)
input.close();
} catch (IOException ignored) {
}
if (connection != null)
connection.disconnect();
}
return file;
}
protected void onPostExecute(File result) {
pd.dismiss();
imageView.setImageURI(Uri.fromFile(result));
}
}
NOTE: To use Google Cloud Storage you need to enable billing. Also you need to create bucket in GCS.
When i try to asyn to upload a file use ListenableFuture,but it is not work.
if it is syn, it work ok.
ListenableFuture not support to excute a post to remote? i found a sample using URL.openStream() work ok for get.
private void asynUploadFileToRemote(final String uptoken, final String key, final File file, final PutExtra extra) {
final ListenableFuture<String> future = pool.submit(new Callable<String>() {
#Override
public String call() throws Exception {
logger.info("starting to upload file");
// here is to upload a file to remote.
PutRet putRet = IoApi.put(uptoken, key, file, extra);
logger.info("end to upload file"); //this log never excute.
return putRet.getKey();
}
});
future.addListener(new Runnable() {
#Override
public void run() {
try {
//recerve nothing here
String key = future.get();
logger.info("uploadkey:" + key);
} catch (InterruptedException e) {
logger.error("Interrupted", e);
} catch (ExecutionException e) {
logger.error("Exception in task", e.getCause());
}
}
}, MoreExecutors.sameThreadExecutor());
}
IoApi.put:Just to upload a file.
public static PutRet put(String uptoken, String key, File file,
PutExtra extra) {
if (!file.exists() || !file.canRead()) {
return new PutRet(new CallRet(400, new Exception(
"File does not exist or not readable.")));
}
if (key == null) {
key = UNDEFINED_KEY;
}
MultipartEntity requestEntity = new MultipartEntity();
try {
requestEntity.addPart("token", new StringBody(uptoken));
FileBody fileBody = new FileBody(file);
requestEntity.addPart("file", fileBody);
requestEntity.addPart("key", new StringBody(key));
if (extra.checkCrc != NO_CRC32) {
if (extra.crc32 == 0) {
return new PutRet(new CallRet(400, new Exception("no crc32 specified!")));
}
requestEntity.addPart("crc32", new StringBody(extra.crc32 + ""));
}
} catch (Exception e) {
e.printStackTrace();
return new PutRet(new CallRet(400, e));
}
String url = Config.UP_HOST;
CallRet ret = new Client().callWithMultiPart(url, requestEntity);
return new PutRet(ret);
}