My client code is in java. It uploads file to different server based on request. These request go between countries like "India to US", "US to UK" etc. Also, sometimes files are as large as 2 GB. My current code is ineffective as it is slow in uploading file and requires you to give 1 GB heap space even to upload just 60 MB file.
String lineEnd = "\r\n";
String twoHyphens = "--";
String boundary = "*****";
int bytesRead, bytesAvailable, bufferSize;
byte[] buffer;
int maxBufferSize = 1 * 1024 * 1024;
//System.out.println("Uploading the file");
String responseFromServer = "";
String urlString = "http://stcss.us.com:8888/codesign/UploadServlet";
try {
//------------------ CLIENT REQUEST
FileInputStream fileInputStream =
new FileInputStream(new File(exsistingFileName));
int file_size = fileInputStream.available();
if (file_size > 1000 * 1024 * 1024) {
System.out.println("File Size Error\n Max Size allowed is 500MB");
System.exit(1);
}
// open a URL connection to the Jsp
URL url = new URL(urlString);
// Open a HTTP connection to the URL
conn = (HttpURLConnection)url.openConnection();
// Allow Inputs
conn.setDoInput(true);
// Allow Outputs
conn.setDoOutput(true);
// Don't use a cached copy.
conn.setUseCaches(false);
// Use a post method.
conn.setRequestMethod("POST");
conn.setRequestProperty("Connection", "Keep-Alive");
conn.setRequestProperty("Content-Type",
"multipart/form-data;boundary=" +
boundary);
//conn.setRequestProperty("Accept", "application/octet-stream");
conn.setRequestProperty("password", password);
conn.setRequestProperty("signingParameters", signingParameters);
dos = new DataOutputStream(conn.getOutputStream());
dos.writeBytes(twoHyphens + boundary + lineEnd);
dos.writeBytes("Content-Disposition: form-data; name=\"upload\";" +
" filename=\"" + exsistingFileName + "\"" +
lineEnd);
dos.writeBytes(lineEnd);
// create a buffer of maximum size
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
buffer = new byte[bufferSize];
// read file and write it into form...
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
while (bytesRead > 0) {
// System.out.println ("file size is"+file_size+"avaiable is"+bytesAvailable);
dos.write(buffer, 0, bufferSize);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
}
// send multipart form data necesssary after file data...
dos.writeBytes(lineEnd);
dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);
// close streams
fileInputStream.close();
dos.flush();
dos.close();
} catch (MalformedURLException ex) {
System.out.println("From CLIENT CLIENT REQUEST Malformed:" + ex);
System.exit(0);
} catch (IOException ioe) {
System.out.println("From CLIENT CLIENT REQUEST:" + ioe);
System.exit(0);
}
What are the ways I can change this code to make upload faster and also get rid of heap space for considerably smaller file? Is something neeeded to be changed on server side? I am using apache-commons-fileupload on the server side.
Related
I am trying to send some multi-part form data to my server API from an Android app. Here is my code where I face parsing issue. I have three values that are posted. Two of them are text values and one is a file. Out of the three, the infill text value doesn't get parsed correctly.
What should I possibly fix in my code to be able to send all values correctly?
try
{
Log.e(Tag);
// Open a HTTP connection to the URL
HttpURLConnection conn = (HttpURLConnection)connectURL.openConnection();
// conn.setConnectTimeout();
// conn.setReadTimeout();
// Allow Inputs
conn.setDoInput(true);
// Allow Outputs
conn.setDoOutput(true);
// Don't use a cached copy.
conn.setUseCaches(false);
// Use a post method.
conn.setRequestMethod("POST");
conn.setRequestProperty("Connection", "Keep-Alive");
conn.setRequestProperty("Content-Type", "multipart/form-data;boundary="+boundary);
DataOutputStream dos = new DataOutputStream(conn.getOutputStream());
// dos.writeBytes(twoHyphens + boundary + lineEnd);
dos.writeBytes("Content-Disposition: form-data; name=\"infill\""+ lineEnd);
dos.writeBytes(lineEnd);
dos.writeInt(infill_);
dos.writeBytes(lineEnd);
dos.writeBytes("Content-Disposition: form-data; name=\"material-type\""+ lineEnd);
dos.writeBytes(lineEnd);
dos.writeInt(materialType);
dos.writeBytes(lineEnd);
dos.writeBytes(twoHyphens + boundary + lineEnd);
dos.writeBytes("Content-Disposition: form-data; name=\"uploadedfile\";filename=\"" + iFileName +"\"" + lineEnd);
dos.writeBytes(lineEnd);
// Log.e(Tag,"Headers are written");
// create a buffer of maximum size
int bytesAvailable = fileInputStream.available();
int maxBufferSize = 1024;
int bufferSize = Math.min(bytesAvailable, maxBufferSize);
byte[ ] buffer = new byte[bufferSize];
// read file and write it into form...
int bytesRead = fileInputStream.read(buffer, 0, bufferSize);
while (bytesRead > 0)
{
dos.write(buffer, 0, bufferSize);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable,maxBufferSize);
bytesRead = fileInputStream.read(buffer, 0,bufferSize);
}
dos.writeBytes(lineEnd);
dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);
// close streams
fileInputStream.close();
dos.flush();
Log.i("STL File Sent, Response code: "+String.valueOf(conn.getResponseCode()));
Log.i("STL File Sent, Response message : "+String.valueOf(conn.getResponseMessage()));
InputStream is = conn.getInputStream();
// retrieve the response from server
int ch;
StringBuffer b =new StringBuffer();
while( ( ch = is.read() ) != -1 ){ b.append( (char)ch ); }
String response=b.toString();
Log.i("STL Response + "+response);
dos.close();
// Set cost and time
getDataFromJsonResponse(response);
}
dos.writeInt(infill_);
This should be sent as text, not binary. HTML is a text protocol. No reason to use a DataOutputStream at all really.
// create a buffer of maximum size
int bytesAvailable = fileInputStream.available();
int maxBufferSize = 1024;
int bufferSize = Math.min(bytesAvailable, maxBufferSize);
byte[ ] buffer = new byte[bufferSize];
// read file and write it into form...
int bytesRead = fileInputStream.read(buffer, 0, bufferSize);
while (bytesRead > 0)
{
dos.write(buffer, 0, bufferSize);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable,maxBufferSize);
bytesRead = fileInputStream.read(buffer, 0,bufferSize);
}
You don't need all this. Just:
byte[] buffer = new byte[8192];
int count;
while ((count = fileInputStream.read(buffer)) > 0)
{
dos.write(buffer, 0, count);
}
There are few if any correct usages of available(), and this isn't one of them.
Trying to send POST variable and a FILE , file is getting stored at location but I am not able to send post data.
HttpURLConnection conn = null;
DataOutputStream dos = null;
String lineEnd = "\r\n";
String twoHyphens = "--";
String boundary = "*****";
int bytesRead, bytesAvailable, bufferSize;
byte[] buffer;
int maxBufferSize = 1 * 2024 * 2024;
File sourceFile = new File(sourceFileUri);
String appName = "basic";
// open a URL connection to the Servlet
FileInputStream fileInputStream = new FileInputStream(sourceFile);
URL url = new URL("http://X.y.z.q/upload2.php");
// Open a HTTP connection to the URL
conn = (HttpURLConnection) url.openConnection();
conn.setDoInput(true); // Allow Inputs
conn.setDoOutput(true); // Allow Outputs
conn.setUseCaches(false); // Don't use a Cached Copy
conn.setRequestMethod("POST");
conn.setRequestProperty("Connection", "Keep-Alive");
conn.setRequestProperty("ENCTYPE", "multipart/form-data");
conn.setRequestProperty("Content-Type", "multipart/form-data;boundary=" + boundary);
conn.setRequestProperty("file", fileName);
conn.setRequestProperty("description", appName); ----->> trying to send appName
String description = appName;
dos = new DataOutputStream(conn.getOutputStream());
dos.writeBytes(twoHyphens + boundary + lineEnd);
dos.writeBytes("Content-Disposition: form-data; name=\"description\";filename=\""
+ appName + "\""+ lineEnd);
dos.writeBytes(lineEnd);
dos.writeBytes(twoHyphens + boundary + lineEnd);
dos.writeBytes("Content-Disposition: form-data; name=\"file\";filename=\""
+ fileName + "\""+ lineEnd);
dos.writeBytes(lineEnd);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
buffer = new byte[bufferSize];
// read file and write it into form...
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
while (bytesRead > 0) {
dos.write(buffer, 0, bufferSize);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
}
// send multipart form data necesssary after file data...
dos.writeBytes(lineEnd);
dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);
// Responses from the server (code and message)
serverResponseCode = conn.getResponseCode();
String serverResponseMessage = conn.getResponseMessage();
AT PHP Side
$description = $_POST["description"];
it says description undefiened in logs.
What is wrong with my code
am using this code to upload an image to my web server:
public void send (View view)
{
HttpURLConnection connection = null;
DataOutputStream outputStream = null;
DataInputStream inputStream = null;
String pathToOurFile = "sdcard/yo.jpg";
String urlServer = "http://192.168.1.4/uplaod.php";
String lineEnd = "\r\n";
String twoHyphens = "--";
String boundary = "*****";
int bytesRead, bytesAvailable, bufferSize;
byte[] buffer;
int maxBufferSize = 1 * 1024 * 1024;
try {
FileInputStream fileInputStream = new FileInputStream(new File(pathToOurFile));
URL url = new URL(urlServer);
connection = (HttpURLConnection) url.openConnection();
// Allow Inputs & Outputs
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setUseCaches(false);
// Enable POST method
connection.setRequestMethod("POST");
connection.setRequestProperty("Connection", "Keep-Alive");
connection.setRequestProperty("Content-Type", "multipart/form-data;boundary=" + boundary);
outputStream = new DataOutputStream(connection.getOutputStream());
outputStream.writeBytes(twoHyphens + boundary + lineEnd);
outputStream.writeBytes("Content-Disposition: form-data; name=\"uploadedfile\";filename=\"" + pathToOurFile + "\"" + lineEnd);
outputStream.writeBytes(lineEnd);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
buffer = new byte[bufferSize];
// Read file
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
while (bytesRead > 0) {
outputStream.write(buffer, 0, bufferSize);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
}
outputStream.writeBytes(lineEnd);
outputStream.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);
// Responses from the server (code and message)
int serverResponseCode = connection.getResponseCode();
String serverResponseMessage = connection.getResponseMessage();
fileInputStream.close();
outputStream.flush();
outputStream.close();
Log.i("ODPOWIEDZ", serverResponseMessage);
Context context = getApplicationContext();
Toast.makeText(context, "its here 1 ", Toast.LENGTH_LONG).show();
} catch (Exception ex) {
Context context = getApplicationContext();
Toast.makeText(context, "its in catsh", Toast.LENGTH_LONG).show();
Log.i("WYJATEK", ex.getMessage());
}
}
and i get this message from my LogCat where is the problem here ??
59-124/system_process D/SntpClient﹕ request time failed: java.net.SocketException: Address family not supported by protocol
this is my php file
<?php
$target_path = "uploadedfile/";
$target_path = $target_path . basename( $_FILES['uploadedfile']['name']);
if(move_uploaded_file($_FILES['uploadedfile']['tmp_name'], $target_path)) {
echo "The file ". basename( $_FILES['uploadedfile']['name'])." has been uploaded";
}
else {
echo "There was an error uploading the file, please try again!";
}
?>
The SNTP error relates to the time protocol, which, incidentally has nothing to do with your app. It looks like it's just the emulator trying to fetch the latest timestamp off an ntp server. Just discard it.
The issue is still something else. You did notice that there might be a typo in the name of the php script? In your code you refer to it as uplaod.php.
Also, make sure you have enabled internet access in your manifest file:
<uses-permission android:name="android.permission.INTERNET"></uses-permission>
I am developing an Android application in which users can upload photos of venues you visit. I'm using the endpoints and have begun testing https://api.foursquare.com/v2/photos/PHOTO_ID through Apigee console. The problem I have is that if I send an image via the console returns me an error 400 problems with mime type as you can in this still image is correct, I think.
https://docs.google.com/file/d/0B9uUMZ3ZVbl_bG5rOUNhM01wb2M/edit?usp=sharing (console error)
I have also tried uploading through the application I am developing and I returned the same error. The execute method in the app is as follows:
TokenStore tokenStore=new TokenStore(getApplicationContext());
String token=tokenStore.getSavedToken();
HttpURLConnection connection = null;
DataOutputStream outputStream = null;
DataInputStream inputStream = null;
String pathToOurFile = params[0];
String urlServer = "https://api.foursquare.com/v2/photos/add?public=1&venueId=5252da418bbd79f3aaa70ae6&oauth_token="+token;
String lineEnd = "\r\n";
String twoHyphens = "--";
String boundary = "*****";
int bytesRead, bytesAvailable, bufferSize;
byte[] buffer;
int maxBufferSize = 1*1024*1024;
try {
FileInputStream fileInputStream = new FileInputStream(new File(pathToOurFile) );
URL url = new URL(urlServer);
connection = (HttpURLConnection) url.openConnection();
// Allow Inputs & Outputs
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setUseCaches(false);
// Enable POST method
connection.setRequestMethod("POST");
connection.setRequestProperty("Connection", "Keep-Alive");
connection.setRequestProperty("Content-Type", "image/jpeg");
connection.setRequestProperty("Content-Type", "multipart/form-data;boundary="+boundary);
outputStream = new DataOutputStream( connection.getOutputStream() );
outputStream.writeBytes(twoHyphens + boundary + lineEnd);
outputStream.writeBytes("Content-Disposition: form-data; name=\"uploadedfile\";filename=\"" + pathToOurFile +"\"" + lineEnd);
//outputStream.writeBytes("Content-Type: image/jpeg");
outputStream.writeBytes(lineEnd);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
buffer = new byte[bufferSize];
// Read file
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
while (bytesRead > 0)
{
outputStream.write(buffer, 0, bufferSize);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
}
outputStream.writeBytes(lineEnd);
outputStream.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);
// Responses from the server (code and message)
int serverResponseCode = connection.getResponseCode();
String serverResponseMessage = connection.getResponseMessage();
Log.i("serverResponse", "serverresponse "+ serverResponseMessage);
Log.i("serverResponseCode", "serverResponseCode "+ serverResponseCode);
fileInputStream.close();
outputStream.flush();
outputStream.close();
}
catch (Exception ex) {
//Exception handling
}
In my tests online server, upload the image class works fine so the problem must be with some kind of data that does not put right for the Foursquare API. Anyone have experience with this? thanks
I my app I am trying to send an image over a PHP server. The problem is the PHP programmers say that it should be sent as an data. Following is the code which I am currently using, please help me to convert the image to a data and get a response from the server.
InputStream is;
private int serverResponseCode;
private String serverResponseMessage;
#Override
public void onCreate(Bundle icicle)
{
super.onCreate(icicle);
setContentView(R.layout.main);
HttpURLConnection connection = null;
DataOutputStream outputStream = null;
DataInputStream inputStream = null;
String pathToOurFile = "/sdcard/siva.PNG";
Log.e("pathToOurFile",""+pathToOurFile);
String urlServer = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/upload.php";
String lineEnd = "\r\n";
String twoHyphens = "--";
String boundary = "*****";
Log.e("URL Server",""+urlServer);
int bytesRead, bytesAvailable, bufferSize;
byte[] buffer;
int maxBufferSize = 1*1024*1024;
Log.e("maxBufferSize",""+maxBufferSize);
try
{
FileInputStream fileInputStream = new FileInputStream(new File(pathToOurFile) );
Log.e("FIS",""+fileInputStream);
URL url = new URL(urlServer);
connection = (HttpURLConnection) url.openConnection();
// Allow Inputs & Outputs
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setUseCaches(false);
// Enable POST method
connection.setRequestMethod("POST");
connection.setRequestProperty("Connection", "Keep-Alive");
connection.setRequestProperty("Content-Type", "multipart/form-data;boundary="+boundary);
outputStream = new DataOutputStream( connection.getOutputStream() );
outputStream.writeBytes(twoHyphens + boundary + lineEnd);
outputStream.writeBytes("Content-Disposition: form-data; name=\"uploadedfile\";filename=\"" + pathToOurFile +"\"" + lineEnd);
outputStream.writeBytes(lineEnd);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
buffer = new byte[bufferSize];
// Read file
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
while (bytesRead > 0)
{
outputStream.write(buffer, 0, bufferSize);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
}
outputStream.writeBytes(lineEnd);
outputStream.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);
// Responses from the server (code and message)
Log.e("con",String.valueOf(connection.getDoOutput()));
serverResponseCode = connection.getResponseCode();
serverResponseMessage = connection.getResponseMessage();
Log.e("response",""+serverResponseCode);
Log.e("serverResponseMessage",""+serverResponseMessage);
fileInputStream.close();
outputStream.flush();
outputStream.close();
}
catch (Exception ex)
{
Log.e("Exception Handling",""+ex);
}
}
Atlast i found the answer, i did a small mistake....In the above code i have changed only one word in the following line
outputStream.writeBytes("Content-Disposition: form-data; name=\"uploadedfile\";filename=\"" + pathToOurFile +"\"" + lineEnd);
In the above line the main part is the word "uploadedfile". This word must be specified by the php programmer or else the file which we are sending will not be replaced.
Please refer the here