I am trying to upload an UTF-8 text file to a server in Blackberry. The upload works great but when I check the file in the server it's an ASCII file and what I need is an UTF-8 file.
This is the code that I use when I create the file:
FileConnection fc = (FileConnection)Connector.open(fileName);
if (!fc.exists()){
fc.create();
}
long byteOffset = fc.usedSize();
OutputStream outStream = fc.openOutputStream(byteOffset);
outStream.write(line.getBytes("UTF-8"));
outStream.close();
fc.close();
To send the file I use this:
public void run (){
httpConnection = null;
_connectionURL = null;
String lineEnd = "\r\n";
String twoHyphens = "--";
String boundary = "*****";
int rc = -1;
OutputStream os = null;
try {
_connectionURL = Constants.UPLOAD_URL + getConnectionString();
httpConnection = (HttpConnection)Connector.open(_connectionURL);
byte [] postDataBytes = getData();
httpConnection.setRequestMethod("POST");
httpConnection.setRequestProperty("Connection", "Keep-Alive");
httpConnection.setRequestProperty("User-Agent", "BlackBerry");
httpConnection.setRequestProperty("Content-Type", "multipart/form-data;boundary=*****");
httpConnection.setRequestProperty(HttpProtocolConstants.HEADER_CONTENT_LANGUAGE, "en-US");
httpConnection.setRequestProperty(HttpProtocolConstants.HEADER_CACHE_CONTROL,"no-cache, no-store, no-transform");
os = httpConnection.openOutputStream();
os.write((twoHyphens + boundary + lineEnd).getBytes());
os.write(("Content-Disposition: form-data; name=\"uploadedfile\";filename=\"" + fileName +"\"" + lineEnd).getBytes());
os.write(lineEnd.getBytes());
os.write(postDataBytes);
os.write(lineEnd.getBytes());
os.write((twoHyphens + boundary + twoHyphens + lineEnd).getBytes());
os.flush();
// Response
rc = httpConnection.getResponseCode();
InputStream in = httpConnection.openInputStream();
int ch;
StringBuffer stringBuffer = new StringBuffer();
while( ( ch = in.read() ) != -1 ){
stringBuffer.append( (char)ch );
}
String responseString = stringBuffer.toString();
...
}catch (IOException ioe){
...
}
}
...
private byte[] getData() throws IOException {
int _c;
StringBuffer _stringBuffer = new StringBuffer("UTF-8");
FileConnection fileForUpload = (FileConnection) Connector.open(Constants.FOLDER_FILES+this.fileName, Connector.READ);
this.fileInputStream = fileForUpload.openDataInputStream();
this.postData = new URLEncodedPostData("UTF-8", false);
while( (_c = this.fileInputStream.read()) != -1){
_stringBuffer.append((char)_c);
}
postData.setData(_stringBuffer);
byte [] _postData = postData.getBytes();
fileForUpload.close();
return _postData;
}
I guess there is something wrong in getData() method or in the httpConnection properties, but i don't know what is it.
Thanks for your help
In addition to Jon Skeet's answer.
To read byte array from file you can simply use net.rim.device.api.io.IOUtilities:
FileConnection fileForUpload =
(FileConnection) Connector.open(path, Connector.READ);
InputStream stream = fileForUpload.openInputStream();
byte[] data = IOUtilities.streamToBytes(stream);
Look at this code, which appears twice:
while( ( ch = in.read() ) != -1 ){
stringBuffer.append( (char)ch );
}
That's treating each byte as a separate character, effectively in ISO-8859-1.
If you really want to convert the content to text, you should be using an InputStreamReader with an encoding of UTF-8, then ideally reading blocks of characters (rather than one character at a time).
This isn't helping either:
byte [] _postData = postData.getBytes();
That will be using the platform default encoding to convert a string to bytes - that's almost never what you want.
Given that your getData method is trying to read a file as a byte array, you shouldn't be converting it to text at all, IMO. If you know the file length beforehand, you should just create a byte array of the right size and repeatedly call InputStream.read(byte[], int, int), noting the return value to see how far you've read. If you don't, you can repeatedly read into a smallish buffer, then write the data you've just read into a ByteArrayOutputStream which you can later get the byte array from.
Additionally, you don't appear to ever close any of your streams - which you should do in finally statements, so that the streams are closed even if an exception is thrown.
Related
I have Base64 String. I am trying to decode it, then decompress it.
String textToDecode = "H4sIAAAAAAAAAAEgAN//0JTQtdGC0LDQu9C40LfQuNGA0L7QstCw0L3QvdGL0LmRCuyiIAAAAA==\n";
byte[] data = Base64.decode(textToDecode, Base64.DEFAULT);
String result = GzipUtil.decompress(data);
Code that I am using for decompression:
public static String decompress(byte[] compressed) throws IOException {
final int BUFFER_SIZE = 32;
ByteArrayInputStream is = new ByteArrayInputStream(compressed);
GZIPInputStream gis = new GZIPInputStream(is, BUFFER_SIZE);
StringBuilder string = new StringBuilder();
byte[] data = new byte[BUFFER_SIZE];
int bytesRead;
while ((bytesRead = gis.read(data)) != -1) {
string.append(new String(data, 0, bytesRead));
}
gis.close();
is.close();
return string.toString();
}
I should get this String:
Детализированный
Insteam of it, I am getting this String with question mark symbols:
Детализирован��ый
What is my mistake? And how to solve it?
One problem is that when converting from bytes to String (internally Unicode)
the encoding is not given. And for a multi-byte encoding like UTF-8 one cannot take a fixed number of bytes (like 32) and then at the end have a valid sequence.
You experienced the loss of evidently a half sequence. Hence the encoding probably is UTF-8.
final int BUFFER_SIZE = 32;
ByteArrayInputStream is = new ByteArrayInputStream(compressed);
GZIPInputStream gis = new GZIPInputStream(is, BUFFER_SIZE);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] data = new byte[BUFFER_SIZE];
int bytesRead;
while ((bytesRead = gis.read(data)) != -1) {
baos.write(data, 0, bytesRead);
}
gis.close();
return baos.toString("UTF-8"); // Or "Windows-1251" ...
The above does away with buffer boundary problems, and specifies the encoding, so the same code runs on different computers.
And mind:
new String(bytes, encoding)
string.getBytes(encoding)
It is possible that the problem is here:
string.append(new String(data, 0, bytesRead))
You are using the default character encoding to decode bytes into a Java String. If the (current) default encoding is different to the encoding used when encoding the original characters to bytes (prior to compression, etc), then you could get bytes that don't decode correctly. The decoder will then replace them with the decoder's replacement character; i.e. '\uFFFD' by default.
If this is the problem, then the solution is to find out what the correct character encoding is and use String(byte[], int, int, Charset) to create the String.
If you work only with streams you can avoid encoding problems, this few line of code should do the job well
public static String decompress(byte[] compressed) throws IOException {
try (ByteArrayOutputStream bos = new ByteArrayOutputStream()) {
try (GZIPInputStream gis = new GZIPInputStream(
new ByteArrayInputStream(compressed))) {
org.apache.commons.compress.utils.IOUtils.copy(gis, bos);
}
return bos.toString();
}
}
There is client and server components, the client is sending the data in more secure way by converting the data in blob using POST method to the server.
Can any suggest me how to convert that blob data to string object in server side(Java).i have tried some code below
Way 1):
==============================
String streamLength = request.getHeader("Content-Length");
int streamIntLength = Integer.parseInt(streamLength);
byte[] bytes = new byte[streamIntLength];
request.getInputStream().read(bytes, 0, bytes.length);
String content = DatatypeConverter.printBase64Binary(bytes);
System.out.println(content);
Output for above code is : some junk data is displaying.
dABlAG0AcABsAGEAdABlAD0AMgAzADUAUgBfAFAAcgBvAHYAaQBkAGUAcgBfA
Way 2) :
======
BufferedReader reader = new BufferedReader(new InputStreamReader(
request.getInputStream()));
StringBuilder sb = new StringBuilder();
for (String line; (line = reader.readLine()) != null;) {
String str = new String(line.getBytes());
System.out.println(str);
}
Please suggest me any one, above both ways are not worked out.
Below code works for me.
StringBuilder stringBuilder = new StringBuilder();
BufferedReader bufferedReader = null;
try {
String streamLength = request.getHeader("Content-Length");
int streamIntLength = Integer.parseInt(streamLength);
InputStream inputStream = request.getInputStream();
if (inputStream != null) {
bufferedReader = new BufferedReader(new InputStreamReader(
inputStream));
char[] charBuffer = new char[streamIntLength];
int bytesRead = -1;
while ((bytesRead = bufferedReader.read(charBuffer)) > 0) {
stringBuilder.append(charBuffer, 0, bytesRead);
}
} else {
stringBuilder.append("");
}
} catch (IOException ex) {
throw ex;
}
String body = stringBuilder.toString();
//System.out.println(body);
byte[] bytes = body.getBytes();
System.out.println(StringUtils.newStringUtf16Le(bytes));
From the first approach, it looks like the data is encoded (possibly in Base64 format). After decoding it, what is the problem you are facing ? If the data is String and then encoded to Base64, you should get the actual string after decoding it. (Assuming platform locales on client and server side are same).
If its a binary data, better you keep it inside a byte stream only. If you anyhow want it to convert to a string, then the first approach looks okay.
If this binary data represents some kind of file, you can get the related information using the HTTP headers and write it to temp location for further use.
Im trying to upload file from blackberry app to Amazon s3.
Here is my code
private synchronized void uploadFileToAmazon(CreateFileIdBean createFileIdBean) throws UnsupportedEncodingException, IOException,ConnectionNotFoundException, ConnectionClosedException, BackupCancelledException, InterruptedException, BackupInterruptedException {
String BOUNDARY = "----------V2ymHFg03ehbqgZCaKO6jy";
String Policy = "{\"expiration\": \"2020-12-01T12:00:00.000Z\","
+ "\"conditions\": ["
+ "{\"bucket\": \"" + BeanFactory.getUserCreateBackupidBean().getBucketName() + "\"},"
+ "{\"x-amz-security-token\": \"" + BeanFactory.getUserCreateBackupidBean().getAmazonToken() + "\"},"
+ "{\"success_action_status\": \"201\"},"
+ "[\"starts-with\", \"$Content-Type\", \"\"],"
+ "[\"starts-with\", \"$key\", \"" + BeanFactory.getUserCreateBackupidBean().getBackupPath() + "\"]"
+ "]"
+ "}";
String encodePolicy = Base64.encode(Policy.getBytes());
String signature = uploadSignature(Policy, BeanFactory.getUserCreateBackupidBean().getAmazonSecret());
Hashtable params = new Hashtable();
params.put("key", BeanFactory.getUserCreateBackupidBean().getBackupPath() + "/" + BeanFactory.getUserCreateFileIdBean().getFileId());
params.put("AWSAccessKeyId", BeanFactory.getUserCreateBackupidBean().getAmazonKey());
params.put("Content-Type", createFileIdBean.getFileTypeContent());
params.put("x-amz-security-token", BeanFactory.getUserCreateBackupidBean().getAmazonToken());
params.put("policy", encodePolicy);
params.put("success_action_status", "201");
params.put("Signature", signature);
send(BOUNDARY, "http://" + BeanFactory.getUserCreateBackupidBean().getBucketName() + ".s3.amazonaws.com/", params, "file", BeanFactory.getUserCreateFileIdBean().getFileId(), createFileIdBean.getFileTypeContent(), createFileIdBean.getFileByte(), createFileIdBean);
}
private synchronized String getBoundaryMessage(String boundary, Hashtable params, String fileField, String fileName, String fileType, byte[] fileBytes, CreateFileIdBean createFileIdBean) {
StringBuffer res = new StringBuffer("--").append(boundary).append("\r\n");
Enumeration keys = params.keys();
while (keys.hasMoreElements()) {
String key = (String) keys.nextElement();
String value = (String) params.get(key);
res.append("Content-Disposition: form-data; name=\"").append(key).append("\"\r\n")
.append("\r\n").append(value).append("\r\n")
.append("--").append(boundary).append("\r\n");
}
return res.toString();
}
private synchronized void send(String boundarry, String url, Hashtable params, String fileField, String fileName, String fileType, byte[] fileBytes, CreateFileIdBean createFileIdBean) throws IOException,ConnectionClosedException,ConnectionNotFoundException, BackupCancelledException, InterruptedException, BackupInterruptedException {
StringBuffer buffer = new StringBuffer();
HttpConnection hc = null;
InputStream is = null;
InputStream inputFileDataStream = null;
DataOutputStream dout = null;
String boundary = boundarry;
StringBuffer res = new StringBuffer();
int ch;
String boundaryMessage = getBoundaryMessage(boundary, params, fileField, fileName, fileType, fileBytes, createFileIdBean);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
bos.write(boundaryMessage.getBytes());
res.append("Content-Disposition: form-data; name=\"").append(fileField).append("\"; filename=\"").append(fileName).append("\"\r\n")
.append("Content-Type: ").append(fileType).append("\r\n\r\n");
bos.write(res.toString().getBytes());
String end = "\r\n"+"--"+boundary+"\r\n"+"Content-Disposition: form-data; name=\""+"submit"+"\"\r\n"+"\r\n"+"Upload to Amazon S3"+"\r\n"+"--"+boundary+"--\r\n";
try {
hc = (HttpConnection) Connector.open(url+Resources.getConnectionString(), Connector.READ_WRITE,true);
hc.setRequestMethod(HttpConnection.POST);
hc.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundarry);
hc.setRequestProperty("User-Agent", Resources.getUserAgentString());
//hc.setRequestProperty("User-Agent", "Profile/MIDP-1.0 Confirguration/CLDC-1.0");
hc.setRequestProperty("Content-Language", "en-US");
hc.setRequestProperty("Connection", "Keep-Alive");
hc.setRequestProperty("Keep-Alive", "300");
hc.setRequestProperty("Expect", "100-continue");
hc.setRequestProperty("Content-Length", (bos.toByteArray().length+createFileIdBean.getFileSize()+end.getBytes().length)+"");
hc.setRequestProperty("Content-length", (bos.toByteArray().length+createFileIdBean.getFileSize()+end.getBytes().length)+"");
dout = new DataOutputStream(hc.openDataOutputStream());
dout.write(bos.toByteArray());
inputFileDataStream = readInputStream(createFileIdBean.getFilePath());
while ((ch = inputFileDataStream.read()) != -1) {
dout.write(ch);
}
dout.write(end.getBytes());
dout.flush();
//dout.close();
is = hc.openDataInputStream();
BeanFactory.getUserUploadFileBean().setResponseCode(hc.getResponseCode() + "");
BeanFactory.getUserUploadFileBean().setResponseMessage(hc.getResponseMessage());
while ((ch = is.read()) != -1) {
buffer.append((char) ch);
}
System.out.println("buffer"+buffer);
}
catch (IOException e) {
throw new BackupInterruptedException(Constants.ERROR_IN_UPLOAD);
} finally {
try {
if (is != null) {
is.close();
}
if (hc != null) {
hc.close();
}
if(inputFileDataStream !=null)
{
inputFileDataStream.close();
}
if(dout !=null)
{
dout.close();
}
} catch (IOException e2) {
System.out.println("aa"+e2.getMessage());
throw e2;
}
}
}
private synchronized String uploadSignature(String policy, String secretKey) throws UnsupportedEncodingException {
String encodePolciy = Base64.encode(policy.getBytes());
HMac m = new HMac(new SHA1Digest());
m.init(new KeyParameter(secretKey.getBytes("UTF-8")));
byte[] bytes = encodePolciy.getBytes("UTF-8");
m.update(bytes, 0, bytes.length);
byte[] mac = new byte[m.getMacSize()];
m.doFinal(mac, 0);
String signature = Base64.encode(mac);
return signature;
}
private synchronized InputStream readInputStream(String path) throws IOException {
FileConnection fc = null;
InputStream is = null;
fc = (FileConnection) Connector.open(path.toString(), Connector.READ);
if (!fc.exists()) {
Settings.ERROR_MESSAGE = "File doesn't exist!";
//throw new BackupInterruptedException(Settings.ERROR_MESSAGE);
} else {
is = fc.openInputStream();
}
if(fc !=null)
{
fc.close();
}
return is;
}
Its always getting connection closed exception when i trying to get response code(in the line BeanFactory.getUserUploadFileBean().setResponseCode(hc.getResponseCode() + "");) after uploading data. for files less than 2MB its working fine. please help me...
My experience with HttpConnection is that it always buffers the full payload before sending anything. For the BlackBerry Curve 8520, 16mb of data ends up making the device unusable. I worked around this by using a raw SocketConnection, and writing the HTTP parts directly in my app, so I could make sure there wasn't any excessive buffering before the bytes hit the socket.
One area I later realized might work, is using HTTP-Chunked mode with the built-in HttpConnection object. Since HTTP-Chunked is fundamentally a streaming mechanism, it may allow you to get out of the 'buffer it all' logic you get by default.
If you try to transfer large files, you might get the dreaded HTTP error 413. The max allowed file size was between 1 and 30 MB depending on the transport type (BIS,BES, etc) if my memory is correct.
In addition to this, there was a RAM quota available to an app. For older 83XX models, we empirically found that the max RAM for an app was about 12MB.
But we were successful transferring large files using chunked connections (using the HTTP Range header), and for the buffers we either used small arrays (<< 12 MB) or temporary files.
I'll post the classic links on the topic:
How to Download large files using the BlackBerry Mobile Data System
What is: HTTP 413 Request entity too large
You might find something about them in the BB forum or in other sites.
I'm having problems with downloading binary file (zip file) in my app from te internet. I have to use basic access authentication to authorize acces to file, but server response is always HTTP/1.0 400 Bad request.
String authentication = this._login+":"+this._pass;
String encoding = Base64.encodeToString(authentication.getBytes(), 0);
String fileName = "data.zip";
URL url = new URL("http://10.0.2.2/androidapp/data.zip");
HttpURLConnection ucon = (HttpURLConnection) url.openConnection();
ucon.setRequestMethod("GET");
ucon.setDoOutput(true);
ucon.setRequestProperty ("Authorization", "Basic " + encoding);
ucon.connect();
/*
* Define InputStreams to read from the URLConnection.
*/
InputStream is = ucon.getInputStream();
BufferedInputStream bis = new BufferedInputStream(is);
/*
* Read bytes to the Buffer until there is nothing more to read(-1).
*/
ByteArrayBuffer bab = new ByteArrayBuffer(50);
int current = 0;
while ((current = bis.read()) != -1) {
bab.append((byte) current);
}
bis.close();
/* Convert the Bytes read to a String. */
FileOutputStream fos = this._context.openFileOutput(fileName, this._context.MODE_WORLD_READABLE);
fos.write(bab.toByteArray());
fos.close();
Could it be caused by whitespaces in password?
I might be a bit late but I just came across a similar problem.
The problem lies in the following line:
String encoding = Base64.encodeToString(authentication.getBytes(), 0);
If you change that line to look like this it should work:
String encoding = Base64.encodeToString(authentication.getBytes(), Base64.NO_WRAP);
By default the Android Base64 util adds a newline character to the end of the encoded string. This invalidates the HTTP headers and causes the "Bad request".
The Base64.NO_WRAP flag tells the util to create the encoded string without the newline character thus keeping the HTTP headers intact.
I have the following Java code to fetch the entire contents of an HTML page at a given URL. Can this be done in a more efficient way? Any improvements are welcome.
public static String getHTML(final String url) throws IOException {
if (url == null || url.length() == 0) {
throw new IllegalArgumentException("url cannot be null or empty");
}
final HttpURLConnection conn = (HttpURLConnection) new URL(url).openConnection();
final BufferedReader buf = new BufferedReader(new InputStreamReader(conn.getInputStream()));
final StringBuilder page = new StringBuilder();
final String lineEnd = System.getProperty("line.separator");
String line;
try {
while (true) {
line = buf.readLine();
if (line == null) {
break;
}
page.append(line).append(lineEnd);
}
} finally {
buf.close();
}
return page.toString();
}
I can't help but feel that the line reading is less than optimal. I know that I'm possibly masking a MalformedURLException caused by the openConnection call, and I'm okay with that.
My function also has the side-effect of making the HTML String have the correct line terminators for the current system. This isn't a requirement.
I realize that network IO will probably dwarf the time it takes to read in the HTML, but I'd still like to know this is optimal.
On a side note: It would be awesome if StringBuilder had a constructor for an open InputStream that would simply take all the contents of the InputStream and read it into the StringBuilder.
As seen in the other answers, there are many different edge cases (HTTP peculiarities, encoding, chunking, etc) that should be accounted for in any robust solution. Therefore I propose that in anything other than a toy program you use the de facto Java standard HTTP library: Apache HTTP Components HTTP Client.
They provide many samples, "just" getting the response contents for a request looks like this:
HttpClient httpclient = new DefaultHttpClient();
HttpGet httpget = new HttpGet("http://www.google.com/");
ResponseHandler<String> responseHandler = new BasicResponseHandler();
String responseBody = httpclient.execute(httpget, responseHandler);
// responseBody now contains the contents of the page
System.out.println(responseBody);
httpclient.getConnectionManager().shutdown();
OK, edited once more. Be sure to put your try-finally blocks around it, or catch IOException
...
final static int BUFZ = 4096;
StringBuilder page = new StringBuilder();
HttpURLConnection conn =
(HttpURLConnection) new URL(url).openConnection();
InputStream is = conn.getInputStream()
// perhaps allocate this one time and reuse if you
//call this method a lot.
byte[] buf = new byte[BUFZ] ;
int nRead = 0;
while((nRead = is.read(buf, 0, BUFZ) > 0) {
page.append(new String(buf /* , Charset charset */));
// uses local default char encoding for now
}
Here try this one:
...
final static int MAX_SIZE = 10000000;
HttpURLConnection conn =
(HttpURLConnection) new URL(url).openConnection();
InputStream is = conn.getInputStream()
// perhaps allocate this one time and reuse if you
//call this method a lot.
byte[] buf = new byte[MAX_SIZE] ;
int nRead = 0;
int total = 0;
// you could also use ArrayList so that you could dynamically
// resize or there are other ways to resize an array also
while(total < MAX_SIZE && (nRead = is.read(buf) > 0) {
total += nRead;
}
...
// do something with buf array of length total
OK the code below was not working for you because the Content-length header line is not being sent in the beginning due to HTTP/1.1 "chunking"
...
HttpURLConnection conn =
(HttpURLConnection) new URL(url).openConnection();
InputStream is = conn.getInputStream()
int cLen = conn.getContentLength() ;
byte[] buf = new byte[cLen] ;
int nRead=0 ;
while(nRead < cLen) {
nRead += is.read(buf, nRead, cLen - nRead) ;
}
...
// do something with buf array
You could do your own buffering on top of InputStreamReader by reading bigger chunks into a character array and appending the array contents to the StringBuilder.
But it would make your code slightly harder to understand, and I doubt it would be worth it.
Note that the proposal by Sean A.O. Harney reads raw bytes, so you'd need to do the conversion to text on top of that.