We Keep Coding

how to read a pdf file online and save on local machine using java

Hi I was trying to read a PDF file online but after reading and writing on local. after viewing the document I am getting an error that content is not supported .
URL url1 =
new URL("http://www.gnostice.com/downloads/Gnostice_PathQuest.pdf");
byte[] ba1 = new byte[1024];
int baLength;
FileOutputStream fos1 = new FileOutputStream("/mnt/linuxabc/research_paper/Gnostice_PathQuest.pdf");
try {
URLConnection urlConn = url1.openConnection();
/* if (!urlConn.getContentType().equalsIgnoreCase("application/pdf")) {
System.out.println("FAILED.\n[Sorry. This is not a PDF.]");
} else {*/
try {
InputStream is1 = url1.openStream();
while ((baLength = is1.read(ba1)) != -1) {
fos1.write(ba1, 0, baLength);
}
fos1.flush();
fos1.close();
is1.close();
} catch (ConnectException ce) {
System.out.println("FAILED.\n[" + ce.getMessage() + "]\n");
}
// }

Your Pdf Link actually redirects to https://www.gnostice.com/downloads.asp, so there is no pdf directly behind the link.
Try with another link: check first in a browser of your choice that invoking the pdf's url render a real pdf in the browser.
The code below is practically the same as yours except for the pdf's url and the output's path, and I am also adding exception throws to the main method's signature and simply printing the content type.
It works as expected:
public class PdfFileReader {
public static void main(String[] args) throws IOException {
URL pdfUrl = new URL("http://www.crdp-strasbourg.fr/je_lis_libre/livres/Anonyme_LesMilleEtUneNuits1.pdf");
byte[] ba1 = new byte[1024];
int baLength;
try (FileOutputStream fos1 = new FileOutputStream("c:\\mybook.pdf")) {
URLConnection urlConn = pdfUrl.openConnection();
System.out.println("The content type is: " + urlConn.getContentType());
try {
InputStream is1 = pdfUrl.openStream();
while ((baLength = is1.read(ba1)) != -1) {
fos1.write(ba1, 0, baLength);
}
fos1.flush();
fos1.close();
is1.close();
} catch (ConnectException ce) {
System.out.println("FAILED.\n[" + ce.getMessage() + "]\n");
}
}
}
}
Output:
The content type is: application/pdf

private static String readPdf() throws MalformedURLException, IOException {
URL url = new URL("https://colaboracion.dnp.gov.co/CDT/Sinergia/Documentos/Informe%20al%20Congreso%20Presidencia%202017_Baja_f.pdf");
BufferedReader read = new BufferedReader(
new InputStreamReader(url.openStream()));
String i;
StringBuilder stringBuilder = new StringBuilder();
while ((i = read.readLine()) != null) {
stringBuilder.append(i);
}
read.close();
return stringBuilder.toString();
}

Android - Size in chars of an http response

I am not an pro developing android. I wanted to download a JSON object from my server, but only code I could find was this:
private String downloadUrl(String myurl) throws IOException {
InputStream is = null;
// Only display the first 500 characters of the retrieved
// web page content.
int len = 500;
try {
URL url = new URL(myurl);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setReadTimeout(10000 /* milliseconds */);
conn.setConnectTimeout(15000 /* milliseconds */);
conn.setRequestMethod("GET");
conn.setDoInput(true);
// Starts the query
conn.connect();
int response = conn.getResponseCode();
Log.d("ServerConnection", "The response is: " + response);
is = conn.getInputStream();;
//is.
// Convert the InputStream into a string
String contentAsString = readIt(is, len);
return contentAsString;
// Makes sure that the InputStream is closed after the app is
// finished using it.
} catch (MalformedURLException e) {
//
return "error";
} catch (IOException e) {
//
return "error";
} finally {
if (is != null) {
is.close();
}
}
}
And it works fine, I cant understand. But it has a int len = 500, and my returned json is cropped to 500 chars. I tried changing to a great number, but it puts spaces at the end. How can I know the size in chars of the String contained by the InputSteam?
Thanks

You can check the Content-Length header value of your response:
Map<String, List<String>> headers = connection.getHeaderFields();
for (Entry<String, List<String>> header : headers.entrySet()) {
if(header.getKey().equals("Content-Legth")){
len=Integer.parseInt(header.getValue());
}
}
or you can your response in a buffered reader like this:
InputStream is = connection.getInputStream();
InputStreamReader reader = new InputStreamReader(is);
StringBuilder builder = new StringBuilder();
int c = 0;
while ((c = reader.read()) != -1) {
builder.append((char) c);
}

Yout can use Apache Commons IO IOUtils.toString to convert InputStream to String or use Gson to read object from input stream directly:
return gson.fromJson(new InputStreamReader(inputStream), YourType.class);

Connection gets closed while uploading file(more than 2 MB) in blacberry rim app

Im trying to upload file from blackberry app to Amazon s3.
Here is my code
private synchronized void uploadFileToAmazon(CreateFileIdBean createFileIdBean) throws UnsupportedEncodingException, IOException,ConnectionNotFoundException, ConnectionClosedException, BackupCancelledException, InterruptedException, BackupInterruptedException {
String BOUNDARY = "----------V2ymHFg03ehbqgZCaKO6jy";
String Policy = "{\"expiration\": \"2020-12-01T12:00:00.000Z\","
+ "\"conditions\": ["
+ "{\"bucket\": \"" + BeanFactory.getUserCreateBackupidBean().getBucketName() + "\"},"
+ "{\"x-amz-security-token\": \"" + BeanFactory.getUserCreateBackupidBean().getAmazonToken() + "\"},"
+ "{\"success_action_status\": \"201\"},"
+ "[\"starts-with\", \"$Content-Type\", \"\"],"
+ "[\"starts-with\", \"$key\", \"" + BeanFactory.getUserCreateBackupidBean().getBackupPath() + "\"]"
+ "]"
+ "}";
String encodePolicy = Base64.encode(Policy.getBytes());
String signature = uploadSignature(Policy, BeanFactory.getUserCreateBackupidBean().getAmazonSecret());
Hashtable params = new Hashtable();
params.put("key", BeanFactory.getUserCreateBackupidBean().getBackupPath() + "/" + BeanFactory.getUserCreateFileIdBean().getFileId());
params.put("AWSAccessKeyId", BeanFactory.getUserCreateBackupidBean().getAmazonKey());
params.put("Content-Type", createFileIdBean.getFileTypeContent());
params.put("x-amz-security-token", BeanFactory.getUserCreateBackupidBean().getAmazonToken());
params.put("policy", encodePolicy);
params.put("success_action_status", "201");
params.put("Signature", signature);
send(BOUNDARY, "http://" + BeanFactory.getUserCreateBackupidBean().getBucketName() + ".s3.amazonaws.com/", params, "file", BeanFactory.getUserCreateFileIdBean().getFileId(), createFileIdBean.getFileTypeContent(), createFileIdBean.getFileByte(), createFileIdBean);
}
private synchronized String getBoundaryMessage(String boundary, Hashtable params, String fileField, String fileName, String fileType, byte[] fileBytes, CreateFileIdBean createFileIdBean) {
StringBuffer res = new StringBuffer("--").append(boundary).append("\r\n");
Enumeration keys = params.keys();
while (keys.hasMoreElements()) {
String key = (String) keys.nextElement();
String value = (String) params.get(key);
res.append("Content-Disposition: form-data; name=\"").append(key).append("\"\r\n")
.append("\r\n").append(value).append("\r\n")
.append("--").append(boundary).append("\r\n");
}
return res.toString();
}
private synchronized void send(String boundarry, String url, Hashtable params, String fileField, String fileName, String fileType, byte[] fileBytes, CreateFileIdBean createFileIdBean) throws IOException,ConnectionClosedException,ConnectionNotFoundException, BackupCancelledException, InterruptedException, BackupInterruptedException {
StringBuffer buffer = new StringBuffer();
HttpConnection hc = null;
InputStream is = null;
InputStream inputFileDataStream = null;
DataOutputStream dout = null;
String boundary = boundarry;
StringBuffer res = new StringBuffer();
int ch;
String boundaryMessage = getBoundaryMessage(boundary, params, fileField, fileName, fileType, fileBytes, createFileIdBean);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
bos.write(boundaryMessage.getBytes());
res.append("Content-Disposition: form-data; name=\"").append(fileField).append("\"; filename=\"").append(fileName).append("\"\r\n")
.append("Content-Type: ").append(fileType).append("\r\n\r\n");
bos.write(res.toString().getBytes());
String end = "\r\n"+"--"+boundary+"\r\n"+"Content-Disposition: form-data; name=\""+"submit"+"\"\r\n"+"\r\n"+"Upload to Amazon S3"+"\r\n"+"--"+boundary+"--\r\n";
try {
hc = (HttpConnection) Connector.open(url+Resources.getConnectionString(), Connector.READ_WRITE,true);
hc.setRequestMethod(HttpConnection.POST);
hc.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundarry);
hc.setRequestProperty("User-Agent", Resources.getUserAgentString());
//hc.setRequestProperty("User-Agent", "Profile/MIDP-1.0 Confirguration/CLDC-1.0");
hc.setRequestProperty("Content-Language", "en-US");
hc.setRequestProperty("Connection", "Keep-Alive");
hc.setRequestProperty("Keep-Alive", "300");
hc.setRequestProperty("Expect", "100-continue");
hc.setRequestProperty("Content-Length", (bos.toByteArray().length+createFileIdBean.getFileSize()+end.getBytes().length)+"");
hc.setRequestProperty("Content-length", (bos.toByteArray().length+createFileIdBean.getFileSize()+end.getBytes().length)+"");
dout = new DataOutputStream(hc.openDataOutputStream());
dout.write(bos.toByteArray());
inputFileDataStream = readInputStream(createFileIdBean.getFilePath());
while ((ch = inputFileDataStream.read()) != -1) {
dout.write(ch);
}
dout.write(end.getBytes());
dout.flush();
//dout.close();
is = hc.openDataInputStream();
BeanFactory.getUserUploadFileBean().setResponseCode(hc.getResponseCode() + "");
BeanFactory.getUserUploadFileBean().setResponseMessage(hc.getResponseMessage());
while ((ch = is.read()) != -1) {
buffer.append((char) ch);
}
System.out.println("buffer"+buffer);
}
catch (IOException e) {
throw new BackupInterruptedException(Constants.ERROR_IN_UPLOAD);
} finally {
try {
if (is != null) {
is.close();
}
if (hc != null) {
hc.close();
}
if(inputFileDataStream !=null)
{
inputFileDataStream.close();
}
if(dout !=null)
{
dout.close();
}
} catch (IOException e2) {
System.out.println("aa"+e2.getMessage());
throw e2;
}
}
}
private synchronized String uploadSignature(String policy, String secretKey) throws UnsupportedEncodingException {
String encodePolciy = Base64.encode(policy.getBytes());
HMac m = new HMac(new SHA1Digest());
m.init(new KeyParameter(secretKey.getBytes("UTF-8")));
byte[] bytes = encodePolciy.getBytes("UTF-8");
m.update(bytes, 0, bytes.length);
byte[] mac = new byte[m.getMacSize()];
m.doFinal(mac, 0);
String signature = Base64.encode(mac);
return signature;
}
private synchronized InputStream readInputStream(String path) throws IOException {
FileConnection fc = null;
InputStream is = null;
fc = (FileConnection) Connector.open(path.toString(), Connector.READ);
if (!fc.exists()) {
Settings.ERROR_MESSAGE = "File doesn't exist!";
//throw new BackupInterruptedException(Settings.ERROR_MESSAGE);
} else {
is = fc.openInputStream();
}
if(fc !=null)
{
fc.close();
}
return is;
}
Its always getting connection closed exception when i trying to get response code(in the line BeanFactory.getUserUploadFileBean().setResponseCode(hc.getResponseCode() + "");) after uploading data. for files less than 2MB its working fine. please help me...

My experience with HttpConnection is that it always buffers the full payload before sending anything. For the BlackBerry Curve 8520, 16mb of data ends up making the device unusable. I worked around this by using a raw SocketConnection, and writing the HTTP parts directly in my app, so I could make sure there wasn't any excessive buffering before the bytes hit the socket.
One area I later realized might work, is using HTTP-Chunked mode with the built-in HttpConnection object. Since HTTP-Chunked is fundamentally a streaming mechanism, it may allow you to get out of the 'buffer it all' logic you get by default.

If you try to transfer large files, you might get the dreaded HTTP error 413. The max allowed file size was between 1 and 30 MB depending on the transport type (BIS,BES, etc) if my memory is correct.
In addition to this, there was a RAM quota available to an app. For older 83XX models, we empirically found that the max RAM for an app was about 12MB.
But we were successful transferring large files using chunked connections (using the HTTP Range header), and for the buffers we either used small arrays (<< 12 MB) or temporary files.
I'll post the classic links on the topic:
How to Download large files using the BlackBerry Mobile Data System
What is: HTTP 413 Request entity too large
You might find something about them in the BB forum or in other sites.

How to get VIEWSTATE before posting using HttpURLConnection (android/java)?

I am writing an Android application, and have been looking for a way to get the _VIEWSTATE from the server I want to post to so I can put it in my Http post content. A few people recommended regex, but then some other pros were strongly opposed to parsing HTML with regex. So, how to parse the _VIEWSTATE ? I am using HttpURLConnection/HttpsURLConnection in an AsyncTask. Also, don't I need to put the InputStream reader first, to get the _VIEWSTATE first? All the android examples put the input stream after the output stream. Here is what my code looks like so far (posting to one site that has three pages that have to be "clicked through"):
In my Activity, I call the Async task like this:
//execute AsyncTask for all three reports
submit_report.execute(report1, report2, report3);
My Async task doInBackground method:
class UploadReportTask extends AsyncTask<HashMap<String,String>, ProgressBar, Void> {
//this is called on task.execute
protected Void doInBackground(HashMap<String,String>...maps) {
System.out.println("Report is being uploaded");
URL url = null;
try {
url = new URL(getString(R.string.url_dnc));
} catch (MalformedURLException e) {
e.printStackTrace();
}
HttpURLConnection urlConnection = null;
try {
urlConnection = (HttpURLConnection) url.openConnection();
} catch (IOException e) {
e.printStackTrace();
}
try {
urlConnection.setDoOutput(true);
urlConnection.setRequestProperty("Accept-Charset", utf);
urlConnection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=" + utf);
urlConnection.setChunkedStreamingMode(0);
//For each map in maps, encode the map,
//get the headers, add the headers to the map, convert to bytes,
//then post the bytes,
//get response.
for (HashMap<String,String> map : maps){
byte[] payload = makePayload(map);
OutputStream out = new BufferedOutputStream(urlConnection.getOutputStream());
//urlConn.connect //I think this happens here
out.write(payload);
InputStream in = new BufferedInputStream(urlConnection.getInputStream());
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
int length = in.read();
String result, line = reader.readLine();
result = line;
while (length != -1){
result+=line;
}
System.out.println(result);
out.flush();
out.close();
}
} catch (IOException e) {
e.printStackTrace();
}
finally{
urlConnection.disconnect();
}
return null;
}

protected String parseViewstate(String response){
int i = 0;
String viewstate = "";
while (true){
int found = response.indexOf("\"__VIEWSTATE\"", i);
if (found == -1) break;
int start = found + 38; //check numbers from start of "__V"
int end = (response.indexOf("/>", start)) -2;
viewstate = response.substring(start, end);
i = end + 1;
}return viewstate;
}

Java uploading .png to server using php POST data

This is the method I have in my java application. It is reading the bytes correctly, I have logged to see if it was. The problem is that the php is not realizing the data is there. I have tested and the .php reads that $_POST is set, but is empty.
public void screenshot(BufferedImage screenshot) {
try {
ImageIO.write(screenshot, "png",
new File(Environment.getStorageDirectory().toString()
.concat(File.separator + SCRIPT_NAME + ".png")));
HttpURLConnection httpUrlConnection;
OutputStream outputStream;
BufferedInputStream fileInputStream;
BufferedReader serverReader;
int totalBytes;
String response = "";
String serverResponse = "";
String localFileName = Environment.getStorageDirectory().toString()
.concat(File.separator + SCRIPT_NAME + ".png");
// Establish a connection
httpUrlConnection = (HttpURLConnection) new URL(
"http://www.scripted.it/scriptoptions/utils/saveScreenshot.php?user="
+ SupraCrafter.statHandler.getUser())
.openConnection();
httpUrlConnection.setDoOutput(true);
httpUrlConnection.setDoInput(true);
httpUrlConnection.setRequestMethod("POST");
httpUrlConnection.setRequestProperty("Content-type",
"application/x-www-form-urlencoded");
outputStream = httpUrlConnection.getOutputStream();
// Buffered input stream
fileInputStream = new BufferedInputStream(new FileInputStream(
localFileName));
// Get the size of the image
totalBytes = fileInputStream.available();
// Loop through the files data
for (int i = 0; i < totalBytes; i++) {
// Write the data to the output stream
outputStream.write(fileInputStream.read());
}
// Close the output stream
outputStream.close();
// New reader to get server response
serverReader = new BufferedReader(new InputStreamReader(
httpUrlConnection.getInputStream()));
// Read the servers response
serverResponse = "";
while ((response = serverReader.readLine()) != null) {
serverResponse = serverResponse + response;
}
System.out.println(serverResponse);
// Close the buffered reader
serverReader.close();
} catch (Exception e) {
e.printStackTrace();
}
try {
URL url = new URL(
"http://scripted.it/scriptoptions/utils/setScreenshotStatus.php?user="
+ SupraCrafter.statHandler.getUser() + "&pass="
+ SupraCrafter.statHandler.getPass() + "&script="
+ SCRIPT_NAME + "&status=1");
BufferedReader in = new BufferedReader(new InputStreamReader(
url.openStream()));
in.close();
} catch (MalformedURLException e) {
} catch (IOException e) {
}
}
Here is the .php file:
<?
// Config
$uploadBase = "../screenshots/";
$uploadFilename = $_GET['user'] . ".png";
$uploadPath = $uploadBase . $uploadFilename;
// Upload directory
if(!is_dir($uploadBase))
mkdir($uploadBase);
// Grab the data
$incomingData = file_get_contents('php://input');
// Valid data?
if(!$incomingData)
die("No input data");
// Write to disk
$fh = fopen($uploadPath, 'w') or die("Error opening file");
fwrite($fh, $incomingData) or die("Error writing to file");
fclose($fh) or die("Error closing file");
echo "Success";
?>
It always echos 'no input data.'

You are not encoding the content with application/x-www-form-urlencoded. You should not simply copy the bytes into the HTTP payload, but instead encode it correctly.
application/x-www-form-urlencoded is not the only possible way of encoding it, multipart/form-data is another common choice. Both are supported by almost all webservers, and as a consequence by PHP.
A tutorial on how to encode using Java is here : http://www.devx.com/Java/Article/17679
Why don't you use Apache's HttpClient or similar library that already do that tedious work for you?
Apache HttpClient : http://hc.apache.org/httpcomponents-client-ga/