I'm trying to fetch some data using Google feed api. But line = reader.readLine() is always null.
URL url = new URL("https://ajax.googleapis.com/ajax/services/feed/find?" +
"v=1.0&q=Official%20Google%20Blog");
URLConnection connection = url.openConnection();
String line;
StringBuilder builder = new StringBuilder();
BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
while((line = reader.readLine()) != null) {
builder.append(line);
}
JSONObject json = new JSONObject(builder.toString());
Try this
URL url = new URL("https://ajax.googleapis.com/ajax/services/feed/find?" +
"v=1.0&q=Official%20Google%20Blog");
URLConnection connection = url.openConnection();
ByteArrayOutputStream content = new ByteArrayOutputStream();
InputStream is = connection.getInputStream();
int len = 0;
byte[] buffer = new byte[1024];
while ((len = is.read(buffer)) >= 0)
{
content.write(buffer, 0, len);
}
byte[] finalContent = content.toByteArray();
String str = new String(finalContent, "UTF8");
System.out.print(str);
Or other way is you can read the content-length header and read that till you get that much data.
Related
I sent a GET message with socket. And I received response message as string. But I want to receive as hexadecimal. But I didn't accomplish. This is my code block as string. Can you help me ?
dos = new DataOutputStream(socket.getOutputStream());
dis = new BufferedReader(new InputStreamReader(socket.getInputStream()));
dos.write(requestMessage.getBytes());
String data = "";
StringBuilder sb = new StringBuilder();
while ((data = dis.readLine()) != null) {
sb.append(data);
}
when you use BufferedReader you'll get the input into String format..so better way to use InputStream...
here is sample code to achieve this.
InputStream in = socket.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] read = new byte[1024];
int len;
while((len = in.read(read)) > -1) {
baos.write(read, 0, len);
}
// this is the final byte array which contains the data
// read from Socket
byte[] bytes = baos.toByteArray();
after getting the byte[] you can convert it to hex string using the following function
StringBuilder sb = new StringBuilder();
for (byte b : bytes) {
sb.append(String.format("%02X ", b));
}
System.out.println(sb.toString());// here sb is hexadecimal string
reference from java-code-to-convert-byte-to-hexadecimal
I need to download a .txt file from a website, the problem is the downloaded file doesn't respect the same line wrapping as the original file.
File:
Word1
Word2
Word3
File downloaded:
Word1Word2Word3
I use this method to download (this isn't mine) :
#Override
protected String doInBackground(String... f_url) {
int count;
try {
URL url = new URL(f_url[0]);
URLConnection conection = url.openConnection();
conection.connect();
int lenghtOfFile = conection.getContentLength();
InputStream input = new BufferedInputStream(url.openStream(), 8192);
OutputStream output = new FileOutputStream( MegaMethods.FolderPath+"downloadedfile.txt");
byte data[] = new byte[1024];
long total = 0;
while ((count = input.read(data)) != -1) {
total += count;
publishProgress(""+(int)((total*100)/lenghtOfFile));
output.write(data, 0, count);
}
output.flush();
output.close();
input.close();
} catch (Exception e) {
Log.e("Error: ", e.getMessage());
}
return null;
}
Try using a BufferedReader to read it in something like
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line = null;
StringBuilder responseData = new StringBuilder();
while((line = in.readLine()) != null) {
responseData.append(line);
}
then output the lines as necessary. I'm no where near a station where I can test this so you might have to do some fiddling.
here what's the problem
I have problem when I tried to get String from my StringBuilder
BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent()), 128 * 1024);
StringBuilder dataResponseSB = new StringBuilder();
String line ;
while ((line = reader.readLine()) != null) {
dataResponseSB.append(line);
if (DataFactory.DEBUG_MODE) {
// all data here are complete
Log.i("===LoadDataActivity","line: "+line);
}
}
String rawdata = new String(dataResponseSB); // dataResponseSB.toString(); also not work
if (DataFactory.DEBUG_MODE) {
// data here are lost
Log.i("===LoadDataActivity","rawdata: "+rawdata);
}
(-) I receive a huge data from BufferedReader .readLine()
(-) I use Log to check and sure that I got about 5 line of 8000 Buffer Size per line and I am very sure that I have receive all data properly
(1) I append each line to StringBuilder Here
(-) after I append all the line to StringBuilder
(2) I try to convert it back to String
(-) Now, the problem, the when I check to new String here, the data have only 8192 (it should contain at least 30,000 or more)
What is the problem ? I am not sure it lost when it append to StringBuilder(1) or it lost when it convert back to String (2)
I add the code that I have tried below here ,, I have tried both UTF8 and without UTF8
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
//params.setParameter(CoreProtocolPNames.PROTOCOL_VERSION, );
params.setParameter(CoreConnectionPNames.SOCKET_BUFFER_SIZE, 128 * 1024);
HttpClient client = new DefaultHttpClient(params);
// HttpClient client = new DefaultHttpClient(new BasicHttpParams());
HttpPost httppost = new HttpPost(DataFactory.REQUEST_API_URL + "?id=" + DataFactory.USER_ID );
// Depends on your web service
HttpConnectionParams.setConnectionTimeout(client.getParams(), 10000); //Timeout Limit
HttpConnectionParams.setSocketBufferSize(client.getParams(), 128 * 1024);
HttpResponse response = client.execute(httppost);
//response.setParams(client.getParams().setParameter(CoreConnectionPNames.SOCKET_BUFFER_SIZE, 128 * 1024));
//String rawdata = IOUtils.toString(response.getEntity().getContent(), "UTF-8");
// String rawdata = EntityUtils.toString(response.getEntity());
String rawdata = getResponseBody(response.getEntity());
//Scanner s = new Scanner(response.getEntity().getContent()).useDelimiter("\\A");
//String rawdata = s.hasNext() ? s.next() : "";
/*
//BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
// ===================
BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent()), 128 * 1024);
StringBuilder dataResponseSB = new StringBuilder();
String line ;
while ((line = reader.readLine()) != null) {
dataResponseSB.append(line);
if (DataFactory.DEBUG_MODE) {
Log.i("===LoadDataActivity","line: "+line);
}
}
dataResponseSB.trimToSize();
String rawdata = new String(dataResponseSB);
/*
InputStreamReader reader = new InputStreamReader(response.getEntity().getContent());
StringBuffer sb = new StringBuffer();
int c;
while ((c = reader.read()) != -1) {
sb.append((char)c);
if (DataFactory.DEBUG_MODE) {
//Log.i("===LoadDataActivity","line: "+line);
}
}
*/
I'm pretty sure this is the problem:
Log.i("===LoadDataActivity","rawdata: "+rawdata);
You're assuming that a log entry can include all of your data - I believe each log entry is limited to 8192 characters.
I suggest you log rawdata.length() and you'll see that it's actually got all of the data - it's just logging it that's failing.
try this,,
public String getResponseBody(final HttpEntity entity) throws IOException, ParseException {
if (entity == null) {
throw new IllegalArgumentException("HTTP entity may not be null");
}
InputStream instream = entity.getContent();
if (instream == null) {
return "";
}
if (entity.getContentLength() > Integer.MAX_VALUE) {
throw new IllegalArgumentException(
"HTTP entity too large to be buffered in memory");
}
StringBuilder buffer = new StringBuilder();
BufferedReader reader = new BufferedReader(new InputStreamReader(instream, HTTP.UTF_8));
String line = null;
try {
while ((line = reader.readLine()) != null) {
buffer.append(line);
}
} finally {
instream.close();
reader.close();
}
System.out.println("GEN END : " + Calendar.getInstance().getTimeInMillis());
return buffer.toString();
}
// Try this way,hope this will help you to solve your problem...
StringBuilder buffer = new StringBuilder();
BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent(), HTTP.UTF_8));
String line = null;
try {
while ((line = reader.readLine()) != null) {
buffer.append(line);
}
} finally {
instream.close();
reader.close();
}
System.out.println("Buffer : " + buffer.toString());
I am reading the contents of buffered reader in the below method:
public static String readBuffer(Reader reader, int limit) throws IOException
{
StringBuilder sb = new StringBuilder();
for (int i = 0; i < limit; i++) {
int c = reader.read();
if (c == -1) {
return ((sb.length() > 0) ? sb.toString() : null);
}
if (((char) c == '\n') || ((char) c == '\r')) {
break;
}
sb.append((char) c);
}
return sb.toString();
}
I am invoking this method later to test -
URL url = new URL("http://www.oracle.com/");
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
StringBuffer sb = new StringBuffer();
String line=null;
while((line=readBuffer(in, 2048))!=null) {
sb.append(line);
}
System.out.println(sb.toString());
My question here is, I am returning the contents of bufferedreader into a string in my first method, and appending these String contents into a StringBuffer again in the second method, and reading out of it. Is this the right way? Any other way I can read the String contents that has contents from url?Please advise.
I hope this works -
public static String readFromURL(){
URL url = new URL("http://www.oracle.com/");
StringBuilder responseBuilder = new StringBuilder();
HttpURLConnection httpCon = (HttpURLConnection) url.openConnection();
httpCon.setDoOutput(true);
httpCon.setDoInput(true);
int resCode = httpCon.getResponseCode();
InputStream is = null;
if (resCode == 200) {
is = httpCon.getInputStream();
BufferedReader reader = new BufferedReader(
new InputStreamReader(is));
String response = null;
while (true) {
response = reader.readLine();
if (response == null)
break;
responseBuilder.append(response);
}
}
return responseBuilder.toString();
}
I have a url of a file on the Internet. I need to calculate the SHA1 hash, and read this file by each line. I know how to do this, but I read this file twice which probably isn't a very good solution.
How can I do this more effectively?
Here is my code:
URL url = new URL(url);
URLConnection urlConnection = url.openConnection();
urlConnection.setConnectTimeout(1000);
urlConnection.setReadTimeout(1000);
logger.error(urlConnection.getContent() + " ");
InputStream is = urlConnection.getInputStream();
// first reading of file is:
int i;
File file = new File("nameOfFile");
BufferedInputStream bis = new BufferedInputStream(is);
BufferedOutputStream bos =
new BufferedOutputStream(new FileOutputStream(file.getName()));
while ((i = bis.read()) != -1) {
bos.write(i);
}
bos.flush();
bis.close();
sha1(file);
// second reading of file is:
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
String line;
while ((line = reader.readLine()) != null) {
// do something
}
protected byte[] sha1(final File file) throws Exception {
if (file == null || !file.exists()) {
return null;
}
final MessageDigest messageDigest = MessageDigest.getInstance(SHA1);
InputStream is = new BufferedInputStream(new FileInputStream(file));
try {
final byte[] buffer = new byte[1024];
for (int read = 0; (read = is.read(buffer)) != -1;) {
messageDigest.update(buffer, 0, read);
}
} finally {
IOUtils.closeQuietly(is);
}
return messageDigest.digest();
}
If you pass it through a DigestInputStream, it'll do the MessageDigest and still be usable as an InputStream.
DigestInputStream dis = new DigestInputStream(is,
MessageDigest.getInstance(SHA1));
BufferedInputStream bis = new BufferedInputStream(dis);
BufferedOutputStream bos = new BufferedOutputStream(
new FileOutputStream(file.getName()));
while ((i = bis.read()) != -1) {
bos.write(i);
}
bos.close();
return dis.getMessageDigest().digest();