I have to download more than 40,000 json records from my web service, but it is taking so long time to download. The downloaded data is storing into a string, but I am not able parse that string. Please tell me a solution to solve this problem.
URL: https://buzoonga.co.uk/appapi/contacts.php?user_id=531
public JSONObject getJSONFromUrl(String url) {
// Making HTTP request
try {
// defaultHttpClient
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(url);
HttpResponse httpResponse = httpClient.execute(httpPost);
HttpEntity httpEntity = httpResponse.getEntity();
is = httpEntity.getContent();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(
is, "iso-8859-1"), 8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
json = sb.toString();
} catch (Exception e) {
Log.e("Buffer Error", "Error converting result " + e.toString());
}
// try parse the string to a JSON object
try {
jObj = new JSONObject(json);
} catch (JSONException e) {
Log.e("JSON Parser", "Error parsing data " + e.toString());
}
// return JSON String
return jObj;
}
check out gzip. excellent text compressor. you are going to remove ~70% of the size :) you just have to gzip it serverside and unzip it clientside. if this overhead increases the time but the download is a third of the time, you might win in the end.
Also consider designing a better schema to reduce the size of your jsons.
Like, instead of "animalBodyParts" put "abp" or something. you have not put an example of your jsons so I am just guessing.
You can use the JSONTokener class and pass the InputStream
jObj = new JSONObject(new JSONTokener(httpEntity.getContent()));
To reduce the time to download you need to support gzipped downloads on the server side or change your JSON structure to a more compact array based format
Related
I use the following function in java to connect to my php script in a server.
The script queries for an image (BLOB), encodes it (base64) and sends it back in JSON format.
The problem is that I can get some of the encoded data but not all of them. Should I use any headers on my httpRequest? What am I doing wrong?
Here is my java function:
public JSONObject makeHttpRequest(String url, String method,
List<NameValuePair> params) {
// Making HTTP request
try {
// request method is POST
// defaultHttpClient
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(url);
httpPost.setEntity(new UrlEncodedFormEntity(params));
HttpResponse httpResponse = httpClient.execute(httpPost);
HttpEntity httpEntity = httpResponse.getEntity();
is = httpEntity.getContent();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
try {
BufferedReader reader = new BufferedReader(
new InputStreamReader(is, "UTF-8"), 8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
json = sb.toString();
System.out.println("Print1: "+json);
} catch (Exception e) {
Log.e("Buffer Error", "Error converting result " + e.toString());
return null;
}
// try parse the string to a JSON object
try {
jObj = new JSONObject(json);
} catch (JSONException e) {
Log.e("JSON Parser", "Error parsing data " + e.toString());
return null;
}
System.out.println("Print2: "+jObj.toString());
// return JSON String
return jObj;
}
Here is my PHP code showing the JSON structure:
$response = array(
"success" => 0,
"message" => "",
"data" => "");
$stmt->bindColumn(1, $binary, PDO::PARAM_LOB);
$stmt->bindColumn(2, $professorName);
$stmt->fetch(PDO::FETCH_BOUND);
$pic = base64_encode($binary);
$data = array(
"photo" => $pic,
"professorName" => $professorName
);
$response["success"] = 1;
$response["message"] = "We have results!";
$response["data"] = $data;
echo json_encode($response);
After testing I realized that if I use a small image (tested size < 1Kb) everything works as it should.
If I try to obtain a larger image (here 5 Kb) the data I obtain is cut in the middle.
The json structure doesn't arrive complete.
Here is an example of a successful transfer:
Print1: {"success":1,"message":"We have results!","data":
{"photo":"iVBORw0KGgoAAAANS~~~~more data~~~~N6GGYzAAAAAElFTkSuQmCC","professorName":"cluvas"}}
Print2: {"data":{"kathigitisName":"cluvas","photo":"iVBORw0K~~~~more data~~~~AElFTkSuQmCC"},
"message":"We have results!","success":1}
Here an example of an unsucessful one
Print1: {"success":1,"message":"We have results!","data":
{"photo":"\/9j\/4AAQS~~~~more data~~~~6KKogkH\/ //unexpected end of data
Print2: {"data":{"professorName":"cluvas",
"photo":"\/9j\/4AAQS~~~~more data~~~~K+U\/Ce //unexpected end of data
Hi I am new to android programming and I am working on a project that converts a list into json object and stores it in the server and retrieves it. I am able to send the json object to the server and store it but i am not able to retrieve it. What approach should i use to retrieve the json object stored on the server?
You can reffer bellow class ...... call getJSONfromURL(YOUR_JOSN_URL) and method will return you JSONObject.
I hope this will work
public class JSONfunctions {
public static JSONObject getJSONfromURL(String url) {
InputStream is = null;
JSONObject jObj = null;
String json = "";
// Making HTTP request
try {
// defaultHttpClient
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpGet httpget = new HttpGet(url);
HttpResponse httpResponse = httpClient.execute(httpget);
HttpEntity httpEntity = httpResponse.getEntity();
is = httpEntity.getContent();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(
is, "UTF-8"), 8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
json = sb.toString();
} catch (Exception e) {
Log.e("Buffer Error", "Error converting result " + e.toString());
}
// try parse the string to a JSON object
try {
jObj = new JSONObject(json);
} catch (JSONException e) {
Log.e("JSON Parser", "Error parsing data " + e.toString());
}
// return JSON String
return jObj;
}
}
To retrieve JSON from server you have to establish a connection with server first by using DefaultHttpClient Hope you did that. If yes, post your code and if no then have look on
www.androidhive.info/2012/01/android-json-parsing-tutorial
Or google it you will lot of help to parse the JSON when you have URL with you.
i am new to programming and I am making a json parser, but my thread method is not accessible, how do I go about doing this. When I debug, it goes into my getjson method but then skips the run method.
I searched on stack over flow already but I am very confused about threading is there any good way to do this?
public class jsonParser {
static InputStream is = null;
static JSONObject jObj = null;
static String json = "";
// constructor
public jsonParser() {
}
public JSONObject getJSONFromUrl(final String url) {
Thread t = new Thread() {
public void run() {
Looper.prepare(); //For Preparing Message Pool for the child Thread
// Making HTTP request
try {
// defaultHttpClient
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(url);
HttpResponse httpResponse = httpClient.execute(httpPost);
HttpEntity httpEntity = httpResponse.getEntity();
is = httpEntity.getContent();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(
is, "iso-8859-1"), 8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
json = sb.toString();
} catch (Exception e) {
Log.e("Buffer Error", "Error converting result " + e.toString());
}
// try parse the string to a JSON object
try {jObj = new JSONObject(json);
} catch (JSONException e) {
// TODO Auto-generated catch block
Log.e("JSON Parser", "Error parsing data " + e.toString());
}
Looper.loop();
return;
}
}; t.start();
return jObj;
}
}
It goes to the run() method. But the principle of a thread is that it's executed in parallel to the current thread. So here's what happens on the time line:
main thread
_______________________________________________________________________
| |
start parser thread return jObj
parser thread
_________________________________________________________________
| |
execute an HTTP request and parse json assign JSON object to jObj
When you return jObj from the current thread, you have just started the parser thread. Maybe it hasn't even started executing yet. Maybe it's still waiting for the HTTP response. What's sure is that there is 0% chance that it has finished executing and stored the result in the static jObj variable yet. So, the value returned from the current thread is null.
It's hard to tell you what you should do, except read more on threading and on the supporting classes provided by android.
I created a json result in mvc and I'm building an Android app to get the json result. This is what my json result looks like
{"name":"Mr. Spock","gender":"Male"}
This is my controller
public ActionResult Index()
{
var result = new { name = "Mr. Spock", gender = "Male" };
return Json(result, JsonRequestBehavior.AllowGet);
}
And this I'm using in android
// Creating JSON Parser instance
JSONParser jParser = new JSONParser();
// getting JSON string from URL
JSONObject json = jParser.getJSONFromUrl(url);
JSONParser class
public class JSONParser {
static InputStream is = null;
static JSONObject jObj = null;
static String json = "";
// constructor
public JSONParser() {
}
public JSONObject getJSONFromUrl(String url) {
// Making HTTP request
try {
// defaultHttpClient
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(url);
HttpResponse httpResponse = httpClient.execute(httpPost);
HttpEntity httpEntity = httpResponse.getEntity();
is = httpEntity.getContent();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(
is, "iso-8859-1"), 8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
json = sb.toString();
} catch (Exception e) {
Log.e("Buffer Error", "Error converting result " + e.toString());
}
// try parse the string to a JSON object
try {
jObj = new JSONObject(json);
} catch (JSONException e) {
Log.e("JSON Parser", "Error parsing data " + e.toString());
}
// return JSON String
return jObj;
}
}
I have a declared variable url. Every time I debug, the json variable does not have any values and says "errors during the evaluation"
Anyone with a tip? I tried working with Gson, but no succes
Kind regards
I'll give you some code for Gson. It really is much easier to work with than the built in JSON parsing code. Here's a minimal example using your JSON.
Person.class:
package com.example.tutorial.models;
import com.google.gson.annotations.SerializedName;
public class Person {
#SerializedName("gender")
public String gender = "";
#SerializedName("name")
public String name = "";
}
The annotations are really only necessary when your variable and JSON name differ, but I tend to always include them as it reinforces that they are coming from JSON.
To deserialize:
Gson gson = new GsonBuilder().create();
Person person = gson.fromJson(json, Person.class);
It really is that simple. If this does not work, log the result from the web server and make sure it really is the valid JSON string you expect it to be.
I do have one question, where is your AsyncTask? Your attempt to open a connection to the webserver in the UI thread will definitely cause a NetworkOnMainThreadException. I created a library to do RESTful calls on Android. It's licensed under BSD, so feel free to use it as a guide or outright use it: https://github.com/nedwidek/Android-Rest-API
I am not new to android platform, i have parsed many json pages before, but one link is creating problem. I am using following function to return the json object. The program stops at reader.readline() and loading does not stop.
public JSONObject getJSONFromUrl(String url) {
DefaultHttpClient httpClient = new DefaultHttpClient();
try {
HttpResponse httpResponse = httpClient.execute(new HttpGet(url));
if (httpResponse.getStatusLine().getStatusCode()==200)
{
HttpEntity httpEntity = httpResponse.getEntity();
is = httpEntity.getContent();
}
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
System.out.println("Unsupported Encoding");
} catch (ClientProtocolException e) {
e.printStackTrace();
System.out.println("Client Protocol");
} catch (IOException e) {
e.printStackTrace();
System.out.println("IO");
}
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(
is, "iso-8859-1"),8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line);
}
is.close();
jsonStr = sb.toString();
} catch (Exception e) {
System.out.println("Error here");
Log.e("Buffer Error", "Error converting result " + e.toString());
}
// try parse the string to a JSON object
try {
jObj = new JSONObject(jsonStr);
} catch (JSONException e) {
Log.e("JSON Parser", "Error parsing data " + e.toString());
}
return jObj;
}
can any one suggest something?
I am pretty sure that this is caused by the server not finishing the response in your case. Try to isolate that using "curl" as a client first.
Also there is an easier way to fetch body into String if you use EntitUtils like this:
EntityUtils.toString(entity);
You can also specify encoding as second parameter.
http://developer.android.com/reference/org/apache/http/util/EntityUtils.html