We Keep Coding

How to stop a string from being cut off in Java

I have a java code to call a REST API which returns a JWT token as a response. I send a GET call to the API and it will return a JWT token as a response. The token is being returned fine. However, I've noticed somehow the token is being trimmed.
I tried everything online and nothing seems to be working for me. Below is my code :
try {
URL url = new URL(proxyService.getProperty("proxy.url") + "/" + sessionToken);
log.logText("Connection URL: " + url, logLevel);
String readLine = null;
HttpURLConnection conn = (HttpURLConnection)url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");
int responseCode = conn.getResponseCode();
if (responseCode == HttpURLConnection.HTTP_OK) {
InputStream in = ((URLConnection)conn).getInputStream();
int length = 0;
StringBuffer response = new StringBuffer();
byte[] data1 = new byte[1024];
while (-1 != (length = in.read(data1))) {
response.append(new String(data1, 0, length));
}
log.logText("JSON String Result: " + response.toString(), logLevel);
}
conn.disconnect();
} catch(MalformedURLException e) {
e.printStackTrace();
} catch(IOException e) {
e.printStackTrace();
}
oauthToken = oauthToken.replaceAll("^\"|\"$", "");
log.logText("OAuth2 Token: " + oauthToken, logLevel);
return oauthToken;
Regards,
Learnmore

As #markspace mentioned, please specify the data type for oauthToken (I believe it is of type String). Print the total String and then the length before replaceALL and after replaceALL. compare whats replace adds to the total length if yes then there is no issue of string getting trimmed.

You are not assigning response value to anything. I assume you should be assigning it to oauthToken variable.
Also please close the InputStream instance in finally clause, otherwise you will cause resource leakage.

I think you have to close InputStream first, to flush internal buffer.
public static String getOauthToken() throws IOException {
URL url = new URL(proxyService.getProperty("proxy.url") + "/" + sessionToken);
log.logText("Connection URL: " + url, logLevel);
String oauthToken = readInputString(url);
oauthToken = oauthToken.replaceAll("^\"|\"$", "");
log.logText("OAuth2 Token: " + oauthToken, logLevel);
return oauthToken;
}
private static String readInputString(URL url) throws IOException {
HttpURLConnection conn = (HttpURLConnection)url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");
if (conn.getResponseCode() != HttpURLConnection.HTTP_OK)
throw new RuntimeException("Not expected response code");
try (InputStream in = conn.getInputStream()) {
StringBuffer buf = new StringBuffer();
byte[] b = new byte[1024];
while (true) {
int readBytes = in.read(b);
if (readBytes == -1)
break;
buf.append(new String(b, 0, readBytes));
}
log.logText("JSON String Result: " + buf, logLevel);
return buf.toString();
}
}

It looks like the actual application that I'm calling from is cutting off the response value. I shortened the length of the JWT token and it's not cutting it off. The application must have a limit for a maximum number of characters allowed in a string could be for performance reasons.

Processing large input Stream to List String in Java

Hey I am having a file nearly 110MB size at apache. I am reading that file into input stream and then converting that input stream to List of String based on all suggestion i find on stack overflow. But still i am facing out of memory issue.
Below is my code.
private List<String> readFromHttp(String url, PlainDiff diff) throws Exception {
HttpUrlConnection con = new HttpUrlConnection();
con.setGetUrl(url);
List<String> lines = new ArrayList<String>();
final String PREFIX = "stream2file";
final String SUFFIX = ".tmp";
final File tempFile = File.createTempFile(PREFIX, SUFFIX);
tempFile.deleteOnExit();
StringBuilder sb = new StringBuilder();
try {
InputStream data = con.sendGetInputStream();
if(data==null)
throw new UserAuthException("diff is not available at the location");
else {
try (FileOutputStream out = new FileOutputStream(tempFile)) {
IOUtils.copy(data, out);
LineIterator it = FileUtils.lineIterator(tempFile, "UTF-8");
try {
while (it.hasNext()) {
String line = it.nextLine();
lines.add(line);
sb.append(line);
}
} finally {
LineIterator.closeQuietly(it);
}
}
data.close();
diff.setLineAsString(sb.toString());
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
//System.out.println(lines);
return lines;
}
public InputStream sendGetInputStream() throws IOException {
String encoding = Base64.getEncoder().encodeToString(("abc:$xyz$").getBytes("UTF-8"));
URL obj = new URL(getGetUrl());
// Setup the connection
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
// Set the parameters from the headers
con.setRequestMethod("GET");
con.setDoOutput(true);
con.setRequestProperty ("Authorization", "Basic " + encoding);
InputStream is;
int responseCode = con.getResponseCode();
logger.info("GET Response Code :: " + responseCode);
if (responseCode == HttpURLConnection.HTTP_OK) {
is = con.getInputStream();
}
else {
is = null;
}
return is;
}
Is something in memory i am doing that is consuming lot of heap? Is there a better way to do it?

Your code has multiple issues. I am not going to solve each and every issue but point that out so that you can review your code and learn to write better code.
In method readFromHttp(..):
There is no need to create a new file by IOUtils.copy(data, out);
No use of String Builder StringBuilder sb = new StringBuilder();
No use of line iterator LineIterator
And there are multiple other memory-related issues but for the time being correct these points and test with the below-mentioned code.
Change your reading lines from file to very simple way after correcting the above mistakes:
try(BufferedReader reader = new BufferedReader(new InputStreamReader(data, StandardCharsets.UTF_8))) {
for (String line; (line = reader.readLine()) != null;) {
lines.add(line);
}
}

Android receiving json, buffer isn't filled properly

I have a symfony webservice returning the content of a database table as json.
This code comes directly from the Android HttpUrlConnection tutorials themselves.
When called from the webbrowser, it works as verified by json checking tools
No escape characters or \n anywhere
The buffer cuts off after 1190 characters
Shorter json responses work flawlessly
org.json.JSONException: Unterminated string at character 50000 of... {my jason goes on for a while and is terminated by ����������������������������������������� which I assume represents the empty char[] section
This is where it goes wrong somehow, the reader isn't filling the buffer properly
// Reads an InputStream and converts it to a String.
public String readIt(InputStream stream, int len) throws IOException {
Reader reader = new InputStreamReader(stream, "UTF-8");
char[] buffer = new char[len];
reader.read(buffer);
return new String(buffer);
}
HttpUrlConnection call code:
/*
Given a URL, establishes an HttpUrlConnection and retrieves
the web page content as a InputStream, which it returns as
a string.
*/
private String downloadUrl(String myurl) throws IOException {
InputStream inputStream = null;
/*
Only display the first 500 characters of the retrieved
web page content.
*/
int len = 5000;
String result = "[]";
try {
System.out.print("TRYING TO OPEN: " + myurl + "\n");
URL url = new URL(myurl);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setReadTimeout(30000 /* milliseconds */);
connection.setConnectTimeout(30000 /* milliseconds */);
connection.setRequestMethod("POST");
connection.setDoInput(true);
// Starts the query
connection.connect();
int response = connection.getResponseCode();
Log.d("HTTP CONNECTION :", "The response is: " + response);
inputStream = connection.getInputStream();
// Convert the InputStream into a string
result = readIt(inputStream, len);
} catch (Exception e) {
e.printStackTrace();
} finally {
if (inputStream != null) {
inputStream.close();
}
}
return result;
}
Most other SO answers I've come across refer to now deprecated code, thank you for reading.

How to get VIEWSTATE before posting using HttpURLConnection (android/java)?

I am writing an Android application, and have been looking for a way to get the _VIEWSTATE from the server I want to post to so I can put it in my Http post content. A few people recommended regex, but then some other pros were strongly opposed to parsing HTML with regex. So, how to parse the _VIEWSTATE ? I am using HttpURLConnection/HttpsURLConnection in an AsyncTask. Also, don't I need to put the InputStream reader first, to get the _VIEWSTATE first? All the android examples put the input stream after the output stream. Here is what my code looks like so far (posting to one site that has three pages that have to be "clicked through"):
In my Activity, I call the Async task like this:
//execute AsyncTask for all three reports
submit_report.execute(report1, report2, report3);
My Async task doInBackground method:
class UploadReportTask extends AsyncTask<HashMap<String,String>, ProgressBar, Void> {
//this is called on task.execute
protected Void doInBackground(HashMap<String,String>...maps) {
System.out.println("Report is being uploaded");
URL url = null;
try {
url = new URL(getString(R.string.url_dnc));
} catch (MalformedURLException e) {
e.printStackTrace();
}
HttpURLConnection urlConnection = null;
try {
urlConnection = (HttpURLConnection) url.openConnection();
} catch (IOException e) {
e.printStackTrace();
}
try {
urlConnection.setDoOutput(true);
urlConnection.setRequestProperty("Accept-Charset", utf);
urlConnection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=" + utf);
urlConnection.setChunkedStreamingMode(0);
//For each map in maps, encode the map,
//get the headers, add the headers to the map, convert to bytes,
//then post the bytes,
//get response.
for (HashMap<String,String> map : maps){
byte[] payload = makePayload(map);
OutputStream out = new BufferedOutputStream(urlConnection.getOutputStream());
//urlConn.connect //I think this happens here
out.write(payload);
InputStream in = new BufferedInputStream(urlConnection.getInputStream());
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
int length = in.read();
String result, line = reader.readLine();
result = line;
while (length != -1){
result+=line;
}
System.out.println(result);
out.flush();
out.close();
}
} catch (IOException e) {
e.printStackTrace();
}
finally{
urlConnection.disconnect();
}
return null;
}

protected String parseViewstate(String response){
int i = 0;
String viewstate = "";
while (true){
int found = response.indexOf("\"__VIEWSTATE\"", i);
if (found == -1) break;
int start = found + 38; //check numbers from start of "__V"
int end = (response.indexOf("/>", start)) -2;
viewstate = response.substring(start, end);
i = end + 1;
}return viewstate;
}

Grabbing JSON works from one link, not from another

I'm doing a simple JSON grab from two links with the same code. I'm doing it two separate times, so the cause of my issue isn't because they're running into each other or something.
Here is my code:
#Override
protected String doInBackground(Object... params) {
try {
URL weatherUrl = new URL("my url goes here");
HttpURLConnection connection = (HttpURLConnection) weatherUrl
.openConnection();
connection.connect();
responseCode = connection.getResponseCode();
if (responseCode == HttpURLConnection.HTTP_OK) {
InputStream inputStream = connection.getInputStream();
Reader reader = new InputStreamReader(inputStream);
int contentLength = connection.getContentLength();
char[] charArray = new char[contentLength];
reader.read(charArray);
String responseData = new String(charArray);
Log.v("test", responseData);
When I try this with:
http://www.google.com/calendar/feeds/developer-calendar#google.com/public/full?alt=json
I get an error of having an array lenth of -1
For this link:
http://api.openweathermap.org/data/2.5/weather?id=5815135
It returns fine and I get a log of all of the JSON. Does anyone have any idea why?
Note: I tried stepping through my code in debug mode, but I couldn't catch anything. I also downloaded a Google chrome extension for parsing json in the browser and both urls look completely valid. I'm out of ideas.

Log this: int contentLength = connection.getContentLength();
I don't see the google url returning a content-length header.
If you just want String output from a url, you can use Scanner and URL like so:
Scanner s = new Scanner(new URL("http://www.google.com").openStream(), "UTF-8").useDelimiter("\\A");
out = s.next();
s.close();
(don't forget try/finally block and exception handling)
The longer way (which allows for progress reporting and such):
String convertStreamToString(InputStream is) throws UnsupportedEncodingException {
BufferedReader reader = new BufferedReader(new
InputStreamReader(is, "UTF-8"));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null)
sb.append(line + "\n");
} catch (IOException e) {
// Handle exception
} finally {
try {
is.close();
} catch (IOException e) {
// Handle exception
}
}
return sb.toString();
}
}
and then call String response = convertStreamToString( inputStream );