We Keep Coding

Android app connecting to shutterstock api throws IOException with Error 401

I am trying to connect my android app to shutterstock api so that it can search for some images there. It uses https scheme + Basic Authentication header to allow users for all search requests. I implemented the functionality in a regular java project using HttpsURLConnection and was able to get correct JSON responses.
The java code looks like this:
HttpsURLConnection urlConnection = (HttpsURLConnection) url.openConnection();//proxy);
String username = "62c01aa824222683004b", password = "dc4ad748a75e4e69ec853ad2435a62b700e66164";
String encoded = Base64.getEncoder().encodeToString((username+":"+password).getBytes("UTF-8"));
System.out.println(encoded.equals("Nj0jMDFhZWE4ZmE4MjY4MzAwNGI6ZGM0YWQ3NDhhNzVlNGU2gWVjODUzYWQ0ZmEzYTYyYjc7MGU2NjE2NA==")); // prints true
urlConnection.setRequestProperty("Authorization", "Basic "+encoded);
When ported this into Android, it was throwing an IOException with 401 error code. As explained in many posts on SO (like the one here), I modified the code accordingly with an extra try-catch as below:
String username = "62c01aa824222683004b", password = "dc4ad748a75e4e69ec853ad2435a62b700e66164", encoded = "";
encoded = Base64.encodeToString((username+":"+password).getBytes("UTF-8"), Base64.URL_SAFE);
Log.e("test", "encoded strings match:" + encoded.equals("Nj0jMDFhZWE4ZmE4MjY4MzAwNGI6ZGM0YWQ3NDhhNzVlNGU2gWVjODUzYWQ0ZmEzYTYyYjc7MGU2NjE2NA==") + "\n" + encoded); // prints false but string is same!!
URL url = new URL(reqUrl);
connection = (HttpsURLConnection) url.openConnection();
connection.setRequestProperty("Authorization", "Basic "+encoded);
try {
if (connection != null) {
connection.connect();
if (200 == connection.getResponseCode()) { // ---> throws IOException
BufferedReader br = new BufferedReader(new InputStreamReader((connection.getInputStream())));
String output;
while ((output = br.readLine()) != null) {
Log.e("test", output);
response.append(output);
}
connection.disconnect();
return response.toString();
}
}
} catch (IOException e) {
try {
Log.e("test", e.getMessage()); // ---> prints "No authentication challenges found"
Log.e("test", connection.getResponseCode() + ":" + connection.getResponseMessage() + connection.getHeaderFields());
//---> prints 401:Unauthorized{null=[HTTP/1.1 401 Unauthorized], cache-control=[no-cache], Connection=[keep-alive], Content-Length=[38], Content-Type=[application/json; charset=utf8], Date=[Tue, 31 May 2016 14:11:28 GMT], Server=[nginx], X-Android-Received-Millis=[1464703888222], X-Android-Sent-Millis=[1464703887592], x-end-user-request-id=[f754ec7f-c344-431b-b641-360aabb70184], x-shutterstock-app-version=[apitwo-625], x-shutterstock-resource=[/v2/images/search]}
if (401 == connection.getResponseCode()) {
InputStream es = connection.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(es));
String output;
while ((output = br.readLine()) != null) {
Log.e("test", output); // ---> prints {"message":"Invalid auth credentials"}
response.append(output);
}
connection.disconnect();
return response.toString();
} else {
Log.e("test","Could not connect! " + connection.getResponseCode() + ":" + connection.getResponseMessage() + ". " + connection.getRequestMethod());
}
}catch (Exception e1){e1.printStackTrace();}
}
I was unable to check the response headers in Firefox's Rest client because it does not send the request to server when I add the Authentication header.
So the questions here are:
Is this the right way to handle the 401 error in Android? Will I get the JSON response in the inner try-catch?
The java program uses exactly the same encoded string as in Android. How come the String.equals() returns "true" in java but "false" in android?
The error message from the server says "Invalid auth credentials". Does the encoded string differ between Android and Java for any reason? If yes, then point 2 makes sense.

I copied the encoded string from the java program into the Android variable and was able to authenticate successfully with shutterstock. So Indeed the encoded strings on Android and Java were different though in UTF-8 format. This also explains the "false" in Android and the "Invalid credentials" message from the server.
Just not sure why/how it differs when both the encoded strings are the same for the human eyes!

Why did my application stop connecting using HTTP, and how can I debug it?

I have a program that includes HTTP connection to a PLM application that runs on SQL Server. The program is scheduled to run daily. It collects data from few sources, then issues a query to the PLM to store the data, and finally reads the PLM's reply to verify if the data was properly stored.
The application ran OK, until we upgraded both the DB (into SQL Server 2012) and the PLM.
Since then the upgrade, when the program establishes the connection it receives OK status; however, the data setting query does not affect the data base, and there is no answer received. There are no error messages - just malfunction.
My major question is - how to debug it. I know whet I send and what I receive. How can I get more data on what happens in between?
I attach the code for review. What I didn't add here is the query itself, which is WML-like string. The PLM should fire an answer regardless the query it receives, even if it is an error message. However, I get only NULL.
public Boolean amlArasCommunication (String data , int targetDbType, String passWord)
{
final String url = "http://plm-srv/InnovatorServer/Server/InnovatorServer2012.aspx";
final String schemeUrl = "'http://schemas.xmlsoap.org/soap/envelope/'";
String answer =””;
String dataBase = data base name;
Writer wout;
HttpURLConnection amlConnection = null;
try
{
// instantiate the HttpURLConnection with the URL object - A new connection is
// opened every time by calling the openConnection method of the protocol
// handler for this URL. This is the point where the connection is opened.
amlConnection = (HttpURLConnection) new URL(url).openConnection();
amlConnection.setDoOutput(true);
amlConnection.setDoInput(true);
amlConnection.setRequestMethod("POST");
amlConnection.setRequestProperty("SOAPAction", "ApplyAML");
amlConnection.setConnectTimeout(10000);
amlConnection.setReadTimeout(10000);
amlConnection.setRequestProperty("AUTHUSER", "Admin");
amlConnection.setRequestProperty("AUTHPASSWORD", calcMD5(passWord));
amlConnection.setRequestProperty("DATABASE", dataBase);
String query = "<?xml version='1.0'?>\r\n" +
"<SOAP-ENV:Envelope xmlns:SOAP-ENV=" + schemeUrl + ">\r\n" +
" <SOAP-ENV:Body>\r\n" +
data +
" </SOAP-ENV:Body>\r\n" +
"</SOAP-ENV:Envelope>\r\n";
// instantiate OutputStreamWriter using the output stream, returned from getOutputStream, that writes
// to this connection. If an I/O error occurs while creating the output stream, IOException will be fired.
wout = new OutputStreamWriter(amlConnection.getOutputStream());
wout.write (query);
wout.close();
// At this point, we've sent all the data. The outputStream was closed, while the connection is still open
int result;
if ((result = amlConnection.getResponseCode()) == HttpURLConnection.HTTP_OK)
{
// Get the communication results from the PLM
InputStream ac = amlConnection.getInputStream();
InputStreamReader isr = new InputStreamReader (ac);
BufferedReader in = new BufferedReader(isr);
String readResult = in.readLine ();
int count = 0;
while (readResult != null)
{
answer += readResult + "\n";
readResult = in.readLine ();
count++;
}
in.close();
if (answer.contains("fault"))
System.out.println ("Error message: " + answer + "\nQuery: " + query);
else
log.message ("Lines count=" + count + "; com status=" + result + "; reply: " + answer, false);
}
else
// Error code is returned, or no status code is returned, do stuff in the else block
System.out.println("Connection failed with the following code: " + result);
}
catch (IOException e) { ; }
if (amlConnection != null)
amlConnection.disconnect ();
return true;
}

I think you will have to look into the log files of the PLM application to find out why you do not get an HTTP response. There might be a number of possible reasons why the application is not working anymore after the upgrade.
I guess that it will be difficult to debug the problem based on the client code only. As the server seems to accept your HTTP, I would expect that this event and errors would be written to a log file somewhere. You might also want to try some graphical tool like SOAP UI to test the SOAP service.

Http Client hanging for some reason

Working on a HTTP client program using Netbeans.
So far I have gotten to here in my HttpClient class:
public class MyHttpClient {
MyHttpRequest request;
String host;
public MyHttpResponse execute(MyHttpRequest request) throws IOException {
//Creating the response object
MyHttpResponse response = new MyHttpResponse();
//Get web server host and port from request.
String host = request.getHost();
int port = request.getPort();
//Check 1: HOST AND PORT NAME CORRECT!
System.out.println("host: " + host + " port: " + String.valueOf(port));
//Get resource path on web server from requests.
String path = request.getPath();
//Check 2: ENSURE PATH IS CORRECT!
System.out.println("path: " + path);
//Open connection to the web server
Socket s = new Socket(host, port);
//Get Socket input stream and wrap it in Buffered Reader so it can be read line by line.
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(s.getInputStream()));
//Get Socket output stream and wrap it in a DataOutputStream so it can be written to line by line.
DataOutputStream outToServer = new DataOutputStream(s.getOutputStream());
//Get request method
String method = request.getMethod();
//Check 3: ENSURE REQUEST IS CORRECT GET/POST!
System.out.println("Method: " + method);
//GET REQUEST
if(method.equalsIgnoreCase("GET")){
//Send request to server
outToServer.writeChars("GET " + path + " HTTP/1.0");
//HTTP RESPONSE
System.out.println("WAITING FOR RESPONSE!");
String line = inFromServer.readLine();
System.out.println("Line: " + line);
}
//Returning the response
return response;
}
}
I have checked to ensure my request line is constructed correctly, as seen in the print statements throughout. However when I get to this line the program hangs:
System.out.println("WAITING FOR RESPONSE!");
String line = inFromServer.readLine();
I have no idea why... My server is localhost WAMP. It is up and running correctly. I have the file I am requesting stored on the localhost. I can access it through browser.
Any ideas what might be going wrong??

No CR or LF is one of your problems. You should be writing ASCII characters and may be a Host header.
outToServer.write(("GET " + path + " HTTP/1.0\r\n").getBytes("ASCII"));
outToServer.write("Host: myhost.com\r\n\r\n".getBytes("ASCII"));
outToServer.flush();

Getting java.io.IOException: Error writing to server at getInputStream

If the temp string is very large I get java.io.IOException: Error writing to server at getInputStream
String tmp = js.deepSerialize(taskEx);
URL url = new URL("http://"
+ "localhost"
+ ":"
+ "8080"
+ "/Myproject/TestServletUpdated?command=startTask&taskeId=" +taskId + "'&jsonInput={\"result\":"
+ URLEncoder.encode(tmp) + "}");
URLConnection conn = url.openConnection();
InputStream is = conn.getInputStream();
Why is that?
This call goes to the servlet mentioned in the URL.

Use HTTP POST method instead of putting all the data in the URL for GET method. There is an upper limit for the length of the URL, so you need to use POST method if you want to send arbitrary length data.
You may want to modify the URL to http://localhost:8080/Myproject/TestServletUpdated, and put the rest
command = "startTask&taskeId=" + taskId + "'&jsonInput={\"result\":" + URLEncoder.encode(tmp) + "}"
in the body of the POST request.

I think you might have a "too long url", the maximum number of characters are 2000 (see this SO post for more info). GET requests are not made to handle such long data input.
You can, if you can change the servlet code also, change it into a POST instead of a GET request (as you have today). The client code would look pretty simular:
public static void main(String[] args) throws IOException {
URL url = new URL("http", "localhost:8080", "/Myproject/TestServletUpdated");
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write("command=startTask" +
"&taskeId=" +taskId +
"&jsonInput={\"result\":" + URLEncoder.encode(tmp) + "}");
wr.flush();
.... handle the answer ...
}
I didn't see it first but it seems like you have a single quote character in your request string.
...sk&taskeId=" + taskId + "'&jso.....
^
try removing it, it might help you!

getInputStream() is used to read data. Use getOutputStream()

It could be because the request is being sent as a GET which has a limitation of a very few characters. When the limit exceeds you get an IOException. Convert that to POST and it should work.
For POST
URLConnection conn = url.openConnection().
OutputStream writer = conn.getOutputSteam();
writer.write("yourString".toBytes());
Remove the temp string from the url that you are passing. Move the "command" string to the "yourString".toBytes() section in the code above

Java constructing an http request message

I asked a similar question in another thread but I think I'm just having trouble getting the syntax right at this point. I basically want to open a socket in Java, send a HTTP request message to get the header fields of a specific web page. My program looks like this so far:
String server = "www.w3.org";
int port = 80;
String uri = "/Protocols/rfc2616/rfc2616-sec5.html#sec5.1"
Socket socket = new Socket(server, port);
PrintStream output = new PrintStream(socket.getOutputStream());
BufferedReader socketInput = new BufferedReader(new InputStreamReader(socket.getInputStream()));
output.println("HEAD " + uri + " HTTP/1.1");
//String response = "";
String line = "";
while((line = socketInput.readLine()) != null){
System.out.println(line);
}
socketInput.close();
socket.close();
It doesn't really work. Or it doesn't work for all websites. If someone could just tell me the immediate problems with what I'm doing, that would be great. Thank you!

Change
output.println("HEAD " + uri + " HTTP/1.1");
to
output.println("HEAD " + uri + " HTTP/1.1");
output.println("Host: " + server);
output.println();
You have to send the Host header because usually there are more than one virtual host on one IP address. If you use HTTP/1.0 it works without the Host header.

I would use some higher-level component, like HttpURLConnection (see here) or apache http components.