We Keep Coding

Can't access server socket with android app

I want to see the exact headers my android app is sending while making a web request so I thought I'd simply create a simple server app in java on my local machine and have my android app make a call to it. Then simply dump the request to the console so I could see what the app is sending. However when I tried to connect, the app hangs and stops responding.
I created a simple server the only accepts a connection and sysouts the data it gets. The server runs fine and if I hit it from a web browser on my computer will print the headers from the web browsers request. So I know the server works fine.
Here's the code from my app:
URL url = new URL("http://192.168.1.11:9000");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.connect();
PrintWriter writer = new PrintWriter(connection.getOuputStream(), true);
writer.write("hi");
writer.close();
Simple. I only want the headers after all. Now I started without a post and using:
URL url = new URL("http://192.168.1.11:9000");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
in.close();
but that doesn't work. The app stops responding on the getInputStream() request. It just stops and won't continue. The server gets no connection request either.
So in all, the app is blocking on the url connection's getInputStream and I can't figure out why.
Now I've searched for awhile and found these:
Android app communicating with server via sockets
socket exception socket not connected android
Android embedded browser cant connect to server on LAN
Using java.net.URLConnection to fire and handle HTTP requests
Client Socket cannot connect to running Socket server
But nothing helps. I'm not using the localhost like everyone with this problem seems to be and I've tried using the androids 10.0.0.2 but that doesnt work either.
I'm not on a network that restricts anything (I'm home) and I've tried using the first set of code shown in order to send a message to my server but not even that works (it runs fine but the server never gets a client. Hows that work?).
I tried using both URLConnection and HttpURLConnection, they both have the same problem.
I'm also using the internet permission in my app, so it does have the permission needed.
I'm at a loss at this point. Why can't I make a simple call to my server?
EDIT
I used the exact code from androids documentation:
private String downloadUrl(String myurl) throws IOException {
InputStream is = null;
// Only display the first 500 characters of the retrieved
// web page content.
int len = 500;
try {
URL url = new URL("http://10.0.2.2:9000");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setReadTimeout(10000 /* milliseconds */);
conn.setConnectTimeout(15000 /* milliseconds */);
conn.setRequestMethod("GET");
conn.setDoInput(true);
// Starts the query
conn.connect();
int response = conn.getResponseCode();
is = conn.getInputStream();
// Convert the InputStream into a string
String contentAsString = readIt(is, len);
return contentAsString;
// Makes sure that the InputStream is closed after the app is
// finished using it.
} finally {
if (is != null) {
is.close();
}
}
}
but even that doesn't work. It still hangs. Only now it hangs on the getResponseCode(). Then throws a timeout exception. The server never gets a request though.

Your address must start with 'http://", try again!

I think the root of your issue is that Android is FCing your app before the connection completes, because I assume you haven't wrapped this in a Loader, AsyncTask or Thread. I suggest you follow the training guide Google provides, wrapping your call in an AsyncTask and seeing if that corrects the issue.

I have a Java class I use for making HTTP GET requests, I'm guessing its near identical to the android code your using so below I've dumped the relevant part of the code. I've used this class many times in Java applications (not on Android).
currentUrl = new URL(getUrl);
conn = (HttpURLConnection)currentUrl.openConnection();
conn.setRequestProperty("Cookie", getCookies(currentUrl.getHost()));
conn.setRequestProperty("User-Agent", "robadob.org/crawler");
if(referrer!=null){conn.setRequestProperty("Referrer",referrer);}
conn.setRequestMethod("GET");
conn.connect();
//Get response
String returnPage = "";
String line;
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
while ((line = rd.readLine()) != null) {
returnPage+=line+"\n";
}
rd.close();
I can't see anything obvious that would be causing your code to fail, but hopefully you can spot something from this. The setRequestProperty is me setting headers, so you shouldn't need those.
If that fails, flood your code with System.out's so you can see which statement its stalling at.

How can post data to form which is in other server

Iam creating automated system to post the data to the form for registering into the web site
URL url = new URL("https://www.walmart.com/subflow/YourAccountLoginContext/1471476370/sub_generic_login/create_account.do");
String postData = "firstName="+xlsDataList.get(0)+"&lastName="+xlsDataList.get(1)+"&userName="+xlsDataList.get(2)+"&userNameConfirm="+xlsDataList.get(3)+"&pwd="+xlsDataList.get(5)+"&pwdConfirm="+xlsDataList.get(6);
HttpsURLConnection uc = (HttpsURLConnection) url.openConnection();
uc.setDoInput(true);
uc.setDoOutput(true);
uc.setRequestMethod("POST");
uc.setRequestProperty("Accept", "*/*");
uc.setRequestProperty("Content-Length", Integer.toString(postData.getBytes().length));
uc.setRequestProperty("Content-Type", "text/html; charset=utf-8");
OutputStreamWriter outputWriter = new OutputStreamWriter(uc.getOutputStream());
outputWriter.write(postData);
outputWriter.flush();
outputWriter.close();
I thought that those above postdata are just request attributes , and coded accordingly. But after closely checking the view source, i came to know that those are form attributes.
I dnt have access to that form. Now how can i post the data to the form, so that the user get registered by the site?
i have to set the values to formbean.
Please provide your suggesions.

Your are using the wrong Content-Type in your POST: you need to use application/x-www-form-urlencoded. Once you change that, the server will interpret your request body as request parameters and (likely) your "formbean" will be filled with the data.
The above code may be a test case, but you really ought to take care to properly encode all of your data that you are trying to POST. Otherwise, you run the risk of either having a syntactically invalid request (in which case, the server will either reject the request, or ignore important parameters) or introducing a security vulnerability where a user can inject arbitrary request parameters into your POST. I highly recommend code that looks like this:
import java.net.URLEncoder;
String charset = "UTF-8"; // Change this if you want some other encoding
StringBuilder postData = new StringBuilder();
postData.append(URLEncoder.encode("firstName", charset));
postData.append("=");
postData.append(URLEncoder.encode(xlsDataList.get(0)), charset);
postData.append("&");
postData.append(URLEncoder.encode("lastName", charset));
postData.append("=");
postData.append(URLEncoder.encode(xlsDataList.get(1), charset));
postData.append("&");
postData.append(URLEncoder.encode("userName", charset));
postData.append("=");
postData.append(URLEncoder.encode(xlsDataList.get(2), charset));
postData.append("&");
postData.append(URLEncoder.encode("userNameConfirm", charset));
postData.append("=");
postData.append(URLEncoder.encode(xlsDataList.get(3), charset));
postData.append("&");
postData.append(URLEncoder.encode("pwd", charset));
postData.append("=");
postData.append(URLEncoder.encode(xlsDataList.get(5), charset));
postData.append("&");
postData.append(URLEncoder.encode("pwdConfirm", charset));
postData.append("=");
postData.append(xlsDataList.get(6), charset));
It seems silly to encode the static strings like "userNameConfirm", but if you get into that habit, you'll end up using it all the time and your code will be a lot safer.
Also, you need to make sure that the data you send through the OutputStream has the right Content-Length: you are computing the content-length properly, but then you aren't using the bytes you used for the computation to send to the client. You want your code to look more like this:
byte[] postDataBytes = postData.getBytes(charset);
uc.setRequestProperty("Content-Length", Integer.toString(postDataBytes.length));
uc.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
OutputStream outputStream = uc.getOutputStream();
outputStream.write(postDataBytes);
outputStream.flush();
outputStream.close();
You can find a very comprehensive HTTPUrlConnection tutorial in the community wiki: Using java.net.URLConnection to fire and handle HTTP requests

I recommend to use Apache HttpClient. its faster and easier to implement.
PostMethod post = new PostMethod("https://www.walmart.com/subflow/YourAccountLoginContext/1471476370/sub_generic_login/create_account.do");
NameValuePair[] data = {
new NameValuePair("firstName", "joe"),
new NameValuePair("lastName", "bloggs")
};
post.setRequestBody(data);
InputStream in = post.getResponseBodyAsStream();
// handle response.
For details you can refer http://hc.apache.org/

If your project uses Spring 3.x or later I would recommend using the Spring RestTemplate its pretty handy for doing http, code below will log do a form post.
public String login(String username, String password)
{
MultiValueMap<String, String> form = new LinkedMultiValueMap<>();
form.add(usernameInputFieldName, username);
form.add(passwordInputFieldName, password);
RestTemplate template = new RestTemplate();
URI location = template.postForLocation(loginUrl(), form);
return location.toString();
}

The "HTTP Error 500" you've described in your comment is an "Internal server error".
This means that the server either can't use your request (GET/POST) or there's a problem specific to the server you are trying to call.
Taking a look at the URL you're calling, I immediately the same Error 500.
Same happens for both GET and POST requests at httqs://www.walmart.com/subflow/YourAccountLoginContext/1471476370/sub_generic_login/create_account.do (Live link deactivated; replace "q" with "p" to make it work.)
In short: the generally returned "HTTP Error 500" from WallMart's servers prevents your call to succeed.
By the way:
It's not uncommon to get an error 500 instead of a 403 if they are locking your access down.
As you probably don't own the WallMart website and since you're trying to access levels of their websites that are worth to be protected from 3rd party acces, this might well be the case. ;)
PS: I'm not sure if it's wise to show the AccountLogin number in public like this. After all, it's the client ID of a specific WallMart account holder. But hey, that's your choice, not mine.

Also, double check the parameters you are sending. There may be some validations on input data the server is doing. Eg, some fields are mandatory, some are numbers only, etc.

Try spoofing as a browser by modifying the User Agent. WalMart may have a security mechanism that detects that you are doing this in an automated way.
(If you have problems setting the user agent see this post: Setting user agent of a java URLConnection)

403 error in accessing an URL but works fine in browsers

String url = "http://maps.googleapis.com/maps/api/directions/xml?origin=Chicago,IL&destination=Los+Angeles,CA&waypoints=Joplin,MO|Oklahoma+City,OK&sensor=false";
URL google = new URL(url);
HttpURLConnection con = (HttpURLConnection) google.openConnection();
and I use BufferedReader to print the content I get 403 error
The same URL works fine in the browser. Could any one suggest.

The reason it works in a browser but not in java code is that the browser adds some HTTP headers which you lack in your Java code, and the server requires those headers. I've been in the same situation - and the URL worked both in Chrome and the Chrome plugin "Simple REST Client", yet didn't work in Java. Adding this line before the getInputStream() solved the problem:
connection.addRequestProperty("User-Agent", "Mozilla/4.0");
..even though I have never used Mozilla. Your situation might require a different header. It might be related to cookies ... I was getting text in the error stream advising me to enable cookies.
Note that you might get more information by looking at the error text. Here's my code:
try {
HttpURLConnection connection = ((HttpURLConnection)url.openConnection());
connection.addRequestProperty("User-Agent", "Mozilla/4.0");
InputStream input;
if (connection.getResponseCode() == 200) // this must be called before 'getErrorStream()' works
input = connection.getInputStream();
else input = connection.getErrorStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(input));
String msg;
while ((msg =reader.readLine()) != null)
System.out.println(msg);
} catch (IOException e) {
System.err.println(e);
}

HTTP 403 is a Forbidden status code. You would have to read the HttpURLConnection.getErrorStream() to see the response from the server (which can tell you why you have been given a HTTP 403), if any.

This code should work fine. If you have been making a number of requests, it is possible that Google is just throttling you. I have seen Google do this before. You can try using a proxy to verify.

Most browsers automatically encode URLs when you enter them, but the Java URL function doesn't.
You should Encode the URL with URLEncoder URL Encoder

I know this is a bit late, but the easiest way to get the contents of a URL is to use the Apache HttpComponents HttpClient project: http://hc.apache.org/httpcomponents-client-ga/index.html

you original page (with link) and the targeted linked page are not the same domain.
original-domain and target-domain.
I found the difference is in request header:
with 403 forbidden error,
request header have one line:
Referer: http://original-domain/json2tree/ipfs/ipfsList.html
when I enter url, no 403 forbidden,
the request header does NOT have above line referer: original-domain
I finally figure out how to fix this error!!!
on your original-domain web page, you have to add
<meta name="referrer" content="no-referrer" />
it will remove or prevent sending the Referer in header, works both for links and for Ajax requests made

Better way to send lots of text to a server?

I have a java application that sends text to a sql database on a server. Currently my java application takes the text, puts it into the url, then sends it to a php page on the server that takes it with GET and puts it in the database. that works fine to an extent, the problem is, that i need to be able to send lots of text, and i keep getting 414, uri to long errors. is there a better way to do this?
ok, i tried what you said, and read the tutorial, but something is not working. here is my code that i tried
public void submitText(String urls,String data) throws IOException{
URL url = new URL(urls);
URLConnection con = url.openConnection();
con.setRequestProperty("Content-Type", "text/plain; charset=utf-8");
con.setDoOutput(true);
OutputStreamWriter out = new OutputStreamWriter(con.getOutputStream(), "UTF-8");
out.write(data);
out.flush();
}
submitText(server + "publicPB.php", "param=" + text);
here is my php code
$param = $_POST['param'];
$sql = "UPDATE table SET cell='{$param}' WHERE 1";
mysql_query($sql);
...
im pretty sure its not a problem with the php as the php worked fine with GET, and thats all i change with it, my problem i think is that im not 100% sure how to send data to it with the java

Use a POST instead of a GET and send the text as the request body. You can only pass so much data to a URL. E.g.:
// Assuming 'input' is a String and contains your text
URL url = new URL("http://hostname/path");
URLConnection con = url.openConnection();
con.setRequestProperty("Content-Type", "text/plain; charset=utf-8");
con.setDoOutput(true);
OutputStreamWriter out = new OutputStreamWriter(con.getOutputStream(), "UTF-8");
out.write(input);
out.close();
See Reading from and Writing to a URLConnection for more details.

Why don't you use POST to send data across to PHP page? GET does have a smaller limit of content.

Use POST requests, which do not have content length limits.

POST requests do not have length content limits and are much secure than GET requests ;)

If using SQL Server I would look into leveraging BCP. You can write the file and call BCP from within Java, and it will send the information directly to your database.

java.io.IOException: Server returned HTTP response code: 500

I'm facing this problem with Java. I want to get some HTML informations from a URL. This code was working for so long, but suddenly, it stopped working.
When I access this URL using the browser, it opens with no problem.
The code:
URL site = new URL(this.url);
java.net.URLConnection yc = site.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(yc.getInputStream()));
String inputLine;
String objetivo = "<td height=\"28\" colspan=\"2\"";
while ((inputLine = in.readLine()) != null && !inputLine.contains(objetivo)) {
}
inputLine = in.readLine();
The Exception:
java.io.IOException: Server returned HTTP response code: 500 for URL: http://www.myurl.com
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at Sites.websites.Site1.getData(Site1.java:53)
at util.Util.lerArquivo(Util.java:278)
at util.Util.main(Util.java:983)
What's wrong? Did the host block me?

HTTP status code 500 usually means that the webserver code has crashed. You need to determine the status code beforehand using HttpURLConnection#getResponseCode() and in case of errors, read the HttpURLConnection#getErrorStream() instead. It may namely contain information about the problem.
If the host has blocked you, you would rather have gotten a 4nn status code like 401 or 403.
See also:
How to use URLConnection to fire and handle HTTP requests?

This Status Code 500 is an Internal Server Error. This code indicates that a part of the server (for example, a CGI program) has crashed or encountered a configuration error.
i think the problem does'nt lie on your side, but rather on the side of the Http server.
the resources you used to access may have been moved or get corrupted, or its configuration just may have altered or spoiled

I had this problem i.e. works fine when pasted into browser but 505s when done through java. It was simply the spaces that needed to be escaped/encoded.

The problem must be with the parameters you are passing(You must be passing blank parameters). For example : http://www.myurl.com?id=5&name=
Check if you are handling this at the server you are calling.

Change the content-type to "application/x-www-form-urlencoded", i solved the problem.

You may look within the first server response and see if the server sent you a cookie.
To check if the server sent you a cookie, you can use HttpURLConnection#getHeaderFields() and look for headers named "Set-Cookie".
If existing, here's the solution for your problem. 100% Working for this case!

In my case, I had changed the Content-Type to Accept and it resolved the issue.
URL url = new URL(GET_URL);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("GET");
con.setRequestProperty("Accept", "application/json; charset=utf-8");