We Keep Coding

Getting URL from site after submitting form

So the site that Im targeting works like this:
The sites name is https://masterbattlerite.com/. https://masterbattlerite.com/ works like this:
Enter username in search bar of the site
Press the search button (or hit enter)
The site directs you to the userpage with the username you entered.
the URL is now something like https://masterbattlerite.com/238338348
that number (238338348) is the id of the user
I want to know how to get that id as a string in my app
I found out of Selendroid but you need some sort of server to be able to emulate a browser? My app will be released on the play store so I don't want my users needing to start their own selendroid server everytime to do this (maybe Im wrong)

In your code, can you use an HTTP library/framework and POST to the same location the site posts to and retrieve the id from the redirect the server sends? You may have to set some extra (possibly hidden on the webpage) parameters to get it to work.
EDIT: ok, it's even easier with the site you mentioned... if you search for a username of bob, you just GET this URL: http://masterbattlerite.com/profile/bob/lookup
and it returns some nice JSON with the id in it like so: {"status":"success","player":{"id":1268,"user_id":"5688","name":"bob","title":504,"avatar":30016}}
BTW - I figured all of this out using the Chrome developer's tools. Learn to use it - it is your friend.

Options for passing data across HTTP redirects

I am working on a Web application and need to pass data across HTTP redirects. For example:
http://foo.com/form.html
POSTs to
http://foo.com/form/submit.html
If there is an issue with the data, the Response is redirected back to
http://foo.com/form.html?error=Some+error+message
and the query param "error"'s value is displayed on the page.
Is there any other reliable way to pass data across redirects (ie HTTP headers, etc.).
Passing the data as query params works but isn't ideal because:
its cleartext (and in the query string, so SSL cant be relied on to encyrpt) so I wouldn't want to pass sensitive data
URIs are limited in length by the browser (albiet the length is generally fairly long).
IMPORTANT: This platform is state-less and distributed across many app servers, so I can't track the data in a server-side session object.

From the client-server interaction point of view, this is a server internal dispatch issue.
Browsers are not meant to re-post the entity of the initial request automatically according to the HTTP specification: "The action required MAY be carried out by the user agent without interaction with the user if and only if the method used in the second request is GET or HEAD."
If it's not already the case, make form.html dynamic so that it's an HTML static file. Send the POST request to itself and pre-fill the value in case of error. Alternatively, you could make submit.html use the same template as form.html if there is a problem.
its cleartext (and in the query string, so SSL cant be relied on to
encyrpt) so I wouldn't want to pass sensitive data
I'm not sure what the issue is here. You're submitting everything over plain HTTP anyway. Cookie, query parameters and request entity will all be visible. Using HTTPS would actually protect all this, although query parameters can still be an issue with browser history and server logs (that's not part of the connection, which is what TLS protects).

I think using cookies would be a reasonable solution depending on the amount of data. As you can't track it on the server side (by using a sessions for example, which would be much simpler)

You can store error message in database on server and reference to it by id:
http://foo.com/form.html?error_id=42
If error texts are fixed you even don't need to use a database.
Also, you can use Web Storage. Instead of redirection with "Location" header you can display output page with this JavaScript:
var error_message = "Something is wrong";
if( typeof(Storage) !== "undefined" ) {
localStorage.error_message = error_message;
else {
// fallback for IE < 8
alert(error_message);
}
location.href = "new url";
And after redirection you can read localStorage.error_message using JavaScript and display the message.

how do i know whether my html link is clicked or not from Servlet?

My Servlet response type is html and my response contains a hyperlink to another web site.So, now i want to capture the information about whether the user clicked the link or not? and also calculate the total clicks? i am using Tomcat 7 as a server.
Is this possible in setting response header (302 or 404)?...
Please Guide me to get out of this issue?

Yes, you can use a 302: instead of providing the link to the other website, you provide a link to your own servlet, do your accounting and then send back a redirection (301/302) http status with the other web-site URL in the response Location header.
This maybe a bit simplistic though, since the user will leave your original page (is this what you want ?) and search engines may not like this if your web app is public.

I think right now you are redirecting the request(link for another website) at client side.In this approach your server cannot get the information about the click.
What you can do create a servlet and call this servlet on click now this servlet is responsible to redirect the request to another website. Add an static integer counter and increment this when servlet call each time.

Use the method setStatus():-
setStatus(HttpServletResponse.SC_FOUND);
or
setStatus(HttpServletResponse.SC_NOT_FOUND);

Simple example of JQuery Address to manage application state

I'm using the jQuery Address library to re-write my URL depending on what the user is doing in the page. The intention is to allow users to bookmark pages and come back later. The page never refreshes as all server interaction is done via ajax.
jQuery Address is writing URLs like this:
http://localhost:9000/#/u/scott_tiger
I need a to set up a route in Play to be able to route that request through to the appropriate controller. So I set this up:
GET /#/u/{username} Controller.showUser
This doesn't work though, the route definition gets ignored. I've tried loads of things such as trying to escape the "#" and replacing it with a variable that I've populated with Character.toString(35). None of this works.
Does anyone know how I can either define the route properly or get jQuery Address not to write the "#".
EDIT: The "#" doesn't get sent to the server does it. Doh! OK, question is revised.

No. The # and the part of the URL after that is not sent to the server. So your play app on the server will never see such URLs.
HTML5 solution
You need to handle these URLs on the client side using JavaScript. In modern browsers with good HTML5 support, you can modify the address without reloading the page. See Manipulating the browser history on how to do it for these browsers. And see When can I use... for browser support.
#-URLs
On Internet Explorer and older versions of other browsers you need to use # URLs and use JavaScript to load the state (e.g. get the user page /u/scott_tiger in your example). See How to run a JavaScript function when the user is visiting an hash link (#something) using JQuery? for how to do this in JavaScript. Also if a user bookmarks a page with a #-URL you need to reload the state.
See also: What's the shebang/hashbang (#!) in Facebook and new Twitter URLs for?
JavaScript libraries
You may use JavaScript libraries to handle this for you history.js is an example. Also bigger frameworks like Backbone.js handle this.
Does anyone know how I can get jQuery Address not to write the "#".
If you don't write the #-part of the URL, the state can not be linked. So you can not get back to e.g. Scott Tigers profile page if you bookmark the page, because the URL is only http://localhost:9000/ and you will arrive on the front page, while the user though he would arrive on the profile page.

Armed with my new understanding of URLs (thanks #Jonas) I realised that I'd missed half of the story.
I'm using JQuery Address to change the URL depending on what you click in the application. This works great and on lots of browsers. What I was missing was using JQuery Address to watch for external address changes (bookmarks, history, back/forward) and respond accordingly. i.e. set the page up correctly by firing the appropriate Ajax calls and rendering that data appropriately.
Changing the address
$.address.title("new title describing application state");
$.address.parameter("q", "val1");
$.address.parameter("g", "val2");
$.address.update();
Restoring the state
$.address.externalChange(function(event) {
var val1 = event.parameters["q"];
var val2 = event.parameters["g"];
// do something with those values
});

Why do I get an error 500 when I send POST data to an ASP.NET MVC site through Android?

I'm trying to create an android app to check my tests scores of my engineering school. In order to download the Word containing the scores, I need to login to the portal.
I thought it would be simple to do it by sending a POST request.
After bypassing the problem of the self-signed certificate (or whatever) thanks to the code on this page : Self-signed SSL acceptance on Android
I still get an 500 error while trying to send any POST request to the login page, which is here : https://e-campus.hei.fr/ERP-prod/pc_mv_login.aspx
I tried various codes from the web to send the POST data (especially How to do a HTTP Post in Android? this one). And even on a pure java app, I get a 500.
When I point the URL to another testing page, I manage to get it working, but not on https://e-campus.hei.fr/ERP-prod/pc_mv_login.aspx
Could anyone explain to me why it doesn't work or help me get rid of this error ?
EDIT:
This is what is being sent through my browser (According to chrome developper tools)
__EVENTTARGET:
__EVENTARGUMENT:
__VIEWSTATE:dDwxNDU4ODc4MDI5O3Q8O2w8aTwwPjs+O2w8dDw7bDxpPDE+O2k8Nz47aTwxMz47aTwxNT47aTwxNz47aTwxOT47PjtsPHQ8cDxwPGw8VGV4dDs+O2w8SWRlbnRpZmlhbnQgOjs+Pjs+Ozs+O3Q8cDxwPGw8VGV4dDs+O2w8TW90IGRlIHBhc3NlIDo7Pj47Pjs7Pjt0PHA8cDxsPFRleHQ7PjtsPExhbmd1ZSA6Oz4+Oz47Oz47dDx0PDt0PGk8Mj47QDxBbmdsYWlzO0ZyYW7Dp2Fpczs+O0A8ZW47ZnI7Pj47bDxpPDE+Oz4+Ozs+O3Q8cDxwPGw8VGV4dDs+O2w8Vm91cyBuJ8OqdGVzIHBhcyBhdXRvcmlzw6kgIMOgIHZvdXMgY29ubmVjdGVyLjs+Pjs+Ozs+O3Q8cDxwPGw8VGV4dDs+O2w8Q29ubmVjdGVyIDo7Pj47Pjs7Pjs+Pjs+Pjs+inmhCwE9zfymuEXDXGORShkB1GI=
Username:******
Password:******
Langues:fr
Button1:Connecter :
This is the string that i send :
String parameters = "__EVENTTARGET=&__EVENTARGUMENT=&__VIEWSTATE="
+ URLEncoder
.encode("dDwxNDU4ODc4MDI5O3Q8O2w8aTwwPjs+O2w8dDw7bDxpPDE+O2k8Nz47aTwxMz47aTwxNT47aTwxNz47aTwxOT47PjtsPHQ8cDxwPGw8VGV4dDs+O2w8SWRlbnRpZmlhbnQgOjs+Pjs+Ozs+O3Q8cDxwPGw8VGV4dDs+O2w8TW90IGRlIHBhc3NlIDo7Pj47Pjs7Pjt0PHA8cDxsPFRleHQ7PjtsPExhbmd1ZSA6Oz4+Oz47Oz47dDx0PDt0PGk8Mj47QDxBbmdsYWlzO0ZyYW7Dp2Fpczs+O0A8ZW47ZnI7Pj47bDxpPDE+Oz4+Ozs+O3Q8cDxwPGw8VGV4dDs+O2w8Vm91cyBuJ8OqdGVzIHBhcyBhdXRvcmlzw6kgIMOgIHZvdXMgY29ubmVjdGVyLjs+Pjs+Ozs+O3Q8cDxwPGw8VGV4dDs+O2w8Q29ubmVjdGVyIDo7Pj47Pjs7Pjs+Pjs+Pjs+inmhCwE9zfymuEXDXGORShkB1GI=",
"UTF-8") + "&Username="
+ URLEncoder.encode(mUsername, "UTF-8") + "&Password="
+ URLEncoder.encode(mPassword, "UTF-8")
+ "&Langues=fr&Button1="
+ URLEncoder.encode("Connecter :", "UTF-8");

HTTP error 500 just means that the server side code failed. It has a bug, for example a NullPointerException was been thrown over there. If the response body doesn't contain anything sensible (e.g. a stacktrace) so that you could learn how it is caused and so change the request accordingly, then your best bet is to contact the server admin and report about this bug in the server code and ask how to correctly perform a programmatic login.
If that is not an option for some reason, then you should doublecheck if you don't forget to send a specific cookie, header and/or parameter. Probably the server side code was expecting it, but it was null and the code was buggy and hence it totally broke with a 500. I'd suggest to use Firebug to track the entire HTTP traffic and compare it with the headers/parameters you've set. Probably you need to send a specific cookie back? Or you need to send the name=value pair of the submit button? Etcetera.
Update: you're sending the wrong __VIEWSTATE value along. The website runs on ASP.NET MVC which is a component based MVC framework (like as JSF in Java EE). It stores the component tree as "view state". You should not send a random/non-existing/invalidated view state back as paramter, but a valid one. You need to rewrite the HTTP client so that it first fires a GET request on the page with the form and then use a HTML parser (Jsoup?) to extract the value of the hidden __VIEWSTATE input field and finally fire a POST request with exactly that value (and exactly the same cookie in the request header!).
Like as in JSF, the view state is part of CSRF attack prevention. You cannot submit the form without first requesting the form from the website itself in the same session.