We Keep Coding

HTTP/1.1 400 Bad Request using base64 encoding in java

I am using http request. I have two image to call api. Image are base64encoding..but in postman it is working fine. It is not working in my java code..it shows bellow error
response = HTTP/1.1 400 Bad Request [Server: nginx/1.14.0 (Ubuntu), Date: Fri, 13 Nov 2020 09:24:20 GMT, Content-Type: application/json; charset=utf-8, Content-Length: 3866, Connection: keep-alive, X-Powered-By: Express, Access-Control-Allow-Origin: *, Access-Control-Expose-Headers: Origin, X-Requested-With, Content-Type, Accept, x-auth-token, x-login-token, x-verification-token, ETag: W/"f1a-xqn4nWVQUqRNj8g2mv5av6fbTrg"]
statusCode = 400
Postman header image
Postman body
I have used bellow code..
byte[] photo1=null;
byte[] photo2=null;
String base64encodedString = Base64.getEncoder().encodeToString(photo1);
String base64encodedString2 = Base64.getEncoder().encodeToString(photo2);
try {
DefaultHttpClient httpclient = new DefaultHttpClient(new BasicHttpParams());
HttpPost httppost = new HttpPost("url");
JSONObject jsonObject = new JSONObject();
try {
jsonObject.put("photo", base64encodedString);
jsonObject.put("nidFront", base64encodedString2);
} catch (Exception e) {
e.printStackTrace();
}
System.out.println("jsonObject = " + jsonObject);
StringEntity params = new StringEntity(String.valueOf(jsonObject));
httppost.addHeader("content-type", "application/json");
httppost.setHeader("x-auth-token", token);
httppost.setEntity(params);
HttpResponse response = httpclient.execute(httppost);
int statusCode = response.getStatusLine().getStatusCode();
System.out.println("statusCode = " + statusCode);
Please help me what is wrong in code

Apache HttpClient 4.5.5 - wrong headers in response after a GET request

When I'm using Apache HttpClient and loading a webpage via GET request after the page is loaded in the response I have the headers that are different from ones I have when loading the same page in browser. Here is the example of the page: http://empoweredfoundation.org/wp-login.php?action=register
In browser I have the following headers:
Status code: 302
Content-Type: text/html; charset=UTF-8
X-Port: port_10210
X-Cacheable: YES:Forced
Location: http://empoweredfoundation.org/register/
Content-Encoding: gzip
Transfer-Encoding: chunked
Date: Thu, 22 Feb 2018 04:04:35 GMT
Age: 0
Vary: User-Agent
X-Cache: uncached
X-Cache-Hit: MISS
X-Backend: all_requests
When I use HttpClient in my application I have these headers in response:
Status code: 200
Content-Type: text/html; charset=UTF-8
X-Port: port_10210
X-Cacheable: YES:Forced
Transfer-Encoding: chunked
Date: Thu, 22 Feb 2018 04:44:58 GMT
Age: 28224
Vary: Accept-Encoding, User-Agent
X-Cache: cached
X-Cache-Hit: HIT
X-Backend: all_requests
Server: nginx/1.12.1
Date: Thu, 22 Feb 2018 04:45:24 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.4.45
So, I should have a 302 status code but I have 200. And also I see other headers are different than the ones from the browser. I can't figure out why and what should I do to fix this.
Here is the code:
HttpClient httpclient = null;
HttpClientBuilder builder = HttpClients.custom();
Builder requestConfigBuilder = RequestConfig.custom();
// here goes the cookie store creation, ssl configuration etc
builder.setDefaultRequestConfig(requestConfigBuilder.build());
httpclient = builder.build();
HttpResponse response = null;
HttpGet httpget = null;
Escaper escaper = UrlEscapers.urlFragmentEscaper();
httpget = new HttpGet(escaper.escape(url));
httpget.getParams().setParameter("http.socket.timeout", new Integer(socketTimeout));
httpget.getParams().setParameter("http.connection.timeout", new Integer(connectTimeout));
httpget.addHeader("Accept", "text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1");
httpget.addHeader("Accept-Language", "en-US,en;q=0.9");
httpget.addHeader("Accept-Encoding", "identity, *;q=0");
response = httpclient.execute(httpget);
I also tried CloseableHttpClient, had the same result.

I resolved this issue, this solution works: https://memorynotfound.com/apache-httpclient-redirect-handling-example/
I still have 200 status code, not 302. But now I can handle 302 redirects (even when response.getStatusLine() shows 200).
Here is the code from the article:
package com.memorynotfound.httpclient;
import org.apache.http.HttpHost;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.client.utils.URIUtils;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.client.LaxRedirectStrategy;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.List;
/**
* This example demonstrates the use of {#link HttpGet} request method.
* and handling redirect strategy with {#link LaxRedirectStrategy}
*/
public class HttpClientRedirectHandlingExample {
public static void main(String... args) throws IOException, URISyntaxException {
CloseableHttpClient httpclient = HttpClients.custom()
.setRedirectStrategy(new LaxRedirectStrategy())
.build();
try {
HttpClientContext context = HttpClientContext.create();
HttpGet httpGet = new HttpGet("http://httpbin.org/redirect/3");
System.out.println("Executing request " + httpGet.getRequestLine());
System.out.println("----------------------------------------");
httpclient.execute(httpGet, context);
HttpHost target = context.getTargetHost();
List<URI> redirectLocations = context.getRedirectLocations();
URI location = URIUtils.resolve(httpGet.getURI(), target, redirectLocations);
System.out.println("Final HTTP location: " + location.toASCIIString());
} finally {
httpclient.close();
}
}
}
And also I added builder.setRedirectStrategy(new LaxRedirectStrategy()); when created the HttpClient class object.
If you know any solution to get the correct status code (which should be 302), please tell me.

Apache HTTP Client Post appears to work but variable not received

I have the following code that executes a httpclient post request
public void upload() throws Exception{
//HTTP POST Service
try{
HttpClient httpclient = HttpClientBuilder.create().build();
URI uri = new URIBuilder()
.setScheme("http")
.setHost("www.mysite.com")
.setPath("/mypage.php")
.setParameter("Username", userID)
.setParameter("Password", password)
.build();
HttpPost httppost = new HttpPost(uri);
httpclient.execute(httppost);
BasicHttpContext localContext = new BasicHttpContext();
HttpResponse response = httpclient.execute(httppost, localContext);
HttpUriRequest currentReq = (HttpUriRequest) localContext.getAttribute(ExecutionContext.HTTP_REQUEST);
HttpHost currentHost = (HttpHost)localContext.getAttribute(ExecutionContext.HTTP_TARGET_HOST);
String currentUrl = currentHost.toURI() + currentReq.getURI();
System.out.println(currentUrl);
System.out.println(response);
HttpEntity httpEntity = response.getEntity();
String str = "";
if (httpEntity != null) {
str = EntityUtils.toString(httpEntity);
System.out.println(str);
}
}catch (Exception e) {
e.printStackTrace();
}
}
which returns
HTTP/1.1 200 OK [Date: Sat, 11 Jan 2014 16:17:22 GMT, Server: Apache, Expires: Thu, 19 Nov 1981 08:52:00 GMT, Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0, Pragma: no-cache, Vary: Accept-Encoding, Keep-Alive: timeout=10, max=30, Content-Type: text/html, Via: 1.1 NDC1-B2-CE01, Connection: keep-alive]
As if everything had worked fine but my php script on the other end doesn't seem to pick up the variable.
I've tried something as simple as:
<?php
error_log($_POST["Username"]);
?>
But get an index undefined error printed

You are setting the query parameters of the URI which builds a URI your URI like http://www.mysite.com/mypage.php?Username=userId&Password=pass
You need to set the parameters of the HttpPost with NameValuePair.
HttpPost post = new HttpPost("http://www.mysite.com/mypage.php");
List<NameValuePair> params = new ArrayList<NameValuePair>();
params.add(new BasicNameValuePair("Username", userId));
params.add(new BasicNameValuePair("Password", pass));
post.setEntity(new UrlEncodedFormEntity(params));
HttpResponse response = post.execute(post);
Additionally, I would recommend handling authentication with an Authorization header, such as Basic authentication as well as sending credentials over HTTPS.

Download a cookie to make new GET request

I am trying to do a PHP GET request to a website:
The problem is that this website will only process my request if I attach Cookie information to the header of the request.
Or in picture terms, if I disable cookies in my browser, I get this:
Which means the website recognises that it's my first time 'visiting' the site.
Problem is, that if I now use the search bar on the top right, it will not process this request:
it will just show the same (general) screen.
E.g.: if I have cookies disabled and I search for "AAPL", it will not show any results.
Now if I have cookies enabled, the request is handled just fine:
And so the "AAPL" results are shown.
You can try this yourself as well:
With cookies enabled, visit http://www.pennystocktweets.com/user_posts/feeds?cat=search&lptyp=prep&usrstk=AAPL
With cookies disabled, visit the link again: http://www.pennystocktweets.com/user_posts/feeds?cat=search&lptyp=prep&usrstk=AAPL
Now compare the responses, only the first one is correct.
This means that the website only works after the client has downloaded a cookie, and then has made another (new) GET request to the server with this Cookie information attached.
(Does this imply that the website needs a session-cookie to function correctly?)
Now what I'm trying to do is imitate the request with Apache HttpClient like so:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import java.net.CookieHandler;
import java.net.CookieManager;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.Date;
import java.util.List;
import java.util.StringTokenizer;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.HttpClient;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.BasicNameValuePair;
public class downloadTweets {
private String cookies;
private HttpClient client = new DefaultHttpClient();
private final String USER_AGENT = "Mozilla/5.0";
public static void main(String[] args) throws Exception {
String ticker = "AAPL";
String lptyp = "prep";
int opid = 0;
int lpid = 0;
downloadTweets test = new downloadTweets();
String url = test.constructURL(ticker, lptyp, opid, lpid);
// make sure cookies is turn on
CookieHandler.setDefault(new CookieManager());
downloadTweets http = new downloadTweets();
String page = http.GetPageContent(url, ticker);
System.out.println(page);
}
public String constructURL(String ticker, String lptyp, int opid, int lpid)
{
String link = "http://www.pennystocktweets.com/user_posts/feeds?cat=search" +
"&lptyp=" + lptyp +
"&usrstk=" + ticker;
if (opid != 0)
{
link = link +
"&opid=" + opid +
"&lpid=" + lpid;
}
return link;
}
private String GetPageContent(String url, String ticker) throws Exception {
HttpGet request = new HttpGet(url);
String RefererLink = "http://www.pennystocktweets.com/search/post/" + ticker.toUpperCase();
request.setHeader("Host", "www.pennystocktweets.com");
request.setHeader("Connection", "Keep-alive");
request.setHeader("Accept", "*/*");
request.setHeader("X-Requested-With", "XMLHttpRequest");
request.setHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57 Safari/537.36");
request.setHeader("Referer", RefererLink);
request.setHeader("Accept-Language", "nl-NL,nl;q=0.8,en-US;q=0.6,en;q=0.4,fr;q=0.2");
HttpResponse response = client.execute(request);
int responseCode = response.getStatusLine().getStatusCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
BufferedReader rd = new BufferedReader(
new InputStreamReader(response.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
result.append(line);
}
// set cookies
setCookies(response.getFirstHeader("Set-Cookie") == null ? "" :
response.getFirstHeader("Set-Cookie").toString());
return result.toString();
}
public String getCookies() {
return cookies;
}
public void setCookies(String cookies) {
this.cookies = cookies;
}
}
Now, the same thing holds: if I attach (my) cookie information, the response works just fine, and if I don't the response doesn't work.
But I don't know how to get the cookie information and then use it in a new GET request.
So my question is:
How can I make 2 requests to a website such that:
On the first GET request, I get cookie information from the website and store this in my Java program
On the second GET request, I use the stored cookie information (as a Header) to make a new request.
Note: I don't know if the cookie is a normal cookie or a session cookie but I suspect it's a session-cookie!
All help is greatly appreciated!

As the documents of Apache commons httpclient states in the HttpClient Cookie handling part:
HttpClient supports automatic management of cookies, including allowing the server to set cookies and automatically return them to the server when required. It is also possible to manually set cookies to be sent to the server.
Whenever the http client receives cookies they are persisted into HttpState and added automatically to the new request. This is the default behavior.
In the following example code, we can see the cookies returned by two GET requests. We can't see directly the cookies sent to the server, but we can use a tool such as a protocol/net sniffer or ngrep to see the data transmitted over the network:
import java.io.IOException;
import org.apache.commons.httpclient.Cookie;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpException;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.HttpState;
import org.apache.commons.httpclient.cookie.CookiePolicy;
import org.apache.commons.httpclient.methods.GetMethod;
public class HttpTest {
public static void main(String[] args) throws HttpException, IOException {
String url = "http://www.whatarecookies.com/cookietest.asp";
HttpClient client = new HttpClient();
client.getParams().setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY);
HttpMethod method = new GetMethod(url);
int res = client.executeMethod(method);
System.out.println("Result: " + res);
printCookies(client.getState());
method = new GetMethod(url);
res = client.executeMethod(method);
System.out.println("Result: " + res);
printCookies(client.getState());
}
public static void printCookies(HttpState state){
System.out.println("Cookies:");
Cookie[] cookies = state.getCookies();
for (Cookie cookie : cookies){
System.out.println(" " + cookie.getName() + ": " + cookie.getValue());
}
}
}
This is the output:
Result: 200
Cookies:
active_template::468: %2Fresponsive%2Fthree_column_inner_ad3b74de5a1c2f311bee7bca5c368aaa4e:b326b5062b2f0e69046810717534cb09
Result: 200
Cookies:
active_template::468: %2Fresponsive%2Fthree_column_inner_ad%2C+3b74de5a1c2f311bee7bca5c368aaa4e%3Db326b5062b2f0e69046810717534cb09
3b74de5a1c2f311bee7bca5c368aaa4e: b326b5062b2f0e69046810717534cb09
Here is an excerpt of ngrep:
MacBook$ sudo ngrep -W byline -d en0 "" host www.whatarecookies.com
interface: en0 (192.168.11.0/255.255.255.0)
filter: (ip) and ( dst host www.whatarecookies.com )
#####
T 192.168.11.70:56267 -> 54.228.218.117:80 [AP]
GET /cookietest.asp HTTP/1.1.
User-Agent: Jakarta Commons-HttpClient/3.1.
Host: www.whatarecookies.com.
.
####
T 54.228.218.117:80 -> 192.168.11.70:56267 [A]
HTTP/1.1 200 OK.
Server: nginx/1.4.0.
Date: Wed, 27 Nov 2013 10:22:14 GMT.
Content-Type: text/html; charset=iso-8859-1.
Content-Length: 36397.
Connection: keep-alive.
Vary: Accept-Encoding.
Vary: Cookie,Host,Accept-Encoding.
Set-Cookie: active_template::468=%2Fresponsive%2Fthree_column_inner_ad; expires=Fri, 29-Nov-2013 10:22:01 GMT; path=/; domain=whatarecookies.com; httponly.
Set-Cookie: 3b74de5a1c2f311bee7bca5c368aaa4e=b326b5062b2f0e69046810717534cb09; expires=Thu, 27-Nov-2014 10:22:01 GMT.
X-Middleton-Response: 200.
Cache-Control: max-age=0, no-cache.
X-Mod-Pagespeed: 1.7.30.1-3609.
.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/1998/REC-html40-19980424/loose.dtd">
...
##
T 192.168.11.70:56267 -> 54.228.218.117:80 [AP]
GET /cookietest.asp HTTP/1.1.
User-Agent: Jakarta Commons-HttpClient/3.1.
Host: www.whatarecookies.com.
Cookie: active_template::468=%2Fresponsive%2Fthree_column_inner_ad.
Cookie: 3b74de5a1c2f311bee7bca5c368aaa4e=b326b5062b2f0e69046810717534cb09.
.
##
T 54.228.218.117:80 -> 192.168.11.70:56267 [A]
HTTP/1.1 200 OK.
Server: nginx/1.4.0.
Date: Wed, 27 Nov 2013 10:22:18 GMT.
Content-Type: text/html; charset=iso-8859-1.
Content-Length: 54474.
Connection: keep-alive.
Vary: Accept-Encoding.
Vary: Cookie,Host,Accept-Encoding.
Set-Cookie: active_template::468=%2Fresponsive%2Fthree_column_inner_ad%2C+3b74de5a1c2f311bee7bca5c368aaa4e%3Db326b5062b2f0e69046810717534cb09; expires=Fri, 29-Nov-2013 10:22:05 GMT; path=/; domain=whatarecookies.com; httponly.
Set-Cookie: 3b74de5a1c2f311bee7bca5c368aaa4e=b326b5062b2f0e69046810717534cb09; expires=Thu, 27-Nov-2014 10:22:05 GMT.
X-Middleton-Response: 200.
Cache-Control: max-age=0, no-cache.
X-Mod-Pagespeed: 1.7.30.1-3609.
.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/1998/REC-html40-19980424/loose.dtd">
...

HttpClient 4.x how to use cookies?

I'm trying to use the cookies I get in a response to my post method using HttpClient 4.0.3;
Here is my code:
public void generateRequest()
{
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost("http://mysite.com/login");
httpclient.getParams().setParameter("http.useragent", "Custom Browser");
httpclient.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION,
HttpVersion.HTTP_1_1);
httpclient.getParams().setParameter(ClientPNames.COOKIE_POLICY,
CookiePolicy.BROWSER_COMPATIBILITY);
CookieStore cookieStore = new BasicCookieStore();
HttpContext localContext = new BasicHttpContext();
localContext.setAttribute(ClientContext.COOKIE_STORE, cookieStore);
try
{
LOG.info("Status Code: sending");
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(2);
nameValuePairs.add(new BasicNameValuePair("email", "john%40gmail.com"));
nameValuePairs.add(new BasicNameValuePair("password", "mypassword"));
httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
httppost.setHeader("ContentType", "application/x-www-form-urlencoded");
HttpResponse response = httpclient.execute(httppost, localContext);
HttpEntity entity = response.getEntity();
if (entity != null)
{
entity.consumeContent();
}
iterateCookies(httpclient);
}
catch (ClientProtocolException e)
{
LOG.error("ClientProtocolException", e);
}
catch (IOException e)
{
LOG.error("IOException", e);
}
}
private void iterateCookies(DefaultHttpClient httpclient)
{
List<Cookie> cookies = httpclient.getCookieStore().getCookies();
if (cookies.isEmpty())
{
System.out.println("No cookies");
}
else
{
for (Cookie c : cookies)
{
System.out.println("-" + c.toString());
}
}
}
But I keep getting the No cookies logged out even though when I use web-sniffer.net, I get this response:
Status: HTTP/1.1 302 Found
Cache-Control: private, no-store
Content-Type: text/html; charset=utf-8
Location: http://www.mysite.com/loginok.html
Server: Microsoft-IIS/7.0
X-AspNet-Version: 2.0.50727
Set-Cookie: USER=DDA5FF4E1C30661EC61CFA; domain=.mysite.com; expires=Tue, 08-Jan-2013 18:39:53 GMT; path=/
Set-Cookie: LOGIN=D6CC13A23DCF56AF81CFAF; domain=.mysite.com; path=/ Date: Mon, 09 Jan 2012 18:39:53 GMT
Connection: close
Content-Length: 165
All the examples I've found online that make any sort of sense refer to HttpClient 3.x where you can set the CookiePolicy to IGNORE and handle the Set-Cookie header manually. I can't understand why this is so difficult in 4.x. I need access to the USER hash for a number of reasons. Can anyone please tell me how in the hell I can get access to it?
UPDATE
I have found the following C# code which does the same thing and works correctly.
private static string TryGetCookie(string user, string pass, string baseurl)
{
string body = string.Format("email={0}&password={1}", user, pass);
byte[] bodyData = StringUtils.StringToASCIIBytes(body);
HttpWebRequest req = WebRequest.Create(baseurl) as HttpWebRequest;
if (null != req.Proxy)
{
req.Proxy.Credentials = CredentialCache.DefaultCredentials;
}
req.AllowAutoRedirect = false;
req.Method = "Post";
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = bodyData.Length;
using (Stream reqBody = req.GetRequestStream())
{
reqBody.Write(bodyData, 0, bodyData.Length);
reqBody.Close();
}
HttpWebResponse resp1 = req.GetResponse() as HttpWebResponse;
string cookie = resp1.Headers["Set-Cookie"];
if( string.IsNullOrEmpty(cookie))
{
if (0 < resp1.ContentLength)
{
// it's probably not an event day, and the server is returning a singlecharacter
StreamReader stringReader = new StreamReader(resp1.GetResponseStream());
return stringReader.ReadToEnd();
}
return null;
}
return ParseCookie(cookie);
}
I believe my java code is not forming the post request correctly because when I use a URLConnection and print the request header from web-sniffer.net below:
POST /reg/login HTTP/1.1[CRLF]
Host: live-timing.formula1.com[CRLF]
Connection: close[CRLF]
User-Agent: Web-sniffer/1.0.37 (+http://web-sniffer.net/)[CRLF]
Accept-Charset: ISO-8859-1,UTF-8;q=0.7,*;q=0.7[CRLF]
Cache-Control: no-cache[CRLF]
Accept-Language: de,en;q=0.7,en-us;q=0.3[CRLF]
Referer: http://web-sniffer.net/[CRLF]
Content-type: application/x-www-form-urlencoded[CRLF]
Content-length: 53[CRLF]
[CRLF]
email=john%40gmail.com&password=mypassword
I get a response from the server that contains the set-cookies header. Is my java code not generating the request the same as web-sniffer.net?
I have seen a post method generated using this code:
PostMethod authPost = new PostMethod("http://localhost:8000/webTest/j_security_check");
// authPost.setFollowRedirects(false);
NameValuePair[] data = {
new NameValuePair("email", "john%40gmail.com"),
new NameValuePair("password", "mypassword")
};
authPost.setRequestBody(data);
status = client.executeMethod(authPost);
The main difference here being that the NameValuePair data is set in the request body rather than set as the entity. Does this make a difference? Would this produce the correct request header?

Both cookies look suspicious. Both use outdated Netscape cookie draft format. Both have invalid domain attribute value. The LOGIN appears malformed (semicolon is missing after the path attribute) on top of that. So, most likely both cookies got rejected by HttpClient.
You can find out whether this is the case by running HttpClient with the context logging turned on as described here:
http://hc.apache.org/httpcomponents-client-ga/logging.html
One last remark. Generally one should not meddle with cookie policies when using HttpClient 4.x. The default BEST_MATCH policy will automatically delegate processing of cookies to a particular cookie spec implementation based on the composition of the Set-Cookie header value. In order to disable cookie processing entirely one should remove cookie processing protocol interceptors from the protocol processing chain.
Hope this helps.

I believe the problem is that you are mixing two "styles" here: on one hand you create your own BasicCookieStore and put that in your HttpContext; on the other hand, when print the cookies, you loop over the cookie store in the DefaultHttpClient.
So either change the iterateCookies to use your local cookie store, or just use the one provided by the DefaultHttpClient. As you can see in the javadoc of DefaultHttpClient, it should automatically add response cookies to its internal cookie store.

It's always a simple answer! After debugging the C# and another C program I found. I was jumping the gun and doing my own encoding on the email address to remove the # character. This was the problem!!! Nothing else seemed to make a difference whether it was there or not! The code now looks like:
public void postData(final String email, final String password)
{
DefaultHttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(LOGIN_URL);
client.getParams().setParameter("http.useragent", "Custom Browser");
client.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
try
{
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(2);
nameValuePairs.add(new BasicNameValuePair("email", email));
nameValuePairs.add(new BasicNameValuePair("password", password));
UrlEncodedFormEntity entity = new UrlEncodedFormEntity(nameValuePairs, HTTP.UTF_8);
entity.setContentType("application/x-www-form-urlencoded");
post.setEntity(entity);
printHeaders(client.execute(post));
printCookies(client.getCookieStore());
}
catch (ClientProtocolException e)
{
LOG.error("ClientProtocolException", e);
}
catch (IOException e)
{
LOG.error("IOException", e);
}
}
and the output now looks like:
Response Headers:
-Cache-Control: private, no-store
-Content-Type: text/html; charset=utf-8
-Server: Microsoft-IIS/7.0
-X-AspNet-Version: 2.0.50727
-Set-Cookie: USER=3D907C0EB817FD7...92F79616E6E96026E24; domain=.mysite.com; expires=Thu, 10-Jan-2013 20:22:16 GMT; path=/
-Set-Cookie: LOGIN=7B7028DC2DA82...5CA6FB6CD6C2B1; domain=.mysite.com; path=/
-Date: Wed, 11 Jan 2012 20:22:16 GMT
-Content-Length: 165
-Cookie:: [version: 0][name: USER][value: 3D907C0E...E26E24][domain: .mysite.com][path: /][expiry: Thu Jan 10 20:22:16 GMT 2013]
-Cookie:: [version: 0][name: LOGIN][value: 7B7028D...D6C2B1][domain: .mysite.com][path: /][expiry: null]