We Keep Coding

post can't reach on target with Heroku - Fixie

I'm making REST API service with another REST API service.
When I post a request to use external REST API service, the post can't reach on target.
The target host requires static IP address and HTTP protcol, so I use Fixie on Heroku.
I get following messages on heroku.
o.apache.http.impl.execchain.RetryExec : I/O exception (org.apache.http.NoHttpResponseException) caught when processing request to {tls}->http://xxxxx.usefixie.com:80->https://my-target-host.com:443: The target server failed to respond
o.apache.http.impl.execchain.RetryExec : Retrying request to {tls}->http://xxxxx.usefixie.com:80->https://my-target-host.com:443 org.apache.http.NoHttpResponseException: my-target-host.com:443 failed to respond
Currentry, the host don't expose access log.
These are my Java code.
Code-A.
HttpPost post = new HttpPost("/action");
HttpHost targetHost = new HttpHost("my-target-host.com",443,"https");
post.setHeader("Content-Type", "application/json; charset=UTF-8");
try{
URL proxyUrl = new URL(System.getenv("FIXIE_URL"));
HttpHost proxy = new HttpHost(proxyUrl.getHost(), proxyUrl.getPort());
HttpRoutePlanner routePlanner = new HttpRoutePlanner() {
#Override
public HttpRoute determineRoute(HttpHost target, org.apache.http.HttpRequest request,
org.apache.http.protocol.HttpContext context) throws HttpException {
return new HttpRoute(target, null, new HttpHost(proxyUrl.getHost(), proxyUrl.getPort()), true);
}
};
try (CloseableHttpClient httpclient = HttpClients.custom().setRoutePlanner(routePlanner).build();) {
try {
post.setEntity(new StringEntity(CONTENT_JSON, StandardCharsets.UTF_8));
CloseableHttpResponse res = httpclient.execute(targetHost,post);
} catch (Exception e) {
e.printStackTrace();
}
}catch(Exception e){
e.printStackTrace();
}
}catch(Exception e){
e.printStackTrace();
}
Code-B.
HttpPost post = new HttpPost("/action");
HttpHost targetHost = new HttpHost("my-target-host.com",443,"https");
post.setHeader("Content-Type", "application/json; charset=UTF-8");
try{
URL proxyUrl = new URL(System.getenv("FIXIE_URL"));
String userInfo = proxyUrl.getUserInfo();
String user = userInfo.substring(0, userInfo.indexOf(':'));
String password = userInfo.substring(userInfo.indexOf(':') + 1);
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(
new AuthScope(proxyUrl.getHost(),proxyUrl.getPort()),
new UsernamePasswordCredentials(user,password));
try (CloseableHttpClient httpclient = HttpClients.custom().setDefaultCredentialsProvider(credsProvider).build();) {
HttpHost proxy = new HttpHost(proxyUrl.getHost(), proxyUrl.getPort());
RequestConfig config = RequestConfig.custom().setProxy(proxy).build();
post.setConfig(config);
String encodedAuth = Base64.getEncoder().encodeToString(userInfo.getBytes());
post.setHeader("Proxy-Authorization", "Basic " + encodedAuth);
try {
post.setEntity(new StringEntity(CONTENT_JSON, StandardCharsets.UTF_8));
CloseableHttpResponse res = httpclient.execute(targetHost,post);
} catch (Exception e) {
e.printStackTrace();
}
}catch(Exception e){
e.printStackTrace();
}
}catch(Exception e){
e.printStackTrace();
}
Code-C.
HttpPost post = new HttpPost("/action");
HttpHost targetHost = new HttpHost("my-tareget-host.com",443,"https");
try{
URL proxyUrl = new URL(System.getenv("FIXIE_URL"));
String userInfo = proxyUrl.getUserInfo();
String user = userInfo.substring(0, userInfo.indexOf(':'));
String password = userInfo.substring(userInfo.indexOf(':') + 1);
DefaultHttpClient httpclient_ = new DefaultHttpClient();
try {
httpclient_.getCredentialsProvider().setCredentials(
new AuthScope(proxyUrl.getHost(), proxyUrl.getPort()),
new UsernamePasswordCredentials(user, password));
HttpHost proxy = new HttpHost(proxyUrl.getHost(), proxyUrl.getPort());
httpclient_.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
String encodedAuth = Base64.getEncoder().encodeToString(userInfo.getBytes());
post.setEntity(new StringEntity(CONTENT_JSON, StandardCharsets.UTF_8));
post.setHeader("Proxy-Authorization", "Basic " + encodedAuth);
HttpResponse rsp = httpclient_.execute(targetHost, post);
} finally {
httpclient_.getConnectionManager().shutdown();
}
}catch(Exception e){
e.printStackTrace();
}
These code cause same result.
On same environment, this ruby code is reached.
RestClient.proxy = ENV['FIXIE_URL'] if ENV['FIXIE_URL']
RestClient.post("https://my-target-host.com/action", CONTENT_JSON, {
'Content-Type' => 'application/json; charset=UTF-8'
})
What's happend on access? What should I modify on Java?

State management in httpAsyncClient

I am trying to set cookie in httpAsyncClient library but problem is i am not figure out to find example of it..here is my code.
public JSONArray sendRequest(List<BasicNameValuePair> postPairs){
HttpAsyncClient httpclient = null;
try {
httpclient = new DefaultHttpAsyncClient();
httpclient.start();
HttpPost post = new HttpPost("http://10.0.0.62:8080/IDocWS"+postPairs.get(0).getValue());
BasicCookieStore cookie = new BasicCookieStore();
HttpContext httpContext = new BasicHttpContext();
httpContext.setAttribute(ClientContext.COOKIE_STORE,cookie);
post.setEntity(new UrlEncodedFormEntity( postPairs, HTTP.UTF_8 ) );
Future<HttpResponse> future = httpclient.execute(post, null);
HttpResponse resp = future.get();
HttpEntity entity = resp.getEntity();
JSONArray jArray = CovnertToJson(entity);
return jArray;
and here is my ConvertToJson method
public JSONArray CovnertToJson(HttpEntity entity){
try{
InputStream inputStream1= entity.getContent();
BufferedReader reader1 = new BufferedReader(new InputStreamReader(inputStream1, "UTF-8"));
StringBuilder sb = new StringBuilder();
String line1;
while ((line1 = reader1.readLine()) != null) {
sb.append(line1);
}
JSONArray jArray = new JSONArray(sb.toString());
return jArray;
}

First get cookie from request header:
Future<HttpResponse> future = httpclient.execute(post, null);
HttpResponse resp = future.get();
header = resp.getHeaders("cookie");
myCookies = header.toString();
After getting cookie put it in HTTPContext and send it and it shoud work.
CookieStore cookieStore = new BasicCookieStore();
Cookie cookie = new BasicClientCookie("cookie", myCookies);
cookieStore.addCookie(cookie);
httpContext = new BasicHttpContext();
httpContext.setAttribute(ClientContext.COOKIE_STORE,cookieStore);

Enable cookies with HTTPClient

I am trying to implement a Java Code that fetch urls like this one:
http://trendistic.indextank.com/_trending-topics-archive/2011-12-25
The code I wrote is based in HTTPClient as follow:
HttpClient httpclient = new DefaultHttpClient();
// Create a local instance of cookie store
CookieStore cookieStore = new BasicCookieStore();
// Create local HTTP context
HttpContext localContext = new BasicHttpContext();
// Bind custom cookie store to the local context
localContext.setAttribute(ClientContext.COOKIE_STORE, cookieStore);
HttpGet httpget = new HttpGet(urls);
try {
System.out.println("executing request " + httpget.getURI());
HttpResponse response = httpclient.execute(httpget, localContext);
String line = "";
StringBuilder total = new StringBuilder();
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
while ((line = rd.readLine()) != null) {
total.append(line+"\n");
}
System.out.println(total);
return (total.toString());
} catch (Exception e) {
e.printStackTrace();
return(e.getMessage());
}
}
Nevertheless, the response I got is:
if (tz != tzless.readCookie('local_tz')) {
// no cookies!
document.write("<h1>Cookies are not enabled for this site</h1>");
document.write("<p>In order for Trendistic to work in Internet Explorer you need to enable cookies.</p>");
document.write("<ul><li>Please enable cookies for this site and refresh this page</li><li>Or try Trendistic in a different browser such as Safari, Firefox or Chrome.</li></ul>");
} else {
self.location.reload();
}
How can I enable cookies for fetching the HTML with the dynamical results.

How to do HTTP CONNECT Method by HTTPClient behind Proxy

I am programing a Browser using HTTPClient to access HTTPS on Android.
The following source code has implemented access HTTPS by HTTPClient.
// Proxy Host and Port
String proxyHost = "127.0.0.1";
int proxyPort = 10000;
// HTTPS Site
String strURL_https = "https://172.17.4.37:8443/apache.html";
URL url_https = null;
HttpClient httpClient = null;
HttpGet get = null;
try{
url_https = new URL(strURL_https);
MySSLSocketFactory sf = MySSLSocketFactory.getSocketFactory();
get = new HttpGet(strURL_https);
try {
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpProtocolParams.setContentCharset(params, HTTP.UTF_8);
// Add two scheme for Port 443 and 8443
httpClient = new DefaultHttpClient();
Scheme sch = new Scheme("https", sf, 443);
Scheme sch2 = new Scheme("https", sf, 8443);
httpClient.getConnectionManager().getSchemeRegistry().register(sch);
httpClient.getConnectionManager().getSchemeRegistry().register(sch2);
// Set Proxy
httpClient.getParams().setParameter(ConnRouteParams.DEFAULT_PROXY, new HttpHost(proxyHost, proxyPort));
} catch (Exception e) {
e.printStackTrace();
}
HttpContext context = new BasicHttpContext();
// Execute GET Method
HttpResponse response = httpClient.execute(get, context);
HttpEntity entity = response.getEntity();
String charSet = null;
String contentType = null;
StringBuffer sb = null;
if(entity!= null){
charSet=EntityUtils.getContentCharSet(entity);
contentType=entity.getContentType().getValue();
InputStream is = entity.getContent();
BufferedReader br = new BufferedReader(new InputStreamReader(is, "utf8"));
String text = null;
String line = br.readLine();
sb = new StringBuffer();
while(line != null) {
sb.append(line+"\r\n");
line = br.readLine();
}
}
// Load Data on Webview
webview.loadDataWithBaseURL(strURL_https, sb.toString(), "text/html", charSet, null);
But when do httpClient.execute(get, context), my proxy server will receive HTTP CONNECT Method as the following:
CONNECT 172.17.4.37:8443 HTTP/1.1
User-Agent: Apache-HttpClient/UNAVAILABLE (java 1.4)
Host: 172.17.4.37:8443
My proxy server send the HTTP Method to VPN, but cannot get any response.
In my opinion, I should get like HTTP/1.0 200 Connection Established.
My question is how to do HTTP CONNECT Method by HTTPClient without proxy.
After create connection established, set proxy to HTTPClient.
I am looking forward your answer.
Thank you.

Apache HttpClient 4.1.1 NTLM authentication not SPNEGO

The problem here is consuming a web resource that has NTLM authentication while using the Apache HttpClient on the client side. The issue I am having is forcing the client to use NTLM authentication. here is a code sapmle.
DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.getAuthSchemes().register("ntlm",new NTLMSchemeFactory());
NTCredentials creds = new NTCredentials("_myUSer_","_myPass_","_myWorkstation_","_myDomain_");
httpclient.getCredentialsProvider().setCredentials( new AuthScope("serverName",80), creds);
List<String> authpref = new ArrayList<String>();
authpref.add(AuthPolicy.NTLM);
httpclient.getParams().setParameter(AuthPNames.PROXY_AUTH_PREF, authpref);
HttpHost target = new HttpHost("serverName", 80, "http");
HttpGet httpget = new HttpGet("webResource");
HttpContext localContext = new BasicHttpContext();
HttpResponse response = httpclient.execute(target, httpget, localContext);
Here is the error from Java:
org.apache.http.client.protocol.RequestTargetAuthentication process
SEVERE: Authentication error: Invalid name provided (Mechanism level: Could not load configuration file C:\WINDOWS\krb5.ini (The system cannot find the file specified))
The web server response is a 401.
Any ideas on why the auth policy not being set correctly?
Am I missing something in the code?

I have a similar situation and I suspect that you are setting the wrong parameter: AuthPNames.PROXY_AUTH_PREF. I use AuthPNames.TARGET_AUTH_PREF and all seems to work fine.

Here is my solution to this Problem: And "evandongen" is right.
Please note the use of the URIBuilder.
String username = "uid";
String pwd = "pwd";
String servername = "www.someserver.com";
String workstation = "myworkstation";
String domain = "somedomain";
String relativeurl = "/util/myservice.asmx";
String oldimagePath = "\\mypath\\image.jpg";
DefaultHttpClient httpclient = new DefaultHttpClient();
try {
httpclient.getAuthSchemes().register("ntlm",new NTLMSchemeFactory());
NTCredentials creds = new NTCredentials(username,pwd,workstation,domain);
httpclient.getCredentialsProvider().setCredentials(new AuthScope(servername,80), creds);
List authpref = new ArrayList();
authpref.add(AuthPolicy.NTLM);
URIBuilder builder = new URIBuilder();
builder.setScheme("http")
.setHost(servername)
.setPath(relativeurl + "/DeleteImage")
.setParameter("imagePath", oldimagePath);
URI uri = builder.build();
httpclient.getParams().setParameter(AuthPNames.TARGET_AUTH_PREF, authpref);
HttpHost target = new HttpHost(servicename, 80, "http");
HttpGet httpget = new HttpGet(uri);
HttpContext localContext = new BasicHttpContext();
HttpResponse response1 = httpclient.execute(target, httpget, localContext);
BufferedReader reader = new BufferedReader(new InputStreamReader(response1.getEntity().getContent()));
String line = reader.readLine();
while (line != null)
{
System.out.println(line);
line = reader.readLine();
}
} catch (Exception e) {
System.out.println("Exception:"+e.toString());
} finally {
// End
}

I think this is because of a defect, see here.

HttpClient did not work for me but the below snippet did.
Reference - http://docs.oracle.com/javase/7/docs/technotes/guides/net/http-auth.html
public static String getResponse(String url, String userName, String password) throws IOException {
Authenticator.setDefault(new Authenticator() {
#Override
public PasswordAuthentication getPasswordAuthentication() {
System.out.println(getRequestingScheme() + " authentication");
return new PasswordAuthentication(userName, password.toCharArray());
}
});
URL urlRequest = new URL(url);
HttpURLConnection conn = (HttpURLConnection) urlRequest.openConnection();
conn.setDoOutput(true);
conn.setDoInput(true);
conn.setRequestMethod("GET");
StringBuilder response = new StringBuilder();
InputStream stream = conn.getInputStream();
BufferedReader in = new BufferedReader(new InputStreamReader(stream));
String str = "";
while ((str = in.readLine()) != null) {
response.append(str);
}
in.close();
return response.toString();
}