Fiddler doesn't capture Apache HttpClient post - java

Somehow Fiddler doesn't capture the posts I send from my HttpClient provided by Apache.
But when I send the same post in C# using the HttpClient to the same server, Fiddler does intercept the sessions.
My Java code:
private DefaultHttpClient client = new DefaultHttpClient();
private HttpContext context = new BasicHttpContext();
private BasicCookieStore store = new BasicCookieStore();
public Client() throws URISyntaxException {
context.setAttribute(ClientContext.COOKIE_STORE, store);
logIn();
}
private void logIn() throws URISyntaxException {
HttpUriRequest login = RequestBuilder.post()
.setUri(new URI("http://www.derpforum.nl"))
.addParameter("username", "Kattoor4")
.addParameter("password", "XXXX")
.addHeader("Referer", "http://www.derpforum.nl/")
.build();
try (CloseableHttpResponse response = client.execute(login, context)) {
HttpEntity entity = response.getEntity();
BufferedReader reader = new BufferedReader(new InputStreamReader(entity.getContent()));
String line;
while ((line = reader.readLine()) != null)
System.out.println(line);
} catch (IOException e) {
e.printStackTrace();
}
}
Any thoughts? Thanks!

I am usring Apache HttpClient(4.5.5), SWT4 and Fiddler4, and the VM arguments method does not work for me.
So I set the proxy settings in the code and it works.
HttpHost proxy = new HttpHost("localhost", 8888, "http");
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
CloseableHttpClient httpclient = HttpClients.custom()
.setRoutePlanner(routePlanner)
.build();

You probably need to configure Java to use Fiddler as a proxy either in code or by setting the relevant Java system properties as below. See this question.
-Dhttp.proxyHost=127.0.0.1
-Dhttp.proxyPort=8888

Related

Upload File from GWT to another domain , response is always null

I am uploading a File from GWT to a different domain
File Uploads well , But the response i sent from the server always reaches as "null" at the client side
response.setContentType("text/html");
response.setHeader("Access-Control-Allow-Origin", "*");
response.getWriter().print("TEST");
response is NULL only when i upload the file on a different domain ... (on same domain all is OK)
I also see this in GWT documentation
Tip:
The result html can be null as a result of submitting a form to a different domain.
http://www.gwtproject.org/javadoc/latest/com/google/gwt/user/client/ui/FormPanel.SubmitCompleteEvent.html
Is there any way I can receive back a response at my client side when i am uploading file to a different domain
There are 2 possible answer:
Use JSONP Builder
JsonpRequestBuilder requestBuilder = new JsonpRequestBuilder();
requestBuilder.requestObject(url, new AsyncCallback<FbUser>() {
#Override
public void onFailure(Throwable ex) {
throw SOMETHING_EXCEPTION(ex);
}
#Override
public void onSuccess(ResponseModel resp) {
if (resp.isError()) {
// on response error on something
log.error(resp.getError().getMessage())
log.error(resp.getError().getCode())
}
log.info(resp.getAnyData())
}
Not to use GWT to upload, rather use other client like apache HttpClient
public uploadFile() {
HttpClient httpClient = new DefaultHttpClient();
HttpContext localContext = new BasicHttpContext();
HttpPost httpPost = new HttpPost(url);
FileBody bin = new FileBody(new File(UPLOADED_FILE));
long size = bin.getContentLength();
MultipartEntity reqEntity = new MultipartEntity();
reqEntity.addPart("PART", bin);
String content = "-";
try {
httpPost.setEntity(reqEntity);
HttpResponse response = httpClient.execute(httpPost, localContext);
HttpEntity ent = response.getEntity();
InputStream st = ent.getContent();
StringWriter writer = new StringWriter();
IOUtils.copy(st, writer);
content = writer.toString();
} catch (IOException e) {
return "false";
}
return content;
}
Hope it helps

Error 407 when trying to send a request in Java

I have the following code:
public class SendRequest {
public static void main(String[] args) throws ClientProtocolException, IOException {
String url = "http://backoffice.xyz";
HttpHost proxy = new HttpHost("proxy.proxy", 8080, "http");
HttpClient client = HttpClientBuilder.create().setProxy(proxy).build();
HttpGet request = new HttpGet(url);
//request.addHeader("User-Agent", "USER-AGENT");
HttpResponse response = client.execute(request);
System.out.println("Response Code: " + response.getStatusLine().getStatusCode());
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while((line = rd.readLine()) != null){
result.append(line);
}
System.out.println(result.toString());
}
}
This is returning a 407 Unauthorized Access/Cache Access Denied Error. What code do i need to include so i can authenticate through the proxy?
Does your proxy require username/password based authentication? If so, try implementing java.net.Authenticator. I guess you will need to set useSystemProperties
Authenticator.setDefault(new Authenticator(){
PasswordAuthentication getPasswordAuthentication(){
return new PasswordAuthentication(username, password);
}
});
You might need to add setDefaultCredentialsProvider(CredentialsProvider credentialsProvider) to the HTTPClientBuilder and use SystemDefaultCredentialsProvider instance for that.
1.Can you check your proxy ?? I'm not sure if proxy.proxy is correct.
Your rest of the code seems fine.
2.And also make sure you are updating httpcore version to 4.4 or above. And also you can update httpcore to latest version.
3.You could check the authentication of your proxy as #Goutham mentioned
Hope this helps!!

how to make a valid rest call with authentication in Java

I am trying to write a java class file that authenticates to a system using a HTTP Rest call )post in this case).
I have tried the following code, but I get an error stating:
{"errors":[{"message":"The request could not be understood","developerMessage":"The request body did not contain valid JSON"}]}
here is my code:
public class simplePost {
private final String USER_AGENT = "Mozilla/5.0";
public static void main(String[] args) throws IOException {
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost("https://xxxxxx/xxxxx/token");
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials("username", "password"));
List nameValuePairs = new ArrayList(1);
nameValuePairs.add(new BasicNameValuePair("username", "xxxxxxxxxx"));
nameValuePairs.add(new BasicNameValuePair("password", "xxxxxx"));
post.setEntity(new UrlEncodedFormEntity(nameValuePairs));
HttpResponse response = client.execute(post);
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
String line;
while ((line = rd.readLine()) != null) {
System.out.println(line);
}
}
}
I am not sure why I am getting this error.
When I put in the URI and login information in Postman or AdvanceRestClient, I get the proper response.
Can I get a little help on this?
thanks!
ironmantis7x

authentication with Java and apache HttpClient 4.5.1

my problem is, that i don't get, how to log in with Java and Apache HttpComponents (HttpClient v4.5.1) into a specific site: Site im trying to log in. I have the username (test_admin) and the password (testing) to log in but i think this is not enough and i need something more. I think this has something to do with the field security_token i see when i make a get request to the uri, but i dont know how to keep that or how to save that and what to do with it afterwards. There is also a hidden input field with the name login-ticket, but i dont know what's that for either. I want to login, because i need to see the courses and add some new ones. After trying with several code implementations im stick with this code:
public static void setGet(CloseableHttpClient httpClient) throws UnsupportedOperationException, IOException
{
HttpGet httpGet = new HttpGet("http://demo.studip.de/dispatch.php/admin/courses");
CloseableHttpResponse httpResponse = httpClient.execute(httpGet);
System.out.println("GET Response Status:: "
+ httpResponse.getStatusLine().getStatusCode());
showEntity(httpResponse,httpResponse.getEntity());
}
public static HttpEntity setParam(int count, String[] params, String[] values)
{
List<NameValuePair> formparams = new ArrayList<NameValuePair>();
for (int i = 0; i < count; i++)
{
formparams.add(new BasicNameValuePair(params[i],values[i]));
System.out.println("Paramater------------------> "+params[i]+" Values-------------> "+values[i]);
}
UrlEncodedFormEntity entity = new UrlEncodedFormEntity(formparams, Consts.UTF_8);
return entity;
}
public static void setPost(HttpClient httpC) throws ClientProtocolException, IOException
{
HttpPost httppost = new HttpPost("http://demo.studip.de/dispatch.php/admin/courses");
//String[] params = {"loginname", "password"};
//String[] values = {"test_admin", "testing"};
//HttpEntity entity = setParam(2, params, values );
HttpResponse response = httpC.execute(httppost);
System.out.println("POST Response Status:: "
+ response.getStatusLine().getStatusCode());
showEntity(response, response.getEntity());
}
public static void showEntity(HttpResponse httpResp, HttpEntity httpClient) throws IOException
{
httpClient = httpResp.getEntity();
if (httpClient != null)
httpClient = new BufferedHttpEntity(httpClient);
System.out.print(EntityUtils.toString(httpClient));
}
public static void main(String[] args) throws InterruptedException, IOException {
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials("test_admin", "testing"));
CloseableHttpClient hc =
HttpClientBuilder.create().setDefaultCredentialsProvider(credentialsProvider).build();
setGet(hc);
// HttpClient httpclient = HttpClients.createDefault();
setPost(hc);
setGet(hc);
}
The problem now ist that i get everytime the same answer from the server i only see the login page in the response, where the server asks me to login with username and password.
Which code you get from the server 401,403,301,302 or 200?

Java SSLException: hostname in certificate didn't match

I have been using the following code to connect to one of google's service. This code worked fine on my local machine :
HttpClient client=new DefaultHttpClient();
HttpPost post = new HttpPost("https://www.google.com/accounts/ClientLogin");
post.setEntity(new UrlEncodedFormEntity(myData));
HttpResponse response = client.execute(post);
I put this code in a production environment, which had blocked Google.com. On request, they allowed communication with Google server by allowing me to accessing an IP : 74.125.236.52 - which is one of Google's IPs. I edited my hosts file to add this entry too.
Still I could not access the URL, which I wonder why. So I replaced the above code with :
HttpPost post = new HttpPost("https://74.125.236.52/accounts/ClientLogin");
Now I get an error like this :
javax.net.ssl.SSLException: hostname in certificate didn't match:
<74.125.236.52> != <www.google.com>
I guess this is because Google has multiple IPs. I cant ask the network admin to allow me access to all those IPs - I may not even get this entire list.
What should I do now ? Is there a workaround at Java level ? Or is it totally in hands of the network guy ?
You can also try to set a HostnameVerifier as described here. This worked for me to avoid this error.
// Do not do this in production!!!
HostnameVerifier hostnameVerifier = org.apache.http.conn.ssl.SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER;
DefaultHttpClient client = new DefaultHttpClient();
SchemeRegistry registry = new SchemeRegistry();
SSLSocketFactory socketFactory = SSLSocketFactory.getSocketFactory();
socketFactory.setHostnameVerifier((X509HostnameVerifier) hostnameVerifier);
registry.register(new Scheme("https", socketFactory, 443));
SingleClientConnManager mgr = new SingleClientConnManager(client.getParams(), registry);
DefaultHttpClient httpClient = new DefaultHttpClient(mgr, client.getParams());
// Set verifier
HttpsURLConnection.setDefaultHostnameVerifier(hostnameVerifier);
// Example send http request
final String url = "https://encrypted.google.com/";
HttpPost httpPost = new HttpPost(url);
HttpResponse response = httpClient.execute(httpPost);
The certificate verification process will always verify the DNS name of the certificate presented by the server, with the hostname of the server in the URL used by the client.
The following code
HttpPost post = new HttpPost("https://74.125.236.52/accounts/ClientLogin");
will result in the certificate verification process verifying whether the common name of the certificate issued by the server, i.e. www.google.com matches the hostname i.e. 74.125.236.52. Obviously, this is bound to result in failure (you could have verified this by browsing to the URL https://74.125.236.52/accounts/ClientLogin with a browser, and seen the resulting error yourself).
Supposedly, for the sake of security, you are hesitant to write your own TrustManager (and you musn't unless you understand how to write a secure one), you ought to look at establishing DNS records in your datacenter to ensure that all lookups to www.google.com will resolve to 74.125.236.52; this ought to be done either in your local DNS servers or in the hosts file of your OS; you might need to add entries to other domains as well. Needless to say, you will need to ensure that this is consistent with the records returned by your ISP.
I had similar problem. I was using Android's DefaultHttpClient. I have read that HttpsURLConnection can handle this kind of exception. So I created custom HostnameVerifier which uses the verifier from HttpsURLConnection. I also wrapped the implementation to custom HttpClient.
public class CustomHttpClient extends DefaultHttpClient {
public CustomHttpClient() {
super();
SSLSocketFactory socketFactory = SSLSocketFactory.getSocketFactory();
socketFactory.setHostnameVerifier(new CustomHostnameVerifier());
Scheme scheme = (new Scheme("https", socketFactory, 443));
getConnectionManager().getSchemeRegistry().register(scheme);
}
Here is the CustomHostnameVerifier class:
public class CustomHostnameVerifier implements org.apache.http.conn.ssl.X509HostnameVerifier {
#Override
public boolean verify(String host, SSLSession session) {
HostnameVerifier hv = HttpsURLConnection.getDefaultHostnameVerifier();
return hv.verify(host, session);
}
#Override
public void verify(String host, SSLSocket ssl) throws IOException {
}
#Override
public void verify(String host, X509Certificate cert) throws SSLException {
}
#Override
public void verify(String host, String[] cns, String[] subjectAlts) throws SSLException {
}
}
A cleaner approach ( only for test environment) in httpcliet4.3.3 is as follows.
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslContext,SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
CloseableHttpClient httpclient = HttpClients.custom().setSSLSocketFactory(sslsf).build();
In httpclient-4.3.3.jar, there is another HttpClient to use:
public static void main (String[] args) throws Exception {
// org.apache.http.client.HttpClient client = new DefaultHttpClient();
org.apache.http.client.HttpClient client = HttpClientBuilder.create().build();
System.out.println("HttpClient = " + client.getClass().toString());
org.apache.http.client.methods.HttpPost post = new HttpPost("https://www.rideforrainbows.org/");
org.apache.http.HttpResponse response = client.execute(post);
java.io.InputStream is = response.getEntity().getContent();
java.io.BufferedReader rd = new java.io.BufferedReader(new java.io.InputStreamReader(is));
String line;
while ((line = rd.readLine()) != null) {
System.out.println(line);
}
}
This HttpClientBuilder.create().build() will return org.apache.http.impl.client.InternalHttpClient. It can handle the this hostname in certificate didn't match issue.
Thanks Vineet Reynolds. The link you provided held a lot of user comments - one of which I tried in desperation and it helped. I added this method :
// Do not do this in production!!!
HttpsURLConnection.setDefaultHostnameVerifier( new HostnameVerifier(){
public boolean verify(String string,SSLSession ssls) {
return true;
}
});
This seems fine for me now, though I know this solution is temporary. I am working with the network people to identify why my hosts file is being ignored.
The concern is we should not use ALLOW_ALL_HOSTNAME_VERIFIER.
How about I implement my own hostname verifier?
class MyHostnameVerifier implements org.apache.http.conn.ssl.X509HostnameVerifier
{
#Override
public boolean verify(String host, SSLSession session) {
String sslHost = session.getPeerHost();
System.out.println("Host=" + host);
System.out.println("SSL Host=" + sslHost);
if (host.equals(sslHost)) {
return true;
} else {
return false;
}
}
#Override
public void verify(String host, SSLSocket ssl) throws IOException {
String sslHost = ssl.getInetAddress().getHostName();
System.out.println("Host=" + host);
System.out.println("SSL Host=" + sslHost);
if (host.equals(sslHost)) {
return;
} else {
throw new IOException("hostname in certificate didn't match: " + host + " != " + sslHost);
}
}
#Override
public void verify(String host, X509Certificate cert) throws SSLException {
throw new SSLException("Hostname verification 1 not implemented");
}
#Override
public void verify(String host, String[] cns, String[] subjectAlts) throws SSLException {
throw new SSLException("Hostname verification 2 not implemented");
}
}
Let's test against https://www.rideforrainbows.org/ which is hosted on a shared server.
public static void main (String[] args) throws Exception {
//org.apache.http.conn.ssl.SSLSocketFactory sf = org.apache.http.conn.ssl.SSLSocketFactory.getSocketFactory();
//sf.setHostnameVerifier(new MyHostnameVerifier());
//org.apache.http.conn.scheme.Scheme sch = new Scheme("https", 443, sf);
org.apache.http.client.HttpClient client = new DefaultHttpClient();
//client.getConnectionManager().getSchemeRegistry().register(sch);
org.apache.http.client.methods.HttpPost post = new HttpPost("https://www.rideforrainbows.org/");
org.apache.http.HttpResponse response = client.execute(post);
java.io.InputStream is = response.getEntity().getContent();
java.io.BufferedReader rd = new java.io.BufferedReader(new java.io.InputStreamReader(is));
String line;
while ((line = rd.readLine()) != null) {
System.out.println(line);
}
}
SSLException:
Exception in thread "main" javax.net.ssl.SSLException: hostname in certificate didn't match: www.rideforrainbows.org != stac.rt.sg OR stac.rt.sg OR www.stac.rt.sg
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:231)
...
Do with MyHostnameVerifier:
public static void main (String[] args) throws Exception {
org.apache.http.conn.ssl.SSLSocketFactory sf = org.apache.http.conn.ssl.SSLSocketFactory.getSocketFactory();
sf.setHostnameVerifier(new MyHostnameVerifier());
org.apache.http.conn.scheme.Scheme sch = new Scheme("https", 443, sf);
org.apache.http.client.HttpClient client = new DefaultHttpClient();
client.getConnectionManager().getSchemeRegistry().register(sch);
org.apache.http.client.methods.HttpPost post = new HttpPost("https://www.rideforrainbows.org/");
org.apache.http.HttpResponse response = client.execute(post);
java.io.InputStream is = response.getEntity().getContent();
java.io.BufferedReader rd = new java.io.BufferedReader(new java.io.InputStreamReader(is));
String line;
while ((line = rd.readLine()) != null) {
System.out.println(line);
}
}
Shows:
Host=www.rideforrainbows.org
SSL Host=www.rideforrainbows.org
At least I have the logic to compare (Host == SSL Host) and return true.
The above source code is working for httpclient-4.2.3.jar and httpclient-4.3.3.jar.
Updating the java version from 1.8.0_40 to 1.8.0_181 resolved the issue.
SSLConnectionSocketFactory sslConnectionSocketFactory = new SSLConnectionSocketFactory(
SSLContexts.custom().loadTrustMaterial(null, new TrustSelfSignedStrategy()).build(),
SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
CloseableHttpClient httpClient = HttpClients.custom().setSSLSocketFactory(sslConnectionSocketFactory).build();

Categories