I am using HTTPClient Fluent API version 4.3.2 to send multiple request to various addresses defined by user. Each address will use its particular proxy setting. Below is my current code:
try {
final Executor executor = Executor.newInstance().auth(proxy, userName, passWord);
System.out.println(executor.execute(Request.Get(uri).viaProxy(proxy)).returnResponse().getStatusLine());
} catch (final Exception e) {
e.printStackTrace();
}
I am facing a problem, when I connect to Address 1 with Proxy 1 (correct value of proxy username and proxy password), the request send properly as expected.
But when I add Address 2 with Proxy 1 (wrong value of proxy username and proxy password), with the expectation is request will fail, but it still connect successfully.
I though that after send request through a proxy, the very first proxy setting was cached somewhere in JVM, and I can not set another proxy setting for another request.
Is there any solution for this?
Trying setting proxy prior to request execution
Executor executor = Executor.newInstance().auth(proxy, userName, passWord);
System.out.println(executor
.viaProxy(proxy)
.execute(Request.Get(uri))
.returnResponse().getStatusLine());
Related
I am attempting to configure an UAA (https://github.com/cloudfoundry/uaa) SSO server to use CAS5 as an IDP.
The call to 'cas/authorize' is succeeding. The callback URL is called to the UAA server. The UAA server is then trying to call 'cas/accessToken' during the serving of the 'callback' URL endpoint to the browser, i.e. back-end HTTP request is being made back to CAS5.
I have traced through the execution of the 'cas/accessToken' request within CAS5. The request is failing on the Pac4j Profile lookup. The profile is trying to be accessed via either the request object or the session object. For the previous call to 'cas/authorize' the lookup succeeds via the session but for the 'cas/accessToken' the session does not contain the profile because the request is originating from the UAA server backend and not the user's browser.
Is this a bug with the OIDC support in CAS5? How do I get the Pac4j Profile lookup to succeed here?
Here is the stack trace where the profile lookup fails within CAS5:
ProfileManager<U>.retrieveAll(boolean) line: 69
ProfileManager<U>.get(boolean) line: 35
OAuth20AuthorizeEndpointController.isRequestAuthenticated(ProfileManager,J2EContext) line: 142
OidcAuthorizeEndpointController(OAuth20AuthorizeEndpointController).handleRequest(HttpServletRequest, HttpServletResponse) line: 109 OidcAuthorizeEndpointController.handleRequest(HttpServletRequest,HttpServletResponse) line: 86
I am using 5.1.0 not 5.2.2. I had rolledback to 5.1.0 around this time.
The problem still exists in the 5.1.0 code. I resolved by modified the OidcIdTokenGeneratorService to pull the CLAIM_PREFERRED_USERNAME from the TGT:
if (!claims.hasClaim(OidcConstants.CLAIM_PREFERRED_USERNAME)) {
String username = accessTokenId.getGrantingTicket().getAuthentication().getCredentials().get(0).getId();
claims.setClaim(OidcConstants.CLAIM_PREFERRED_USERNAME, username);
}
vs
if (!claims.hasClaim(OidcConstants.CLAIM_PREFERRED_USERNAME)) {
claims.setClaim(OidcConstants.CLAIM_PREFERRED_USERNAME, profile.getId());
}
There were some other calls upstream in the OAuth20AccessTokenEndpointController to get the CLIENT_ID via pac4j. The CLIENT_ID is also available as a request param. Changed calls similar to:
final String clientId = uProfile.getId();
to
final String clientId = request.getParameter(OAuth20Constants.CLIENT_ID);
One of the API (Say XYZ API) call alone requires the proxy to be set, When I run a suite where this API is called before and after every test script in the suite there is java.net.ConnectException seen randomly.
Only for XYZ API, I set and reset the proxy using the line:
RestAssured.proxy(String, int)
and
RestAssured.reset()
Please help me resolve this issue. Could this be a bug?
Note: This API call fails randomly just saying first 10 times it would have passed may be the 11th it fails with exception rest all scripts are then skipped in execution.
Code snapshot:
public void GetAuthenticationToken(String userid, String password)
{
RestAssured.proxy(config.getProperty("ProxyHost"),Integer.parseInt(config.getProperty("ProxyPort"))); //Authentication API is outside network and requires proxy
String APIUrl_Aut = config.getProperty("APIUrl_Aut");
String APIBody_Aut = "grant_type=password&username="+userid+"&password="+password;
//Making post request with authentication
Response response = RestAssured.given().log().all().headers("Accept", "application/json","Content-Type","application/x-www-form-urlencoded").body(APIBody_Aut).
when().post(APIUrl_Aut).then().contentType(ContentType.JSON).extract().response();
if (response.getStatusCode() == 200)
{
WritePropertyToTemp("AUTH_TOKEN", "Bearer "+response.body().jsonPath().get("access_token").toString());
WritePropertyToTemp("AUTH_TOKEN_OnlyToken",response.body().jsonPath().get("access_token").toString());
log.info("Authentication token generated successfully");
}
else
log.info("Authentication token failed to generate");
RestAssured.reset(); //Resetting proxy
}
I'm using Google OAuth2 client-side to authorize a web-app (Liferay Portlet) to use the Calendar Service.
On my Development Server, the whole flow completes successfully:
I start creating a GoogleAuthorizationCodeRequestUrl
Using com.google.api.client.extensions.jetty.auth.oauth2.LocalServerReceiver I create a new redirect URI and set it to wait for Google Response.
a new window/tab opens in user's browser, to googleAuthorizationCodeRequestUrl
user Logs in (if not already logged)
User authorizes the requested scopes
the tab closes automatically, the jetty's Callback URI is fetched from Google,
The flow continues with token exchange etc
But when Deploying on some other remote server (identical environment) the flow gets stuck in step 6. Google seems to be unable to find the redirect_uri. My browsers are landing on an error page, informing that they couldn't establish a connection to the server at localhost:[random port generated from jetty]
Checking the logs, I can see that in both cases (dev/remote server), the redirect_uri created by jetty is localhost:[5digits]/Callback. (not affected by the dns or ip on the remote server) Is this normal ? Did I miss something on the configuration? Maybe jetty was supposed to create another redirect URI, that I should additionally add from Google Dev Console (obviously I cant set to localhost..)
Is it possible that a firewall or proxy setting is blocking the redirect_url?
Any other ideas what I did wrong?
EDIT: posting some code for the URLs creation
GoogleAuthorizationCodeFlow flow = new GoogleAuthorizationCodeFlow.Builder(httpTransport, jsonFactory, secrets, scopes)
.setDataStoreFactory(dataStore).build();
Credential credents = flow.loadCredential(usermail);
String redirect_url = null;
// Checking if the given user is not authorized
if (credents == null) {
// Creating a local receiver
LocalServerReceiver receiver = new LocalServerReceiver();
try {
// Getting the redirect URI
String redirectUri = receiver.getRedirectUri();
// Creating a new authorization URL
AuthorizationCodeRequestUrl authorizationUrl = flow.newAuthorizationUrl();
// Setting the redirect URI
authorizationUrl.setRedirectUri(redirectUri);
// Building the authorization URL
String url = authorizationUrl.build();
// Logging a short message
log.info("Creating the authorization URL : " + url);
//This url will be fetched right after, as a button callback (target:_blank)
//by using :FacesContext.getCurrentInstance().getExternalContext().redirect(googleAuthUrl);
googleAuthUrl = url;
// Receiving authorization code
String code = receiver.waitForCode();
// Exchanging it for an access token
TokenResponse response = flow.newTokenRequest(code).setRedirectUri(redirectUri).execute();
// Storing the credentials for later access
credents = flow.createAndStoreCredential(response, id);
} finally {
// Releasing resources
receiver.stop();
}
}
// Setting up the calendar service client
client = new com.google.api.services.calendar.Calendar.Builder(httpTransport, jsonFactory, credents).setApplicationName(APPLICATION_NAME)
.build();
Instead of creating an instance of LocalServerReceiver by this:
// Creating a local receiver
LocalServerReceiver receiver = new LocalServerReceiver();
You should do this by using a LocalServerReceiver.Builder.
According to documentation non-parameter constructor is: Constructor that starts the server on "localhost" selects an unused port.
So you can use builder, set proper host name (remote server) and build LocalServerReceiver instance. (or you can use LocalServerReceiver(host, port) constructor)
This should set redirect_uri to proper address.
I am having a problem getting the Apache HttpClient to connect to a service external to my virtualised development environment.
To access the internet (e.g. api.twitter.com) I need to call a local URL (e.g. api.twitter.com.dev.mycompany.net), which then forwards the request to real host.
The problem is, that to whatever request I send, I get a 404 Not Found response.
I have tried debugging it using wget, and it appears the problem is, that the destination server identifies the desired resource by using both the request URL and the hostname in the Host header. Since the hostname does not match, it is unable to locate the resource.
I have (unsuccessfully) tried to override the Host header by setting the http.virtual-host parameter on the client like this:
HttpClient client = new DefaultHttpClient();
if (envType.isWithProxy()) {
client.getParams().setParameter(ClientPNames.VIRTUAL_HOST, "api.twitter.com");
}
Technical details:
Client is used as an executor in RESTeasy to call the REST API. So "manually" setting the virtual host (as described here) is not an option.
Everything is done via HTTPS/SSL - not that I think it makes a difference.
Edit 1: Using a HttpHost instead of a String does not have the desired effect either:
HttpClient client = new DefaultHttpClient();
if (envType.isWithProxy()) {
HttpHost realHost = new HttpHost("api.twitter.com", port, scheme);
client.getParams().setParameter(ClientPNames.VIRTUAL_HOST, realHost);
}
Edit 2: Further investigation has revealed, that the parameter needs to be set on the request object. The following is the code v. 4.2-aplha1 of HttpClient setting the virtual host:
HttpRequest orig = request;
RequestWrapper origWrapper = wrapRequest(orig);
origWrapper.setParams(params);
HttpRoute origRoute = determineRoute(target, origWrapper, context);
virtualHost = (HttpHost) orig.getParams().getParameter(
ClientPNames.VIRTUAL_HOST);
paramsare the parameters passed from the client. But the value for 'virtualHost' is read from the request parameters.
So this changes the nature of the question to: How do I set the VIRTUAL_HOST property on the requests?
ClientPNames.VIRTUAL_HOST is the right parameter for overriding physical host name in HTTP requests. I would just recommend setting this parameter on the request object instead of the client object. If that does not produce the desired effect please post the complete wire / context log of the session (see logging guide for instructions) either here or to the HttpClient user list.
Follow-up
OK. Let's take a larger sledge hammer. One can override content of the Host header using an interceptor.
DefaultHttpClient client = new DefaultHttpClient();
client.addRequestInterceptor(new HttpRequestInterceptor() {
public void process(
final HttpRequest request,
final HttpContext context) throws HttpException, IOException {
request.setHeader(HTTP.TARGET_HOST, "www.whatever.com");
}
});
One can make the interceptor clever enough to override the header selectively, only for specific hosts.
I need to access Facebook but all outgoing communication is blocked on our server so I have to use proxy.
I initialize proxies with:
ProxySelector.setDefault(new ConfigurableProxySelector(mapping));
Proxy type is HTTP, proxy host and port are working (confirmed by simple wget test).
I'm trying to do this:
HttpClient httpClient = new HttpClient();
HttpMethod method = new GetMethod("https://graph.facebook.com:443");
int status = httpClient.executeMethod(method);
Now, in my class ConfigurableProxySelector I have select method on which I have breakpoint:
public List<Proxy> select(URI uri) {
...
}
So, using HttpClient I make an request, which should be proxied and code stops at breakpoint in select() method in ConfigurableProxySelector.
But what is strange is that uri.scheme = "socket" and .toString() gives "socket://graph.facebook.com:443" instead of "https://graph.facebook.com:443".
Because ProxySelector have mapping for "https://" and not for "socket://", it does not find it and it ends with "Connection refused". What is strange is that select() method is called 4 times before execution ends with "Connection refused".
Any help would be appreciated.
Apache HTTP Client 3.1 will not natively honor HTTP Proxies returned from the default ProxySelector or user implementations.
Quick Summary of ProxySelector
ProxySelector is a service class which selects and returns a suitable Proxy for a given URL based on its scheme. For example, a request for http://somehost will try to provide an HTTP proxy if one is defined. The default ProxySelector can be configured at runtime using System Properties, such as http.proxyHost and http.proxyPort.
HTTPUrlConnection
An instance of HTTPUrlConnection will check against the default ProxySelector multiple times: 1st to select for http or https, then later when it builds the raw tcp socket, using the socket scheme. A SOCKS proxy could be used to proxy a raw tcp socket but are not often found in corporate environments, so a raw tcp socket will usually receive no proxy.
HTTP Client 3.1
HC 3.1, on the other hand, will never check the default ProxySelector for the http/https schemes. It will check, however, at a later points for the socket scheme when it eventually builds the raw socket - This is the request you are seeing. This means the System Properties http.proxyHost and http.proxyPort are ineffective. This is obviously not ideal for most people who only have an HTTP/HTTPS proxy.
To work around this, you have two options: define a proxy on each HC 3.1 connection or implement your own HC 3.1 HTTPConnectionManager.
HTTPConnectionManager
The HTTPConnectionManager is responsible for building connections for the HC 3.1 client.
The default HC 3.1 HTTPConnectionManager can be extended so that it looks for a suitable proxy from a ProxySelector (default or custom) when building the request in the same way HTTPUrlConnection does:
public class MyHTTPConnectionManager extends SimpleHttpConnectionManager {
#Override
public HttpConnection getConnectionWithTimeout(
HostConfiguration hostConfiguration, long timeout) {
HttpConnection hc = super.getConnectionWithTimeout(hostConfiguration, timeout);
try {
URI uri = new URI( hostConfiguration.getHostURL());
List<Proxy> hostProxies = ProxySelector.getDefault().select(uri);
Proxy Proxy = hostProxies.get(0);
InetSocketAddress sa = (InetSocketAddress) Proxy.address();
hc.setProxyHost(sa.getHostName());
hc.setProxyPort(sa.getPort());
} catch (URISyntaxException e) {
return hc;
}
return hc;
}
}
Then, when you create an HC 3.1 client, use your new connection manager:
HttpClient client = new HttpClient(new MyHTTPConnectionManager() );
It's not the ProxySelector that changes the scheme, but the SocketFactory opening a Socket.
If the SocketFactory is null a SOCKS socket will be created by default which only allows SOCKS proxies. I don't know anything about Sockets and cannot tell you if there's a way to make it work with HTTP proxies.
But using another approach may help, since Apache HttpClient seems to have its own way to configure proxies.
client.getHostConfiguration().setProxy(proxyHost, proxyPort);
if (proxyUser != null) {
client.getState().setProxyCredentials(new AuthScope(proxyHost, proxyPort),
new UsernamePasswordCredentials(proxyUser, proxyPassword));
}