We Keep Coding

Accessing kerberos secured WebHDFS without SPnego

I have a working application for managing HDFS using WebHDFS.
I need to be able to do this on a Kerberos secured cluster.
The problem is, that there is no library or extension to negotiate the ticket for my app, I only have a basic HTTP client.
Would it be possible to create a Java service which would handle the ticket exchange and once it gets the Service ticket to just pass it to the app for use in a HTTP request?
In other words, my app would ask the Java service to negotiate the tickets and it would return the Service ticket back to my app in a string or raw string and the app would just attach it to the HTTP request?
EDIT: Is there a similar elegant solution like #SamsonScharfrichter described for HTTPfs? (To my knowledge, it does not support delegation tokens)
EDIT2: Hi guys, I am still completly lost. Im trying to figure out the Hadoop-auth client without any luck. Could you please help me out again? I already spent hours reading upon it without luck.
The examples say to do this:
* // establishing an initial connection
*
* URL url = new URL("http://foo:8080/bar");
* AuthenticatedURL.Token token = new AuthenticatedURL.Token();
* AuthenticatedURL aUrl = new AuthenticatedURL();
* HttpURLConnection conn = new AuthenticatedURL(url, token).openConnection();
* ....
* // use the 'conn' instance
* ....
Im lost already here. What initial connection do I need? How can
new AuthenticatedURL(url, token).openConnection();
take two parameters? there is no constructor for such a case. (im getting error because of this). Shouldnt a principal be somewhere specified? It is probably not going to be this easy.
URL url = new URL("http://<host>:14000/webhdfs/v1/?op=liststatus");
AuthenticatedURL.Token token = new AuthenticatedURL.Token();
HttpURLConnection conn = new AuthenticatedURL(url, token).openConnection(url, token);

Using Java code plus the Hadoop Java API to open a Kerberized session, get the Delegation Token for the session, and pass that Token to the other app -- as suggested by #tellisnz -- has a drawback: the Java API requires quite a lot of dependencies (i.e. a lot of JARs, plus Hadoop native libraries). If you run you app on Windows, in particular, it will be a tough ride.
Another option is to use Java code plus WebHDFS to run a single SPNEGOed query and GET the Delegation Token, then pass it to the other app -- that option requires absolutely no Hadoop library on your server. The barebones version would be sthg like
URL urlGetToken = new URL("http://<host>:<port>/webhdfs/v1/?op=GETDELEGATIONTOKEN") ;
HttpURLConnection cnxGetToken =(HttpURLConnection) urlGetToken.openConnection() ;
BufferedReader httpMessage = new BufferedReader( new InputStreamReader(cnxGetToken.getInputStream()), 1024) ;
Pattern regexHasToken =Pattern.compile("urlString[\": ]+(.[^\" ]+)") ;
String httpMessageLine ;
while ( (httpMessageLine =httpMessage.readLine()) != null)
{ Matcher regexToken =regexHasToken.matcher(httpMessageLine) ;
if (regexToken.find())
{ System.out.println("Use that template: http://<Host>:<Port>/webhdfs/v1%AbsPath%?delegation=" +regexToken.group(1) +"&op=...") ; }
}
httpMessage.close() ;
That's what I use to access HDFS from a Windows Powershell script (or even an Excel macro). Caveat: with Windows you have to create your Kerberos TGT on the fly, by passing to the JVM a JAAS config pointing to the appropriate keytab file. But that caveat also applies to the Java API, anyway.

You could take a look at the hadoop-auth client and create a service which does the first connection, then you might be able to grab the 'Authorization' and 'X-Hadoop-Delegation-Token' headers and cookie from it and add it to your basic client's requests.
First you'll need to have either used kinit to authenticate your user for application before running. Otherwise, you're going to have to do a JAAS login for your user, this tutorial provides a pretty good overview on how to do that.
Then, to do the login to WebHDFS/HttpFS, we'll need to do something like:
URL url = new URL("http://youhost:8080/your-kerberised-resource");
AuthenticatedURL.Token token = new AuthenticatedURL.Token();
HttpURLConnection conn = new AuthenticatedURL().openConnection(url, token);
String authorizationTokenString = conn.getRequestProperty("Authorization");
String delegationToken = conn.getRequestProperty("X-Hadoop-Delegation-Token");
...
// do what you have to to get your basic client connection
...
myBasicClientConnection.setRequestProperty("Authorization", authorizationTokenString);
myBasicClientConnection.setRequestProperty("Cookie", "hadoop.auth=" + token.toString());
myBasicClientConnection.setRequestProperty("X-Hadoop-Delegation-Token", delegationToken);

JSF - How can I take full control of a deployed managed bean if I want to troubleshoot?

I was given a URL that returns a name of person based on an unique id parameter (rest web service).
When I entered below url to a browser I was able to get expected name wrapped in json format via GET method. If I enter "id=1234" then name comes back as "DaeYoung". However it has to be https instead of http so that I have to download a CA cert from the server and import it to my browser otherwise it won't work.
https://[host:port]/fooService/queryName?id=1234
What I want to accomplish is create a managed bean that makes rest ws call in JSF (2.x) web application. First I ran a test code from a standalone Java class below from eclipse IDE to verify if I could get name value back. And yes, I was able to get expected name back.
However I had to add a CA cert from the server (rest ws service provider) to a local Java_Home/jre/lib/security/cacerts file and include HostnameVerifier in the code otherwise I would get java.security.cert.CertificateException: No subject alternative names matching IP address xxx.xxx.xxx.xxx found.
public void findName() throws Exception {
HttpsURLConnection.setDefaultHostnameVerifier(new HostnameVerifier()
{
public boolean verify(String hostname, SSLSession session) {
// ip address of rest service provider
if (hostname.equals(127.0.0.1))
return true;
return false;
}
});
String url = "https://[host:port]/fooService/queryOIN";
String query = "id=1234";
String charset = "utf-8";
URLConnection conn = new URL(url + "?" + query).openConnection();
conn.setRequestProperty("Accept-Charset", charset);
InputStream in = conn.getInputStream();
String name = IOUtils.toString(in,charset);
System.out.println("name=" + name);
}
Now when I put above code into a bean and deployed a web app into an app server (websphere), I observed two events.
1. with a valid id "1234" provided, name "DaeYoung" expected however return value was empty string
2. no exception was thrown with above condition which I expected
I was tailing system log file but nothing abnormal behaviors are thrown.
My questions is...
How can I troubleshoot or verify above code running inside of the app server in order to grasp what is happening inside of findName() method?

HttpServletRequest#getHeader("User-Agent") returns null browser name

I'm using Java 6. I have very less knowledge of JSP and Servlets.
I'm using the following code to get browser name in which my application is running:
String browserName = requestProvider.get().getHeader("User-Agent");
Also I'm using the following code to get IP address of the machine on which my application is running:
String ipAdd = requestProvider.get().getRemoteAddr();
In both the cases requestProvider is a reference variable of type Provider<HttpServletRequest>. And I'm assured that it is never NULL.
Now the problem is some times I get both values (browserName and ipAdd) NULL.
I've written sometimes because I don't have a test case.
So my question is, what are the cases in Java, when these values can be NULL?
What care should I take in coding to avoid this issue?
Is there any alternate way to get IP address & browser name every time?

String browserName = requestProvider.get().getHeader("User-Agent");
null means whoever sent the request didn't include a "User-Agent" header.
String ipAdd = requestProvider.get().getRemoteAddr();
is unlikely to return null under normal circumstances, but there are reports the it may do so in edge cases, like after the response has already been sent. Regardless, "get IP address of the machine on which my application is running" doesn't sound like what getRemoteAddr() is for. It's for getting the address of the most recent client or proxy that sent the request.
Is there any alternate way to get IP address & browser name every time?
No. You're entirely dependent on the behavior of the HTTP client and/or any intervening proxies to get information like this.

Try using user-agent as lowercase, because it works if we directly access from header.
String browserName = requestProvider.get().getHeader("user-agent");
alternate way to get IP address is
String ip = requestProvider.get().getHeader("True-Client-IP"); this works if we have akamai integeration.

how to call a resource url running on the same AppEngine instance from a servlet

I have implemented some RESTful resource urls such as http://myapp.appspot.com/temperature/get with Jersey. I also have a servlet running on the same AppEngine instance with which I want to call the resource urls. My problem is that I don't know how to refer to the running Appengine instance the servlet is within. In other words, I'd like to use the AppEngine equivalent of 127.0.0.1 or $SERVER_HOME.
My initial approach was to do something like this, in the servlet:
URL url = new URL(getServletContext().getContextPath());
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
, but getContextPath() returns an empty string (as it should, since I don't place the application name in the url.
How do I get a reference to the app's base location, so I can reference url resources running within it? Thanks in advance!

To check if the URL exists in the same server machine do this:
String host = remoteHost;
int port = remotePort;
String remoteContext = //the application root you want to access;
String remtoePath = // the servlet or path you want to access;
if((host.equals(request.getServerName()) || host.equals("localhost"))
&& port == request.getServerPort(){
....
ServletContext context = servletContext.get(remoteContext);
RequestDispatcher dispatcher = context.getRequestDispatcher(remotePath);
//if you want to forward
//dispatcher.forward(request, response);
//if you want to include
//dispatcher.include(request, response);
}

The same instance or the same version of the code? You can't target the same instance. There's no way to do that. You can target the same version (yourdomain.appspot.com or latest.yourdomain.appspot.com). If you want to know if your app is running in the dev server you can use the SystemProperty class
https://developers.google.com/appengine/docs/java/javadoc/com/google/appengine/api/utils/SystemProperty
If you want to target a version, you should really know the url... if not, use Ramesh's suggestion. Generally we set up a static properties class in our servlets that hold the name of the currently active domain for this purpose. Or just use TaskQueues... they are great for this purpose.

illegal character with 10.0.2.2 when I call my webservice from android ADB

I made a webservice with netbeans (Java EE) and I want to communicate with my Android app (made with eclipse).
When I'm trying to communicate, for example, I call my webservice through my browser :
localhost:9999/SupRails_Start/resources/rest/mail#gmail.com/password
... And it give me a JSON, ...
With the ADB, I call that URI :
10.0.2.2:9999/SupRails_Start/resources/rest/mail#gmail.com/password
But it give me
java.lang.IllegalArgumentException: Illegal character in scheme at index 0: 10.0.2.2:0000/SupRails_Start/resources/rest/...
When I copy-paste the JSON (received from the browser) into a php page and that I host it on the web, it's works perfectly !
Here is a part of my Android project :
WebService webService = new WebService("10.0.2.2:9999/SupRails_Start/resources/rest/wil.nad#gmail.com/root");
Map<String, String> params = new HashMap<String, String>();
String temp = webService.webGet("", params);
Log.v("", "JSON received = "+temp);
Please help me I don't understand my mistake... Thanks a lot!

Your web service URL is not complete. The syntax of a valid URL is like:
scheme://domain:port/path?query_string#fragment_id
Every URL consists of some of the following: the scheme name (commonly
called protocol), followed by a colon, two slashes,[note 1] then,
depending on scheme, a server name (exp. ftp., www., smtp., etc)
followed by a dot (.) then a domain name[note 2] (alternatively, IP
address), a port number, the path of the resource to be fetched or the
program to be run, then, for programs such as Common Gateway Interface
(CGI) scripts, a query string,[7][8] and an optional fragment
identifier.
[Wikipedia]
So change this:
new WebService("10.0.2.2:9999/SupRails_Start/resources/rest/wil.nad#gmail.com/root")
to this:
new WebService("http://10.0.2.2:9999/SupRails_Start/resources/rest/wil.nad#gmail.com/root")