I am a newbie to ICEfaces and i have a requirement where i need to download a document from a given url (http://ipaddress/formexec?objectid=201).
This URL uses a form based authentication that is deployed through ICEFaces.
i tracked the request of this URL and i get the following line:
&ice.submit.partial=false&ice.event.target=loginForm%3Aj_id33&ice.event.captured=loginForm%3Aj_id33
Is there any libraries or code to download the document by successfully passing the username and password.
You need to extract the jsessionid from the Set-Cookie response header and append it as URL attribute to the subsequent requests as http://example.com/path/page.jsf;jsessionid=XXX.
Here's a kickoff example with help of "plain vanilla" java.net.URLConnection:
// Prepare stuff.
String loginurl = "http://example.com/login";
String username = "itsme";
String password = "youneverguess";
URLConnection connection = null;
InputStream response = null;
// First get jsessionid (do as if you're just opening the login page).
connection = new URL(loginurl).openConnection();
response = connection.getInputStream(); // This will actually send the request.
String cookie = connection.getHeaderField("Set-Cookie");
String jsessionid = cookie.split(";")[0].split("=")[1]; // This assumes JSESSIONID is first field (normal case), you may need to change/finetune it.
String jsessionidurl = ";jsessionid=" + jsessionid;
response.close(); // We're only interested in response header. Ignore the response body.
// Now do login.
String authurl = loginurl + "/j_security_check" + jsessionidurl;
connection = new URL(authurl).openConnection();
connection.setDoOutput(true); // Triggers POST method.
PrintWriter writer = new PrintWriter(new OutputStreamWriter(connection.getOutputStream()));
writer.write("j_username=" + URLEncoder.encode(username, "UTF-8")
+ "&j_password=" + URLEncoder.encode(password, "UTF-8"));
writer.close();
response = connection.getInputStream(); // This will actually send the request.
response.close();
// Now you can do any requests in the restricted area using jsessionid. E.g.
String downloadurl = "http://example.com/download/file.ext" + jsessionidurl;
InputStream download = new URL(downloadurl).openStream();
// ...
To achieve the same with less bloated code, consider Apache Commons HttpComponents Client.
Form based auth is not much different from other requests. All you have to do is to submit an a request to the auth form providing required parameters, such as user and password and in some cases additional token that you would have to get from the source page. Then you need to get cookies from the auth response or session id parameter and copy them to your next request that will fetch the data.
Related
UPDATE
My problem turned out to be due to some super-class weirdness. However, the urlEncode in this example is wrong. In this case it was also unnecessary. If needed it should be done by each key and value without encoding & or =.
Other than that, this code is correct.
The (scala) code below fails because the server does not seem to receive the body.
The warning on the server log is:
WARNING *** OAuthTokenProcessor caught a OAuthRequestProblemException with message OAuthProblemException{error='invalid_request', description='Missing grant_type parameter value', uri='null', state='null', scope='null', redirectUri='null', responseStatus=0, parameters={}}
I am certain that the params are correct. The url is correct, The headers are correct
(This call already works in Postman and Python)
Any hints deeply appreciated!
private def doPostRequest(): Unit ={
try {
val connectionforPost = (new URL(url)).openConnection.asInstanceOf[HttpURLConnection]
val params = "grant_type=password&client_id=xxxx&client_secret=seccret&username=admin&password=xxxx"
val encodedString = URLEncoder.encode(params,"UTF-8")
import java.nio.charset.StandardCharsets
val postData = encodedString.getBytes(StandardCharsets.UTF_8)
val contentlength = Integer.toString(encodedString.length)
connectionforPost.setConnectTimeout(50000)
connectionforPost.setReadTimeout(50000)
connectionforPost.setRequestMethod("POST")
connectionforPost.setDoOutput(true)
connectionforPost.setDoInput(true)
connectionforPost.setInstanceFollowRedirects(false)
connectionforPost.setRequestProperty("User-agent", "test bot")
connectionforPost.setRequestProperty("Content-Type", "application/x-www-form-urlencoded")
connectionforPost.setRequestProperty("Accept", "*/*")
connectionforPost.setRequestProperty("Connection", "keep-alive")
connectionforPost.setRequestProperty("Content-Length", contentlength)
connectionforPost.setRequestProperty("Accept-Encoding", "gzip, deflate")
connectionforPost.setUseCaches(false)
import java.io.DataOutputStream
val wr = new DataOutputStream(connectionforPost.getOutputStream)
wr.write(postData)
if (wr != null) wr.close()
} catch{
case e:Exception => logger.error(e.getMessage,e)
throw e
}
}
It appears that the server complains that you are not logged in. You get an authentication problem exception. How it usually works is that you send a request to the server to login with the user name and password. If the server authenticates you (recognizes your user name and password as valid ones) it sends you a response with an authentication token in the headers. You may see those 2 headers in your response to successful login:
access-control-expose-headers=[Authorization], Authorization=[Bearer ....]
This means that in all your subsequent requests you will need to add a header "Authorization" that will hold the value "Bearer ....". Then the server will recognize this request as coming from an authenticated user.
Also, I suggest that you may use a 3d party HTTP client. Some well-known clients are Appache HTTP client and OK HTTP client. I use my own Http client that I wrote. It is also available as part of a MgntUtils Open Source Library. Here is the link to Maven artifacts and it is also available on Github with source code and JavaDoc. And here is a javadoc page for HttpClient. Here is the code sample on how you may obtain the token and use it:
try {
HttpClient loginClient = new HttpClient();
loginClient.setConnectionUrl("http://your_url/login");
loginClient.setRequestProperty("accept", "application/json;charset=UTF-8");
loginClient.setContentType("application/json");
String result = loginClient.sendHttpRequest(HttpMethod.POST, "{ \"username\": \"your_user_name\", \"password\": \"Your_password\"}");
System.out.println(result);
System.out.println("HTTP " + loginClient.getLastResponseCode() + " " + loginClient.getLastResponseMessage());
System.out.println("Response headers: " + loginClient.getLastResponseHeaders());
String accessControlExposeHeader = loginClient.getLastResponseHeader("access-control-expose-headers").get(0);
String accessKey = loginClient.getLastResponseHeader(accessControlExposeHeader).get(0);
HttpClient workingClient = new HttpClient();
workingClient.setRequestProperty("accept", "application/json;charset=UTF-8");
workingClient.setContentType("application/json");
workingClient.setRequestProperty(accessControlExposeHeader, accessKey);
workingClient.setConnectionUrl("http://yourUrl/yourPath");
System.out.println(workingClient.sendHttpRequest(HttpMethod.GET));
} catch (IOException e) {
System.out.println(TextUtils.getStacktrace(e));
}
Am making webservice calls to HTTPS server from an android application. Below is the code snippet, with which am able to make web service calls successfully and getting response.
My Question is, do we need to perform any additional step to encrypt data before making call to HTTPS server?
Because, from android profiler am able to see all my Web Requests in plain text format. My understanding is that request will gets encrypted before making HTTPS call.
public static WebServiceResp makeWebServiceCall(String XML, String urlPath) throws IOException{
//Code to make a web service HTTP request
String responseString = "";
String outputString = "";
String wsURL = urlPath;
URL url = new URL(wsURL);
URLConnection connection = url.openConnection();
HttpsURLConnection httpConn = (HttpsURLConnection)connection;
ByteArrayOutputStream bout = new ByteArrayOutputStream();
//System.out.println(XML);
byte[] buffer = new byte[XML.length()];
buffer = XML.getBytes();
bout.write(buffer);
byte[] b = bout.toByteArray();
// Set the appropriate HTTP parameters.
httpConn.setRequestProperty("Content-Length",
String.valueOf(b.length));
httpConn.setRequestProperty("Content-Type", "text/xml; charset=utf-8");
httpConn.setRequestMethod("POST");
httpConn.setRequestProperty("Cache-Control", "no-cache");
httpConn.setDoOutput(true);
httpConn.setDoInput(true);
OutputStream out = httpConn.getOutputStream();
//Write the content of the request to the outputstream of the HTTP Connection.
out.write(b);
out.close();
//Ready with sending the request.
//Check the status
int status = httpConn.getResponseCode();
Log.d(TAG, "makeWebServiceCall: "+"Processing Status: "+status);
BufferedReader in;
if (status <= 200) {
//Read the response.
Log.d(TAG, "makeWebServiceCall: Getting Input Stream");
InputStreamReader isr =
new InputStreamReader(httpConn.getInputStream());
in = new BufferedReader(isr);
}else{
//Read the response.
Log.d(TAG, "makeWebServiceCall: Getting Error Stream");
InputStreamReader isr =
new InputStreamReader(httpConn.getErrorStream());
in = new BufferedReader(isr);
}
//Write the SOAP message response to a String.
while ((responseString = in.readLine()) != null) {
outputString = outputString + responseString;
}
Log.d(TAG, "makeWebServiceCall: WebServiceResponse " + outputString);
//Parse the String output to a org.w3c.dom.Document and be able to reach every node with the org.w3c.dom API.
Document document = Utils.parseXmlFile(outputString);
//NodeList nodeLst = document.getElementsByTagName("GetWeatherResult");
// String weatherResult = nodeLst.item(0).getTextContent();
//System.out.println("Weather: " + weatherResult);
//Write the SOAP message formatted to the console.
WebServiceResp webServiceResp = new WebServiceResp();
webServiceResp.setDocument(document);
webServiceResp.setStatus(status);
return webServiceResp;
}
No. If you're sending it to an https website, the encryption is done as part of the protocol. You don't need to do any additional work.
No. The encryption that you see is on the network layer. The client which initiates the https call see what was sent and what was received. That is how https works.
When you look at chrome browser's network tab, you see what was sent and what was received. Now this is not a security problem, https is more about you doing things which make its difficult for anyone between the network to eavesdrop your data.
Now if you still want an additional level of security you can use certificate pinning
https://www.owasp.org/index.php/Certificate_and_Public_Key_Pinning
https://medium.com/#appmattus/android-security-ssl-pinning-1db8acb6621e
How can you add to network_security_config from MainActivity
So in this technique you basically say that the certificate hash that you expected is to have this content. And then if someone uses a trusted proxy with trusted CAs on the system, even after generating a valid certificate for the given domain the connections will not be established.
HTTPS is transparent to your application, all of the magic happens between Transport Layer(so it calls 'Transport Layer Security'), you may imagine encrypted telegrams in the old days, generals tell telegrapher messages in plain text, and telegrapher send them in encrypted form(maybe use some kind of codebook), anyone who didn't have the same codebook can't decrypt the message easily, and anyone who uses telegrams didn't care about the codebook(or even known about it, except those telegraphers on both side of the 'Transport Layer').
The encryption/decryption is done by built-in network client module provided by OS. So you needn't worry about it.
You can view plain texts with some client tools as they know exactly what they are sending/receiving. E.g. chrome developer tool. (Actually they don't care about encryption/decryption either).
I want to read a table from Google Sheets from my android application.
I want to do that by Google Sheets APIs.
I declared the sheet as public, created API key and tried to send the GET service call:
https://sheets.googleapis.com/v4/spreadsheets/{My Sheet key}/values/responses:append?key={My API credential key}
I get 401 code.
Response:
Request is missing required authentication credential. Expected OAuth
2 access token, login cookie or other valid authentication credential.
See
https://developers.google.com/identity/sign-in/web/devconsole-project.
My code:
private static final String SHEET_URL = "https://sheets.googleapis.com/v4/spreadsheets/1d534sQ5xaNbr65wMM_qH2yjXo3EPrrp3o34z-Foledg/values/responses:append?key=AIzaSyDT88Nq6jhtaKH-vIVEuvGO1d9Sx8ewR0w";
public String GetConanimList() throws Exception {
URL url = new URL(SHEET_URL);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("GET");
conn.setRequestProperty("Content-Type", "application/json");
OutputStream os = conn.getOutputStream();
String jsonPayload = null;
//os.write(jsonPayload.getBytes());
os.flush();
os.close();
int statusCode = conn.getResponseCode();
System.out.println("Response from WA Gateway: \n");
System.out.println("Status Code: " + statusCode);
BufferedReader br = new BufferedReader(new InputStreamReader(
(statusCode == 200) ? conn.getInputStream() : conn.getErrorStream()
));
String output;
String response = "";
while ((output = br.readLine()) != null) {
response = response + output;
}
conn.disconnect();
return response;
}
What am I missing? Thanks.
To use Google Sheets API, you need to authorize your requests.
There are two ways to identify your application: using an OAuth 2.0
token (which also authorizes the request) and/or using the
application's API key. Here's how to determine which of those
options to use:
If the request requires authorization (such as a request for an individual's private data), then the application must provide an OAuth 2.0 token with the request. The application may also provide the API key, but it doesn't have to.
If the request doesn't require authorization (such as a request for public data), then the application must provide either the API key or an OAuth 2.0 token, or both—whatever option is most convenient for you.
You can also refer to the quickstart projects available as a guide on how to correctly implement this.
how to get httponly cookies? What tools should I use org.apache.http or jsoup, may be url connection... please anybody give me example. I tryed to get cookies using jsoup, but jsoup doesn't return httponly cookies.
Thanks!!!
You can use java.net.CookieHandler, referenced at: Java Cookie Handling
As an example taken directly from http://www.hccp.org/java-net-cookie-how-to.html:
Retrieving cookies from a response:
Open a java.net.URLConnection to the server:
URL myUrl = new URL("http://www.hccp.org/cookieTest.jsp");
URLConnection urlConn = myUrl.openConnection();
urlConn.connect();
Loop through response headers looking for cookies:
Since a server may set multiple cookies in a single request, we will need to loop through the response headers, looking for all headers named "Set-Cookie".
String headerName=null;
for (int i=1; (headerName = uc.getHeaderFieldKey(i))!=null; i++) {
if (headerName.equals("Set-Cookie")) {
String cookie = urlConn.getHeaderField(i);
...
Extract cookie name and value from cookie string:
The string returned by the getHeaderField(int index) method is a series of name=value separated by semi-colons (;). The first name/value pairing is actual data string we are interested in (i.e. "sessionId=0949eeee22222rtg" or "userId=igbrown"), the subsequent name/value pairings are meta-information that we would use to manage the storage of the cookie (when it expires, etc.).
cookie = cookie.substring(0, cookie.indexOf(";"));
String cookieName = cookie.substring(0, cookie.indexOf("="));
String cookieValue = cookie.substring(cookie.indexOf("=") + 1, cookie.length());
This is basically it. We now have the cookie name (cookieName) and the cookie value (cookieValue).
please anybody give me example
Here is one with Jsoup.
SAMPLE CODE
Response response;
try {
response = Jsoup //
.connect("https://httpbin.org/cookies/set?http-only-cookie=test;%20httponly") //
.ignoreContentType(true) //
.execute();
System.out.println(response.cookies());
} catch (IOException e) {
throw new RuntimeException(e);
}
OUTPUT
{http-only-cookie="test\073 httponly"}
I have searched for a while and I am not finding a clear answer. I am trying to log into a webstie.
https://hrlink.healthnet.com/
This website redirects to a login page that is not consitent. I have to post my login credentials to the redirected URL.
Im am trying to code this in Java but I do not understand how to get the URL from the response. It may look a bit messy but I have it this way while I am testing.
HttpGet httpget = new HttpGet("https://hrlink.healthnet.com/");
HttpResponse response = httpclient.execute(httpget);HttpEntity entity = response.getEntity();
String redirectURL = "";
for(org.apache.http.Header header : response.getHeaders("Location")) {
redirectURL += "Location: " + header.getValue()) + "\r\n";
}
InputStream is;
is = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(is,"iso-8859-1"),8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
String result = sb.toString();
I know i get redirected because my result string shows be the actual login page but I am not able to get the new URL.
In FireFox I am using TamperData. When I navigate to this website https://hrlink.healthnet.com/ I have a GET with a 302 - Found and the Location of the Login Page. Then another GET to the actual Login Page
Any help is greatly appreciated thank you.
Check out w3c documentation:
10.3.3 302 Found
The temporary URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s).
If the 302 status code is received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.
One solution is to use POST method to break auto-redirecting at client side:
HttpPost request1 = new HttpPost("https://hrlink.healthnet.com/");
HttpResponse response1 = httpclient.execute(request1);
// expect a 302 response.
if (response1.getStatusLine().getStatusCode() == 302) {
String redirectURL = response1.getFirstHeader("Location").getValue();
// no auto-redirecting at client side, need manual send the request.
HttpGet request2 = new HttpGet(redirectURL);
HttpResponse response2 = httpclient.execute(request2);
... ...
}
Hope this helps.