I'm porting an Android app on GAE.
The app compile and post a login form to enter in a
password protected site.
I'm using UrlFetchTransport instead of DefaultHttpClient.
I can get web page without problem using the following piece of code:
HttpTransport HTTP_TRANSPORT = new UrlFetchTransport();
HttpRequestFactory requestFactory = HTTP_TRANSPORT.createRequestFactory();
GenericUrl genericUrl = new GenericUrl(url);
HttpRequest request = requestFactory.buildGetRequest(genericUrl);
com.google.api.client.http.HttpHeaders httpHeaders = request.getHeaders();
httpHeaders.setUserAgent("Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57 Safari/537.36");
httpHeaders.setAccept("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
String paginaRicevuta = request.execute().parseAsString();
I would try a POST request using the following:
UrlFetchTransport HTTP_TRANSPORT = new UrlFetchTransport();
HttpRequestFactory requestFactory = HTTP_TRANSPORT.createRequestFactory();
GenericUrl genericUrl = new GenericUrl("https://www.......");
List<BasicNameValuePair> nvps = new ArrayList<BasicNameValuePair>();
nvps.add(new BasicNameValuePair("ReturnUrl", ""));
nvps.add(new BasicNameValuePair("userName", "username"));
nvps.add(new BasicNameValuePair("password","password"));
//HERE HOW DEFINE httpContent????
HttpContent httpContent=null;
final HttpRequest postRequest = requestFactory.buildPostRequest(genericUrl,httpContent);
postRequest.setContent(httpContent);
postRequest.setFollowRedirects(true);
HttpHeaders httpHeaders = postRequest.getHeaders();
httpHeaders.setUserAgent("Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57 Safari/537.36");
httpHeaders.setAccept("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
httpHeaders.setContentType("application/x-www-form-urlencoded");
postRequest.setHeaders(httpHeaders);
String pagePost = postRequest.execute().parseAsString();
but I don't know how format a right POST request?
Thanks for your help.
Related
I'm trying to send a GET request with RestTemplate. My code looks like:-
RestTemplate template = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Arrays.asList(MediaType.APPLICATION_JSON));
headers.add("user-agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36");
headers.add("_rToken", cookie.getValue());
String url = getUrl(contextUrl);
HttpEntity<Object> entity = new HttpEntity<Object>(headers);
ResponseEntity<String> response = null;
try {
response = template.exchange(url, HttpMethod.GET, entity, String.class);
}
catch(RestClientException ex) {
ex.printStackTrace();
}
return response;
On debugging, my URL looks good. What I have as entity is:-
body = null,
headers = <{Accept=[application/json],
user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36],
_rToken=[2a6ac90f-1dfb-4df3-8d23-6d8d948fb9b5]}>
Since my get method doesn't take a parameter, body is supposed to be null. What am I doing wrong here?
Stack trace:
org.springframework.web.client.HttpClientErrorException: 403 null at
org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:94)
at
org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:79)
at
org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63)
at
org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:773)
at
org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:726)
at
org.springframework.web.client.RestTemplate.execute(RestTemplate.java:682)
at
org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:598)
at com.hm.ul.util.PlpUMCommunicator.get(PlpUMCommunicator.java:53)
Can get from the response from Chrome, but when I want to get it with resttemplate it gives me 401 response.
I try to add the exact Content Types with rest template but it still gives the 401 error.
here is the request that i get it from Chrome :
GET URL HTTP/1.1
Host: host
Connection: keep-alive
Cache-Control: max-age=0
Authorization: Basic auth
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en,de;q=0.9,tr;q=0.8
and here is the code that i used in java to connect the service :
RestTemplate restTemplate = new RestTemplate();
List<HttpMessageConverter<?>> messageConverters = new ArrayList<>();
MappingJackson2HttpMessageConverter converter = new MappingJackson2HttpMessageConverter();
converter.setSupportedMediaTypes(Arrays.asList(MediaType.APPLICATION_XHTML_XML, MediaType.APPLICATION_XML, MediaType.TEXT_HTML));
messageConverters.add(converter);
restTemplate.setMessageConverters(messageConverters);
HttpEntity<Map<String, String>> entity = new HttpEntity<>(params,createHeaders(username, password));
restTemplate.getForObject("URL",String.class, entity);
public static HttpHeaders createHeaders(String username, String password) {
String auth = username + ":" + password;
byte[] encodedAuth = Base64.encodeBase64(auth.getBytes(Charset.forName("US-ASCII")));
String authHeader = "Basic " + new String(encodedAuth);
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.set("Authorization", authHeader);
return httpHeaders;
}
It appears to be a problem with encoding the authorization token generation.
If you are sure about the username and password is correct, the only possibility is that the method/encoding used to generate the token is different on the browser(or the UI application which is generating the token) and the implementation of your createHeaders() method.
Please refer to the below links for more details:
What encoding should I use for HTTP Basic Authentication?
https://github.com/request/request/issues/1334
Hope it'll help.
help me with jsoup login, i have to login, then redirect from login page to another page with saved session.
public void parseXhtml() throws IOException{
String sessionID=null;
Map<String, String> cookies = new HashMap<String, String>();
cookies.put("login", "login");
cookies.put("password", "password");
Connection conn=Jsoup.connect("http://localhost:8080/dir/login.xhtml");
Connection.Response res = Jsoup
.connect("http://localhost:8080/dir/login.xhtml")
.data(cookies)
.execute();
res = Jsoup.connect("http://localhost:8080/dir/dir/index.xhtml")
.cookie("JSESSIONID", res.cookies().get("JSESSIONID"))
.method(Method.GET)
.userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36")
.execute();
Document doc = res.parse();
System.out.println(doc.html());
sessionID = res.cookie("JSESSIONID");
Document docu = Jsoup.connect("http://localhost:8080/dir/dir/index.xhtml")
.userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36")
.cookie("JSESSIONID", res.cookies().get("JSESSIONID"))
.method(Connection.Method.GET)
.get();
it throws the below exception
java.io.IOException: 403 error loading URL http://localhost:8080/dir/dir/index.xhtml
if i'm doing like [Sending POST request with username and password and save session cookie it throws the same exception.
ExternalContext ec=FacesContext.getCurrentInstance().getExternalContext();
HttpServletRequest req=(HttpServletRequest) ec.getRequest();
HttpSession sess=(HttpSession) ec.getSession(true);
String url = req.getRequestURL().append(";jsessionid=").append(sess.getId()).toString();
ec.setRequest("http://localhost:8080/dir/dir/index.xhtml");
HttpServletRequest req2=(HttpServletRequest) ec.getRequest();
String url2 = req2.getRequestURL().append(";jsessionid=").append(sess.getId()).toString();
Document doc2=Jsoup.connect(url2).get();
System.out.println(doc2.html());
Finally i got it, I do not know if it is right, but works
I need to login to external web site, and use HttpClient in combination with CookieStore. As a guide I used http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/httpclient/src/examples/org/apache/http/examples/client/ClientCustomContext.java and http://hc.apache.org/httpcomponents-client-4.4.x/tutorial/html/statemgmt.html
Here is my code:
RequestConfig globalConfig = RequestConfig.custom().setCookieSpec(CookieSpecs.BEST_MATCH).build();
CookieStore cookieStore = new BasicCookieStore();
HttpClientContext context = HttpClientContext.create();
context.setCookieStore(cookieStore);
CloseableHttpClient httpClient = HttpClients.custom()
.setDefaultRequestConfig(globalConfig)
.setDefaultCookieStore(cookieStore)
.build();
HttpGet httpGet = new HttpGet(urlAddress);
httpGet.addHeader("Accept", "text/html, application/xhtml+xml, */*");
httpGet.addHeader("Accept-Language", "en-US,en;q=0.7,ru;q=0.3");
httpGet.addHeader("User-Agent", "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)");
httpGet.addHeader("Accept-Encoding","gzip, deflate");
//httpGet.addHeader("Host", host); // should I copy host here?
httpGet.addHeader("Connection", "Keep-Alive");
CloseableHttpResponse response1 = httpClient.execute(httpGet);
List<Cookie> cookies1 = context.getCookieStore().getCookies();
try {
System.out.println(response1.getStatusLine());
HttpEntity entity1 = response1.getEntity();
// do something useful with the response body
streamToFile(entity1.getContent(), "./login_result_1.html");
// and ensure it is fully consumed
EntityUtils.consume(entity1);
} finally {
response1.close();
}
That works fine.
When I do further request to the site, should I do any special code to pass cookies obtained in the request above?
Should I regenerate cookies or that will be done by Apache?
I was not able to find clear answer. Also my further request is not handled by web site properly and it doesn't allow me to login.
HttpPost httpPost = new HttpPost(urlAddress);
httpPost.addHeader("Accept", "text/html, application/xhtml+xml, */*");
httpPost.addHeader("Accept-Language", "en-US,en;q=0.7,ru;q=0.3");
httpPost.addHeader("Referer", "http://virtonomica.ru/");
httpPost.addHeader("User-Agent", "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)");
httpPost.addHeader("Accept-Encoding","gzip, deflate");
httpPost.addHeader("Content-Type", "application/x-www-form-urlencoded");
httpPost.addHeader("Accept-Encoding", "gzip, deflate");
//httpPost.addHeader("Host", "virtonomica.ru");
//httpPost.addHeader("Content-Length", "51");//LENGTH
httpPost.addHeader("Connection", "Keep-Alive");
httpPost.addHeader("Cache-Control", "no-cache");
httpPost.setEntity(new UrlEncodedFormEntity(nvps)); // nvps is List<NameValuePair> which contains POST data
CloseableHttpResponse response = httpClient.execute(httpPost);
Please advise.
try{
HttpEntity entity=null;
HttpGet httpget=null;
HttpResponse response=null;
httpclient = new DefaultHttpClient();
httpget = new HttpGet(credentialsURL);
httpget.setHeader(HttpHeaders.USER_AGENT, "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.120 Safari/535.2");
response = httpclient.execute(httpget);
entity = response.getEntity();
if (entity != null) {
EntityUtils.consume(entity);
}
}
catch(Exception e){
}
return httpclient;
I am using HttpClient to handle session.Here is my code to handle session using httpclient. But httpclient won't handle ajax calls.
I am planning to move to httpunit to overcome my problem.
Basically I have a referral URL(or authentication server) from which I need to get cookies and session and store it in client.
And use this client across website to get logged user information.
What is the appropriate solution for this using httpunit.
credentialsURL="https://www.,,,,.com./sapLogin.aspx?HOOK_URL=https://authentication.server.address&username=&password=
";