I have to upload a file to a server which only exposes a jsf web page with file upload button (over http). I have to automate a process (done as java stand alone process) which generates a file and uploads the file to the server.Sadly the server to where the file has to be uploaded does not provide a FTP or SFTP. Is there a way to do this?
Thanks,
Richie
When programmatically submitting a JSF-generated form, you need to make sure that you take the following 3 things in account:
Maintain the HTTP session (certainly if website has JSF server side state saving turned on).
Send the name-value pair of the javax.faces.ViewState hidden field.
Send the name-value pair of the button which is virtually to be pressed.
Otherwise the action will possibly not be invoked at all. For the remnant it's not different from "regular" forms. The flow is basically as follows:
Send a GET request on the page with the form.
Extract the JSESSIONID cookie.
Extract the value of the javax.faces.ViewState hidden field from the response. If necessary (for sure if it has a dynamically generated name and thus possibly changes every request), extract the name of input file field and the submit buttonas well. Dynamically generated IDs/names are recognizeable by the j_id prefix.
Prepare a multipart/form-data POST request.
Set the JSESSIONID cookie (if not null) on that request.
Set the name-value pair of javax.faces.ViewState hidden field and the button.
Set the file to be uploaded.
You can use any HTTP client library to perform the task. The standard Java SE API offers java.net.URLConnection for this, which is pretty low level. To end up with less verbose code, you could use Apache HttpClient to do the HTTP requests and manage the cookies and Jsoup to extract data from the HTML.
Here's a kickoff example, assuming that the page has only one <form> (otherwise you need to include an unique identifier of that form in Jsoup's CSS selectors):
String url = "http://localhost:8088/playground/test.xhtml";
String viewStateName = "javax.faces.ViewState";
String submitButtonValue = "Upload"; // Value of upload submit button.
HttpClient httpClient = new DefaultHttpClient();
HttpContext httpContext = new BasicHttpContext();
httpContext.setAttribute(ClientContext.COOKIE_STORE, new BasicCookieStore());
HttpGet httpGet = new HttpGet(url);
HttpResponse getResponse = httpClient.execute(httpGet, httpContext);
Document document = Jsoup.parse(EntityUtils.toString(getResponse.getEntity()));
String viewStateValue = document.select("input[type=hidden][name=" + viewStateName + "]").val();
String uploadFieldName = document.select("input[type=file]").attr("name");
String submitButtonName = document.select("input[type=submit][value=" + submitButtonValue + "]").attr("name");
File file = new File("/path/to/file/you/want/to/upload.ext");
InputStream fileContent = new FileInputStream(file);
String fileContentType = "application/octet-stream"; // Or whatever specific.
String fileName = file.getName();
HttpPost httpPost = new HttpPost(url);
MultipartEntity entity = new MultipartEntity();
entity.addPart(uploadFieldName, new InputStreamBody(fileContent, fileContentType, fileName));
entity.addPart(viewStateName, new StringBody(viewStateValue));
entity.addPart(submitButtonName, new StringBody(submitButtonValue));
httpPost.setEntity(entity);
HttpResponse postResponse = httpClient.execute(httpPost, httpContext);
// ...
Try using HttpClient, here's an article that I think describes what you want, towards the bottom there's a section titled "Using HttpClient-Based FileUpload".
Hope this helps.
Probably that webpage just sends a POST request to the server with the contents of the form. You can easily send such a POST request yourself from Java, without using that page. For example this article shows an example of sending POST requests from Java
What you'll need to do is to examine the HTML on the page and work out what parameters are needed to post the form. It'll probably look something like this:
<form action="/RequestURL">
<input type=file name=file1>
<input type=textbox name=value1>
</form>
Based on that you can write some code to do a POST request to the url:
String data = URLEncoder.encode("value1", "UTF-8") + "=" + URLEncoder.encode("value1", "UTF-8");
data += "&" + URLEncoder.encode("file1", "UTF-8") + "=" + URLEncoder.encode(FileData, "UTF-8");
// Send data
URL url = new URL("http://servername.com/RequestURL");
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(data);
wr.flush();
wr.close();
Remember that the person who wrote the page might do some checks to make sure the POST request came from the same site. In that case you might be in trouble, and you might need to set the user agent correctly.
You could try to use HtmlUnit for this. It provides a very simply API for simulating browser actions. I already used this approach for similar requirements. It's very easy. You should give it a try.
Related
i am creating a desktop application which send file to an tomcat server. the servlet receiver and saves file fine.
I need some help to do a java program that post in a https site. I dont know how to put the parameters because it a multpart form data contect type.. Please help! when I do a post with firefox its like this...
This will depend. I've used the following technique to upload a multi-part file to a server before, based on providing a series of form key/name pairs.
This will be depended on you own requirements and what the servlet is actually expecting...
MultipartEntity entity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
String name = file.getName();
entity.addPart(new FormBodyPart("someFormParameter", new StringBody("someFormName")));
/*...*/
entity.addPart("formFileNameParameter", new FileBody(file, mimeType));
HttpClient client = /*...*/
HttpPost post = new HttpPost(url.toURI());
post.setEntity(entity);
HttpResponse response = client.execute(post);
// Process response
Right now I am using Httppost to Post some parameters in the form of xml to a server. When the post occurs, a geotiff or .tif file is downloaded. I have successfully posted the document to the server and successfully downloaded the file simply by attaching the parameters to the url but I can't seem to combine the two. I have to use post because just using the URL leaves out elevation data in the geotiff.
In short, I am not sure how to simultaneously post and retrieve the image of the post. This is what I have thus far...
// Get target URL
String strURL = POST;
// Get file to be posted
String strXMLFilename = XML_PATH;
File input = new File(strXMLFilename);
// Prepare HTTP post
HttpPost post = new HttpPost(strURL);
post.setEntity(new InputStreamEntity(
new FileInputStream(input), input.length()));
// Specify content type and encoding
post.setHeader(
"Content-type", "text/xml");
// Get HTTP client
HttpClient httpclient = new DefaultHttpClient();
//Locate file to store data in
FileEntity entity = new FileEntity(newTiffFile, ContentType.create("image/geotiff"));
post.setEntity(entity);
// Execute request
try {
System.out.println("Connecting to Metoc site...\n");
HttpResponse result = httpclient.execute(post);
I was under the impression that the entity would contain the resulting image. Any help is much appreciated!
Thanks for the help guys. The entity was what was being sent to the server. I had code that was trying to read it from the response as well but it wasn't working because setting the entity to a file entity messed up the post request. By removing that part, it works great!
I want to get the id cookie that Google issues when you opt-in at the ads settings page (if you're already accepting target advertisement, you must opt out first to see the page to which I am referring).
I've found that, in order to get this cookie, you have to perform an HTTP GET to the action URL in the form that is in this page. The problem is that this URL contains a hash that changes for every new HTTP connection so, first, I must go to this page and get this URL and, then, perform the GET to the URL.
I'm using HttpComponents to get http://www.google.com/ads/preferences but when I parse the contents with JSOUP there is only a script and no form can be found.
I'm afraid that this happens becauses contents are loaded dynamically using some sort of timeout... Does anyone know a workaround for this?
EDIT: by the way, the code that I use by now is:
HttpClient httpclient = new DefaultHttpClient();
// Create a local instance of cookie store
CookieStore cookieStore = new BasicCookieStore();
// Bind custom cookie store to the local context
((AbstractHttpClient) httpclient).setCookieStore(cookieStore);
CookieSpecFactory csf = new CookieSpecFactory() {
public CookieSpec newInstance(HttpParams params) {
return new BrowserCompatSpec() {
#Override
public void validate(Cookie cookie, CookieOrigin origin)
throws MalformedCookieException {
// Allow all cookies
System.out.println("Allowed cookie: " + cookie.getName() + " "
+ cookie.getValue() + " " + cookie.getPath());
}
};
}
};
((AbstractHttpClient) httpclient).getCookieSpecs().register("EASY", csf);
// Create local HTTP context
HttpContext localContext = new BasicHttpContext();
// Bind custom cookie store to the local context
localContext.setAttribute(ClientContext.COOKIE_STORE, cookieStore);
HttpGet httpget = new HttpGet(doubleClickURL);
// Override the default policy for this request
httpclient.getParams().setParameter(
ClientPNames.COOKIE_POLICY, "EASY");
// Pass local context as a parameter
HttpResponse response = httpclient.execute(httpget, localContext);
HttpEntity entity = response.getEntity();
if (entity != null) {
InputStream instream = entity.getContent();
BufferedReader reader = new BufferedReader(
new InputStreamReader(instream));
instream.close();
// Find action attribute of form
Document document = Jsoup.parse(reader.readLine());
Element form = document.select("form").first();
String optinURL = form.attr("action");
URL connection = new URL(optinURL);
// ... get id Cookie
}
You may have more chance using HtmlUnit, Selenium or jWebUnit for such a task. JSoup does not interpret Javascript, and the Google page your pointing to is full of Javascript that should be executed by a browser to produce what you're seeing.
HtmlUnit is OS independent and does not need anything else installed, but I've never used it for complicated Javascript sites. HtmlUnit can also extract data from the web page like JSoup does, but you can still feed the html to JSoup if you prefer using it.
Finally I found it! I found the following site describing the doubleclick cookie protocol:
Privacy Advisory
Then, is as easy as setting a cookie in that domain with name id and value A. Then make an HTTP request to http://www.google.com/ads/preferences and they'll set a correct ID value.
It is a very specific question but I hope that serves to future viewers.
By the way, I found that amazon.com is for example a member of the Ad-sense Network. An HTTP request to doubleclick is sent by means of script in the main page to:
http://ad.doubleclick.net/adj/amzn.us.gw.atf
There you can find a script that seems the actual code to give you the id cookie. Nevertheless, if you access this with the cookie with value A it will set the id of doubleclick.
I am trying to find a solution to this the whole evening now...
I write an app which requests data from a web server. The Server answers in JSON format.
Everything works well except when I enter a umlaut like ä into my App.
In the following I assume the request URL is http://example.com/?q= and I am searching for "Jäger"
The correct call would then be h++p://example.com/?q=J%C3%A4ger
(Sorry for plus-signs but the spam protection doesnt let me post it correctly.)
So my problem is now:
When I give my URL String encoded or unencoded over to HttpGet it will always result in a doublee-encoded URL.
The Request to my Server is then http://example.com/?q=J%25C3%25A4ger (It encodes the percent signs)
which leads to the server searching in database for J%C3%A4ger what is obviously wrong.
So my question is how can I achive that if the user enters "Jäger" my app calls the correctly encoded URL?
Thanks for any help!
Here is the currently used code... Ist probably the worst possible idea I had...
URI url = new URI("http", "//example.com/?q=" + ((EditText)findViewById(R.id.input)).getText().toString(), null);
Log.v("MyLogTag", "API Request: " + url);
HttpGet httpGetRequest = new HttpGet(url);
// Execute the request in the client
HttpResponse httpResponse;
httpResponse = defaultClient.execute(httpGetRequest);
Update: Sorry, HttpParams isn't meant for request parameters but for configuring HttpClient.
On Android, you might want to use Uri.Builder, like suggested in this other SO answer:
Uri uri = new Uri.Builder()
.scheme("http")
.authority("example.com")
.path("someservlet")
.appendQueryParameter("param1", foo)
.appendQueryParameter("param2", bar)
.build();
HttpGet request = new HttpGet(uri.toString());
// This looks very tempting but does NOT set request parameters
// but just HttpClient configuration parameters:
// HttpParams params = new BasicHttpParams();
// params.setParameter("q", query);
// request.setParams(params);
HttpResponse response = defaultClient.execute(request);
String json = EntityUtils.toString(response.getEntity());
Outside of Android, your best bet is building the query string manually (with all the encoding hassles) or finding something similar to Android's Uri.Builder.
I'm trying to use the Apache HttpComponents Client to simulate, in Java, a login process on a website. It's failing at the moment and I'm not sure why. I'm using Firebug in Firefox to try and understand the equivalent browser events and I'm sure it's logging in by a particular POST method on a form.
Here's my code. At the end I expect a whole bunch of cookies to be set, but they are not, and the responses I'm getting to subsequent GET requests prove that the login failed.
String email = ...
String password = ...
String url = ...
DefaultHttpClient client = new DefaultHttpClient();
List<NameValuePair> loginParams = new ArrayList<NameValuePair>(2);
loginParams.add(new BasicNameValuePair("logindetails[email]", email));
loginParams.add(new BasicNameValuePair("logindetails[password]", password));
UrlEncodedFormEntity loginEntity = new UrlEncodedFormEntity(loginParams);
HttpPost loginPost = new HttpPost(url);
loginPost.setEntity(loginEntity);
HttpResponse loginResponse = client.execute(loginPost);
HttpEntity loginRepsonseEntity = loginResponse.getEntity();
loginRepsonseEntity.consumeContent();
for (Cookie cookie: client.getCookieStore().getCookies()) {
System.out.println(cookie.getName() + " -> " + cookie.getValue());
}
Where should I start looking for the problem? I'd like to verify that the POST is the same as the one my browser is making -- what's the best way to do that?
We use the Apache HttpClient stuff every day where I work, it's wonderful. What you're doing, where you are encoding the data yourself, looks a little weird to me. There is an easier way. I'll assume you're doing a POST.
String email = ...
String password = ...
String url = ...
PostMethod post = new PostMethod(url);
NameValuePair[] data = {
new NameValuePair("logindetails[email]", email),
new NameValuePair("logindetails[password]", password)
};
post.setRequestBody(data);
// execute method and handle any error responses.
HttpClient httpclient = new HttpClient();
int statusCode = httpclient.executeMethod(post);
// continue as neccessary
I took that code (and modified it to fit yours better) from the Apache HttpClient site. Unless you have a good reason, you shouldn't need to bother packaging up the fields yourself with the UrlEncodedFormEntity.
As for debugging this kind of stuff, it can be easiest to watch using WireShark to watch the packets go back and forth. Any of these will let you watch the requests go back and forth (as long as you don't use HTTPS) so you can easily tell what's being sent incorrectly by comparing it to what a browser sends. It's not the user friendliest program in the world, but it's invaluable when trying to figure out this kind of stuff.
You can use Firebug in Firefox to look at the Net-tab to investigate how the post is done when you use a web page. Looking at your code, my guess is that your parameter names are strange. logindetails[email] and logindetails[password] look weird.
This would be the equivalent of having:
<form action="...url" method="post">
email: <input type="text" name="logindetails[email]"/><br/>
password: <input type="password" name="logindetails[password]"/><br/>
<input type="submit" value="submit"/>
</form>
Make sure that's what you intended...
Look at the wonderful cURL utility -> http://curl.haxx.se/docs/manpage.html.
It's ubiquitous in Unix/Linux and is available under Cygwin on Windows.
It lets you achieve what you are trying to do in Java in a few keystrokes from command line.