I want to upload a file to a general website.
It seems I'll have to use HTTP requests which I can obtain through http://www.fiddler2.com/.
But I'm a complete beginner in handling HTTP or webrequests. Could someone guide me into a good tutorial on HTTP requests using C++ or Java?
Most websites are in need of a login for example. Also, does a general website accept those http request to upload a file? For uploading a file to a general website it seems you will need a login handler, get the correct request to upload the file and then upload it. To get those correct requests, do I have to see what that specific site asks?
The goal is to upload a file to a general website of choice and expand from there on to more websites.
Thanks in advance!
Related
I am working in a jsp file. I have a lot of files available in my Sharepoint intranet with link like this :
http://myIntranet.eu/my%pdf%1.pdf
I want saved the pdf in a folder in my server. This will help me then to merge all PDF.
I had already merge pdf in java, but I don't know how I can save this PDF on my server.
Can you please help me on how to do that.
If there is no web service or rest service from sharepoint to get the pdf, you can try to use an http client http://hc.apache.org/httpcomponents-client-ga/ to make request to sharpoint on java server to get the pdf file. Maybe you need authentication.
There is a link to Google Drive file accessible to anyone without authentication. Is there any way to download that file using just link and some sort of http client? Most of examples around are relying on Drive API and file IDs, however I'd like to stick with more lightweight approach, at the same time no parsing web page that results from http get to the url.
"If you want to share a direct link, simply change the format of the link from this:
drive.google.com/file/d/FILE_ID/edit?usp=sharing
To this:
drive.google.com/uc?export=download&id=FILE_ID
Note that you'll need to grab the file ID from the original link and append it to the end of the new link.
With the new format, any link you share will automatically download to your recipient's computer."
http://lifehacker.com/share-direct-links-to-files-in-google-drive-and-skip-th-1493813665 //Jan 2014
I have a task where I need to autologin and scrape a particualr website.
I have seen people suggesting htmlUnit and HttpClient mostly with Java. htmlUnit looks like a testing tool. I am not sure what to do with that. Is there an example that explains auto login and web scraping with htmlUnit or httpClient?
Im a Java developer. Can anyone who closely works with it please share any ideas?
Your problem can be broken down to
login into the website
Scrape the data from website.
So, for the first part-:
Install livehttp header firefox addon and than read all the http
headers that were sent and received by your browser while trying to
login.
Try to send these headers using your java code, basically you have
to emulate a HTTP POST request using your java code. For that
google->make post request from java
After you have login into the website, than scrap the data using the API of your choice.
I personally use htmlcleaner HtmlCleaner.
To scrape data you can use XPath expressions with htmlcleaner.
Take a look at Xpath+htmlcleaner and here also
You can also use JSoup instead of htmlcleaner. Advantage of using JSoup is it can handle both login[POST Request] and Data scraping. Take a look here http://pastebin.com/E0WzpuhF
I know it seems a lot of work, i have provided you with two alternative solution for your problem but divide your problem into smaller chunks and than try to solve it.
I'm wondering if there is any way to upload files from a java program to a website that has a simple html form.
An example could look like the one on tinypic.com or bayimg.com.
Are there any Java libraries capable of performing this? Please point me in the right direction :).
Thanks.
Mike.
HTML Form uses just HTTP POST/PUT methods to upload file to server, you don't have to work/think in this case with HTML FORM, just do HTTP Connection to script and send data in required format
Read more:
http://www.jguru.com/faq/view.jsp?EID=62798
http://www.theserverside.com/news/1365153/HttpClient-and-FileUpload
http://www.java-tips.org/other-api-tips/httpclient/how-to-use-multipart-post-method-for-uploading.html
http://www.prasannatech.net/2009/03/java-http-post-file-upload-server.html
http://www.google.cz/search?aq=f&sourceid=chrome&ie=UTF-8&q=java+http+post+upload
I am working on small Java project to programmatically connect to a website with username/password, after login, browse to different links on the site to download some data. First, I need to connect to the website with username/password,
second, while I keep the session open, go to other links to download data.
How do I do this in Java?
Any help will be highly appreciated!
Check out the Apache HTTPClient, it can do all this for you.
Edit: Apache HTTPClient has authentication and cookie handling features included, which will save you a lot of work doing this yourself.
If you want to extract some data HtmlUnit can help you a lot it can manage the authentication and also help you with data extraction.
Investigate with your browser how the web page submits the username/pass data? HTTP Form POST, Ajax, etc..? Use a plugin like Firebug to see network traffic.
You can use URLConnection to create HTTP requests. You will neet to simulate a username/pass login and remember the cookie for use in consequent HTTP requests to simulate a session. Here are some examples: send HTTP POST request, get a cookie, send a cookie.