I would like to do an HTTPRequest in Java and then get the data from the server (it's not a webpage the data come from a database).
I try this but the getData doesn't work.
Do you know how I can get the Data?
public static void main(String args[]) throws Exception {
URL url = new URL("http://ip-ad.com");
HttpURLConnection httpCon = (HttpURLConnection) url.openConnection();
System.out.println("Request method is " + httpCon.getData());
}
Thanks
You can get the response body of the web request as an InputStream with:
httpCon.getInputStream();
From there it depends on what the format of the response data is. If it's XML then pass it to a library to parse XML. If you want to read it into a String see: Reading website's contents into string. Here's an example of writing it to a local file:
InputStream in = httpCon.getInputStream();
OutputStream out = new FileOutputStream("file.dat");
out = new BufferedOutputStream(out);
byte[] buf = new byte[8192];
int len = 0;
while ((len = in.read(buf)) != -1) {
out.write(buf, 0, len);
}
out.close();
You can use http://jersey.java.net/ .
It's a simple lib for your needs.
Related
I'm trying to download a gzip pdf from an url, unpacking it and writing it to a file. It almost works, but currently some characters in the pdf made from my code mismatches the real pdf. I checked this by opening both of the pdf's in notepad.
I provide some short text samples from the two pdfs.
From my code:
’8 /qªMiUe°Ä[H`ðKíulýªäqvA®v8;xÒhÖßÚ²ý!Æ¢ØK$áýçpF[¸t1#y$93
From the real pdf:
ƒ8 /qªMiUe°Ä[H`ðKíulªäqvA®—v8;ŸÒhÖßÚ²!ˆ¢ØK$áçpF[¸t1#y$‘‹3
Here is my code:
public void readPDFfromURL(String urlStr) throws IOException {
URL myURL = new URL(urlStr);
HttpURLConnection urlCon = (HttpURLConnection) myURL.openConnection();
urlCon.setRequestProperty("Accept-Encoding", "gzip");
urlCon.setRequestProperty("Content-Type", "application/pdf");
urlCon.setRequestMethod("GET");
urlCon.setDoInput(true);
urlCon.connect();
Reader reader;
if ("gzip".equals(urlCon.getContentEncoding())) {
reader = new InputStreamReader(new GZIPInputStream(urlCon.getInputStream()));
}
else {
reader = new InputStreamReader(urlCon.getInputStream());
}
FileOutputStream fos = new FileOutputStream("document.pdf");
int data = reader.read();
while(data != -1) {
char c = (char) data;
fos.write(c);
data = reader.read();
}
fos.close();
reader.close();
}
I can open the pdf, and it has the correct amount of pages, but the pages are all blank.
My initial thought is that it might got something to do with character codes to do, like some setting in my java project, intellij etc.
Alternatively, I don't actually need to put it in a file. I just need to download it so I can upload it to another place. However, the pdf should of course be working in either case. I'm really just putting it in an actual file to check if it works.
Thank you for your help!
Here is my new implementation, which solves my question:
public void readPDFfromURL(String urlStr) throws IOException {
URL myURL = new URL(urlStr);
HttpURLConnection urlCon = (HttpURLConnection) myURL.openConnection();
urlCon.setRequestProperty("Accept-Encoding", "gzip");
urlCon.setRequestProperty("Content-Type", "application/pdf");
urlCon.setRequestMethod("GET");
urlCon.setDoInput(true);
urlCon.connect();
GZIPInputStream reader = new GZIPInputStream(urlCon.getInputStream());
FileOutputStream fos = new FileOutputStream("document.pdf");
byte[] buffer = new byte[1024];
int len;
while((len = reader.read(buffer)) != -1){
fos.write(buffer, 0, len);
}
fos.close();
reader.close();
}
There is a billion of topics about Connection, I've tried so many ways to download this file and always fail. When I disable cookies on my web browser I can't download it, for that reason I believe my problem is with cookies.
The function of my program is extract the zip, parse the html inside, with Jsoup, insert the content on mysql database and load it on JApplet. Everything is working except the auto-download part, which I have to do manual download in my web browser.
I'm using this class for the cookie, which returns error on
read.CookieManager.storeCookies(CookieManager.java:89)
which corresponds to this line from Cookie class
for (int i=1; (headerName = conn.getHeaderFieldKey(i)) != null; i++) {
and this one from download class
cm.storeCookies(urlConnection);
the download method
public static void main(String args[]) throws Exception {
downloadFromUrl("http://www1.caixa.gov.br/loterias/_arquivos/loterias/D_mgsasc.zip", "Mozilla", "C:/", "D_mgsasc.zip", true);
}
public static void downloadFromUrl(String srcAddress, String userAgent, String destDir, String destFileName, boolean overwrite) throws Exception
{
InputStream is = null;
FileOutputStream fos = null;
try {
CookieManager cm = new CookieManager();
URL url = new URL(srcAddress);
URLConnection urlConnection = url.openConnection();
urlConnection.setRequestProperty("User-Agent", userAgent);
urlConnection.setDoInput(true);
urlConnection.setDoOutput(true);
urlConnection.setConnectTimeout(30000);
urlConnection.setReadTimeout(30000);
urlConnection.setUseCaches(true);
urlConnection.connect();
cm.storeCookies(urlConnection);
cm.setCookies(url.openConnection());
is = urlConnection.getInputStream();
fos = new FileOutputStream(destFile);
byte[] buffer = new byte[1024];
int len, totBytes = 0;
while((len = is.read(buffer)) > 0)
{
totBytes += len;
fos.write(buffer, 0, len);
}
fos.flush();
fos.close();
}
}
*
updated, removed unnecessary code
*
which returns the following error
java.net.SocketException: Connection reset at
zip.DownloadFile.downloadFromUrl(DownloadFile.java:71)
related to this line in code
is = urlConnection.getInputStream();
When I remove cookies set code, the same error of Connection reset persists.
I am trying to make Rest service call in Java. I am new to web and Rest service. I have Rest service which returns JSON as response. I have the following code but I think it's incomplete because I don't know how to process output using JSON.
public static void main(String[] args) {
try {
URL url = new URL("http://example.com:7000/test/db-api/processor");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setInstanceFollowRedirects(false);
connection.setRequestMethod("PUT");
connection.setRequestProperty("Content-Type", "application/json");
OutputStream os = connection.getOutputStream();
//how do I get json object and print it as string
os.flush();
connection.getResponseCode();
connection.disconnect();
} catch(Exception e) {
throw new RuntimeException(e);
}
}
I am new to Rest services and JSON.
Since this is a PUT request you're missing a few things here:
OutputStream os = conn.getOutputStream();
os.write(input.getBytes()); // The input you need to pass to the webservice
os.flush();
...
BufferedReader br = new BufferedReader(new InputStreamReader(
(conn.getInputStream()))); // Getting the response from the webservice
String output;
System.out.println("Output from Server .... \n");
while ((output = br.readLine()) != null) {
System.out.println(output); // Instead of this, you could append all your response to a StringBuffer and use `toString()` to get the entire JSON response as a String.
// This string json response can be parsed using any json library. Eg. GSON from Google.
}
Have a look at this to have a more clear idea on hitting webservices.
Your code is mostly correct, but there is mistake about OutputStream.
As R.J said OutputStream is needed to pass request body to the server.
If your rest service doesn't required any body you don't need to use this one.
For reading the server response you need use InputStream(R.J also show you example) like that:
try (InputStream inputStream = connection.getInputStream();
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();) {
byte[] buf = new byte[512];
int read = -1;
while ((read = inputStream.read(buf)) > 0) {
byteArrayOutputStream.write(buf, 0, read);
}
System.out.println(new String(byteArrayOutputStream.toByteArray()));
}
This way is good if you don't want to depends on third-part libraries. So I recommend you to take a look on Jersey - very nice library with huge amount of very useful feature.
Client client = JerseyClientBuilder.newBuilder().build();
Response response = client.target("http://host:port").
path("test").path("db-api").path("processor").path("packages").
request().accept(MediaType.APPLICATION_JSON_TYPE).buildGet().invoke();
System.out.println(response.readEntity(String.class));
Since your Content-Type is application/json, you could directly cast the response to a JSON object for example
JSONObject recvObj = new JSONObject(response);
JsonKey jsonkey = objectMapper.readValue(new URL("http://echo.jsontest.com/key/value/one/two"), JsonKey.class);
System.out.println("jsonkey.getOne() : "+jsonkey.getOne())
i am writing a small android application which requires some data which is stored on my web server. The file is a .txt file curretly less than 1 MB. Is it advisable to set up a ftp server to get the data or can i just use a http get method to get the contents on a file. If i am using a http get can someone please tell me the java code required for this operation.
This is out of my head (so an error could have sneaked in):
URL url = new URL("http://www.yourserver.com/some/path");
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
try {
InputStream in = new BufferedInputStream(urlConnection.getInputStream());
FileOutputStream out = new FileutputStream("/path/to/your/output/file");
byte[] buffer = new byte[16384];
int len;
while((len = in.read(buffer)) != -1){
out.write(buffer, 0, len);
}
finally {
urlConnection.disconnect();
}
I'm trying to write a HTTP proxy-server in java. My application takes a GET request from a browser and forwards it to its destination. I would like to read the headers of response package and then forward it back to the browser. This works great for me with text/html-content aslong as its not encoded in gzip. I've tried multiple ways to do this and I'm currently using a DataInputStream and a DataOutputStream but the browser only shows weird symbols.
Here is a simplified version of the code:
ArrayList<String> headerlist = new ArrayList<String>();
InputStream input = clientsocket.getInputStream();
dis = new DataInputStream(input);
serverinputstream = new InputStreamReader(input);
bufferreader = new BufferedReader(serverinputstream);
while(!(line = bufferedreader.readLine()).equals("")) {
headerlist.add(line);
}
PrintWriter pw = new PrintWriter(serveroutputstream, false);
DataOutputStream out = new DataOutputStream(serveroutputstream);
for (int i = 0; i < headerlist.size(); i++) {
pw.println(headerlist.get(i));
}
pw.println();
int bit;
while((bit = dis.read()) != -1) {
out.writeByte(bit);
}
out.flush();
dis.close();
out.close();
This code only handles data that isnt plain text but it doesnt seem to be working. Should I use another method or I am just doing something wrong?
I think you may be overcomplicating things a bit. Your proxy is just forwarding a request on to another destination. There's no reason for it to care about whether it is forwarding text or binary data. It should make no difference.
There's also no reason to read and write the headers individually. All you should need to do is copy the entire request body to the new output-stream.
What about something like:
//...
InputStream input = clientsocket.getInputStream();
streamCopy(input, serveroutputstream);
//...
public void streamCopy(InputStream in, OutputStream out) throws IOException {
int read = 0;
byte[] buffer = new byte[4096];
while ((read = in.read(buffer)) != -1) {
out.write(buffer, 0, read);
}
}