We Keep Coding

use Java client like curl with param

I use influxdb 0.9. in this version, i can write database like
curl -XPOST 'http://localhost:8086/write?db=mydb' -d 'cpu,host=server01,region=uswest value=1.0'
Now I convert it to java
URL url = new URL("http", "localhost", 8086, "/write?db=mydb");
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("POST");
con.setDoOutput(true);
OutputStream wr = con.getOutputStream();
Stirng s = "cpu,host=server01,region=uswest value=51.0";
wr.write(s.getBytes(UTF_8));
wr.flush();
wr.close();
but it doesn't work. Is the "-d" meant to represent post parameters? How can I express that in Java?

In this example the curl flag should really be --data-binary, not -d, which can have different encoding. As long as your string is unaltered by the Java code it should be fine. Anything like URL encoding will prevent the line protocol insert from working.

You need to grab the HTTP response as well. The following worked for me:
import java.net.*;
import java.io.*;
import java.util.*;
public class Client {
private static HttpURLConnection client;
public static void main(String[] args) {
try {
Random rn = new Random();
URL input;
while(true) {
input = new URL("http", "localhost", 8086, "/write?db=mydb");
client = (HttpURLConnection) input.openConnection();
client.setRequestMethod("POST");
client.setDoOutput(true);
double thermals = rn.nextDouble();
String s = "cpu_temperature value=" + thermals;
try (OutputStreamWriter writer =
new OutputStreamWriter(client.getOutputStream())) {
writer.write(s);
}
BufferedReader in = new BufferedReader(new InputStreamReader(
client.getInputStream()));
String decodedString;
while ((decodedString = in.readLine()) != null) {
System.out.println(decodedString);
}
in.close();
System.out.println(s); // for debugging
Thread.sleep(1000); // send data every 1 second
}
} catch(MalformedURLException error) {
System.out.println("Malformed URL!");
} catch(SocketTimeoutException error) {
System.out.println("Socket Timeout Exception!");
} catch (IOException error) {
System.out.println("IOException!");
System.out.println(error);
} catch(InterruptedException e) {
System.out.println("InterruptedException!");
} finally {
if(client != null) { // Make sure the connection is not null.
client.disconnect();
}
}
}
}

your SQL query need to be corrected. Remove "," in "cpu,host=" it will work.

Reading URL, getting read times out error

Hi I am using following code to reading URL:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLEncoder;
public class JavaHttpUrlConnectionReader
{
public static void main(String[] args)
throws Exception
{
new JavaHttpUrlConnectionReader();
}
public JavaHttpUrlConnectionReader()
{
try
{
String myUrl = "http://epaperbeta.timesofindia.com/NasData/PUBLICATIONS/THETIMESOFINDIA/Delhi/2015/06/09/PageIndex/09_06_2015.xml";
// if your url can contain weird characters you will want to
// encode it here, something like this:
// myUrl = URLEncoder.encode(myUrl, "UTF-8");
String results = doHttpUrlConnectionAction(myUrl);
System.out.println(results);
}
catch (Exception e)
{
// deal with the exception in your "controller"
}
}
/**
* Returns the output from the given URL.
*/
private String doHttpUrlConnectionAction(String desiredUrl)
throws Exception
{
URL url = null;
BufferedReader reader = null;
StringBuilder stringBuilder;
try
{
// create the HttpURLConnection
url = new URL(desiredUrl);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
// just want to do an HTTP GET here
connection.setRequestMethod("GET");
// uncomment this if you want to write output to this url
//connection.setDoOutput(true);
// give it 15 seconds to respond
connection.setReadTimeout(35*1000);
connection.connect();
// read the output from the server
reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
stringBuilder = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null)
{
stringBuilder.append(line + "\n");
}
return stringBuilder.toString();
}
catch (Exception e)
{
e.printStackTrace();
throw e;
}
finally
{
// close the reader; this can throw an exception too, so
// wrap it in another try/catch block.
if (reader != null)
{
try
{
reader.close();
}
catch (IOException ioe)
{
ioe.printStackTrace();
}
}
}
}
}
It gives me following error:
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:687)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:632)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1072)
at JavaHttpUrlConnectionReader.doHttpUrlConnectionAction(JavaHttpUrlConnectionReader.java:77)
at JavaHttpUrlConnectionReader.<init>(JavaHttpUrlConnectionReader.java:33)
at JavaHttpUrlConnectionReader.main(JavaHttpUrlConnectionReader.java:21)
Kindly tell me the reason why it occurs, and solution for it.
When I run this code outside of my office LAN, it is working fine. but not in office LAN.
Thanks & Regards
Abhishek

Your URL:
http://epaperbeta.timesofindia.com/NasData/PUBLICATIONS/THETIMESOFINDIA/Delhi/2015/06/09/PageIndex/09_06_2015.xml
is not accessible without a proxy (for example I can't access it from here), no wonder why cannot be read from the stream.
Check your proxy settings. you could try the url in the browser with/without proxy and see the difference.
As #Jens commented look at this.

Java, FileNotfound Exception, While reading conn.getInputStream()

Please tell me some one, How to resolve this problem,
Sometime I am getting Filenotfound Exception and Some time this code working fine.
Below is my code,
public String sendSMS(String data, String url1) {
URL url;
String status = "Somthing wrong ";
try {
url = new URL(url1);
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
conn.setRequestProperty("User-Agent","Mozilla/5.0 ( compatible ) ");
conn.setRequestProperty("Accept","*/*");
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(data);
wr.flush();
// Get the response
try {
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String s;
while ((s = rd.readLine()) != null) {
status = s;
}
rd.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
}
wr.close();
} catch (MalformedURLException e) {
status = "MalformedURLException Exception in sendSMS";
e.printStackTrace();
} catch (IOException e) {
status = "IO Exception in sendSMS";
e.printStackTrace();
}
return status;
}

Rewrite like this and let me know how you go... (note closing of reading and writing streams, also the cleanup of streams if an exception is thrown).
public String sendSMS(String data, String url1) {
URL url;
OutputStreamWriter wr = null;
BufferedReader rd = null;
String status = "Somthing wrong ";
try {
url = new URL(url1);
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
conn.setRequestProperty("User-Agent","Mozilla/5.0 ( compatible ) ");
conn.setRequestProperty("Accept","*/*");
wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(data);
wr.flush();
wr.close();
rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String s;
while ((s = rd.readLine()) != null) {
status = s;
}
rd.close();
} catch (Exception e) {
if (wr != null) try { wr.close(); } catch (Exception x) {/*cleanup*/}
if (rd != null) try { rd.close(); } catch (Exception x) {/*cleanup*/}
e.printStackTrace();
}
return status;
}

This issue seems to be known, but for different reasons so its not clear why this happend.
Some threads would recommend closing the OutputStreamWriter as flushing it is not enough, therefor i would try to clos it directly after fushing as you are not using it in the code between the flush and close.
Other threads show that using a different connections like HttpURLConnection are avoiding this problem from occuring (Take a look here)
Another article suggests to use the URLEncoder class’ static method encode. This method takes a string and encodes it to a string that is ok to put in a URL.
Some similar questions:
URL is accessable with browser but still FileNotFoundException with URLConnection
URLConnection FileNotFoundException for non-standard HTTP port sources
URLConnection throwing FileNotFoundException
Wish you good luck.

It returns FileNotFoundException when the server response to HTTP request is code 404.
Check your URL.

Reading from a URL Connection Java

I'm trying to read html code from a URL Connection. In one case the html file I'm trying to read includes 5 line breaks before the actual doc type declaration. In this case the input reader throws an exception for EOF.
URL pageUrl =
new URL(
"http://www.nytimes.com/2011/03/15/sports/basketball/15nbaround.html"
);
URLConnection getConn = pageUrl.openConnection();
getConn.connect();
DataInputStream dis = new DataInputStream(getConn.getInputStream());
//some read method here
Has anyone ran into a problem like this?
URL pageUrl = new URL("http://www.nytimes.com/2011/03/15/sports/basketball/15nbaround.html");
URLConnection getConn = pageUrl.openConnection();
getConn.connect();
DataInputStream dis = new DataInputStream(getConn.getInputStream());
String urlData = "";
while ((urlData = dis.readUTF()) != null)
System.out.println(urlData);
//exception thrown
java.io.EOFException
at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
at java.io.DataInputStream.readUTF(DataInputStream.java:572)
at java.io.DataInputStream.readUTF(DataInputStream.java:547)
in the case of bufferedreader, it just responds null and doesn't continue
pageUrl = new URL("http://www.nytimes.com/2011/03/15/sports/basketball/15nbaround.html");
URLConnection getConn = pageUrl.openConnection();
getConn.connect();
BufferedReader br = new BufferedReader(new InputStreamReader(getConn.getInputStream()));
String urlData = "";
while(true)
urlData = br.readLine();
System.out.println(urlData);
outputs null

You're using DataInputStream to read data that wasn't encoded using DataOutputStream. Examine the documented behavior for your call to DataInputStream#readUtf(); it first reads two bytes to form a 16-bit integer, indicating the number of bytes that follow comprising the UTF-encoded string. The data you're reading from the HTTP server is not encoded in this format.
Instead, the HTTP server is sending headers encoded in ASCII, per RFC 2616 sections 6.1 and 2.2. You need to read the headers as text, and then determine how the message body (the "entity") is encoded.

This works fine:
package url;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.URL;
/**
* UrlReader
* #author Michael
* #since 3/20/11
*/
public class UrlReader
{
public static void main(String[] args)
{
UrlReader urlReader = new UrlReader();
for (String url : args)
{
try
{
String contents = urlReader.readContents(url);
System.out.printf("url: %s contents: %s\n", url, contents);
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
public String readContents(String address) throws IOException
{
StringBuilder contents = new StringBuilder(2048);
BufferedReader br = null;
try
{
URL url = new URL(address);
br = new BufferedReader(new InputStreamReader(url.openStream()));
String line = "";
while (line != null)
{
line = br.readLine();
contents.append(line);
}
}
finally
{
close(br);
}
return contents.toString();
}
private static void close(Reader br)
{
try
{
if (br != null)
{
br.close();
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
}

This:
public class Main {
public static void main(String[] args)
throws MalformedURLException, IOException
{
URL pageUrl = new URL("http://www.google.com");
URLConnection getConn = pageUrl.openConnection();
getConn.connect();
BufferedReader dis = new BufferedReader(
new InputStreamReader(
getConn.getInputStream()));
String myString;
while ((myString = dis.readLine()) != null)
{
System.out.println(myString);
}
}
}
Works perfectly. The URL you are supplying, however, returns nothing.

Read url to string in few lines of java code

I'm trying to find Java's equivalent to Groovy's:
String content = "http://www.google.com".toURL().getText();
I want to read content from a URL into string. I don't want to pollute my code with buffered streams and loops for such a simple task. I looked into apache's HttpClient but I also don't see a one or two line implementation.

Now that some time has passed since the original answer was accepted, there's a better approach:
String out = new Scanner(new URL("http://www.google.com").openStream(), "UTF-8").useDelimiter("\\A").next();
If you want a slightly fuller implementation, which is not a single line, do this:
public static String readStringFromURL(String requestURL) throws IOException
{
try (Scanner scanner = new Scanner(new URL(requestURL).openStream(),
StandardCharsets.UTF_8.toString()))
{
scanner.useDelimiter("\\A");
return scanner.hasNext() ? scanner.next() : "";
}
}

This answer refers to an older version of Java. You may want to look at ccleve's answer.
Here is the traditional way to do this:
import java.net.*;
import java.io.*;
public class URLConnectionReader {
public static String getText(String url) throws Exception {
URL website = new URL(url);
URLConnection connection = website.openConnection();
BufferedReader in = new BufferedReader(
new InputStreamReader(
connection.getInputStream()));
StringBuilder response = new StringBuilder();
String inputLine;
while ((inputLine = in.readLine()) != null)
response.append(inputLine);
in.close();
return response.toString();
}
public static void main(String[] args) throws Exception {
String content = URLConnectionReader.getText(args[0]);
System.out.println(content);
}
}
As #extraneon has suggested, ioutils allows you to do this in a very eloquent way that's still in the Java spirit:
InputStream in = new URL( "http://jakarta.apache.org" ).openStream();
try {
System.out.println( IOUtils.toString( in ) );
} finally {
IOUtils.closeQuietly(in);
}

Or just use Apache Commons IOUtils.toString(URL url), or the variant that also accepts an encoding parameter.

There's an even better way as of Java 9:
URL u = new URL("http://www.example.com/");
try (InputStream in = u.openStream()) {
return new String(in.readAllBytes(), StandardCharsets.UTF_8);
}
Like the original groovy example, this assumes that the content is UTF-8 encoded. (If you need something more clever than that, you need to create a URLConnection and use it to figure out the encoding.)

Now that more time has passed, here's a way to do it in Java 8:
URLConnection conn = url.openConnection();
try (BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream(), StandardCharsets.UTF_8))) {
pageText = reader.lines().collect(Collectors.joining("\n"));
}

Additional example using Guava:
URL xmlData = ...
String data = Resources.toString(xmlData, Charsets.UTF_8);

Java 11+:
URI uri = URI.create("http://www.google.com");
HttpRequest request = HttpRequest.newBuilder(uri).build();
String content = HttpClient.newHttpClient().send(request, BodyHandlers.ofString()).body();

If you have the input stream (see Joe's answer) also consider ioutils.toString( inputstream ).
http://commons.apache.org/io/api-1.4/org/apache/commons/io/IOUtils.html#toString(java.io.InputStream)

The following works with Java 7/8, secure urls, and shows how to add a cookie to your request as well. Note this is mostly a direct copy of this other great answer on this page, but added the cookie example, and clarification in that it works with secure urls as well ;-)
If you need to connect to a server with an invalid certificate or self signed certificate, this will throw security errors unless you import the certificate. If you need this functionality, you could consider the approach detailed in this answer to this related question on StackOverflow.
Example
String result = getUrlAsString("https://www.google.com");
System.out.println(result);
outputs
<!doctype html><html itemscope="" .... etc
Code
import java.net.URL;
import java.net.URLConnection;
import java.io.BufferedReader;
import java.io.InputStreamReader;
public static String getUrlAsString(String url)
{
try
{
URL urlObj = new URL(url);
URLConnection con = urlObj.openConnection();
con.setDoOutput(true); // we want the response
con.setRequestProperty("Cookie", "myCookie=test123");
con.connect();
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
StringBuilder response = new StringBuilder();
String inputLine;
String newLine = System.getProperty("line.separator");
while ((inputLine = in.readLine()) != null)
{
response.append(inputLine + newLine);
}
in.close();
return response.toString();
}
catch (Exception e)
{
throw new RuntimeException(e);
}
}

Here's Jeanne's lovely answer, but wrapped in a tidy function for muppets like me:
private static String getUrl(String aUrl) throws MalformedURLException, IOException
{
String urlData = "";
URL urlObj = new URL(aUrl);
URLConnection conn = urlObj.openConnection();
try (BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream(), StandardCharsets.UTF_8)))
{
urlData = reader.lines().collect(Collectors.joining("\n"));
}
return urlData;
}

URL to String in pure Java
Example call to get payload from http get call
String str = getStringFromUrl("YourUrl");
Implementation
You can use the method described in this answer, on How to read URL to an InputStream and combine it with this answer on How to read InputStream to String.
The outcome will be something like
public String getStringFromUrl(URL url) throws IOException {
return inputStreamToString(urlToInputStream(url,null));
}
public String inputStreamToString(InputStream inputStream) throws IOException {
try(ByteArrayOutputStream result = new ByteArrayOutputStream()) {
byte[] buffer = new byte[1024];
int length;
while ((length = inputStream.read(buffer)) != -1) {
result.write(buffer, 0, length);
}
return result.toString(UTF_8);
}
}
private InputStream urlToInputStream(URL url, Map<String, String> args) {
HttpURLConnection con = null;
InputStream inputStream = null;
try {
con = (HttpURLConnection) url.openConnection();
con.setConnectTimeout(15000);
con.setReadTimeout(15000);
if (args != null) {
for (Entry<String, String> e : args.entrySet()) {
con.setRequestProperty(e.getKey(), e.getValue());
}
}
con.connect();
int responseCode = con.getResponseCode();
/* By default the connection will follow redirects. The following
* block is only entered if the implementation of HttpURLConnection
* does not perform the redirect. The exact behavior depends to
* the actual implementation (e.g. sun.net).
* !!! Attention: This block allows the connection to
* switch protocols (e.g. HTTP to HTTPS), which is <b>not</b>
* default behavior. See: https://stackoverflow.com/questions/1884230
* for more info!!!
*/
if (responseCode < 400 && responseCode > 299) {
String redirectUrl = con.getHeaderField("Location");
try {
URL newUrl = new URL(redirectUrl);
return urlToInputStream(newUrl, args);
} catch (MalformedURLException e) {
URL newUrl = new URL(url.getProtocol() + "://" + url.getHost() + redirectUrl);
return urlToInputStream(newUrl, args);
}
}
/*!!!!!*/
inputStream = con.getInputStream();
return inputStream;
} catch (Exception e) {
throw new RuntimeException(e);
}
}
Pros
It is pure java
It can be easily enhanced by adding different headers as a map (instead of passing a null object, like the example above does), authentication, etc.
Handling of protocol switches is supported