We Keep Coding

Java alternative for curl -T

i must send one text string using java to a IP web cam, before it take picture. So after I read the camera user manual and searched in google, the only thing i found was using cURL. I install it and its run fine, and everything is okay, the text from the file appear in the video streaming. The command is this
curl -T test.xml http://admin:pass#192.168.0.1/Video/inputs/channels/2/overlays/text/2
and the content of test.xml is:
<TextOverlay xmlns="http://www.hikvision.com/ver10/XMLSchema" version="1.0">
<id>2</id>
<enabled>true</enabled>
<posX>5</posX>
<posY>5</posY>
<message>Text here </message>
</TextOverlay>
So I want to send this content using Java, I already tried using post and java.net but I get an error "Server returned HTTP response code: 403 for URL"
Here is my code:
System.out.println("Starting......");
URL url = new URL("http://192.168.0.1/Video/inputs/channels/2/overlays/text/2/");
String data = "<TextOverlay xmlns=\"http://www.hikvision.com/ver10/XMLSchema\" version=\"1.0\">\n"
+ "<id>2</id>\n"
+ "<enabled>true</enabled>\n"
+ "<posX>5</posX>\n"
+ "<posY>5</posY>\n"
+ "<message>Text here</message>\n"
+ "</TextOverlay>";
HttpURLConnection httpConnection = prepareConn(url, null, "admin", "pass");
httpConnection.setDoOutput(true);
httpConnection.setDoInput(true);
httpConnection.setRequestMethod("POST");
httpConnection.setRequestProperty ( "Content-Type", "text/xml" );
OutputStreamWriter out = new OutputStreamWriter(httpConnection.getOutputStream());
out.write(data);
out.flush();
out.close();
System.out.println("Printing......");
System.out.println(httpConnection.getResponseCode());
System.out.println(httpConnection.getResponseMessage());
InputStreamReader reader = new InputStreamReader(httpConnection.getInputStream());
StringBuilder buf = new StringBuilder();
char[] cbuf = new char[2048];
int num;
while(-1 != (num = reader.read(cbuf)))
{
buf.append(cbuf, 0, num);
}
String result = buf.toString();
System.out.println("\nResponse received from server after POST" + result);
}
static private HttpURLConnection prepareConn(final URL url, Properties request_props, String username, String password) throws Error, IOException
{
System.out.println("Authorization......");
if (!url.getProtocol().equalsIgnoreCase("http"))
throw new Error(url.toString() + " is not HTTP!");
final HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setConnectTimeout(300);
conn.setRequestMethod("POST");
final Properties DEFAULT_REQUEST_PROPS = new Properties();
DEFAULT_REQUEST_PROPS.setProperty("charset", "utf-8");
final Properties props = new Properties(DEFAULT_REQUEST_PROPS);
if (request_props != null)
for (final String name : request_props.stringPropertyNames())
props.setProperty(name, request_props.getProperty(name));
for (final String name : props.stringPropertyNames())
conn.setRequestProperty(name, props.getProperty(name));
if(null != username && null != password)
conn.setRequestProperty("Authorization", "Basic " + new BASE64Encoder().encode((username+":"+password).getBytes()));
return conn;
}
Hope someone can help :)
All the best !

I just use wrong RequestMethod, after deep research I found that i must use PUT not POST request. Now just change setRequestMethod("POST") to setRequestMethod("PUT") and works like a charm.

how to detect encoding when i'm using bufferedReader

i know this question was asked many times however i'm stuck with this problem and nothing i've read helped me.
i have this code:
BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while((line = reader.readLine()) != null)content += line+"\r\n";
reader.close();
i'm trying to get content of this webpage http://www.garazh.com.ua/tires/catalog/Marangoni/E-COMM/description/ and all nonlatin symbols have been displayed wrong.
i tried set encoding like:
BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream(), "WINDOWS-1251"));
and at this point everething was well! but i cant change encoding for each website i try to parse and i need some solution.
so guys, i know that there is not that easy to detect encoding as it seems but i'm realy need it. if someone had such problem please explain me how you have solved it!
any help appriciated!
this is entire code of the function i'm using to get content:
protected Map<String, String> getFromUrl(String url){
Map<String, String> mp = new HashMap<String, String>();
String newCookie = "", redirect = null;
try{
String host = this.getHostName(url), content = "", header = "", UA = this.getUA(), cookie = this.getCookie(host, UA), referer = "http://"+host+"/";
URL U = new URL(url);
URLConnection conn = U.openConnection();
conn.setRequestProperty("Host", host);
conn.setRequestProperty("User-Agent", UA);
conn.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
conn.setRequestProperty("Accept-Language", "ru-ru,ru;q=0.8,en-us;q=0.5,en;q=0.3");
conn.setRequestProperty("Accept-Encoding", "gzip,deflate");
conn.setRequestProperty("Accept-Charset", "utf-8;q=0.7,*;q=0.7");
conn.setRequestProperty("Keep-Alive", "115");
conn.setRequestProperty("Connection", "keep-alive");
conn.setRequestProperty("Connection", "keep-alive");
if(referer != null)conn.setRequestProperty("Referer", referer);
if(cookie != null && !cookie.contentEquals(""))conn.setRequestProperty("Cookie", cookie);
for(int i=0; ; i++){
String name = conn.getHeaderFieldKey(i);
String value = conn.getHeaderField(i);
if(name == null && value == null)break;
else if(name != null)if(name.contentEquals("Set-Cookie"))newCookie += value + " ";
else if(name.toLowerCase().trim().contentEquals("location"))redirect = value;
header += name + ": " + value + "\r\n";
}
if(!newCookie.contentEquals("") && !newCookie.contentEquals(cookie))this.setCookie(host, UA, newCookie.trim());
try{
BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while((line = reader.readLine()) != null)content += line+"\r\n";
reader.close();
}
catch(Exception e){/*System.out.println(url+"\r\n"+e);*/}
mp.put("url", url);
mp.put("header", header);
mp.put("content", content);
}
catch(Exception e){
mp.put("url", "");
mp.put("header", "");
mp.put("content", "");
}
if(redirect != null && this.redirectCount < 3){
mp = getFromUrl(redirect);
this.redirectCount++;
}
return mp;
}

Use jsoup for example. Detecting character encoding of a random website is complex issue because of lying/non-existent headers and 2 different meta tags. For example, the page you linked doesn't send the charset in Content-Type header.
And you're going to need a HTML parser anyway, you didn't think of going with a regex, did you?
Here's example usage:
Connection connection = Jsoup.connect("http://www.garazh.com.ua/tires/catalog/Marangoni/E-COMM/description/");
connection
.header("Host", host)
.header("User-Agent", UA)
.header("Accept", "text/html,application/xhtml+xml,application/xmlq=0.9,*/*q=0.8")
.header("Accept-Language", "ru-ru,ruq=0.8,en-usq=0.5,enq=0.3")
.header("Accept-Encoding", "gzip,deflate")
.header("Accept-Charset", "utf-8q=0.7,*q=0.7")
.header("Keep-Alive", "115")
.header("Connection", "keep-alive");
connection.followRedirects(true);
Document doc = connection.get();
Map<String, String> cookies = connection.response().cookies();
Elements titles = doc.select(".title");
for( Element title : titles ) {
System.out.println(title.ownText());
}
Output:
Шины Marangoni E-COMM
Описание шины Marangoni E-COMM

You want to look for the 'Content-Type' header:
Content-Type: text/html; charset=utf-8
The "charset" part there is what you're looking for.

DataOutputStream seems not send parameters

I trying to send POST data with DataOutputStream and get response data.
I coded like this.
String urlParameters = "table=page&format=xml";
out.println(urlParameters+"<br/><br/><br/>");
String searchUrl = "http://localhost:8081/WebTest/test.jsp";
URL url = new URL(searchUrl);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setInstanceFollowRedirects(false);
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
connection.setRequestProperty("charset", "utf-8");
connection.setRequestProperty("Content-Length", ""+Integer.toString(urlParameters.getBytes().length));
connection.setUseCaches (false);
DataOutputStream wr = new DataOutputStream(connection.getOutputStream());
wr.writeUTF(urlParameters);
wr.flush();
wr.close();
if( connection.getResponseCode() == 200){
XMLInputFactory xmlFactory = XMLInputFactory.newInstance();
XMLStreamReader reader = xmlFactory.createXMLStreamReader(new InputStreamReader(connection.getInputStream(),"UTF-8"));
try{
while (reader.hasNext()) {
Integer eventType = reader.next();
if (eventType.equals(XMLStreamReader.START_ELEMENT)){
out.print(" " + reader.getName() + "<br/>");
}else if (eventType.equals(XMLStreamReader.CHARACTERS)){
out.print(" " + reader.getText() + "<br/>");
}else if (eventType.equals(XMLStreamReader.ATTRIBUTE)){
out.print(" " + reader.getName() + " <br/>");
}else if (eventType.equals(XMLStreamReader.END_ELEMENT)){
out.print(" " + reader.getName() + " <br/>");
}
}
} catch (XMLStreamException e){
e.printStackTrace();
} finally{
connection.disconnect();
reader.close();
}
}
and this is test.jsp
<?xml version="1.0" encoding="UTF-8" ?>
<%# page language="java" contentType="text/xml; charset=UTF-8" %>
<response>
<table><%=request.getParameter("table") %></table>
</response>
But, result was
response
table
null
table
response
Why request.getParameter("table") cannot get (or DataOutputStream doesn't send) data?
I'm so confused.
Thanks everyone's help.

You should not use DataOutputStream.writeUTF, see API
First, two bytes are written to the output stream as if by the writeShort method giving the number of bytes to follow. This value is the number of bytes actually written out, not the length of the string. Following the length, each character of the string is output, in sequence, using the modified UTF-8 encoding for the character. If no exception is thrown, the counter written is incremented by the total number of bytes written to the output stream. This will be at least two plus the length of str, and at most two plus thrice the length of str.
That is, what DataOutputStream.writeUTF writes can be read only with DataInputStream.readUTF
I suggest to use
OutputStreamWriter w = new OutputStreamWriter(connection.getOuputStream(), "UTF-8");
w.write(urlParameters);
w.flush();

Java uploading .png to server using php POST data

This is the method I have in my java application. It is reading the bytes correctly, I have logged to see if it was. The problem is that the php is not realizing the data is there. I have tested and the .php reads that $_POST is set, but is empty.
public void screenshot(BufferedImage screenshot) {
try {
ImageIO.write(screenshot, "png",
new File(Environment.getStorageDirectory().toString()
.concat(File.separator + SCRIPT_NAME + ".png")));
HttpURLConnection httpUrlConnection;
OutputStream outputStream;
BufferedInputStream fileInputStream;
BufferedReader serverReader;
int totalBytes;
String response = "";
String serverResponse = "";
String localFileName = Environment.getStorageDirectory().toString()
.concat(File.separator + SCRIPT_NAME + ".png");
// Establish a connection
httpUrlConnection = (HttpURLConnection) new URL(
"http://www.scripted.it/scriptoptions/utils/saveScreenshot.php?user="
+ SupraCrafter.statHandler.getUser())
.openConnection();
httpUrlConnection.setDoOutput(true);
httpUrlConnection.setDoInput(true);
httpUrlConnection.setRequestMethod("POST");
httpUrlConnection.setRequestProperty("Content-type",
"application/x-www-form-urlencoded");
outputStream = httpUrlConnection.getOutputStream();
// Buffered input stream
fileInputStream = new BufferedInputStream(new FileInputStream(
localFileName));
// Get the size of the image
totalBytes = fileInputStream.available();
// Loop through the files data
for (int i = 0; i < totalBytes; i++) {
// Write the data to the output stream
outputStream.write(fileInputStream.read());
}
// Close the output stream
outputStream.close();
// New reader to get server response
serverReader = new BufferedReader(new InputStreamReader(
httpUrlConnection.getInputStream()));
// Read the servers response
serverResponse = "";
while ((response = serverReader.readLine()) != null) {
serverResponse = serverResponse + response;
}
System.out.println(serverResponse);
// Close the buffered reader
serverReader.close();
} catch (Exception e) {
e.printStackTrace();
}
try {
URL url = new URL(
"http://scripted.it/scriptoptions/utils/setScreenshotStatus.php?user="
+ SupraCrafter.statHandler.getUser() + "&pass="
+ SupraCrafter.statHandler.getPass() + "&script="
+ SCRIPT_NAME + "&status=1");
BufferedReader in = new BufferedReader(new InputStreamReader(
url.openStream()));
in.close();
} catch (MalformedURLException e) {
} catch (IOException e) {
}
}
Here is the .php file:
<?
// Config
$uploadBase = "../screenshots/";
$uploadFilename = $_GET['user'] . ".png";
$uploadPath = $uploadBase . $uploadFilename;
// Upload directory
if(!is_dir($uploadBase))
mkdir($uploadBase);
// Grab the data
$incomingData = file_get_contents('php://input');
// Valid data?
if(!$incomingData)
die("No input data");
// Write to disk
$fh = fopen($uploadPath, 'w') or die("Error opening file");
fwrite($fh, $incomingData) or die("Error writing to file");
fclose($fh) or die("Error closing file");
echo "Success";
?>
It always echos 'no input data.'

You are not encoding the content with application/x-www-form-urlencoded. You should not simply copy the bytes into the HTTP payload, but instead encode it correctly.
application/x-www-form-urlencoded is not the only possible way of encoding it, multipart/form-data is another common choice. Both are supported by almost all webservers, and as a consequence by PHP.
A tutorial on how to encode using Java is here : http://www.devx.com/Java/Article/17679
Why don't you use Apache's HttpClient or similar library that already do that tedious work for you?
Apache HttpClient : http://hc.apache.org/httpcomponents-client-ga/

Image writing over URLConnection

I am trying to write an image over an HttpURLConnection.
I know how to write text but I am having real problems trying
to write an image
I have succeeded in writing to the local HD using ImageIO:
But I am trying to write Image by ImageIO on url and failed
URL url = new URL(uploadURL);
connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.setDoInput(true);
connection.setUseCaches(false);
connection.setRequestProperty("Content-Type", "multipart/form-data;
boundary=" + boundary);
output = new DataOutputStream(connection.getOutputStream());
output.writeBytes("--" + boundary + "\r\n");
output.writeBytes("Content-Disposition: form-data; name=\"" + FIELD_NAME + "\";
filename=\"" + fileName + "\"\r\n");
output.writeBytes("Content-Type: " + dataMimeType + "\r\n");
output.writeBytes("Content-Transfer-Encoding: binary\r\n\r\n");
ImageIO.write(image, imageType, output);
the uploadURL is the url to an asp page on the server which will upload the image with the file name given in "content-Disposition: part.
now when I send this then asp page find the request and find the name of file. but does not find the file to be uploaded.
The problem is that when writing by ImageIO on URL what will the name of the file on which the ImageIO is writing,
So please help me how ImageIO will write an image on URLConnection and how can I know the name of the file which I have to use in the asp page to upload the file
Thanks for taking the time to read this post
Dilip Agarwal

First I believe that you should call io.flush() and then io.close() after writing image.
Second content type seems strange for me. It seems that you are trying to submit form while it is actually image. I do not know what does your asp expect but typically when I write code that should transfer file over HTTP I send appropriate content type, e.g. image/jpeg.
Here is for example code snippet I extracted from one small utility that I wrote and I am using during my current work:
URL url = new URL("http://localhost:8080/handler");
HttpURLConnection con = (HttpURLConnection)url.openConnection();
con.setDoInput(true);
con.setDoOutput(true);
con.setUseCaches(false);
con.setRequestProperty("Content-Type", "image/jpeg");
con.setRequestMethod("POST");
InputStream in = new FileInputStream("c:/temp/poc/img/mytest2.jpg");
OutputStream out = con.getOutputStream();
copy(in, con.getOutputStream());
out.flush();
out.close();
BufferedReader r = new BufferedReader(new InputStreamReader(con.getInputStream()));
// obviously it is not required to print the response. But you have
// to call con.getInputStream(). The connection is really established only
// when getInputStream() is called.
System.out.println("Output:");
for (String line = r.readLine(); line != null; line = r.readLine()) {
System.out.println(line);
}
I used here method copy() that I took from Jakarta IO utils. Here is the code for reference:
protected static long copy(InputStream input, OutputStream output)
throws IOException {
byte[] buffer = new byte[12288]; // 12K
long count = 0L;
int n = 0;
while (-1 != (n = input.read(buffer))) {
output.write(buffer, 0, n);
count += n;
}
return count;
}
Obviously the server side must be ready to read the image content directly from POST body.
I hope this helps.

The OP seems lost into oblivion but for the benefit of Mister Kite :
// main method
URL url = new URL(uploadURL);
connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true); // triggers "POST"
// connection.setDoInput(true); // only if needed
connection.setUseCaches(false); // dunno
final String boundary = Long.toHexString(System.currentTimeMillis());
connection.setRequestProperty("Content-Type", "multipart/form-data; boundary="
+ boundary);
output = new DataOutputStream(connection.getOutputStream());
try {
// image must be a File instance
flushMultiPartData(image, output, boundary);
} catch (IOException e) {
System.out.println("IOException in flushMultiPartData : " + e);
return;
}
// ...
private void flushMultiPartData(File file, OutputStream serverOutputStream,
String boundary) throws FileNotFoundException, IOException {
// SEE https://stackoverflow.com/a/2793153/281545
PrintWriter writer = null;
try {
// true = autoFlush, important!
writer = new PrintWriter(new OutputStreamWriter(serverOutputStream,
charsetForMultipartHeaders), true);
appendBinary(file, boundary, writer, serverOutputStream);
// End of multipart/form-data.
writer.append("--" + boundary + "--").append(CRLF);
} finally {
if (writer != null) writer.close();
}
}
private void appendBinary(File file, String boundary, PrintWriter writer,
OutputStream output) throws FileNotFoundException, IOException {
// Send binary file.
writer.append("--" + boundary).append(CRLF);
writer.append(
"Content-Disposition: form-data; name=\"binaryFile\"; filename=\""
+ file.getName() + "\"").append(CRLF);
writer.append("Content-Type: "
+ URLConnection.guessContentTypeFromName(file.getName()))
.append(CRLF);
writer.append("Content-Transfer-Encoding: binary").append(CRLF);
writer.append(CRLF).flush();
InputStream input = null;
try {
input = new FileInputStream(file);
byte[] buffer = new byte[1024];
for (int length = 0; (length = input.read(buffer)) > 0;) {
output.write(buffer, 0, length);
}
output.flush(); // Important! Output cannot be closed. Close of
// writer will close output as well.
} finally {
if (input != null) try {
input.close();
} catch (IOException logOrIgnore) {}
}
writer.append(CRLF).flush(); // CRLF is important! It indicates end of
// binary boundary.
}
You may want to add Gzip compression - see file corrupted when I post it to the servlet using GZIPOutputStream for a working class with or without Gzip. The ImageIO has no place here - just write the bytes past the wire and use ImageIO to your heart's content on the server. Based on #BalusC answer