I've read through the other excellent stack overflow articles and tried a lot of them and variations on them but must be making some basic error time and time again? The page I'm posting to works but when I run my java program I just get an empty set on the mySQL database that the data is being posted to. The direct URL that works would be:
http://myURL.co.uk/enteremail.php?email=value
the code
String data = URLEncoder.encode("email", "UTF-8") + "=" + URLEncoder.encode("value1", "UTF-8");
URL url = new URL("http://myURL.co.uk/enteremail.php");
URLConnection conn = url.openConnection();
conn.setRequestMethod("POST");
conn.setDoOutput(true);
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(data);
wr.flush();
wr.close();
I know that there are much better ways of doing this using java but I have to use this way as its a bit of a workaround for another problem.
TIA
You need to close the URL connection as well.
I'd suggest checking out the Apache Http client which will do all the heavy lifting for you.
I'd still like to know what was up with my original code, but because what I wanted to do was so simple I've just cut a corner and done this which works for sending the post request to a php page but may not be the solution for anything else:
String email = "myEmail";
URL post= new URL("http://myURL.co.uk/enteremail.php?email="+email);
URLConnection goPost = post.openConnection();
new InputStreamReader(goPost.getInputStream());
Hopefully of some help to someone else down the line, KISS
Related
stockURL = 'https://finviz.com/screener.ashx?v=152&f=cap_smallover&ft=4&c=0,1,2,6,7,10,11,13,14,45,65';
buffer = java.io.BufferedReader(...
java.io.InputStreamReader(...
openStream(...
java.net.URL(stockURL))));
I don't know how to go about this^ at all, so any other suggestions or improvements to the code would be welcome. Here is another thread that wasn't answered https://www.mathworks.com/matlabcentral/answers/434084-reading-in-stock-data-from-a-website
I haven't tried this in a while.
URL url = new URL("http://localhost:8080/foobar");
URLConnection hc = url.openConnection();
hc.setRequestProperty("User-Agent", " ");
You'll use your user agent as the second parameter, where I have the space.
I am sending commands to a server using http, and I currently need to parse a response that the server sends back (I am sending the command via the command line, and the servers response appears in my browser).
There are a lot of resources such as this: Saving a web page to a file in Java, that clearly illustrate how to scrape a page such as cnn.com. However, since this is a response page that is only generated when the camera receives a specific command, my attempts to use the method described by Mike Deck (in the link above) have met with failure. (Specifically, when my program requests the page again the server returns a 401 error.)
The response from the server opens a new tab in my browser. Essentially, I need to know how to save the current web page using java, since reading in a file is probably the most simple way to approach this. Do any of you know how to do this?
TL;DR How do you save the current webpage to a webpage.html or webpage.txt file using java?
EDIT: I used Base64 from the Apache commons codec, which solved my 401 authentication issue. However, I am still getting a 400 error when I attempt to connect my InputStream (see below). Does this mean a connection isn't being established in the first place?
URL url = new URL ("http://"+ipAddress+"/axis-cgi/record/record.cgi?diskid=SD_DISK");
byte[] encodedBytes = Base64.encodeBase64("root:pass".getBytes());
String encoding = new String (encodedBytes);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("POST");
connection.setDoInput (true);
connection.setRequestProperty ("Authorization", "Basic " + encoding);
connection.connect();
InputStream content = (InputStream)connection.getInputStream();
BufferedReader in = new BufferedReader (new InputStreamReader (content));
String line;
while ((line = in.readLine()) != null) {
System.out.println(line);
}
EDIT 2: Changing the request to a GET resolved the issue.
So while scrutinizing my code above, I decided to change
connection.setRequestMethod("POST");
to
connection.setRequestMethod("GET");
This solved my problem. In hindsight, I think the server was not recognizing the HTTP because it is not set up to handle the various trappings that come along with post.
Don't know how to explain it better but i'm trying to get a response from an URL containing a function (right?).
I'm working on this for a lot of hours and progressing a little every time but can't get this finally working.
This is the request and response headers from chrome dev tools:
Headers
My code is:
String params = "{\"prefixText\":\"" + city
+ "\",\"count\":10,\"contextKey\":\"he\"}";
conn = (HttpURLConnection) new URL(
"http://bus.gov.il/WebForms/wfrmMain.aspx/GetCompletionList")
.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("POST");
conn.setChunkedStreamingMode(0);
// conn.setFixedLengthStreamingMode(params.length());
conn.addRequestProperty("Accept", "*/*");
conn.addRequestProperty("Content-Type", "application/json; charset=UTF-8");
conn.addRequestProperty("Content-Length", String.valueOf(params.length()));
conn.addRequestProperty("Host", "bus.gov.il");
conn.addRequestProperty("Origin", "http://bus.gov.il");
conn.addRequestProperty("X-Requested-With", "XMLHttpRequest");
conn.addRequestProperty("Referer",
"http://bus.gov.il/WebForms/wfrmMain.aspx?width=1024&company=1&language=he&state=");
OutputStream os = new BufferedOutputStream(conn.getOutputStream());
os.write(params.getBytes());
String answer = readStream(conn.getInputStream());
I get the exception (I see in the stack trace) when calling "getinputstream" on this line:
String answer = readStream(conn.getInputStream());
before entering the readStream function!
I don't know how to solve it...
Tried searching about xmlhttprequest but understood that it's only in JS.
Also: I know I have a lot of unnecessary request properties but I can't figure out which are unnecessary until the code will work.
Thanks in advance :)
Sadly, it used to be (and probably still is) that the HttpURLConnection throws a FileNotFoundException when you get a 404 error. When you are doing the getInputStream() that's when it's first connecting, so any error from the server will show up there.
Get Wireshark or something if you want to see what's really going on in HTTP land as you make the request.
I have a java application that sends text to a sql database on a server. Currently my java application takes the text, puts it into the url, then sends it to a php page on the server that takes it with GET and puts it in the database. that works fine to an extent, the problem is, that i need to be able to send lots of text, and i keep getting 414, uri to long errors. is there a better way to do this?
ok, i tried what you said, and read the tutorial, but something is not working. here is my code that i tried
public void submitText(String urls,String data) throws IOException{
URL url = new URL(urls);
URLConnection con = url.openConnection();
con.setRequestProperty("Content-Type", "text/plain; charset=utf-8");
con.setDoOutput(true);
OutputStreamWriter out = new OutputStreamWriter(con.getOutputStream(), "UTF-8");
out.write(data);
out.flush();
}
submitText(server + "publicPB.php", "param=" + text);
here is my php code
$param = $_POST['param'];
$sql = "UPDATE table SET cell='{$param}' WHERE 1";
mysql_query($sql);
...
im pretty sure its not a problem with the php as the php worked fine with GET, and thats all i change with it, my problem i think is that im not 100% sure how to send data to it with the java
Use a POST instead of a GET and send the text as the request body. You can only pass so much data to a URL. E.g.:
// Assuming 'input' is a String and contains your text
URL url = new URL("http://hostname/path");
URLConnection con = url.openConnection();
con.setRequestProperty("Content-Type", "text/plain; charset=utf-8");
con.setDoOutput(true);
OutputStreamWriter out = new OutputStreamWriter(con.getOutputStream(), "UTF-8");
out.write(input);
out.close();
See Reading from and Writing to a URLConnection for more details.
Why don't you use POST to send data across to PHP page? GET does have a smaller limit of content.
Use POST requests, which do not have content length limits.
POST requests do not have length content limits and are much secure than GET requests ;)
If using SQL Server I would look into leveraging BCP. You can write the file and call BCP from within Java, and it will send the information directly to your database.
I am trying to write a servlet that will send a XML file (xml formatted string) to another servlet via a POST.
(Non essential xml generating code replaced with "Hello there")
StringBuilder sb= new StringBuilder();
sb.append("Hello there");
URL url = new URL("theservlet's URL");
HttpURLConnection connection = (HttpURLConnection)url.openConnection();
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Length", "" + sb.length());
OutputStreamWriter outputWriter = new OutputStreamWriter(connection.getOutputStream());
outputWriter.write(sb.toString());
outputWriter.flush();
outputWriter.close();
This is causing a server error, and the second servlet is never invoked.
This kind of thing is much easier using a library like HttpClient. There's even a post XML code example:
PostMethod post = new PostMethod(url);
RequestEntity entity = new FileRequestEntity(inputFile, "text/xml; charset=ISO-8859-1");
post.setRequestEntity(entity);
HttpClient httpclient = new HttpClient();
int result = httpclient.executeMethod(post);
I recommend using Apache HTTPClient instead, because it's a nicer API.
But to solve this current problem: try calling connection.setDoOutput(true); after you open the connection.
StringBuilder sb= new StringBuilder();
sb.append("Hello there");
URL url = new URL("theservlet's URL");
HttpURLConnection connection = (HttpURLConnection)url.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Length", "" + sb.length());
OutputStreamWriter outputWriter = new OutputStreamWriter(connection.getOutputStream());
outputWriter.write(sb.toString());
outputWriter.flush();
outputWriter.close();
The contents of an HTTP post upload stream and the mechanics of it don't seem to be what you are expecting them to be. You cannot just write a file as the post content, because POST has very specific RFC standards on how the data included in a POST request is supposed to be sent. It is not just the formatted of the content itself, but it is also the mechanic of how it is "written" to the outputstream. Alot of the time POST is now written in chunks. If you look at the source code of Apache's HTTPClient you will see how it writes the chunks.
There are quirks with the content length as result, because the content length is increased by a small number identifying the chunk and a random small sequence of characters that delimits each chunk as it is written over the stream. Look at some of the other methods described in newer Java versions of the HTTPURLConnection.
http://java.sun.com/javase/6/docs/api/java/net/HttpURLConnection.html#setChunkedStreamingMode(int)
If you don't know what you are doing and don't want to learn it, dealing with adding a dependency like Apache HTTPClient really does end up being much easier because it abstracts all the complexity and just works.
Don't forget to use:
connection.setDoOutput( true)
if you intend on sending output.