I want to have an application which parses various RSS feeds and send the information to a remote server. The information is sent in xml format via http. At first I tried to deploy this application on my own server, so I send the xml using the method shown in this tutorial by Java Tips. Here is my code which is replicated from the example:
First Method
String strURL = "http://localhost/readme/readme_xml";
String strXMLFilename = "output.xml";
File input = new File(strXMLFilename);
PostMethod post = new PostMethod(strURL);
post.setRequestEntity(new InputStreamRequestEntity(
new FileInputStream(input), input.length()));
post.setRequestHeader(
"Content-type", "text/xml; charset=ISO-8859-1");
HttpClient httpclient = new HttpClient();
try {
int result = httpclient.executeMethod(post);
System.out.println("Response status code: " + result);
System.out.println("Response body: ");
System.out.println(post.getResponseBodyAsString());
} finally {
post.releaseConnection();
}
This works perfectly (I even tested using a remote server outside the localhost). Then, somehow I cant use my own server to deploy this application, so I decided to migrate to Google Apps Engine. One thing about it, as we know it, is that not all libraries are allowed in the environment. So I try another method shown in ExampleDepot.com (I can't find where the exact url though) as below:
Second Method
try {
/* fill up this url with the remote server url */
URL url = new URL("http://localhost/readme/readme_xml");
FileReader fr = new FileReader("output.xml");
char[] buffer = new char[1024*10];
int len = 0;
if ((len = fr.read(buffer)) != -1){
/* send http request to remote server */
URLConnection conn = url.openConnection();
conn.setRequestProperty("Content-Type","text/xml;charset=ISO-8859-1"); /* need to specify the content type */
conn.setDoOutput(true);
conn.setDoOutput(true);
PrintWriter pw = new PrintWriter(conn.getOutputStream());
pw.write(buffer, 0, len);
pw.flush();
/* receive response from remote server*/
BufferedReader bf = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String input = null;
while ((input = bf.readLine()) != null){
System.out.println(input);
}
}
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
The second method though, doesn't work and gives the following error (I use SimpleXMLElement (php) object to parse xml in the remote hosting):
Error message from remote server
Here's the php code from the remote server (In here, I just want the SimpleXMLElement to parse the xml without doing anything else fancy for now)
$xml = new SimpleXMLElement('php://input', NULL, TRUE);
foreach ($xml -> attributes() as $name => $val){
echo "[".$name."] = ".$val."\n";
}
I thought the cause of this problem is the malfunction xml file (because the eclipse IDE indicates there's error of "invalid byte 1 of 1-byte utf-8 sequence"). Then I use the same exact input xml file to the first method, but it still works perfectly.
So is there any adjustment that I need to make to the second method? Or is there any other method that I can use to send xml file to remote server? Let me know if I need to add some other details. Thanks for your help.
NOTE: I actually solved this problem by using the solution given in the comments. I didn't use approaches suggested in the answers, even though those answers are pretty useful. So, I didn't select the best answer out of those answers given. Nonetheless, I still appreciate all of your helps, thus deserve my upvote. Cheers!
I guess you need to change the content type to multipart/form-data. See an already answered question in detailed. The file upload is discussed at the bottom of this example
I would, as the first answer suggest, read the file with an InputStream. Converting from byte to char and back again is unnecessary and a source of error. Also, verify that the input file really is using the ISO-8859-1 encoding.
UPDATE:
When using a FileReader, you accept the default encoding (i.e. how to make chars from bytes). This encoding must match the encoding used for the input file, otherwise there's a great risk that the result is corrupted. The default Java encoding is different for different platforms, so it is generally not a good idea to rely on it.
In your second example, there's no reason to read the file as characters, since it will be sent on the wire as bytes anyway. Using byte streams all the way also avoids the encoding issue (apart from the information in the content-type header).
never read a file as chars unless you are reading a text file. xml is not text, it is a binary format. copy the file using normal InputStreams and byte[]s.
also, as #beny23 suggested in his comment, make sure you always copy streams using a loop, not a single read() (even if your buffer is big enough, it is not guaranteed that the InputStream will give you all the bytes in one call, even for a FileInputStream).
Related
I'm working on a Spring Boot project and I need users to be able to send a GET request to my service and get multiple files as response without pages involved. This is what I have right now:
public void get(HttpServletResponse response,FileInputStream[] streams){
String boundary = "------customBoundary";
response.setContentType("multipart/mixed; boundary="+boundary);
String contentType = "Content-type: audio/mpeg";
var outputStream = response.getOutputStream();
for(FileInputStream ois:streams){
if(ois!=null){
outputStream.println("--"+boundary);
outputStream.println(contentType);
outputStream.println("Content-disposition: attachment; filename=\"random.mp3\"");
outputStream.println();
try{
BufferedInputStream inputStream = new BufferedInputStream(ois);
int swap;
while ((swap = inputStream.read()) != -1) {
outputStream.write(swap);
}
outputStream.println();
outputStream.flush();
ois.close();
inputStream.close();
}catch (Exception s){
s.printStackTrace();
}
}
}
outputStream.println("--"+boundary+"--");
outputStream.flush();
outputStream.close();
}
If you access the service on Chrome, it'll automatically download a file name "download", as the browser sees the whole this as one giant file attachment.
I think the problem may be at the ContentType part, so far I tried "multipart/x-mixed-replace" as some post here suggested, only to find out later that it was removed several years ago. I tried "multipart/form-data" out of desperation and that didn't work.
The files I'm dealing with are media files, it's quite large and I'm running this on a crappy server, so putting them in one zip file is not an option.
Most of the result I found more or less have something to do with html or javascript, but what I'm doing here is intended as an API for other services to call so I'm hoping it can be done with pure java.
Up till early this year the US Treasury web site posted monthly US Receipts and Outlays data in txt format. It was easy to write a program to read and store the info. All I use were:
URL url = new URL("https://www.fiscal.treasury.gov/fsreports/rpt/mthTreasStmt/mts1214.txt")
URLConnection connection.openConnection();
InputStream is = connection.getInputStream();
Then I just read the InputStream into a local file.
Now when I try same code, for May, I get an InputStream with nothing in it.
Just clicking on "https://www.fiscal.treasury.gov/fsreports/rpt/mthTreasStmt/mts0415.xlsx" opens an excel worksheet (the download path has since changed).
Which is great if you don't mind clicking on each link separately ... saving the file somewhere ... opening it manually to enable editing ... then saving it again as a real .xlsx file (because they really hand you an .xls file.)
But when I create a URL from that link, and use it to get an InputStream, the is empty. I also tried url.openStream() directly. No different.
Can anyone see a way I can resume using a program to read the new format?
In case its of interest I now use this code to write the stream to the file bit by bit... but there are no bits, so I don't know if it works.
static void copyInputStreamToFile( InputStream in, File file ) {
try {
OutputStream out = new FileOutputStream(file);
byte[] buf = new byte[1024];
System.out.println("reading: " + in.read(buf));
//This is what tells me it is empty, i.e. the loop below is ignored.
int len;
while((len=in.read(buf))>0){
out.write(buf,0,len);
}
out.close();
in.close();
} catch (Exception e) {
e.printStackTrace();
}
}
Any help is appreciated.
I know there are a lot of questions similar to this all around SO, but they either provide a very case-specific solution that I don't get to adapt to my issue or simply don't seem to work at all.
I have a multi-language app that downloads certain information from the internet and stores it into a file for later usage. This is how the storage is done:
public static void writeStringToFile(String string, File file)
throws IOException {
if (!file.exists()) {
file.createNewFile();
}
FileOutputStream outputStream = new FileOutputStream(file);
outputStream.write(string.getBytes("UTF-8"));
outputStream.close();
}
But later, when the spanish version of the file is read, the app displays the special characters, like ñ, as the black diamond with the question mark inside I-ve tried to:
Download the information in my computer to check that the file is fine and put it manually in the app so that it reads from it instead of downloading it itself. The file is fine, but the app shows no change.
Replace the argument of getBytes by "ISO 8859-1", but the only difference in the result is that the weird character is this time the regular question mark.
Copy the file, once downloaded, from the device to the computer to check if it was fine, and it was already wrong (there are "empty square" characters shown in place of the question marks, that are not shown if I wget the file).
So I'm almost sure that the problem is in how I write the file since it gets out of the server fine but is stored wrong. But I have been so much time looking at the method and I can't find what the problem is...any clues?
EDIT: This is how I download the information.
public static InputStream performGetRequest(String uri)
throws IOException, URISyntaxException, ServerIsCheckingException {
HttpResponse response;
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet();
request.setURI(new URI(uri));
response = client.execute(request);
if (response.getStatusLine().getStatusCode() == 409) {
throw new ServerIsCheckingException();
}
else {
return response.getEntity().getContent();
}
}
To convert it to a String object that I later pass to the method writeStringToFile, I use
public static String inputStreamAsString(InputStream is) throws IOException {
java.util.Scanner s = new java.util.Scanner(is);
String ret;
ret = s.useDelimiter("\\A").hasNext() ? s.next() : "";
return ret;
}
I also thought that writeStringToFile could be the problem, but I tried another alternative which specifies UTF-8 to be used and didn't work either.
You'll have to make sure that the document you are trying to write is being read in the same charset. In your case, if the document you're downliading is in spanish, it will probably be written in UTF-8 or ISO-8859-1, so you'll have to set the corresponding enconding both in the reading and writing.
You might use HttpProtocolParams.setContentCharset() to set the corresponding charset to the BasicHttpParams object.
This might help:
Android Java UTF-8 HttpClient Problem
I'm working on a HTTP server in Java, which for testing purposes is running under Windows 8.1.
The way it's coded makes it so when a certain parameter is set, it changes the header of the HTTP file and sends the file through the socket with something that works kind of like:
socket.outputStream.write(filter.read());
Assume that the communication works fine, since I have tested it with various other filters and it works perfectly.
One of the filters is supposed to grab the HTML file, zip it and then send it to the client, without creating the file in the server machine. This is the header:
"HTTP/1.1 200 OK\nContent-Type: application/zip\nContent-Disposition: filename=\"" + request + ".zip\"\n";
Afterwards, I set my filter to a class I created (which is copied below) and send the file. My problem is that even though the server is definitively sending data, the client only downloads an empty zip file, with nothing inside.
I've been stuck with this issue for a few days, I can't seem to figure out what's wrong. I think that there's something wrong with how I create the entry or maybe how I close the outputs. I can't be sure.
I'd really appreciate any advice that could be given to me on this issue. Thanks for your attention.
class ZipFilterInputStream extends FilterInputStream
{
protected ZipFilterInputStream(InputStream inputToFilter) throws IOException
{
super(inputToFilter);
//Get the stuff ready for compression
ByteArrayOutputStream out = new ByteArrayOutputStream();
ZipOutputStream zout = new ZipOutputStream(out);
zout.putNextEntry(new ZipEntry("file.html"));
//Compress the stream
int data = in.read();
while (data != -1)
{
zout.write(data);
data = in.read();
}
zout.closeEntry();
zout.finish();
//Get the stream ready for reading.
in = new ByteArrayInputStream(out.toByteArray());
out.close();
}
public int read() throws IOException
{
return in.read();
}
}
So what I am trying to achieve is reading the contents of a .txt file from a url:
BufferedReader reader = null;
File f = new File ("www.website.com/filename.txt");
if (f.exists()) {
try {
reader = new BufferedReader(new FileReader(f));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
String line = "";
try {
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
} catch (IOException e) {
e.printStackTrace();
}
}
Even though I have content in the .txt file (only one line), when I print the line nothing shows up. Is reading a file from a URL or from your hard drive different, or am I doing something wrong?
The File class is for files on a "normal" file system (usually local, but potentially networked) - not URLs. Basically it's for the sort of file you can use (e.g. read or edit) directly on a command line, with no HTTP involved1.
That's what the URL class is for. So you can either use that (with URLConnection) or use a dedicated HTTP 3rd party library, such as the Apache HttpClient library.
1 I'm sure there are some shells which allow the use of URLs as if they were local filenames, but I'm talking about a more traditional approach.
I tried own my own and this worked...
URL urlObj=new URL("http://www.example.com/index.html"); //This can be any website' index.html or an available file
//we basically get HTML page/file
Scanner fGetter=new Scanner(urlObj.openStream());
while(fGetter.hasNext()){
System.out.println(""+fGetter.nextLine());
}
And I think "example.com" can be used without any legal issues :)
I don't actually know but I don't think you can read a file from a URL like that. You need to send a HTTP GET request to that url to read the information in.
See object HttpURLConnection.
http://docs.oracle.com/javase/7/docs/api/java/net/HttpURLConnection.html