I know there are a lot of questions similar to this all around SO, but they either provide a very case-specific solution that I don't get to adapt to my issue or simply don't seem to work at all.
I have a multi-language app that downloads certain information from the internet and stores it into a file for later usage. This is how the storage is done:
public static void writeStringToFile(String string, File file)
throws IOException {
if (!file.exists()) {
file.createNewFile();
}
FileOutputStream outputStream = new FileOutputStream(file);
outputStream.write(string.getBytes("UTF-8"));
outputStream.close();
}
But later, when the spanish version of the file is read, the app displays the special characters, like ñ, as the black diamond with the question mark inside I-ve tried to:
Download the information in my computer to check that the file is fine and put it manually in the app so that it reads from it instead of downloading it itself. The file is fine, but the app shows no change.
Replace the argument of getBytes by "ISO 8859-1", but the only difference in the result is that the weird character is this time the regular question mark.
Copy the file, once downloaded, from the device to the computer to check if it was fine, and it was already wrong (there are "empty square" characters shown in place of the question marks, that are not shown if I wget the file).
So I'm almost sure that the problem is in how I write the file since it gets out of the server fine but is stored wrong. But I have been so much time looking at the method and I can't find what the problem is...any clues?
EDIT: This is how I download the information.
public static InputStream performGetRequest(String uri)
throws IOException, URISyntaxException, ServerIsCheckingException {
HttpResponse response;
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet();
request.setURI(new URI(uri));
response = client.execute(request);
if (response.getStatusLine().getStatusCode() == 409) {
throw new ServerIsCheckingException();
}
else {
return response.getEntity().getContent();
}
}
To convert it to a String object that I later pass to the method writeStringToFile, I use
public static String inputStreamAsString(InputStream is) throws IOException {
java.util.Scanner s = new java.util.Scanner(is);
String ret;
ret = s.useDelimiter("\\A").hasNext() ? s.next() : "";
return ret;
}
I also thought that writeStringToFile could be the problem, but I tried another alternative which specifies UTF-8 to be used and didn't work either.
You'll have to make sure that the document you are trying to write is being read in the same charset. In your case, if the document you're downliading is in spanish, it will probably be written in UTF-8 or ISO-8859-1, so you'll have to set the corresponding enconding both in the reading and writing.
You might use HttpProtocolParams.setContentCharset() to set the corresponding charset to the BasicHttpParams object.
This might help:
Android Java UTF-8 HttpClient Problem
Related
I'm working on a HTTP server in Java, which for testing purposes is running under Windows 8.1.
The way it's coded makes it so when a certain parameter is set, it changes the header of the HTTP file and sends the file through the socket with something that works kind of like:
socket.outputStream.write(filter.read());
Assume that the communication works fine, since I have tested it with various other filters and it works perfectly.
One of the filters is supposed to grab the HTML file, zip it and then send it to the client, without creating the file in the server machine. This is the header:
"HTTP/1.1 200 OK\nContent-Type: application/zip\nContent-Disposition: filename=\"" + request + ".zip\"\n";
Afterwards, I set my filter to a class I created (which is copied below) and send the file. My problem is that even though the server is definitively sending data, the client only downloads an empty zip file, with nothing inside.
I've been stuck with this issue for a few days, I can't seem to figure out what's wrong. I think that there's something wrong with how I create the entry or maybe how I close the outputs. I can't be sure.
I'd really appreciate any advice that could be given to me on this issue. Thanks for your attention.
class ZipFilterInputStream extends FilterInputStream
{
protected ZipFilterInputStream(InputStream inputToFilter) throws IOException
{
super(inputToFilter);
//Get the stuff ready for compression
ByteArrayOutputStream out = new ByteArrayOutputStream();
ZipOutputStream zout = new ZipOutputStream(out);
zout.putNextEntry(new ZipEntry("file.html"));
//Compress the stream
int data = in.read();
while (data != -1)
{
zout.write(data);
data = in.read();
}
zout.closeEntry();
zout.finish();
//Get the stream ready for reading.
in = new ByteArrayInputStream(out.toByteArray());
out.close();
}
public int read() throws IOException
{
return in.read();
}
}
I'm having a problem with my applet. I have a school project where I'm supposed to make a pong-online game. It runs fine offline but when I try to load it from a server I just get an empty frame with a red text in it. when I click the text I get the message:
incompatible magic value 1013478509
I'm using jetty-all-8.1.8.v20121106.jar, and servlet-api-3.0.jar
The class that starts up the server looks like this:
public class TheServer extends HttpServlet {
private static final long serialVersionUID = 1L;
private Scanner sc;
private String webSite;
private PrintWriter out;
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
resp.setContentType("text/html; charset=UTF-8");
resp.setStatus(HttpServletResponse.SC_OK);
out = resp.getWriter();
sc = new Scanner(new File("F:\\Users\\Johan\\Workspace Kurs 5\\PongOnline\\bin\\pong.html"));
webSite = "";
while(sc.hasNext())
webSite += sc.nextLine();
sc.close();
out.println(webSite);
System.out.println(webSite);
}
public static void main(String...args) throws Exception {
ServletContextHandler context = new ServletContextHandler( ServletContextHandler.SESSIONS);
context.addServlet(TheServer.class, "/");
Server server = new Server(666);
server.setHandler(context);
server.start();
server.join();
}
}
The magic value of a valid Java class is 0xCAFEBABE (the hex value of 3405691582), which is the first 4 bytes. But you're getting 0x3C68746D (the hex value of 1013478509) which in turn stands for the ASCII characters <, h, t and m. To see it yourself, run this piece of code:
int magic = 1013478509;
ByteBuffer b = ByteBuffer.allocate(4);
b.putInt(magic);
System.out.println(new String(b.array()));
This in combination with the applet being served by a website suggests that it's the start of a <html> tag which in turn suggests that it's a HTML document.
So, the HTTP request to applet has apparently actually returned a HTML document. You should be able to see it yourself when you change the current request URI in browser address bar to point to applet's URL. Then you'll see what the browser actually retrieved when it tried to download the applet. Perhaps it's a simple HTTP 404 error document in flavor of a HTML page.
To fix it, just make sure that the URL in the <applet> or <object> tag is correct. It's relative to the current request URL as you see in browser address bar. The way how your servlet works is quite strange. You're streaming a HTML file from outside the deploy folder. This suggests that the applet is also outside the deploy folder and thus not reachable by a valid URL at all. You should put both the HTML page and the applet in the web content folder. This way you don't need that servlet anymore.
This means you have a file with the .class extension which is not a class. All classes have to start with magic number of 0xCAFEBABE
The first four bytes of your "class" reads
System.out.println(new String(BigInteger.valueOf(1013478509).toByteArray()));
prints
<htm
so I suspect it's a HTML file.
According to the Java Language Specification, a proper .class file has starts with the magic number :
The magic item supplies the magic number identifying the class file
format; it has the value 0xCAFEBABE.
If you open any compiled .class file with a hex editor and inspect its first bytes, they should be 0xCAFEBABE. 1013478509 in ASCII translates to <htm.
Make sure you've got the class properly compiled on the server. And more likely, as BalusC already pointed out in his answer, make sure URL's are correct. The <htm... bytes you're getting might be an HTML error document served by the server.
I am using some file IO and want to know if there is a method to check if a file is an image?
This works pretty well for me. Hope I could help
import javax.activation.MimetypesFileTypeMap;
import java.io.File;
class Untitled {
public static void main(String[] args) {
String filepath = "/the/file/path/image.jpg";
File f = new File(filepath);
String mimetype= new MimetypesFileTypeMap().getContentType(f);
String type = mimetype.split("/")[0];
if(type.equals("image"))
System.out.println("It's an image");
else
System.out.println("It's NOT an image");
}
}
if( ImageIO.read(*here your input stream*) == null)
*IS NOT IMAGE*
And also there is an answer: How to check a uploaded file whether it is a image or other file?
In Java 7, there is the java.nio.file.Files.probeContentType() method. On Windows, this uses the file extension and the registry (it does not probe the file content). You can then check the second part of the MIME type and check whether it is in the form <X>/image.
You may try something like this:
String pathname="abc\xyz.png"
File file=new File(pathname);
String mimetype = Files.probeContentType(file.toPath());
//mimetype should be something like "image/png"
if (mimetype != null && mimetype.split("/")[0].equals("image")) {
System.out.println("it is an image");
}
You may try something like this:
import javax.activation.MimetypesFileTypeMap;
File myFile;
String mimeType = new MimetypesFileTypeMap().getContentType( myFile ));
// mimeType should now be something like "image/png"
if(mimeType.substring(0,5).equalsIgnoreCase("image")){
// its an image
}
this should work, although it doesn't seem to be the most elegant version.
There are a variety of ways to do this; see other answers and the links to related questions. (The Java 7 approach seems the most attractive to me, because it uses platform specific conventions by default, and you can supply your own scheme for file type determination.)
However, I'd just like to point out that no mechanism is entirely infallible:
Methods that rely on the file suffix will be tricked if the suffix is non-standard or wrong.
Methods that rely on file attributes (e.g. in the file system) will be tricked if the file has an incorrect content type attribute or none at all.
Methods that rely on looking at the file signature can be tricked by binary files which just happen to have the same signature bytes.
Even simply attempting to read the file as an image can be tricked if you are unlucky ... depending on the image format(s) that you try.
Other answers suggest to load full image into memory (ImageIO.read) or to use standard JDK methods (MimetypesFileTypeMap and Files.probeContentType).
First way is not efficient if read image is not required and all you really want is to test if it is an image or not (and maybe to save it's content type to set it in Content-Type response header when this image will be read in the future).
Inbound JDK ways usually just test file extension and not really give you result that you can trust.
The way that works for me is to use Apache Tika library.
private final Tika tika = new Tika();
private MimeType detectImageContentType(InputStream inputStream, String fileExtension) {
Assert.notNull(inputStream, "InputStream must not be null");
String fileName = fileExtension != null ? "image." + fileExtension : "image";
MimeType detectedContentType = MimeType.valueOf(tika.detect(inputStream, fileName));
log.trace("Detected image content type: {}", detectedContentType);
if (!validMimeTypes.contains(detectedContentType)) {
throw new InvalidImageContentTypeException(detectedContentType);
}
return detectedContentType;
}
The type detection is based on the content of the given document stream and the name of the document. Only a limited number of bytes are read from the stream.
I pass fileExtension just as a hint for the Tika. It works without it. But according to documentation it helps to detect better in some cases.
The main advantage of this method compared to ImageIO.read is that Tika doesn't read full file into memory - only first bytes.
The main advantage compared to JDK's MimetypesFileTypeMap and Files.probeContentType is that Tika really reads first bytes of the file while JDK only checks file extension in current implementation.
TLDR
If you plan to do something with read image (like resize/crop/rotate it), then use ImageIO.read from Krystian's answer.
If you just want to check (and maybe store) real Content-Type, then use Tika (this answer).
If you work in the trusted environment and you are 100% sure that file extension is correct, then use Files.probeContentType from prunge's Answer.
Here's my code based on the answer using tika.
private static final Tika TIKA = new Tika();
public boolean isImageMimeType(File src) {
try (FileInputStream fis = new FileInputStream(src)) {
String mime = TIKA.detect(fis, src.getName());
return mime.contains("/")
&& mime.split("/")[0].equalsIgnoreCase("image");
} catch (IOException e) {
throw new RuntimeException(e);
}
}
So I pushed my java app to a server, pretty excited about that.
Now I want to test something, how can I save the posted data to my servlet to a file, and the filename should be a unique guid.
I have this so far:
public class TestServlet extends javax.servlet.http.HttpServlet {
protected void doPost(javax.servlet.http.HttpServletRequest request, javax.servlet.http.HttpServletResponse response)
throws javax.servlet.ServletException, IOException {
}
protected void doGet(javax.servlet.http.HttpServletRequest request, javax.servlet.http.HttpServletResponse response)
throws javax.servlet.ServletException, IOException {
PrintWriter printWriter = response.getWriter();
printWriter.print("hello, world from testservlet!");
}
}
So assuming the http posted data (say around 50K) will be posted to the field 'payload', how can I grab the posted text, and save it to a file, with the filename being a GUID.
Does java have a construct to clean up an open file, like in c#:
using(var file = new ....)
{
// write to file
}
That closes the connection and cleans up memory etc.
Also, do I need to set special permissions for tomcat to save this file?
I just set things up by default right now (just playing around on a VPS) using ubuntu 11, installed tomcat6.
Thanks.
You can user request to read the "payload", see the API doc for ServletRequest:
request.getParameter("payload");
You can use File to create the file, see AP doc:
File newFile = new File("fileName");
boolean isCreated = newfile.createNewFile();
You can write to the file as follows,
BufferedWriter out = new BufferedWriter(new FileWriter(newFile));
out.write(payLoad);
out.close();
For GUID you see this Create a GUID in Java
And for the clean up, you don't have to worry about it in Java, it's Garbage Collector ( What is the garbage collector in Java? ) does it for you automatically when the reference goes out of scope.
But you should close the resources like out.close to release it back to the system when you are done with it.
Also, do I need to set special permissions for tomcat to save this file?
You do not need to do that because tomcat is just a server, it's more related to the file system (OS). I use Glassfish on Unix and I don't need to do anything like that to create file.
Now I want to test something, how can I save the posted data to my servlet to a file, and the filename should be a unique guid.
Use File#createTempFile() to create a file with an unique ID in the given folder.
File file = File.createTempfile("prefix-", ".ext", "/path/to/files");
// ...
See also:
Saving uploaded file in specific location
Does java have a construct to clean up an open file, like in c#: using?
Only in Java 7 which is already been out for some time.
try (FileWriter writer = new FileWriter(file)) {
writer.write(content);
}
which is equivalent to
FileWriter writer = null;
try {
writer = new FileWriter(file);
writer.write(content);
} finally {
if (writer != null) writer.close();
}
See also:
"using" keyword in java
Also, do I need to set special permissions for tomcat to save this file?
The user who has started Tomcat should indeed have the file write permissions on the given directory.
In the future please ask separate questions in separate SO questions.
Java 7 has a new try with resources construct that will take care of closing the file for you. Otherwise... just close the file; no big deal.
As far as "special permissions", as long as the user Tomcat is running under can access the directory in question, there's no issue. I'd recommend against storing it under the webapp directories, though (and if it's deployed as a war you may not be able to anyway). Keep uploaded files in a known, but separate, directory.
I want to have an application which parses various RSS feeds and send the information to a remote server. The information is sent in xml format via http. At first I tried to deploy this application on my own server, so I send the xml using the method shown in this tutorial by Java Tips. Here is my code which is replicated from the example:
First Method
String strURL = "http://localhost/readme/readme_xml";
String strXMLFilename = "output.xml";
File input = new File(strXMLFilename);
PostMethod post = new PostMethod(strURL);
post.setRequestEntity(new InputStreamRequestEntity(
new FileInputStream(input), input.length()));
post.setRequestHeader(
"Content-type", "text/xml; charset=ISO-8859-1");
HttpClient httpclient = new HttpClient();
try {
int result = httpclient.executeMethod(post);
System.out.println("Response status code: " + result);
System.out.println("Response body: ");
System.out.println(post.getResponseBodyAsString());
} finally {
post.releaseConnection();
}
This works perfectly (I even tested using a remote server outside the localhost). Then, somehow I cant use my own server to deploy this application, so I decided to migrate to Google Apps Engine. One thing about it, as we know it, is that not all libraries are allowed in the environment. So I try another method shown in ExampleDepot.com (I can't find where the exact url though) as below:
Second Method
try {
/* fill up this url with the remote server url */
URL url = new URL("http://localhost/readme/readme_xml");
FileReader fr = new FileReader("output.xml");
char[] buffer = new char[1024*10];
int len = 0;
if ((len = fr.read(buffer)) != -1){
/* send http request to remote server */
URLConnection conn = url.openConnection();
conn.setRequestProperty("Content-Type","text/xml;charset=ISO-8859-1"); /* need to specify the content type */
conn.setDoOutput(true);
conn.setDoOutput(true);
PrintWriter pw = new PrintWriter(conn.getOutputStream());
pw.write(buffer, 0, len);
pw.flush();
/* receive response from remote server*/
BufferedReader bf = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String input = null;
while ((input = bf.readLine()) != null){
System.out.println(input);
}
}
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
The second method though, doesn't work and gives the following error (I use SimpleXMLElement (php) object to parse xml in the remote hosting):
Error message from remote server
Here's the php code from the remote server (In here, I just want the SimpleXMLElement to parse the xml without doing anything else fancy for now)
$xml = new SimpleXMLElement('php://input', NULL, TRUE);
foreach ($xml -> attributes() as $name => $val){
echo "[".$name."] = ".$val."\n";
}
I thought the cause of this problem is the malfunction xml file (because the eclipse IDE indicates there's error of "invalid byte 1 of 1-byte utf-8 sequence"). Then I use the same exact input xml file to the first method, but it still works perfectly.
So is there any adjustment that I need to make to the second method? Or is there any other method that I can use to send xml file to remote server? Let me know if I need to add some other details. Thanks for your help.
NOTE: I actually solved this problem by using the solution given in the comments. I didn't use approaches suggested in the answers, even though those answers are pretty useful. So, I didn't select the best answer out of those answers given. Nonetheless, I still appreciate all of your helps, thus deserve my upvote. Cheers!
I guess you need to change the content type to multipart/form-data. See an already answered question in detailed. The file upload is discussed at the bottom of this example
I would, as the first answer suggest, read the file with an InputStream. Converting from byte to char and back again is unnecessary and a source of error. Also, verify that the input file really is using the ISO-8859-1 encoding.
UPDATE:
When using a FileReader, you accept the default encoding (i.e. how to make chars from bytes). This encoding must match the encoding used for the input file, otherwise there's a great risk that the result is corrupted. The default Java encoding is different for different platforms, so it is generally not a good idea to rely on it.
In your second example, there's no reason to read the file as characters, since it will be sent on the wire as bytes anyway. Using byte streams all the way also avoids the encoding issue (apart from the information in the content-type header).
never read a file as chars unless you are reading a text file. xml is not text, it is a binary format. copy the file using normal InputStreams and byte[]s.
also, as #beny23 suggested in his comment, make sure you always copy streams using a loop, not a single read() (even if your buffer is big enough, it is not guaranteed that the InputStream will give you all the bytes in one call, even for a FileInputStream).