I'm writing an application (for educational purposes), which needs to use database management system (I wrote my own extremely primitive DBMS, it is part of the task). And I want to ensure that at any time my application is running contents of all tables are correct. For that purposes I wrote method, which looks through each file and make necessary checks. The problem is that I want to call this method only once, when application starts and deny access to files to ensure that nobody changed their contents while my program is working.
I use the following approach. When application starts, I initialize InputStreamReader and OutputStreamWriter, store them and close them only when my application is terminated.
Part of initialization method:
FileInputStream fis = new FileInputStream(file);
FileOutputStream fos = new FileOutputStream(file, true);
InputStreamReader isr = new InputStreamReader(fis, "UTF-8");
OutputStreamWriter osw = new OutputStreamWriter(fos, "UTF-8");
this.tables.get(table).put("fis", fis);
this.tables.get(table).put("fos", fos);
this.tables.get(table).put("isr", isr);
this.tables.get(table).put("osw", osw);
Close method:
try {
for(Map<String, Object> table_map: tables.values()) {
OutputStreamWriter osw = (OutputStreamWriter)table_map.get("osw");
InputStreamReader isr = (InputStreamReader)table_map.get("isr");
if (osw != null)
osw.close();
if (isr != null)
isr.close();
}
}
catch (IOException e) {
throw new DBException("Closing error");
}
Partly, this approach works, because when I try to modify any of these files using MS Notepad, I get the following error
"The process cannot access the file because it is being used by
another process"
That's what I want to see. But if I use Notepad++, I can make any modifications when my application is running, that's not what I expect to see. So what can I do to ensure that no other process can modify my files?
I tried to use FileLock, but it denies access only for my process, if I'm not mistaken.
Sorry for my poor English, hope you will understand my question anyway.
I'm not sure this is a problem worth solving. Whatever approach you take, someone with the correct privileges can probably undo your file protection and could make changes anyway.
It is best to focus on gracefully handling invalid data and otherwise trusting what is in the file. Adding some kind of integrity check (per row or table) will make it harder for someone to accidentally or maliciously change your data in a way that leaves it looking "valid".
If you read the section "Platform dependencies" in the java.nio.channels.FileLock docsyou see that:
FileLocks are not (only) for locking inside one JVM but for all processes on the computer.
File locks (note the different spelling) are greatly platform and configuration specific.
So you basicyll have to ask yourself: What protection do I really need?
If you only want to guard against running your programm multiple times on the same data you can assume that your programm "behaves well" and
use FileLocks or
use a marker lock file or
use a "dirty/locked" marker inside the file
If you want to protect against every other program then you are lost as you have seen in the Notepad++ scenario: Considering all platforms and all possible ways to circumvent locks and using Java- you have no chance.
Related
Do I can open a file (linux character device) for read+write, and use the two classes to implement a dialog like client-server?
Something like this:
File file = new File("/dev/ttyS0");
FileOutpuStream fo = new FileOutputStream(file)
FileInputStream fi = new FileInputStream(file)
After the above declarations, can I continuously send pollings (questions) to the file, and read its replies? (Of course, attached to ttyS0 there is a kind of server)
I was not able to test it, but you might want to give RandomAccessFile a try.
It does not give you the opertunity to create streams, but it implements DataInput and DataOutput. Thats maybe good enough for your purpose?
RandomAccessFile docs
String file = "/dev/ttyS0";
try {
RandomAccessFile f = new RandomAccessFile(file, "rwd");
} catch (IOException e){
e.printStackTrace();
}
The /dev/ttyS0 file is a device file for a serial terminal.
If the device has been configured appropriately to connect to a serial terminal line, then you should be able to read and write like that. However, on a typical desktop or laptop, it probably won't work because there won't be connected serial line.
(For example, when I do this on my PC:
$ sudo bash -c "cat < /dev/ttyS0"
I get this:
cat: -: Input/output error
which is saying that the device cannot be read from.)
Note that a /dev/tty* device does not behave like a regular file. The characters that are written in no way relate to the characters that you read back. Also note that it is not possible to make ioctl requests using the standard Java APIs. So configuring the terminal driver from Java would be problematic.
If you were talking abour reading and writing a regular file, it should work too. However, the behavior could be a rather confusing, especially if you have buffering in your streams. One issue you need to deal with is that the two file descriptors are independent of each other.
If you need to do this kind of thing with a regular file, you should probably use RandomAccessFile.
I didn't try RandomAccessFile, it could also work... it worked smoothly with FileInputStream and FileOutputStream, see this answer in SO: https://stackoverflow.com/a/56935267/7332147
I have a temporary file which I want to send the client from the controller in the Play Framework. Can I delete the file after opening a connection using FileInputStream? For example can I do something like this -
File file = getFile();
InputStream is = new FileInputStream(file);
file.delete();
renderBinary(is, "name.txt");
What if file is a large file? If I delete the file, will subsequent reads() on InputStream give an error? I have tried with files of around 1MB I don't get an error.
Sorry if this is a very naive question, but I could not find anything related to this and I am pretty new to Java
I just encountered this exact same scenario in some code I was asked to work on. The programmer was creating a temp file, getting an input stream on it, deleting the temp file and then calling renderBinary. It seems to work fine even for very large files, even into the gigabytes.
I was surprised by this and am still looking for some documentation that indicates why this works.
UPDATE: We did finally encounter a file that caused this thing to bomb. I think it was over 3 Gb. At that point, it became necessary to NOT delete the file while the rendering was in process. I actually ended up using the Amazon Queue service to queue up messages for these files. The messages are then retrieved by a scheduled deletion job. Works out nicely, even with clustered servers on a load balancer.
It seems counter-intuitive that the FileInputStream can still read after the file is removed.
DiskLruCache, a popular library in the Android world originating from the libcore of the Android platform, even relies on this "feature", as follows:
// Open all streams eagerly to guarantee that we see a single published
// snapshot. If we opened streams lazily then the streams could come
// from different edits.
InputStream[] ins = new InputStream[valueCount];
try {
for (int i = 0; i < valueCount; i++) {
ins[i] = new FileInputStream(entry.getCleanFile(i));
}
} catch (FileNotFoundException e) {
....
As #EJP pointed out in his comment on a similar question, "That's how Unix and Linux behave. Deleting a file is really deleting its name from the directory: the inode and the data persist while any processes have it open."
But I don't think it is a good idea to rely on it.
I've made two apps designed to run concurrently (I do not want to combine them), and one reads from a certain file and the other writes to it. When one or the other are running no errors, however if they are both running a get an access is denied error.
Relevant code of the first:
class MakeImage implements Runnable {
#Override
public void run() {
File file = new File("C:/Users/jeremy/Desktop/New folder (3)/test.png");
while (true) {
try{
//make image
if(image!=null)
{
file.createNewFile();
ImageIO.write(image, "png", file);
hello.repaint();}}
catch(Exception e)
{
e.printStackTrace();
}
}
}
}
Relevant code of the second:
BufferedImage image = null;
try {
// Read from a file
image = ImageIO.read(new File("C:/Users/jeremy/Desktop/New folder (3)/test.png"));
}
catch(Exception e){
e.printStackTrace();
}
if(image!=null)
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ImageIO.write( image, "png", baos );
baos.flush();
byte[] imageInByte = baos.toByteArray();
baos.close();
returns=Base64.encodeBase64String(imageInByte);
}
I looked at this: Java: how to handle two process trying to modify the same file, but that is when both are writting to the file where here only one is. I tried the retry later method as suggested in the former's answer without any luck. Any help would be greatly appreciated.
Unless you use OS level file locking of some sort and check for the locks you're not going to be able to reliably do this very easily. A fairly reliable way to manage this would be to use another file in the directory as a semaphore, "touch" it when you're writing or reading and remove it when you're done. Check for the existence of the semaphore before accessing the file. Otherwise you will need to use a database of some sort to store the file lock (guaranteed consistency) and check for it there.
That said, you really should just combine this into 1 program.
Try RandomAccessFile.
This is a useful but very dangerous feature. It goes like this "if you create different instances of RandomAccessFile for a same file you can concurrently write to the different parts of the file."
You can create multiple threads pointing to different parts of the file using seek method and multiple threads can update the file at the same time. Seek allow you to move to any part of the file even if it doesn't exist (after EOF), hence you can move to any location in the newly created file and write bytes on that location. You can open multiple instances of the same file and seek to different locations and write to multiple locations at the same time.
Use synchronized on the method that modify the file.
Edited:
As per the Defination of a Thread safe class, its this way.. " A class is said to be thread safe, which it works correctly in the presence of the underlying OS interleaving and scheduling with NO means of synchronization mechanism from the client side".
I believe there is a File which is to be accessed on to a different machine, so there must be some client-server mechanism, if its there.. then Let the Server side have the synchronization mechanism, and then it doesnt matters how many client access it...
If not, synchronized is more than enough........
I'm trying to delete a file that another thread within my program has previously worked with.
I'm unable to delete the file but I'm not sure how to figure out which thread may be using the file.
So how do I find out which thread is locking the file in java?
I don't have a straight answer (and I don't think there's one either, this is controlled at OS-level (native), not at JVM-level) and I also don't really see the value of the answer (you still can't close the file programmatically once you found out which thread it is), but I think you don't know yet that the inability to delete is usually caused when the file is still open. This may happen when you do not explicitly call Closeable#close() on the InputStream, OutputStream, Reader or Writer which is constructed around the File in question.
Basic demo:
public static void main(String[] args) throws Exception {
File file = new File("c:/test.txt"); // Precreate this test file first.
FileOutputStream output = new FileOutputStream(file); // This opens the file!
System.out.println(file.delete()); // false
output.close(); // This explicitly closes the file!
System.out.println(file.delete()); // true
}
In other words, ensure that throughout your entire Java IO stuff the code is properly closing the resources after use. The normal idiom is to do this in the try-with-resources statement, so that you can be certain that the resources will be freed up anyway, even in case of an IOException. E.g.
try (OutputStream output = new FileOutputStream(file)) {
// ...
}
Do it for any InputStream, OutputStream, Reader and Writer, etc whatever implements AutoCloseable, which you're opening yourself (using the new keyword).
This is technically not needed on certain implementations, such as ByteArrayOutputStream, but for the sake of clarity, just adhere the close-in-finally idiom everywhere to avoid misconceptions and refactoring-bugs.
In case you're not on Java 7 or newer yet, then use the below try-finally idiom instead.
OutputStream output = null;
try {
output = new FileOutputStream(file);
// ...
} finally {
if (output != null) try { output.close(); } catch (IOException logOrIgnore) {}
}
Hope this helps to nail down the root cause of your particular problem.
About this question, I also try to find out this answer, and ask this question and find answer:
Every time when JVM thread lock a file exclusively, also JVM lock
some Jave object, for example, I find in my case:
sun.nio.fs.NativeBuffer
sun.nio.ch.Util$BufferCache
So you need just find this locked Java object and analyzed them and
you find what thread locked your file.
I not sure that it work if file just open (without locked exclusively), but I'm sure that is work if file be locked exclusively by Thread (using java.nio.channels.FileLock, java.nio.channels.FileChannel and so on)
More info see this question
What I am doing is I am reading in a html file and I am looking for a specific location in the html for me to enter some text.
So I am using a bufferedreader to read in the html file and split it by the tag . I want to enter some text before this but I am not sure how to do this. The html would then be along the lines of ...(newText)(/HEAD) (The brackets round head are meant to be angled brackets. Don't know how to insert them)
Would I need a PrintWriter to the same file and if so, how would I tell that to write it in the correct location.
I am not sure which way would be most efficient to do something like this.
Please Help.
Thanks in advance.
Here is part of my java code:
File f = new File("newFile.html");
FileOutputStream fos = new FileOutputStream(f);
PrintWriter pw = new PrintWriter(fos);
BufferedReader read = new BufferedReader(new FileReader("file.html"));
String str;
int i=0;
boolean found = false;
while((str= read.readLine()) != null)
{
String[] data = str.split("</HEAD>");
if(found == false)
{
pw.write(data[0]);
System.out.println(data[0]);
pw.write("</script>");
found = true;
}
if(i < 1)
{
pw.write(data[1]);
System.out.println(data[1]);
i++;
}
pw.write(str);
System.out.println(str);
}
}
catch (Exception e) {
e.printStackTrace( );
}
When I do this it gets to a point in the file and I get these errors:
FATAL ERROR: MERLIN: Unable to connect to EDG API,
Cannot find .edg_properties file.,
java.lang.OutOfMemoryError: unable to create new native thread,
Cannot truncate table,
EXCEPTION:Cannot open connection to server: SQLExceptio,
Caught IOException: java.io.IOException: JZ0C0: Connection is already closed, ...
I'm not sure why I get these or what all of these mean?
please Help.
Should be pretty easy:
Read file into a String
Split into before/after chunks
Open a temp file for writing
Write before chunk, your text, after chunk
Close up, and move temp file to original
Sounds like you are wondering about the last couple steps in particular. Here is the essential code:
File htmlFile = ...;
...
File tempFile = File.createTempFile("foo", ".html");
FileWriter writer = new FileWriter(tempFile);
writer.write(before);
writer.write(yourText);
writer.write(after);
writer.close();
tempFile.renameTo(htmlFile);
Most people suggest writing to a temporary file and then copying the temporary file over the original on successful completion.
The forum thread has some ideas of how to do it.
GL.
For reading and writing you can use FileReaders/FileWriters or the corresponding IO stream classes.
For the editing, I'd suggest to use an HTML parser to handle the document. It can read the HTML document into an internal datastructure which simplifies your effort to search for content and apply modification. (Most?) Parsers can serialize the document to HTML again.
At least you're sure to not corrupt the HTML document structure.
Following up on the list of errors in your edit, a lot of that possibly stems from the OutOfMemoryError. That means you simply ran out of memory in the JVM, so Java was unable to allocate objects. This may be caused by a memory leak in your application, or it could simply be that the work you're trying to do does need more memory transiently than you have allocated it.
You can increase the amount of memory that the JVM starts up with by providing the Xmx argument to the java executable, e.g.:
-Xmx1024m
would set the maximum heap size to 1024 megabytes.
The other issues might possibly caused by this; when objects can't reliably be created or modified, lots of weird things tend to happen. That said, there's a few things that look like you can take action. In particular, whatever MERLIN is it looks like it can't do it's work because it needs a property file for EDG, which it's unable to find in the location it's looking. You'll probably need to either put a config file there, or tell it to look at another location.
The other IOExceptions are fairly self-explanatory. Your program could not establish a connection to the server because of a SQLException (the underlying exception itself will probably be found in the logs); and some other part of the program tried to communicate to a remote machine using a closed connection.
I'd look at fixing the properties file (if it's not a benign error) and the memory issues first, and then seeing if any of the remaining problems still manifest.