Now I'm making a little program in Java which must read a really big file. Due to this thing, I want to access to the file but not read completely each time, then my question is the following: can I change the offset of the file descriptor with a simple instruction or the only solution that I have is read all the previous lines which I don't need?
In other words, can I simulate the lseek command in my input file?
I think it's not necessary this time, but if someone wants code, I'll post it.
Regards!
I think you probably want RandomAccessFile.
Specifically, you want the seek(long) method.
Related
I want to read a file incrementally in java while the file is being modified/written by some other process. So suppose Process "A" is writing/logging a file "X" and another process "B" wants to incrementally read the file "X", say every 1 sec (or even continuously) to find a particular pattern. What's the best way to do this in java? I know I can use RandomAccessFile's 'seek' method but will that interfere with the writing of the file? Is there a better way to do this?
Poll the data modified of the file. Opening it up for reading can prevent other programs from writing to the file at the same time.
If you're able to use Java 7, you could take advantage of the WatchService ... but it doesn't solve having to parse the whole file.
The only thing I can think off is maintaining some kind of "marker" that would indicate the last position you were up to. The next time you came to read the file, you could skip to this point and read from there (updating the marker when you're done)
I would like to read a file from last line using RandomAccessFile. Is this possible or do I have to use another class?
Beside this file changes during the time so the last line doesn't remain last forever. During the reading another, java program write on it. My question is: the program will see in the same time another java program write on the file, the changes?
Edit
Well suppose I have a server that write its faults in a error log file during it's running.another program reads every line.which should be the best way?
Yes reading a file from the bottom up is possible using RandomAccessFile:
Reading the Last Line of a File in Java through Random Access
as for the other part of your question:
Beside this file changes during the time so the last line doesn't
remain last forever.During the reading another java program write on
it.My question is: the program will see in the same time another java
program write on the file, the changes?
I would propose a SSCCE in which you show what you are trying to accomplish and the problem
EDIT:
As Jon Skeets comment suggests, I found a link to a similar question answered by him: Quickly read the last line of a text file?
EDIT 2:
I think I got your second question, I'm not sure it's possible, as a single file cant be accessed by 2 different streams at the same time, one will just throw an error when trying to open the file. Ypu can however monitor if changes occur after the file has been read using Java.NIO Directory Watcher, Unless I misunderstood you.
I have a set of text files providing informations that are parsed, analysed and allow building a model. Sometime, the user of this model wants to know which part of a text file was used to generate a given model item.
For that I am thinking of keeping track of the range of lines (or bytes) ids to be able to read the appropriate text part once required.
My question is: I wonder if it their exists any java Reader able to read a file by using a start and stop line (or byte) id instead of reading the file from the begining and counting the lines (bytes)?
Best regards
If you know exactly amount of bytes, that should be skipped, you can use seek method method of RandomAccessFile
To read from the certain byte - SeekableByteChannel. Of cause, there aren't any Readers able to start from the line id - because positions of line separators are unknown.
You can use InputStream.mark() and InputStream.skip() to navigate to concrete position into the file.
But are you sure you really have to implement this yourself? Take a look on Lucine - the indexing service that probably will help you.
I'm writing arbitrary byte arrays (mock virus signatures of 32 bytes) into arbitrary files, and I need code to overwrite a specific file given an offset into the file. My specific question is: is there source code/libraries that I can use to perform this particular task?
I've had this problem with Python file manipulation as well. I'm looking for a set of functions that can kill a line, cut/copy/paste, etc. My assumptions are that these are extremely common tasks, and I couldn't find it in the Java API nor my google searches.
Sorry for not RTFM well; I haven't come across any information, and I've been looking for a while now.
Maybe you are looking for something like the RandomAccessFile class in the standard Java JDK. It supports reads and writes at some offset, as well as byte arrays.
Java's RandomAccessFile is exactly what you want.
It includes methods like seek(long) that allow you to move wherever you need in the file. It also allows for reading and writing at the same time.
As far as I know, Java has primarily lower level functions for manipulating files directly. Here is the best I've come up with
The actions you describe are standard in the Swing world, and for text comes down to manipulating a Document object. These act on data in memory. The class java.nio.channels.FileChannel has similar methods that act directly on a file. Neither fine the end of lines automatically, but other classes in java.io and java.nio do.
Apache Commons has a sandbox library called Flatfile which looks like it does what you want. The problem is that no code has been released yet. You may, however, want to talk to people working on it to get some more ideas. I didn't do a general check on libraries.
Have you looked into File/FileReader/FileWriter/BufferedReader? You can get the contents of the files and manipulate it as you like, you can search the data in the files, you can overwrite files, create new, append to an existing....
I am not sure this is exactly what you are asking for but I use these APIs all the time for logging, RTF editors, text file creation for email, and many other things.
As far as cut/copy/past goes, I have not come across the ability to do that directly, however, you can output the contents of the file and "copy" what part of it you want and "paste" it into a new file, or append it to an existing.
While writing a byte array to a file is a common task, writing to a give file 32-bytes byte array just once is just not something you are going to find in java.io :)
To get started, would the below method and comments look reasonable to you? I bet someone here, maybe even myself, could whip it out quick like.
public static void writeFauxVirusSignature(File file, byte[] bytes, long offset) {
//open file
//move to offset
//write bytes
//close file
}
Questions:
How big could the potential target files be?
Do you need performance?
I ask because clean, easy to read code would use Apache Commons lib's, but large file writes in a performance sensitive environment will necessitate using java.nio libraries
Has anybody written any classes for reading and writing Palm Database (PDB) files in Java? (I mean on a server, not on the Palm device itself.) I tried to google, but all I got were Protein Data Bank references.
I wrote a Perl program that does it using Palm::PDB.pm, but I want to turn it into a servlet for a GWT app.
The jSyncManager project at http://www.jsyncmanager.org/ is under the LGPL and includes classes to read and write PDB files -- look in jSyncManager/API/Protocol/Util/DLPDatabase.java in its source code. It looks like the core code you need from this could be isolated from the rest of the library with a little effort.
There are a few ways that you can go about this;
Easiest but slowest: Find a perl-> java bridge. This will not be quick, but it will work and it should involve the least amount of work.
Find a C++/C# implementation that you have the source to and convert it (this should be the fastest solution)
Find a Java reader ... there seems to be a few listed under google... however I do not have any experience with these.
Depending on what your intended usage is, you might look into writing a simple reader yourself. The format is pretty simple and you only need to handle a couple of simple fields to parse it.
Basically there is a header for the entire file which has a 2 byte integer at the end which specifies the number of record. So just skip your way through the bytes for all the other fields in the header and then read the last field which is the number of records in the file. Be aware that the PDB format writes integers with most significant byte first.
Following this, there will be a record header for each record, the first field of which is the actual offset into the file for the record itself. Again, be aware of the byte order.
So, now you have the offsets into the file for each record in the file, which should make it very easy to read the actual records as long as you know the format of these for the type of PDB file you are trying to read.
Wikipedia has a nice overview of the header formats.
Maybe JPilot can help? They must have a lot of Java code dealing with Palm OS data.