Can I write to the end of a 5GB file in Java? - java

Can I write to the end of a 5GB file in Java? This question came up in my office and no one is sure what the answer is.

This should be possible fairly easily using a RandomAccessFile. Something like the following should work:
String filename;
RandomAccessFile myFile = new RandomAccessFile(filename, "rw");
// Set write pointer to the end of the file
myFile.seek(myFile.length());
// Write to end of file here

Yes. Take a look at this link RandomAccessFile
http://java.sun.com/javase/6/docs/api/java/io/RandomAccessFile.html#seek(long)
That is , you open the file, and then set the position to the end of the file. And start writing from there.
Tell us how it went.

Actually that would depend on the underlying File System and how the JVM on that platform implements the File Stream. Because, if a file is bigger than 5GB you cannot, with a 32Bit operative system open the whole file and just write to it, because of the 4.3 Billion limit stuff ( 32^2 ).
So, the answer shortly would be, Yes, it is possible, IF Java handles the file correctly and the File System is a good one :)

If you just mean that you need to append to the file, check out the
FileWriter(File file, boolean append)
constructor in the FileWriter class.
Sorry, I don't have a 5GB file handy to test with. :)

5GB? I wonder if the OS is a bigger problem, but that's doubtful.
In theory, you can just open the file in append mode.
OutputStream in = new java.io.FileOutputStream(fileName, true);
and write till the filesystem fills up.
See Bill the Lizard for char data.

Related

Could be file_path external in java? I mean not storing in pc hard disk, but somewhere else (ex: cloud, ...)

Suppose I have file object:
File file = new File("file_path");
could be file_path external?
For example I want to store file on S3 bucket;
So, I checked:
String file_path = "https://"+bucket+".s3.amazonaws.com/" + fileName + ".png";
But it didn't work?
Any suggestions?
Thanks in advance!
Yes and no.
No, in that java.io.File represents a file on a local disk and cannot represent anything else, period.
Yes, in way one, in that the File API has been replaced with the java.nio.file.Path API which has this pluggable concept called a filesystem; you could make an implementation of FileSystem that represents an S3 bucket. It's non-trivial to do this; search the web for it if you are interested in this.
Yes, in way two, in that the vast majority of java libraries and APIs take an InputStream or an OutputStream and never a File. You can turn a file into an IS or OS, and you can turn a bucket entry (be it for writing or for reading) in such a thing as well. Same goes for network sockets, your process' standard in and standard out, DB blobs, and so much more; the very point of IS and OS is to be an abstraction for 'a source that provides a stream of bytes on demand' and 'a source that you send a stream of bytes to'.

How to create an InputStream of files that have a certain extension in Java?

I have a lot of files in a directory but I only want to read the ones with a certain extension (say .txt). I want these files added to the same BufferedInputStream so that I can read them in one go. When I call read() at the end of a file, the next one should begin.
It really feels like there should be an obvious answer to this but I had no luck finding it.
You might want to take a look at SequenceInputStream:
A SequenceInputStream represents the logical concatenation of other
input streams. It starts out with an ordered collection of input
streams and reads from the first one until end of file is reached,
whereupon it reads from the second one, and so on, until end of file
is reached on the last of the contained input streams.
To me the "obvious answer" is:
Just iterate through all the files in the directory using a proper filter. For each file create a FileInputStream, read it and close it.
I don't think there is an obvious answer to this question.
Probably you need to create a Wrapper InputStream with a list of files you want to read from. Internally you will open/close streams as needed, namely when a file is completely read.
It is not obvious but should not be difficult. This way you can work 'only' with one InputStream for all files.

Why doesn't java.io.File have a close method?

While java.io.RandomAccessFile does have a close() method java.io.File doesn't. Why is that? Is the file closed automatically on finalization or something?
The javadoc of the File class describes the class as:
An abstract representation of file and directory pathnames.
File is only a representation of a pathname, with a few methods concerning the filesystem (like exists()) and directory handling but actual streaming input and output is done elsewhere. Streams can be opened and closed, files cannot.
(My personal opinion is that it's rather unfortunate that Sun then went on to create RandomAccessFile, causing much confusion with its inconsistent naming.)
java.io.File doesn't represent an open file, it represents a path in the filesystem. Therefore having close method on it doesn't make sense.
Actually, this class was misnamed by the library authors, it should be called something like Path.
Essentially random access file wraps input and output streams in order to manage the random access. You don't open and close a file, you open and close streams to a file.
A BufferedReader can be opened and closed but a File is never opened, it just represents a path in the filesystem.
Say suppose, you have
File f = new File("SomeFile");
f.length();
You need not close the Files, because its just the representation of a path.
You should always consider to close only reader/writers and in fact streams.
As already stated, the File class does not have a closing method as it's merely a path or a reference to the actual File.
You will usually use this File class as a helper to open the actual file with a FileReader class which you can close. That said, it does close itself on exit but if you read a file from your program and then try to do something to this file externally, it could result in an error on that external call, so it's better to close it
File path = new File(/some/path/file.txt);
FileReader actualFile = new FileReader(path);
...<
if(imDoneWithTheFile)
actualFile.close();

In java, how do i edit 1 line of a text file?

Ok so I know the value of the line, I dont have the line number, how would I edit only 1 line?
Its a config file, i.e
x=y
I want a command to edit x=y to x=y,z.
or even x=z.
In Java you can use `Properties class:
app.config file:
x=y
java:
public void writeConfig() throws Exception {
Properties tempProp = new Properties();
tempProp.load(new FileInputStream("app.config"));
tempProp.setProperty("x", "y,z");
tempProp.store(new FileOutputStream("app.config"), null);
}
If you are using that configuration format, you might want to use
java.util.Properties
component to read/write on that file.
But if you just want to edit it by hand, you can just read the file line by line and match the variable you want to change.
One way to do it is to:
Read the file into memory; e.g. as an array of Strings representing the lines of the file.
Locate the String/line you want to change.
Use a regex (or whatever) to modify the String/line
Write a new version of the file from the in memory version.
There are many variations on this. You also need to take care when you write the new version of the file to guard against losing everything if something goes wrong during the write. (Typically you write the new version to a temporary file, rename the old version out of the way (e.g. as a backup) and rename the new version in place of the old one.)
Unfortunately, there is no way to add or remove characters in the middle of a regular text file without rewriting a large part of the file. This "problem" is not specific to Java. It is fundamental to the way that text files are modelled / represented on most mainstream operating systems.
Unless the new line has the exact same length as the old one, your best bet is to
Open a temporary output file
Read the config file, line by line
Search for your key
If you can't find it, just write the line you just read to the output file
If you can find it, write the new value to the temporary file instead
Until you hit EOF
Delete old file
Rename new file to the old file
IF your config file is small, you can also do the whole parsing/modification step in memory and then write the final result back to the config file, that way you skip the temporary file (although a temporary file is a good way to prevent corruption if something breaks while you write the file).
If this is not what you're looking for, you should edit your question to be a lot more clear. I'm just guessing what you're asking for.
If your data is all key and value pairs, for example ...
key1=value1
key2=value2
... then load them into a Properties object. Off the top of my head, you'll need a FileInputStream to load the properties, modify with myProperties.put(key, value) and then save the properties with the use of a FileOutputStream.
Hope this helps!
rh

Shift the file while writing?

Is it possible to shift the contents of a file while writing to it using FileWriter?
I need to write data constants to the head of the file and if I do that it overwrites the file.
What technique should I use to do this or should I make make copies of the file (with the new data on top) on every file write?
If you want to overwrite certain bytes of the file and not others, you can use seek and write to do so. If you want to change the content of every byte in the file (by, for example, adding a single byte to the beginning of the file) then you need to write a new file and potentially rename it after you've done writing it.
Think of the answer to the question "what will be the contents of the byte at offset x after I'm done?". If, for a large percent of the possible values of x the answer is "not what it used to be" then you need a new file.
Rather than contending ourselves with the question "what will be the contents of the byte at offset x after I'm done?", lets change the mindset and ask why can't the file system or perhaps the hard disk firmware do : a) provide another mode of accessing the file [let's say, inline] b) increase the length of the file by the number of bytes added at the front or in the middle or even at the end c) move each byte that starts from the crossection by the newcontent.length positions
It would be easier and faster to handle these operations at the disk firmware or file system implementation level rather than leaving that job to the application developer. I hope file system writers or hard disk vendors would offer such feature soon.
Regards,
Samba

Categories