We Keep Coding

Writing Strings to a binary file java

I have a list of objects that has some simple String properties. I want to be able to save those strings to binary so that when you open the file outside the program, you only see 1's and 0's.
I have managed to use FileOutputStreamand saved the strings, however, I can't manage to get it to write to binary. The file reads as clean readable text. I have tried getBytes().
What would be the best approach for this? Keep in mind that I want to be able to read the file later and construct back the objects. Would it be better to use Serializable and save a list of objects?
Here is my FileWriter:
NB: The toString() is custom and returns a String with linebreaks for every property.
public class FileWriter {
public void write(String fileName, Savable objectToSave ) throws IOException {
File fileToSave = new File(fileName);
String stringToSave = objectToSave.toString();
byte[] bytesToSave = stringToSave.getBytes(StandardCharsets.UTF_8) ;
try (
OutputStream outputStream = new FileOutputStream(fileToSave);
) {
outputStream.write(bytesToSave);
} catch (IOException e) {
throw new IOException("error");
}
}
}

If your goal is simply serializing, implementing Serializable and writing them would work, but your string is still going to be readable. You can encrypt the stream, but anyone decompiling your code can still devise a way to read the values.

SequenceFile Compactor of several small files in only one file.seq

Novell in HDFS and Hadoop:
I am developing a program which one should get all the files of a specific directory, where we can find several small files of any type.
Get everyfile and make append in a SequenceFile compressed, where the key must be the path of the file, and the value must be the file got, For now my code is:
import java.net.*;
import org.apache.hadoop.fs.*;
import org.apache.hadoop.conf.*;
import org.apache.hadoop.io.*;
import org.apache.hadoop.io.compress.BZip2Codec;
public class Compact {
public static void main (String [] args) throws Exception{
try{
Configuration conf = new Configuration();
FileSystem fs =
FileSystem.get(new URI("hdfs://quickstart.cloudera:8020"),conf);
Path destino = new Path("/user/cloudera/data/testPractice.seq");//test args[1]
if ((fs.exists(destino))){
System.out.println("exist : " + destino);
return;
}
BZip2Codec codec=new BZip2Codec();
SequenceFile.Writer outSeq = SequenceFile.createWriter(conf
,SequenceFile.Writer.file(fs.makeQualified(destino))
,SequenceFile.Writer.compression(SequenceFile.CompressionType.BLOCK,codec)
,SequenceFile.Writer.keyClass(Text.class)
,SequenceFile.Writer.valueClass(FSDataInputStream.class));
FileStatus[] status = fs.globStatus(new Path("/user/cloudera/data/*.txt"));//args[0]
for (int i=0;i<status.length;i++){
FSDataInputStream in = fs.open(status[i].getPath());
outSeq.append(new org.apache.hadoop.io.Text(status[i].getPath().toString()), new FSDataInputStream(in));
fs.close();
}
outSeq.close();
System.out.println("End Program");
}catch(Exception e){
System.out.println(e.toString());
System.out.println("File not found");
}
}
}
But after of every execution I receive this exception:
java.io.IOException: Could not find a serializer for the Value class: 'org.apache.hadoop.fs.FSDataInputStream'. Please ensure that the configuration 'io.serializations' is properly configured, if you're using custom serialization.
File not found
I understand the error must be in the type of the file I am creating and the type of object I define for adding to the sequenceFile, but I don't know which one should add, can anyone help me?

FSDataInputStream, like any other InputStream, is not intended to be serialized. What serializing an "iterator" over a stream of byte should do ?
What you most likely want to do, is to store the content of the file as the value. For example you can switch the value type from FsDataInputStream to BytesWritable and just get all the bytes out of the FSDataInputStream. One drawback of using Key/Value SequenceFile for a such purpose is that the content of each file has to fit in memory. It could be fine for small files but you have to be aware of this issue.
I am not sure what you are really trying to achieve but perhaps you could avoid reinventing the wheel by using something like Hadoop Archives ?

Thanks a lot by your comments, the problem was the serializer like you say, and finally I used BytesWritable:
FileStatus[] status = fs.globStatus(new Path("/user/cloudera/data/*.txt"));//args[0]
for (int i=0;i<status.length;i++){
FSDataInputStream in = fs.open(status[i].getPath());
byte[] content = new byte [(int)fs.getFileStatus(status[i].getPath()).getLen()];
outSeq.append(new org.apache.hadoop.io.Text(status[i].getPath().toString()), new org.apache.hadoop.io.BytesWritable(in));
}
outSeq.close();
Probably there are other better solutions in the hadoop ecosystem but this problem was an exercise of a degree I am developing, and for now We are remaking the wheel for understanding concepts ;-).

saving random numbers in java

I'm doing an animation in Processing. I'm using random points and I need to execute the code twice for stereo vision.
I have lots of random variables in my code, so I should save it somewhere for the second run or re-generate the SAME string of "random" numbers any time I run the program. (as said here: http://www.coderanch.com/t/372076/java/java/save-random-numbers)
Is this approach possible? How? If I save the numbers in a txt file and then read it, will my program run slower? What's the best way to do this?
Thanks.

If you just need to be able to generate the same sequence for a limited time, seeding the random number generator with the same value to generate the same sequence is most likely the easiest and fastest way to go. Just make sure that any parallel threads always request their pseudo random numbers in the same sequence, or you'll be in trouble.
Note though that there afaik is nothing guaranteeing the same sequence if you update your Java VM or even run a patch, so if you want long time storage for your sequence, or want to be able to use it outside of your Java program, you need to save it to a file.

Here is a sample example:
public static void writeRandomDoublesToFile(String filePath, int numbersCount) throws IOException
{
FileOutputStream fos = new FileOutputStream(new File(filePath));
BufferedOutputStream bos = new BufferedOutputStream(fos);
DataOutputStream dos = new DataOutputStream(bos);
dos.writeInt(numbersCount);
for(int i = 0; i < numbersCount; i++) dos.writeDouble(Math.random());
}
public static double[] readRandomDoublesFromFile(String filePath) throws IOException
{
FileInputStream fis = new FileInputStream(new File(filePath));
BufferedInputStream bis = new BufferedInputStream(fis);
DataInputStream dis = new DataInputStream(bis);
int numbersCount = dis.readInt();
double[] result = new double[numbersCount];
for(int i = 0; i < numbersCount; i++) result[i] = dis.readDouble();
return result;
}

Well, there's a couple of ways that you can approach this problem. One of them would be to save the random variables as input into a file and pass that file name as a parameter to your program.
And you could do that in one of two ways, the first of which would be to use the args[] parameter:
import java.io.*;
import java.util.*;
public class bla {
public static void main(String[] args) {
// You'd need to put some verification code here to make
// sure that input was actually sent to the program.
Scanner in = new Scanner(new File(args[1]));
while(in.hasNextLine()) {
System.out.println(in.nextLine());
}
} }
Another way would be to use Scanner and read from the console input. It's all the same code as above, but instead of Scanner in = new Scanner(new File(args[1])); and all the verification code above that. You'd substitute Scanner in = new Scanner(System.in), but that's just to load the file.
The process of generating those points could be done in the following manner:
import java.util.*;
import java.io.*;
public class generator {
public static void main(String[] args) {
// You'd get some user input (or not) here
// that would ask for the file to save to,
// and that can be done by either using the
// scanner class like the input example above,
// or by using args, but in this case we'll
// just say:
String fileName = "somefile.txt";
FileWriter fstream = new FileWriter(fileName);
BufferedWriter out = new BufferedWriter(fstream);
out.write("Stuff");
out.close();
}
}
Both of those solutions are simple ways to read and write to and from a file in Java. However, if you deploy either of those solutions, you're still left with some kind of parsing of the data.
If it were me, I'd go for object serialization, and store a binary copy of the data structure I've already generated to disk rather than having to parse and reparse that information in an inefficient way. (Using text files, usually, takes up more disk space.)
And here's how you would do that (Here, I'm going to reuse code that has already been written, and comment on it along the way) Source
You declare some wrapper class that holds data (you don't always have to do this, by the way.)
public class Employee implements java.io.Serializable
{
public String name;
public String address;
public int transient SSN;
public int number;
public void mailCheck()
{
System.out.println("Mailing a check to " + name
+ " " + address);
}
}
And then, to serialize:
import java.io.*;
public class SerializeDemo
{
public static void main(String [] args)
{
Employee e = new Employee();
e.name = "Reyan Ali";
e.address = "Phokka Kuan, Ambehta Peer";
e.SSN = 11122333;
e.number = 101;
try
{
FileOutputStream fileOut =
new FileOutputStream("employee.ser");
ObjectOutputStream out =
new ObjectOutputStream(fileOut);
out.writeObject(e);
out.close();
fileOut.close();
}catch(IOException i)
{
i.printStackTrace();
}
}
}
And then, to deserialize:
import java.io.*;
public class DeserializeDemo
{
public static void main(String [] args)
{
Employee e = null;
try
{
FileInputStream fileIn =
new FileInputStream("employee.ser");
ObjectInputStream in = new ObjectInputStream(fileIn);
e = (Employee) in.readObject();
in.close();
fileIn.close();
}catch(IOException i)
{
i.printStackTrace();
return;
}catch(ClassNotFoundException c)
{
System.out.println(.Employee class not found.);
c.printStackTrace();
return;
}
System.out.println("Deserialized Employee...");
System.out.println("Name: " + e.name);
System.out.println("Address: " + e.address);
System.out.println("SSN: " + e.SSN);
System.out.println("Number: " + e.number);
}
}
Another alternative solution to your problem, that does not involve storing data, is to create a lazy generator for whatever function that provides you your random values, and provide the same seed each and every time. That way, you don't have to store any data at all.
However, that still is quite a bit slower (I think) than serializing the object to disk and loading it back up again. (Of course, that's a really subjective statement, but I'm not going to enumerate cases where that is not true). The advantage of doing that is so that it doesn't require any kind of storage at all.
Another way, that you may have not possibly thought of, is to create a wrapper around your generator function that memoizes the output -- meaning that data that has already been generated before will be retrieved from memory and will not have to be generated again if the same inputs are true. You can see some resources on that here: Memoization source
The idea behind memoizing your function calls is that you save time without persisting to disk. This is ideal if the same values are generated over and over and over again. Of course, for a set of random points, this isn't going to work very well if every point is unique, but keep that in the back of your mind.
The really interesting part comes when considering the ways that all the previous strategies I've described in this post can be combined together.
It'd be interesting to setup a Memoizer class, like described in the second page of 2 and then implement java.io.Serialization in that class. After that, you can add methods save(String fileName) and load(String fileName) in the memoizer class that make serialization and deserialization easier, so you can persist the cache used to memoize the function. Very useful.
Anyway, enough is enough. In short, just use the same seed value, and generate the same point pairs on the fly.

Java - Load file, replace string, save

I have a program that loads lines from a user file, then selects the last part of the String (which would be an int)
Here's the style it's saved in:
nameOfValue = 0
nameOfValue2 = 0
and so on. I have selected the value for sure - I debugged it by printing. I just can't seem to save it back in.
if(nameOfValue.equals(type)) {
System.out.println(nameOfValue+" equals "+type);
value.replace(value, Integer.toString(Integer.parseInt(value)+1));
}
How would I resave it? I've tried bufferedwriter but it just erases everything in the file.

My suggestion is, save all the contents of the original file (either in memory or in a temporary file; I'll do it in memory) and then write it again, including the modifications. I believe this would work:
public static void replaceSelected(File file, String type) throws IOException {
// we need to store all the lines
List<String> lines = new ArrayList<String>();
// first, read the file and store the changes
BufferedReader in = new BufferedReader(new FileReader(file));
String line = in.readLine();
while (line != null) {
if (line.startsWith(type)) {
String sValue = line.substring(line.indexOf('=')+1).trim();
int nValue = Integer.parseInt(sValue);
line = type + " = " + (nValue+1);
}
lines.add(line);
line = in.readLine();
}
in.close();
// now, write the file again with the changes
PrintWriter out = new PrintWriter(file);
for (String l : lines)
out.println(l);
out.close();
}
And you'd call the method like this, providing the File you want to modify and the name of the value you want to select:
replaceSelected(new File("test.txt"), "nameOfValue2");

I think most convenient way is:
Read text file line by line using BufferedReader
For each line find the int part using regular expression and replace
it with your new value.
Create a new file with the newly created text lines.
Delete source file and rename your new created file.
Please let me know if you need the Java program implemented above algorithm.

Hard to answer without the complete code...
Is value a string ? If so the replace will create a new string but you are not saving this string anywhere. Remember Strings in Java are immutable.
You say you use a BufferedWriter, did you flush and close it ? This is often a cause of values mysteriously disappearing when they should be there. This exactly why Java has a finally keyword.
Also difficult to answer without more details on your problem, what exactly are you trying to acheive ? There may be simpler ways to do this that are already there.

Writing in the beginning of a text file Java

I need to write something into a text file's beginning. I have a text file with content and i want write something before this content. Say i have;
Good afternoon sir,how are you today?
I'm fine,how are you?
Thanks for asking,I'm great
After modifying,I want it to be like this:
Page 1-Scene 59
25.05.2011
Good afternoon sir,how are you today?
I'm fine,how are you?
Thanks for asking,I'm great
Just made up the content :) How can i modify a text file like this way?

You can't really modify it that way - file systems don't generally let you insert data in arbitrary locations - but you can:
Create a new file
Write the prefix to it
Copy the data from the old file to the new file
Move the old file to a backup location
Move the new file to the old file's location
Optionally delete the old backup file

Just in case it will be useful for someone here is full source code of method to prepend lines to a file using Apache Commons IO library. The code does not read whole file into memory, so will work on files of any size.
public static void prependPrefix(File input, String prefix) throws IOException {
LineIterator li = FileUtils.lineIterator(input);
File tempFile = File.createTempFile("prependPrefix", ".tmp");
BufferedWriter w = new BufferedWriter(new FileWriter(tempFile));
try {
w.write(prefix);
while (li.hasNext()) {
w.write(li.next());
w.write("\n");
}
} finally {
IOUtils.closeQuietly(w);
LineIterator.closeQuietly(li);
}
FileUtils.deleteQuietly(input);
FileUtils.moveFile(tempFile, input);
}

I think what you want is random access. Check out the related java tutorial. However, I don't believe you can just insert data at an arbitrary point in the file; If I recall correctly, you'd only overwrite the data. If you wanted to insert, you'd have to have your code
copy a block,
overwrite with your new stuff,
copy the next block,
overwrite with the previously copied block,
return to 3 until no more blocks

As #atk suggested, java.nio.channels.SeekableByteChannel is a good interface. But it is available from 1.7 only.
Update : If you have no issue using FileUtils then use
String fileString = FileUtils.readFileToString(file);

This isn't a direct answer to the question, but often files are accessed via InputStreams. If this is your use case, then you can chain input streams via SequenceInputStream to achieve the same result. E.g.
InputStream inputStream = new SequenceInputStream(new ByteArrayInputStream("my line\n".getBytes()), new FileInputStream(new File("myfile.txt")));

I will leave it here just in case anyone need
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
try (FileInputStream fileInputStream1 = new FileInputStream(fileName1);
FileInputStream fileInputStream2 = new FileInputStream(fileName2)) {
while (fileInputStream2.available() > 0) {
byteArrayOutputStream.write(fileInputStream2.read());
}
while (fileInputStream1.available() > 0) {
byteArrayOutputStream.write(fileInputStream1.read());
}
}
try (FileOutputStream fileOutputStream = new FileOutputStream(fileName1)) {
byteArrayOutputStream.writeTo(fileOutputStream);
}