Main:
package main;
import racreader.RAFReader;
public class RandomAccessFile {
public static void main(String[] args) {
if (args.length != 2) {
System.err.println("Wrong arguments length");
System.exit(1);
}
try {
RAFReader reader = new RAFReader (args[0]);
try {
String output = reader.readUTF(Integer.parseInt(args[1]));
System.out.print(output);
} catch (Exception e) {
System.err.println(e.toString());
} finally {
reader.close();
}
} catch (Exception e) {
System.err.println(e.toString());
}
}
}
RAFReader:
package racreader;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.RandomAccessFile;
public class RAFReader {
private final String fileName;
private final RandomAccessFile reader;
public RAFReader(String fileName) throws FileNotFoundException {
this.fileName = fileName;
this.reader = openFile();
}
private RandomAccessFile openFile() throws FileNotFoundException {
RandomAccessFile reader = new RandomAccessFile(fileName, "r");
return reader;
}
public String readUTF(int offset) throws IOException {
reader.seek(offset);
String output = reader.readUTF();
return output;
}
public void close() throws IOException {
reader.close();
}
}
The problem is in EOFException in every file (even encoded in UTF8) and every offset. Why?
UPD: I try to get my program working with file with this content:
Это тест UTF-8 чтения
It works fine only if offset = 0. Any other offset throws EOFException.
The readUTF()/writeUTF() methods from RandomAccesFile use conventions for writing Java String objects, which are not necessarily honored by UTF encoded text files. readUTF() was not meant to be used for reading arbitrary text file, which was not originally written by using RandomAccesFile.writeUTF().
As method Javadocs specify, readUTF() assumes that the first two bytes it reads contain the number of bytes in the following string. This is the case if the string was written to file by the pairing writeUTF() method, but in case of the text file this will throw intermittent EOFException, since the first two bytes will contain actual characters from the string.
In your case, a different set of classes can solve the problem. Consider rewriting RAFReader class using InputStreamReader:
public String readUTF(int offset) throws IOException {
FileInputStream is = new FileInputStream(fileName);
Reader fileReader = new InputStreamReader(is, "UTF-8");
StringBuilder stringBuilder = new StringBuilder();
fileReader.skip(offset);
int charsRead;
char buf[] = new char[256];
//Read until there is no more characters to read.
while ((charsRead = fileReader.read(buf)) > 0) {
stringBuilder.append(buf, 0, charsRead);
}
fileReader.close();
return stringBuilder.toString();
}
If using RandomAccesFile is a must, you can use input stream which wraps the RandomAccesFile. The simplest way to do it is through FileChannel, encapsulated by RandomAccesFile:
InputStream is = Channels.newInputStream(reader.getChannel());
EOFException - if this file reaches the end before reading all the
bytes.
One possible way in which you got EOFException is at line
reader.seek(offset);
String output = reader.readUTF();
Maybe the offset value is high for file length. Try with offset = 0 and check if you get EOF or not.
I have an valid XML file(valid cause browser can parse it) that I try to parse using JDOM2. The code was running good for other xml files but for this particular xml file it gives me the following exception on builder.build() line : "com.sun.org.apache.xerces.internal.impl.io.MalformedByteSequenceException: Invalid byte 3 of 3-byte UTF-8 sequence. "
My code is as follows
import java.io.*;
import java.util.*;
import java.net.*;
import org.jdom2.*;
import org.jdom2.input.*;
import org.jdom2.output.*;
import org.jdom2.adapters.*;
public class Test
{
public static void main(String st[])
{
String results="N.A.";
SAXBuilder builder = new SAXBuilder();
Document doc;
results = scrapeSite().trim();
try
{
doc = builder.build(new ByteArrayInputStream(results.getBytes()));
}
catch(JDOMException e)
{
System.out.println(e.toString());
}
catch(IOException e)
{
System.out.println(e.toString());
}
}
public static String scrapeSite()
{
String temp="";
try
{
URL url = new URL("http://msu-footprints.org/2011/Aditya/search_5.xml");
URLConnection conn = url.openConnection();
conn.setAllowUserInteraction(false);
InputStream urlStream = url.openStream();
BufferedReader br = new BufferedReader(new InputStreamReader(urlStream));
String t = br.readLine();
while(t!=null)
{
temp = temp + t;
t = br.readLine();
}
}
catch(IOException e)
{
System.out.println(e.toString());
}
return temp;
}
}
why are you reading the xml into a String with a Reader? you are corrupting the xml before you parse it. treat xml as bytes, not chars.
and why are you reading the whole URL InputStream just to convert it into another ByteArrayInputStream? you can reduce that to about 2 lines of code by passing the URL InputStream directly to the builder. (not mention avoid additional memory issues caused by reading the entire stream into memory).
As jtahlborn points out, you should always treat XML as bytes, letting the parser work out the encoding.
But more than that, you should never ever use String.getBytes() to get the bytes of a string: you will not be getting what you think you are.
In this case you can just get the bytes of the site, but even if you were constructing XML in a string and then handing that to a parser as a byte sequence (or, more likely, writing the bytes to a file), you would want to specify the encoding such that it matches the encoding the XML says it's in, which by default is UTF-8:
byte[] bytes = myString.getBytes("UTF-8");
Likewise, if for some reason you needed to use a Writer or Reader, you must specify the encoding to write or read in.
If you need to construct XML, a good way is to use the XMLStreamWriter class:
ByteArrayOutputStream outStream = new ByteArrayOutputStream();
XMLStreamWriter writer =
XMLOutputFactory.newInstance().createXMLStreamWriter(outStream);
I'm trying to save a couple of Strings but I'm currently using ObjectInputStream, which apparently doesn't save my data in a permanent manner. Here is the code which I commented in my project. It saves the string in a temporal manner. Anytime I exit my program, puff the data is gone:
ObjectInputStream FileIn= new ObjectInputStream(new FileInputStream("cars.txt"));
//AND HERE IS THE CODE FOR THE RandomAccessFile VERSION:
RandomAccessFile FileIn = new RandomAccessFile("cars.txt", "rw");
au=(Cars)FileIn.readObject(); //THIS readObject(), is giving me errors
//Cars is a Class
Is there any other alternative that I can use to read RandomAccessFile... Please help and thanks.
Just like FileInputStream, you need to wrap a RandomAccessFile in an ObjectInputStream. i.e. RandomAccessFile doesn't buy you anything.
final RandomAccessFile raf = new RandomAccessFile("file.dat", "r");
ObjectInputStream ois = new ObjectInputStream(new InputStream() {
#Override
public int read(byte[] b, int off, int len) throws IOException {
return raf.read(b, off, len);
}
#Override
public int read() throws IOException {
return raf.read();
}
});
For simple String objects is far easier using plain DataInputStream / DataOutputStream:
package test;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.util.ArrayList;
import java.util.List;
public class TestFile {
static final String FILE = "/tmp/cars.txt";
public static void main(String[] args) {
try {
List<String> strs = new ArrayList<String>();
strs.add("Audi");
strs.add("Seat");
strs.add("Renault");
saveStrings(strs);
strs = loadStrings();
System.out.println("Read strings: " + strs);
} catch (Exception e) {
}
}
static List<String> loadStrings() throws Exception {
DataInputStream dis = null;
List<String> list = new ArrayList<String>();
try {
dis = new DataInputStream(new FileInputStream(FILE));
while (dis.available() > 0) {
list.add(dis.readUTF());
}
} finally {
if (dis != null)
dis.close();
}
return list;
}
static void saveStrings(List<String> list) throws Exception {
DataOutputStream dos = null;
try {
dos = new DataOutputStream(new FileOutputStream(FILE));
for (String str : list) {
dos.writeUTF(str);
}
} finally {
if (dos != null)
dos.close();
}
}
}
If you are asking whether you can use RandomAccessFile to seek around inside an object stream and read objects then the short answer is "no". Serialized object streams are heavily encoded with backwards pointers to previously used objects including previously dumped class definitions, etc..
Serialization stream specification with some format details.
Here's some more details on the serialization format.
Discussion about random writing from Java docs.
We had a similar requirement and wrote some code which closes and re-opens the serialized stream once and a while and recorded the positions of these break points. This didn't give us the ability to read a particular object but it did give us the ability to append to serialized stream and to skip over a particular portion of the file -- skip to the next break.
Well you have to invoke, writeObject() instead of readObject() which is actually to read from disk to memory, and of course when the program ends, so does the memory used by that program.
I have a java ee application where I use a servlet to print a log file created with log4j. When reading log files you are usually looking for the last log line and therefore the servlet would be much more useful if it printed the log file in reverse order. My actual code is:
response.setContentType("text");
PrintWriter out = response.getWriter();
try {
FileReader logReader = new FileReader("logfile.log");
try {
BufferedReader buffer = new BufferedReader(logReader);
for (String line = buffer.readLine(); line != null; line = buffer.readLine()) {
out.println(line);
}
} finally {
logReader.close();
}
} finally {
out.close();
}
The implementations I've found in the internet involve using a StringBuffer and loading all the file before printing, isn't there a code light way of seeking to the end of the file and reading the content till the start of the file?
[EDIT]
By request, I am prepending this answer with the sentiment of a later comment: If you need this behavior frequently, a "more appropriate" solution is probably to move your logs from text files to database tables with DBAppender (part of log4j 2). Then you could simply query for latest entries.
[/EDIT]
I would probably approach this slightly differently than the answers listed.
(1) Create a subclass of Writer that writes the encoded bytes of each character in reverse order:
public class ReverseOutputStreamWriter extends Writer {
private OutputStream out;
private Charset encoding;
public ReverseOutputStreamWriter(OutputStream out, Charset encoding) {
this.out = out;
this.encoding = encoding;
}
public void write(int ch) throws IOException {
byte[] buffer = this.encoding.encode(String.valueOf(ch)).array();
// write the bytes in reverse order to this.out
}
// other overloaded methods
}
(2) Create a subclass of log4j WriterAppender whose createWriter method would be overridden to create an instance of ReverseOutputStreamWriter.
(3) Create a subclass of log4j Layout whose format method returns the log string in reverse character order:
public class ReversePatternLayout extends PatternLayout {
// constructors
public String format(LoggingEvent event) {
return new StringBuilder(super.format(event)).reverse().toString();
}
}
(4) Modify my logging configuration file to send log messages to both the "normal" log file and a "reverse" log file. The "reverse" log file would contain the same log messages as the "normal" log file, but each message would be written backwards. (Note that the encoding of the "reverse" log file would not necessarily conform to UTF-8, or even any character encoding.)
(5) Create a subclass of InputStream that wraps an instance of RandomAccessFile in order to read the bytes of a file in reverse order:
public class ReverseFileInputStream extends InputStream {
private RandomAccessFile in;
private byte[] buffer;
// The index of the next byte to read.
private int bufferIndex;
public ReverseFileInputStream(File file) {
this.in = new RandomAccessFile(File, "r");
this.buffer = new byte[4096];
this.bufferIndex = this.buffer.length;
this.in.seek(file.length());
}
public void populateBuffer() throws IOException {
// record the old position
// seek to a new, previous position
// read from the new position to the old position into the buffer
// reverse the buffer
}
public int read() throws IOException {
if (this.bufferIndex == this.buffer.length) {
populateBuffer();
if (this.bufferIndex == this.buffer.length) {
return -1;
}
}
return this.buffer[this.bufferIndex++];
}
// other overridden methods
}
Now if I want to read the entries of the "normal" log file in reverse order, I just need to create an instance of ReverseFileInputStream, giving it the "revere" log file.
This is a old question. I also wanted to do the same thing and after some searching found there is a class in apache commons-io to achieve this:
org.apache.commons.io.input.ReversedLinesFileReader
I think a good choice for this would be using RandomFileAccess class. There is some sample code for back-reading using this class on this page. Reading bytes this way is easy, however reading strings might be a bit more challenging.
If you are in a hurry and want the simplest solution without worrying too much about performance, I would give a try to use an external process to do the dirty job (given that you are running your app in a Un*x server, as any decent person would do XD)
new BufferedReader(new InputStreamReader(Runtime.getRuntime().exec("tail yourlogfile.txt -n 50 | rev").getProcess().getInputStream()))
A simpler alternative, because you say that you're creating a servlet to do this, is to use a LinkedList to hold the last N lines (where N might be a servlet parameter). When the list size exceeds N, you call removeFirst().
From a user experience perspective, this is probably the best solution. As you note, the most recent lines are the most important. Not being overwhelmed with information is also very important.
Good question. I'm not aware of any common implementations of this. It's not trivial to do properly either, so be careful what you choose. It should deal with character set encoding and detection of different line break methods. Here's the implementation I have so far that works with ASCII and UTF-8 encoded files, including a test case for UTF-8. It does not work with UTF-16LE or UTF-16BE encoded files.
import java.io.BufferedReader;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.RandomAccessFile;
import java.io.Reader;
import java.io.UnsupportedEncodingException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import junit.framework.TestCase;
public class ReverseLineReader {
private static final int BUFFER_SIZE = 8192;
private final FileChannel channel;
private final String encoding;
private long filePos;
private ByteBuffer buf;
private int bufPos;
private byte lastLineBreak = '\n';
private ByteArrayOutputStream baos = new ByteArrayOutputStream();
public ReverseLineReader(File file, String encoding) throws IOException {
RandomAccessFile raf = new RandomAccessFile(file, "r");
channel = raf.getChannel();
filePos = raf.length();
this.encoding = encoding;
}
public String readLine() throws IOException {
while (true) {
if (bufPos < 0) {
if (filePos == 0) {
if (baos == null) {
return null;
}
String line = bufToString();
baos = null;
return line;
}
long start = Math.max(filePos - BUFFER_SIZE, 0);
long end = filePos;
long len = end - start;
buf = channel.map(FileChannel.MapMode.READ_ONLY, start, len);
bufPos = (int) len;
filePos = start;
}
while (bufPos-- > 0) {
byte c = buf.get(bufPos);
if (c == '\r' || c == '\n') {
if (c != lastLineBreak) {
lastLineBreak = c;
continue;
}
lastLineBreak = c;
return bufToString();
}
baos.write(c);
}
}
}
private String bufToString() throws UnsupportedEncodingException {
if (baos.size() == 0) {
return "";
}
byte[] bytes = baos.toByteArray();
for (int i = 0; i < bytes.length / 2; i++) {
byte t = bytes[i];
bytes[i] = bytes[bytes.length - i - 1];
bytes[bytes.length - i - 1] = t;
}
baos.reset();
return new String(bytes, encoding);
}
public static void main(String[] args) throws IOException {
File file = new File("my.log");
ReverseLineReader reader = new ReverseLineReader(file, "UTF-8");
String line;
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
}
public static class ReverseLineReaderTest extends TestCase {
public void test() throws IOException {
File file = new File("utf8test.log");
String encoding = "UTF-8";
FileInputStream fileIn = new FileInputStream(file);
Reader fileReader = new InputStreamReader(fileIn, encoding);
BufferedReader bufReader = new BufferedReader(fileReader);
List<String> lines = new ArrayList<String>();
String line;
while ((line = bufReader.readLine()) != null) {
lines.add(line);
}
Collections.reverse(lines);
ReverseLineReader reader = new ReverseLineReader(file, encoding);
int pos = 0;
while ((line = reader.readLine()) != null) {
assertEquals(lines.get(pos++), line);
}
assertEquals(lines.size(), pos);
}
}
}
you can use RandomAccessFile implements this function,such as:
import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import com.google.common.io.LineProcessor;
public class FileUtils {
/**
* 反向读取文本文件（UTF8）,文本文件分行是通过\r\n
*
* #param <T>
* #param file
* #param step 反向寻找的步长
* #param lineprocessor
* #throws IOException
*/
public static <T> T backWardsRead(File file, int step,
LineProcessor<T> lineprocessor) throws IOException {
RandomAccessFile rf = new RandomAccessFile(file, "r");
long fileLen = rf.length();
long pos = fileLen - step;
// 寻找倒序的第一行:\r
while (true) {
if (pos < 0) {
// 处理第一行
rf.seek(0);
lineprocessor.processLine(rf.readLine());
return lineprocessor.getResult();
}
rf.seek(pos);
char c = (char) rf.readByte();
while (c != '\r') {
c = (char) rf.readByte();
}
rf.readByte();//read '\n'
pos = rf.getFilePointer();
if (!lineprocessor.processLine(rf.readLine())) {
return lineprocessor.getResult();
}
pos -= step;
}
}
use:
FileUtils.backWardsRead(new File("H:/usersfavs.csv"), 40,
new LineProcessor<Void>() {
//TODO implements method
.......
});
The simplest solution is to read through the file in forward order, using an ArrayList<Long> to hold the byte offset of each log record. You'll need to use something like Jakarta Commons CountingInputStream to retrieve the position of each record, and will need to carefully organize your buffers to ensure that it returns the proper values:
FileInputStream fis = // .. logfile
BufferedInputStream bis = new BufferedInputStream(fis);
CountingInputStream cis = new CountingInputSteam(bis);
InputStreamReader isr = new InputStreamReader(cis, "UTF-8");
And you probably won't be able to use a BufferedReader, because it will attempt to read-ahead and throw off the count (but reading a character at a time won't be a performance problem, because you're buffering lower in the stack).
To write the file, you iterate the list backwards and use a RandomAccessFile. There is a bit of a trick: to properly decode the bytes (assuming a multi-byte encoding), you will need to read the bytes corresponding to an entry, and then apply a decoding to it. The list, however, will give you the start and end position of the bytes.
One big benefit to this approach, versus simply printing the lines in reverse order, is that you won't damage multi-line log messages (such as exceptions).
import java.io.File;
import java.io.IOException;
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
/**
* Inside of C:\\temp\\vaquar.txt we have following content
* vaquar khan is working into Citi He is good good programmer programmer trust me
* #author vaquar.khan#gmail.com
*
*/
public class ReadFileAndDisplayResultsinReverse {
public static void main(String[] args) {
try {
// read data from file
Object[] wordList = ReadFile();
System.out.println("File data=" + wordList);
//
Set<String> uniquWordList = null;
for (Object text : wordList) {
System.out.println((String) text);
List<String> tokens = Arrays.asList(text.toString().split("\\s+"));
System.out.println("tokens" + tokens);
uniquWordList = new HashSet<String>(tokens);
// If multiple line then code into same loop
}
System.out.println("uniquWordList" + uniquWordList);
Comparator<String> wordComp= new Comparator<String>() {
#Override
public int compare(String o1, String o2) {
if(o1==null && o2 ==null) return 0;
if(o1==null ) return o2.length()-0;
if(o2 ==null) return o1.length()-0;
//
return o2.length()-o1.length();
}
};
List<String> fs=new ArrayList<String>(uniquWordList);
Collections.sort(fs,wordComp);
System.out.println("uniquWordList" + fs);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
static Object[] ReadFile() throws IOException {
List<String> list = Files.readAllLines(new File("C:\\temp\\vaquar.txt").toPath(), Charset.defaultCharset());
return list.toArray();
}
}
Output:
[Vaquar khan is working into Citi He is good good programmer programmer trust me
tokens[vaquar, khan, is, working, into, Citi, He, is, good, good, programmer, programmer, trust, me]
uniquWordList[trust, vaquar, programmer, is, good, into, khan, me, working, Citi, He]
uniquWordList[programmer, working, vaquar, trust, good, into, khan, Citi, is, me, He]
If you want to Sort A to Z then write one more comparater
Concise solution using Java 7 Autoclosables and Java 8 Streams :
try (Stream<String> logStream = Files.lines(Paths.get("C:\\logfile.log"))) {
logStream
.sorted(Comparator.reverseOrder())
.limit(10) // last 10 lines
.forEach(System.out::println);
}
Big drawback: only works when lines are strictly in natural order, like log files prefixed with timestamps but without exceptions
Suppose I have an InputStream that contains text data, and I want to convert this to a String (for example, so I can write the contents of the stream to a log file).
What is the easiest way to take the InputStream and convert it to a String?
public String convertStreamToString(InputStream is) {
// ???
}
If you want to do it simply and reliably, I suggest using the Apache Jakarta Commons IO library IOUtils.toString(java.io.InputStream, java.lang.String) method.
This is my version,
public static String readString(InputStream inputStream) throws IOException {
ByteArrayOutputStream into = new ByteArrayOutputStream();
byte[] buf = new byte[4096];
for (int n; 0 < (n = inputStream.read(buf));) {
into.write(buf, 0, n);
}
into.close();
return new String(into.toByteArray(), "UTF-8"); // Or whatever encoding
}
String text = new Scanner(inputStream).useDelimiter("\\A").next();
The only tricky is to remember the regex \A, which matches the beginning of input. This effectively tells Scanner to tokenize the entire stream, from beginning to (illogical) next beginning...
- from the Oracle Blog
Since Java 9 InputStream.readAllBytes() even shorter:
String toString(InputStream inputStream) throws IOException {
return new String(inputStream.readAllBytes(), StandardCharsets.UTF_8); // Or whatever encoding
}
Note: InputStream is not closed in this example.
You can use a BufferedReader to read the stream into a StringBuilder in a loop, and then get the full contents from the StringBuilder:
public String convertStreamToString(InputStream is) {
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
Full disclosure: This is a solution I found on KodeJava.org. I am posting it here for comments and critique.
A nice way to do this is using Apache commons IOUtils
IOUtils.toString(inputStream, string);