We Keep Coding

Fastest way to read a large XML file in Java

I'm working on a java project to optimize existing code. Currently i'm using BufferedReader/FileInputStream to read content of an XML file as String in Java.
But my question is , is there any faster way to read XML content.Are SAX/DOM faster than BufferedReader/FileInputStream?
Need help regarding the above issue.
Thanks in advance.

I think that your code shown in other question is faster than DOM-like parsers which would definitely require more memory and likely some computation in order to reconstruct the document in full. You may want to profile the code though.
I also think that your code can be prettified a bit for streaming processing if you would use javax XMLStreamReader, which I found quite helpful for many tasks. That class is "... is designed to be the lowest level and most efficient way to read XML data", according to Oracle.
Here is the excerpt from my code where I parse StackOverflow users XML file distributed as a public data dump:
// the input file location
private static final String fileLocation = "/media/My Book/Stack/users.xml";
// the target elements
private static final String USERS_ELEMENT = "users";
private static final String ROW_ELEMENT = "row";
// get the XML file handler
//
FileInputStream fileInputStream = new FileInputStream(fileLocation);
XMLStreamReader xmlStreamReader = XMLInputFactory.newInstance().createXMLStreamReader(
fileInputStream);
// reading the data
//
while (xmlStreamReader.hasNext()) {
int eventCode = xmlStreamReader.next();
// this triggers _users records_ logic
//
if ((XMLStreamConstants.START_ELEMENT == eventCode)
&& xmlStreamReader.getLocalName().equalsIgnoreCase(USERS_ELEMENT)) {
// read and parse the user data rows
//
while (xmlStreamReader.hasNext()) {
eventCode = xmlStreamReader.next();
// this breaks _users record_ reading logic
//
if ((XMLStreamConstants.END_ELEMENT == eventCode)
&& xmlStreamReader.getLocalName().equalsIgnoreCase(USERS_ELEMENT)) {
break;
}
else {
if ((XMLStreamConstants.START_ELEMENT == eventCode)
&& xmlStreamReader.getLocalName().equalsIgnoreCase(ROW_ELEMENT)) {
// extract the user data
//
User user = new User();
int attributesCount = xmlStreamReader.getAttributeCount();
for (int i = 0; i < attributesCount; i++) {
user.setAttribute(xmlStreamReader.getAttributeLocalName(i),
xmlStreamReader.getAttributeValue(i));
}
// all other user record-related logic
//
}
}
}
}
}
That users file format is quite simple and similar to your Bank.xml file:
<users>
<row Id="1567200" Reputation="1" CreationDate="2012-07-31T23:57:57.770" DisplayName="XXX" EmailHash="XXX" LastAccessDate="2012-08-01T00:55:12.953" Views="0" UpVotes="0" DownVotes="0" />
...
</users>

There are different parser options available.
Consider using a streaming parser, because the DOM may become quite big. I.e. either a push or a pull parser.
It's not as if XML parsers are necessarily slow. Consider your web browser. It does XML parsing all the time, and tries really hard to be robust to syntax errors. Usually, memory is the bigger issue.

Change id3 tag version programatically (pref java)

I need a way to change id3 tag version of mp3 files to some id3v2.x programatically, preferably using java though anything that works is better than nothing. Bonus points if it converts the existing tag so that already existing data isn't destroyed, rather than creating a new tag entirely.
Edit: Jaudiotagger worked, thanks. Sadly I had to restrict it to mp3 files and only saving data contained in previous tags if they were id3. I decided to convert the tag to ID3v2.3 since windows explorer can't handle v2.4, and it was a bit tricky since the program was a bit confused about whether to use the copy constructor or the conversion constructor.
MP3File mf = null;
try {
mf = (MP3File)AudioFileIO.read(new File(pathToMp3File));
} catch (Exception e) {}
ID3v23Tag tag;
if (mf.hasID3v2Tag()) tag = new ID3v23Tag(mf.getID3v2TagAsv24());
else if (mf.hasID3v1Tag()) tag = new ID3v23Tag(mf.getID3v1Tag());
else tag = new ID3v23Tag();

My application must be able to read id3v1 or id3v11, but shall only write v23, so I needed a little bit longer piece of code:
AudioFile mf;
Tag mTagsInFile;
...
mf = ... // open audio file the usual way
...
mTagsInFile = mf.getTag();
if (mTagsInFile == null)
{
//contrary to getTag(), getTagOrCreateAndSetDefault() ignores id3v1 tags
mTagsInFile = mf.getTagOrCreateAndSetDefault();
}
// mp3 id3v1 and id3v11 are suboptimal, convert to id3v23
if (mf instanceof MP3File)
{
MP3File mf3 = (MP3File) mf;
if (mf3.hasID3v1Tag() && !mf3.hasID3v2Tag())
{
// convert ID3v1 tag to ID3v23
mTagsInFile = new ID3v23Tag(mf3.getID3v1Tag());
mf3.setID3v1Tag(null); // remove v1 tags
mf3.setTag(mTagsInFile); // add v2 tags
}
}
Basically we have to know that getTagOrCreateAndSetDefault() and similar unfortunately ignores id3v1, so we first have to call getTag(), and only if this fails, we call the mentioned function.
Additionally, the code must also deal with flac and mp4, so we make sure to do our conversion only with mp3 files.
Finally there is a bug in JaudioTagger. You may replace this line
String genre = "(" + genreId + ") " + GenreTypes.getInstanceOf().getValueForId(genreId);
in "ID3v24Tag.java" with this one
String genre = GenreTypes.getInstanceOf().getValueForId(genreId);
Otherwise genre 12 from idv1 will get "(12) Other" which later is converted to "Other Other" and this is not what we would expect. Maybe someone has a more elegant solution.

You can use different libraries for this purpose, for example this or this.

Read the five score in a textfile then print them back with the new score added to the list - ANDROID

In my program when the player submits a score it gets added to a local text file called localHighScores. This is list of the top five score the player has achieved while on that specific device.
I wasn't sure how to write to a new line using FileOutputStream (if you know please share), so instead I've inputted a space in between each score. Therefore what I am trying to do is when the player clicks submit the program will open the file and read any current data is saved. It will save it to an String Array, each element being one of the five score in the text file and when it hits a 'space' in the fie it will add the score just read to the write array element
The code I currently have is as follows:
String space = " ";
String currentScoreSaved;
String[] score = new String[5];
int i = 0;
try
{
BufferedReader inputReader = new BufferedReader(new InputStreamReader(openFileInput("localHighScore.txt")));
String inputString;StringBuffer stringBuffer = new StringBuffer();
while ((inputString = inputReader.readLine()) != null && i < 6)
{
if((inputString = inputReader.readLine()) != space)
{
stringBuffer.append(inputString + "\n");
i++;
score[i] = stringBuffer.toString();
}
}
currentScoreSaved = stringBuffer.toString();
FileOutputStream fos = openFileOutput("localHighScore.txt", Context.MODE_PRIVATE);
while (i < 6)
{
i++;
fos.write(score[i].getBytes());
fos.write(space.getBytes());
}
fos.write(localHighScore.getBytes());
//fos.newLine(); //I thought this was how you did a new line but sadly I was mistaken
fos.close();
}
catch(Exception e)
{
e.printStackTrace();
}
Now you will notice this doesn't re arrange the score if a new highscore is achieved. That I am planning on doing next. For the moment I am just trying to get the program to do the main thing which is read in the current data, stick it in an Array then print it back to that file along with the new score
Any Ideas how this might work, as currently it's printing out nothing even when I had score in the textfile before hand

I'm only a first year student in Java programming and I am a new user here at stackoverflow.com, so pardon me if coding for android has some special rules I don't know about, which prevents this simple and humble example from working. But here is how I would read from a file in the simplest of ways.
File tempFile = new File("<SubdirectoryIfAny/name_of_file.txt");
Scanner readFile = new Scanner( tempFile );
// Assuming that you can structure the file as you please with fx each bit of info
// on a new line.
int counter = 0;
while ( readFile.hasNextLine() ) {
score[counter] = readFile.nextLine();
counter++;
}
As for the writing back to the file? Put it in an entirely different method and simply make a simplified toString-like method, that prints out all the values the exact way you want them in the file, then create a "loadToFile" like method and use the to string method to print back into the file with a printstream, something like below.
File tempFile = new File("<SubdirectoryIfAny/name_of_file.txt");
PrintStream write = new PrintStream(tempFile);
// specify code for your particular program so that the toString method gets the
// info from the string array or something like that.
write.print( <objectName/this>.toStringLikeMethod() );
// remember the /n /n in the toStringLikeMethod so it prints properly in the file.
Again if this is something you already know, which is just not possible in this context please ignore me, but if not I hope it was useful. As for the exceptions, you can figure that you yourself. ;)

Since you are a beginner, and I assume you are trying to get things off the ground as quickly as possible, I'd recommend using SharedPreferences. Basically it is just a huge persistent map for you to use! Having said that... you should really learn about all the ways of storage in Android, so check out this document:
http://developer.android.com/guide/topics/data/data-storage.html
The Android docs are awesome! FYI SharedPreferences may not be the best and awesomest way to do this... but I'm all for quick prototyping as a learner. If you want, write a wrapper class around SharedPreferences.

Remember the values entered on standalone app on the client side

We have a standalone java swing app, in which the user can print something that he drew, on a printer by giving its IP.
Now the requirement is that the app needs to remember the ip that was given the last time by this user.
What I could think of till now is (a brute one though) - keep a log file kind of storage on the client machine, and that everytime the app comes up it reads the last submitted one.
Any suggestions would be helpful.
Thanks in advance.

Here's a tutorial on using the Java Preferences API to achieve what you want.
From the article:
The Java Preferences API provides a
systematic way to handle user and
system preference and configuration
data, e.g. to save user settings,
remember the last value of a field
etc.
I would use this approach over writing any data out explicitly to a file because its platform agnostic.

More or Less that's it. Still you can review the source code for HistoryTextField component of jEdit.
http://www.jedit.org/api/org/gjt/sp/jedit/gui/HistoryTextField.html
A Sample from jEdit source:
public boolean save(Map<String, HistoryModel> models)
{
Log.log(Log.MESSAGE,HistoryModel.class,"Saving history");
File file1 = new File(MiscUtilities.constructPath(
jEdit.getSettingsDirectory(), "#history#save#"));
File file2 = new File(MiscUtilities.constructPath(
jEdit.getSettingsDirectory(), "history"));
if(file2.exists() && file2.lastModified() != historyModTime)
{
Log.log(Log.WARNING,HistoryModel.class,file2
+ " changed on disk; will not save history");
return false;
}
jEdit.backupSettingsFile(file2);
String lineSep = System.getProperty("line.separator");
BufferedWriter out = null;
try
{
out = new BufferedWriter(new OutputStreamWriter(
new FileOutputStream(file1), "UTF-8"));
if(models != null)
{
Collection<HistoryModel> values = models.values();
for (HistoryModel model : values)
{
if(model.getSize() == 0)
continue;
out.write('[');
out.write(StandardUtilities.charsToEscapes(
model.getName(),TO_ESCAPE));
out.write(']');
out.write(lineSep);
for(int i = 0; i < model.getSize(); i++)
{
out.write(StandardUtilities.charsToEscapes(
model.getItem(i),
TO_ESCAPE));
out.write(lineSep);
}
}
}
out.close();
/* to avoid data loss, only do this if the above
* completed successfully */
file2.delete();
file1.renameTo(file2);
}
catch(IOException io)
{
Log.log(Log.ERROR,HistoryModel.class,io);
}
finally
{
IOUtilities.closeQuietly(out);
}
historyModTime = file2.lastModified();
return true;
}

Since it is a Swing app., you might launch it using Java Web Start then persist the data using the PersistenceService. Here is a demo. of the PersistenceService.

i dont really recommend this, but you could use the registry also.

jump on specific location in binary file

I have a binary file which contains image.i have to jump on different locations in file to read the image file. So far i am using mark and reset methods but these are not helping me as i want.
please somebody help me about that i,ll be really thankful.and i am using Input Stream to read the file.

You can use the java.io.RandomAccessFile to do this. The methods seek(long) and getFilePointer() will help to jump to different offsets in the file and come back to original offsets:
RandomAccessFile f = new RandomAccessFile("/my/image/file", "rw");
// read some data.
long positionToJump = 10L;
long origPos = f.getFilePointer(); // store the original position
f.seek(positionToJump);
// now you are at position 10, start reading from here.
// go back to original position
f.seek(origPos);

Android seems to have RandomAccessFile, have you tried it?

Since Java 7 you can use java.nio.file.Files and SeekableByteChannel
byte[] getRandomAccessResults(Path filePath, long offset) throws IOException
{
try (SeekableByteChannel byte_channel = java.nio.file.Files.newByteChannel(filePath, StandardOpenOption.READ))
{
ByteBuffer byte_buffer = ByteBuffer.allocate(128);
byte_channel.position(offset);
byte_channel.read(byte_buffer);
return byte_buffer.array();
}
}