I am trying to read an old .dat file byte by byte, and have run into an issue: a record is terminated by \n (newline). I'd like to read in the whole byte array, then split it on the character.
I can do this by reading the whole byte array from the file, creating a String with the contents of the byte array, then calling String.split(), but find this to be inefficient. I'd rather split the byte array directly if possible.
Can anyone assist?
Update: Code was requested.
public class NgcReader {
public static void main(String[] args) {
String location;
if (System.getProperty("os.name").contains("Windows")) {
location = "F:\\Programming\\Projects\\readngc\\src\\main\\java\\ngcreader\\catalog.dat";
} else {
location = "/media/My Passport/Programming/Projects/readngc/src/main/java/ngcreader/catalog.dat";
}
File file = new File(location);
InputStream is = null;
try {
is = new FileInputStream(file);
} catch (FileNotFoundException e) {
System.out.println("It didn't work!");
System.exit(0);
}
byte[] fileByteArray = new byte[(int) file.length() - 1];
try {
is.read(fileByteArray);
is.close();
} catch (IOException e) {
System.out.println("IOException!");
System.exit(0);
}
// I do NOT like this. I'd rather split the byte array on the \n character
String bigString = new String(fileByteArray);
List<String> stringList = Arrays.asList(bigString.split("\\n"));
for (String record : stringList) {
System.out.print("Catalog number: " + record.substring(1, 6));
System.out.print(" Catalog type: " + record.substring(7, 9));
System.out.print(" Right Ascension: " + record.substring(10, 12) + "h " + record.substring(13, 17) + "min");
System.out.print(" Declination: " + record.substring(18, 21) + " " + record.substring(22, 24));
if (record.length() > 50) {
System.out.print(" Magnitude: " + record.substring(47, 51));
}
if (record.length() > 93) {
System.out.print(" Original Notes: " + record.substring(54,93));
}
if (record.length() > 150) {
System.out.print(" Palomar Notes: " + record.substring(95,150));
}
if (record.length() > 151) {
System.out.print(" Notes: " + record.substring(152));
}
System.out.println();
}
}
Another Update: Here's a README with a description of the file I'm processing:
http://cdsarc.u-strasbg.fr/viz-bin/Cat?VII/1B
It sounds like this might actually just be a text file to start with, in which case:
InputStream stream = new FileInputStream(location);
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(stream,
"ASCII"));
String line;
while ((line = reader.readLine()) != null) {
// Handle the line, ideally in a separate method
}
} finally {
stream.close();
}
This way you never need to have more than a single line of the file in memory at a time.
if you're set on using byte arrays...
byte[] buff = new byte[1024];//smaller buffer
try {
int ind=0,from=0,read;
while((read=is.read(buff,ind,buff.length-ind))!=-1){
for(int i=ind;i<ind+read;i++){
if(buff[i]=='\n'){
string record = new String(buff,from,i+1);
//handle
from=i+1;
}
}
System.arraycopy(buff,from,buff,0,buff.length-from);
ind=ind+read-from;
from=0;
}
} catch (IOException e) {
System.out.println("IOException!");
//System.exit(0);
throw RunTimeException(e);//cleaner way to die
} finally{
is.close();
}
this also avoids loading in the entire file and it puts the close inside a finally
Related
i have a really suspicious case here, envolving a simple method which is supposed to write into a .txt file.
public void extractCoNLL(int n, String outputFile) throws IOException {
String msg;
PrintWriter pr = new PrintWriter(outputFile);
FileInputStream fConlliN = new FileInputStream(this.txt_CoNLL_in);
BufferedReader readBufferData = new BufferedReader(new InputStreamReader(fConlliN));
try {
while ((msg = readBufferData.readLine()) != null) {
String aMsg[] = msg.split("\\s+");
if (!msg.startsWith("#")) {
//pr.println(msg);
if (aMsg.length >= n) {
pr.print(aMsg[n] + "_"); // DOES NOT WORK
pr.println(aMsg[n] + "_"); // WORKS ?????
System.out.println(aMsg[4] + aMsg.length);
} else {
pr.println();
}
}
}
this.txt_CoNLL = out_Extracted_txt_CoNLL;
} catch (Exception e) {
System.err.println("Error Exception: " + e.getMessage());
}
}
Also, why is it not possible for me to add a simple " " -space but i have to be forced to use "_" to seperate the words.
Very grateful for your Help.
Thank you in advance!
This program is meant to see two files located in a particular folder and then merge those two files and create a third file which is does. From the third merged file it is then searching for a keyword such as "test", once it finds that key word it prints out the location and the line of the keyword which is what is somewhat doing. What is happening is when I run the program it stops after the finds the keyword the first time in a line but it will not continue to search that line. So if there is multiple keyword 'test' in the line it will only find the first one and spit back the position and line. I want it to print both or multiple keywords. I think it is because of the IndexOf logic which is causing the issue.
import com.sun.deploy.util.StringUtils;
import java.io.*;
import java.lang.*;
import java.util.Scanner;
public class Concatenate {
public static void main(String[] args) {
String sourceFile1Path = "C:/Users/me/Desktop/test1.txt";
String sourceFile2Path = "C:/Users/me/Desktop/test2.txt";
String mergedFilePath = "C:/Users/me/Desktop/merged.txt";
File[] files = new File[2];
files[0] = new File(sourceFile1Path);
files[1] = new File(sourceFile2Path);
File mergedFile = new File(mergedFilePath);
mergeFiles(files, mergedFile);
stringSearch(args);
}
private static void mergeFiles(File[] files, File mergedFile) {
FileWriter fstream = null;
BufferedWriter out = null;
try {
fstream = new FileWriter(mergedFile, true);
out = new BufferedWriter(fstream);
} catch (IOException e1) {
e1.printStackTrace();
}
for (File f : files) {
System.out.println("merging: " + f.getName());
FileInputStream fis;
try {
fis = new FileInputStream(f);
BufferedReader in = new BufferedReader(new InputStreamReader(fis));
String aLine;
while ((aLine = in.readLine()) != null) {
out.write(aLine);
out.newLine();
}
in.close();
} catch (IOException e) {
e.printStackTrace();
}
}
try {
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}
private static void stringSearch(String args[]) {
try {
String stringSearch = "test";
BufferedReader bf = new BufferedReader(new FileReader("C:/Users/me/Desktop/merged.txt"));
int linecount = 0;
String line;
System.out.println("Searching for " + stringSearch + " in file");
while (( line = bf.readLine()) != null){
linecount++;
int indexfound = line.indexOf(stringSearch);
if (indexfound > -1) {
System.out.println(stringSearch + " was found at position " + indexfound + " on line " + linecount);
System.out.println(line);
}
}
bf.close();
}
catch (IOException e) {
System.out.println("IO Error Occurred: " + e.toString());
}
}
}
It's because you are searching for the word once per line in your while loop. Each iteration of the loop takes you to the next line of the file because you are calling bf.readLine(). Try something like the following. You may have to tweak it but this should get you close.
while (( line = bf.readLine()) != null){
linecount++;
int indexfound = line.indexOf(stringSearch);
while(indexfound > -1)
{
System.out.println(stringSearch + " was found at position " + indexfound + " on line " + linecount);
System.out.println(line);
indexfound = line.indexOf(stringSearch, indexfound);
}
}
I am trying to design two different methods for a Java application. The first method will pass in a string of the name of a file, and return the text of a text file as a string. The second method will pass in the name of a file and the text, and create a new text file and output the string into the file.
Currently my code works without the methods, but I am trying to design it with a separation of concerns and low coupling. I am trying to modify it so I can just call a method to output any sort of data I have in a string to a text file.
Here is my code without the methods:
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.*;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.Scanner;
public class FileCopier {
public static void main(String[] args) {
//What file should be input for reading?
String inputFile = askForInput("Please enter the name of the file to be read in: ");
//What file should be created to display output ?
String outputFile = askForInput("Please come up with a name of the file to be written backwards: ");
//Check to make sure we got the names
System.out.println("inputFile: " + inputFile + " outputFile: " + outputFile);
// Variables to read and write the files
//Call the readTextFile method to read text file into string data
String line = null;
String total = null;
BufferedReader input = null;
try {
// FileReader reads text files in the default encoding.
FileReader fileReader = new FileReader(inputFile);
// Always wrap FileReader in BufferedReader.
input = new BufferedReader(fileReader);
total = input.readLine() + "\n";
while ((line = input.readLine()) != null && total != null) {
total += line + "\n";
System.out.println("Proof that the file says: " + line);
}
input.close();
//Check to make sure we got the text files data
System.out.println("The total string says: \n" + total);
//Call the reverseWords method to switch 'Hello' with 'World'
String info = reverseWords(total);
//Check to make sure the string was reversed
System.out.println("The reversed string says: \n" + info);
File file = new File(outputFile);
BufferedWriter output = null;
output = new BufferedWriter(new FileWriter(file));
output.write(info);
System.out.println("The output file: " + outputFile + " has been written.");
output.close();
} catch (FileNotFoundException ex) {
System.out.println("Unable to open file '" +
inputFile + "'");
} catch (IOException ex) {
System.out.println("Error reading file '" + inputFile + "'");
// Or we could just do this:
// ex.printStackTrace();
}
}
public static String reverseWords(String sentence) {
String[] parts = sentence.trim().split("\\s+");
StringBuilder builder = new StringBuilder();
builder.append(parts[parts.length - 1]);
for (int i = parts.length - 2; i >= 0; --i) {
builder.append(" ").append(parts[i]);
}
return builder.toString();
}
public static String askForInput(String question) {
System.out.println(question);
Scanner in = new Scanner(System.in);
String inputFile = in.nextLine();
return inputFile;
}
}
When creating a method for each of the "read" and "write" portions of my code, I constantly get errors that I assume are from the exception handling. Any thoughts on how to separate code that has exceptions involved?
Think in terms of single responsibility. You have two distinct operations that need to happen: reading and writing.
Let's start with reading. What you're doing right now to read the file surmises these lines:
// FileReader reads text files in the default encoding.
FileReader fileReader = new FileReader(inputFile);
// Always wrap FileReader in BufferedReader.
input = new BufferedReader(fileReader);
total = input.readLine() + "\n";
while ((line = input.readLine()) != null && total != null) {
total += line + "\n";
System.out.println("Proof that the file says: " + line);
}
input.close();
Move that to a method.
private static String readFile(String inputFile) throws IOException {
BufferedReader input;
String total;
String line;// FileReader reads text files in the default encoding.
FileReader fileReader = new FileReader(inputFile);
// Always wrap FileReader in BufferedReader.
input = new BufferedReader(fileReader);
total = input.readLine() + "\n";
while ((line = input.readLine()) != null) {
total += line + "\n";
System.out.println("Proof that the file says: " + line);
}
input.close();
return total;
}
Here's what we did:
We have a variable total which is used elsewhere in the program, so that usage has to be preserved. We're returning String and will declare total = readFile(inputFile); on the outside.
We've changed nothing. This code will run the same way as it did without the method.
Now, if we want to move the writing functionality, which is:
File file = new File(outputFile);
BufferedWriter output = null;
output = new BufferedWriter(new FileWriter(file));
output.write(info);
System.out.println("The output file: " + outputFile + " has been written.");
output.close();
...we just do.
private static void writeFile(String outputFile, String info) throws IOException {
File file = new File(outputFile);
BufferedWriter output = null;
output = new BufferedWriter(new FileWriter(file));
output.write(info);
System.out.println("The output file: " + outputFile + " has been written.");
output.close();
}
Again, nothing's changed on this method. We don't have any other usages of any of the variables in here to worry about, so we can directly bring it across.
All said, that try block looks a bit anemic:
try {
total = readFile(inputFile);
//Check to make sure we got the text files data
System.out.println("The total string says: \n" + total);
//Call the reverseWords method to switch 'Hello' with 'World'
String info = reverseWords(total);
//Check to make sure the string was reversed
System.out.println("The reversed string says: \n" + info);
writeFile(outputFile, info);
} catch (FileNotFoundException ex) {
System.out.println("Unable to open file '" +
inputFile + "'");
} catch (IOException ex) {
System.out.println("Error reading file '" + inputFile + "'");
// Or we could just do this:
// ex.printStackTrace();
}
...which is a good thing.
I am not sure what are you asking about but try to create your own Exceptions and make your methods throw them like this
package com.qmic.test;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.*;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.Scanner;
public class FileCopier {
public static void main(String[] args) {
// What file should be input for reading?
String inputFile = askForInput("Please enter the name of the file to be read in: ");
// What file should be created to display output ?
String outputFile = askForInput("Please come up with a name of the file to be written backwards: ");
// Check to make sure we got the names
System.out.println("inputFile: " + inputFile + " outputFile: "
+ outputFile);
// Variables to read and write the files
// Call the readTextFile method to read text file into string data
String line = null;
String total = null;
BufferedReader input = null;
try {
String readData = readFileContents(inputFile);
// Check to make sure we got the text files data
System.out.println("The total string says: \n" + readData);
// Call the reverseWords method to switch 'Hello' with 'World'
String reversedContents = reverseWords(readData);
writeToFile(outputFile, reversedContents);
} catch (ReadException ex) {
System.out.println("Error reading file '" + inputFile + "'");
// Or we could just do this:
// ex.printStackTrace();
} catch (WriteException ex) {
System.out.println("Error Writing file '" + outputFile + "'");
// Or we could just do this:
// ex.printStackTrace();
}
}
public static String reverseWords(String sentence) {
String[] parts = sentence.trim().split("\\s+");
StringBuilder builder = new StringBuilder();
builder.append(parts[parts.length - 1]);
for (int i = parts.length - 2; i >= 0; --i) {
builder.append(" ").append(parts[i]);
}
return builder.toString();
}
public static String askForInput(String question) {
System.out.println(question);
Scanner in = new Scanner(System.in);
String inputFile = in.nextLine();
return inputFile;
}
public static void writeToFile(String fileName, String data)
throws WriteException {
BufferedWriter output = null;
try {
// Check to make sure the string was reversed
System.out.println("The reversed string says: \n" + data);
File file = new File(fileName);
output = new BufferedWriter(new FileWriter(file));
output.write(data);
System.out.println("The output file: " + fileName
+ " has been written.");
}catch(IOException e){
throw new WriteException();
}finally{
try {
output.close();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
public static String readFileContents(String fileName) throws ReadException {
// FileReader reads text files in the default encoding.
BufferedReader input = null;
String line = null;
String total = null;
try {
FileReader fileReader = new FileReader(fileName);
// Always wrap FileReader in BufferedReader.
input = new BufferedReader(fileReader);
total = input.readLine() + "\n";
while ((line = input.readLine()) != null && total != null) {
total += line + "\n";
System.out.println("Proof that the file says: " + line);
}
} catch (IOException e) {
throw new ReadException();
}finally{
//This is ugly code, if you are using java 7 you have extra option to better this
try {
input.close();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
return total;
}
}
//make me public and move me to a separate file
class WriteException extends IOException {
}
//make me public and move me to a separate file
class ReadException extends IOException {
}
So I've been working on a program that will display the line number and the line itself of the searched text string. If I search dog, and I have lines in my text file that contain the word dog, those lines and line numbers should be shown. I also have created a method that counts the characters, words, and lines of a text file. However, the problem I am having is that whenever I run my program I don't get the line numbers with the lines of the searched text. I successfully get the text from the text file in the console and I successfully get the number of lines, words, etc.
Here's my written code, I am guessing it has to do something with the fact that I don't have a "return results;" statement, but I am not sure where to put it, and if I add it to the end of "+ characters + " characters. "" line by doing "+ results", it just gives me empty brackets.
Maybe I am doing something wrong? Perhaps something to do with closing the file and stream, not sure. Please help, I've tried moving stuff around but no luck.
public String words() {
try {
int words = 0;
int numbers = 0;
int lines = 1;
int characters = 0;
int total = 0;
String c = " ";
FileReader r = new FileReader(file1);
LineNumberReader lnr = new LineNumberReader(r);
StreamTokenizer t = new StreamTokenizer(r);
ArrayList<String> results = new ArrayList<String>();
t.resetSyntax();
t.wordChars('0', '9');
t.wordChars('A', 'Z');
t.wordChars('a', 'z');
t.whitespaceChars(0, ' ');
t.eolIsSignificant(true);
while (t.nextToken() != StreamTokenizer.TT_EOF) {
switch (t.ttype) {
case StreamTokenizer.TT_NUMBER:
numbers++;
break;
case StreamTokenizer.TT_WORD:
characters += t.sval.length();
words++;
break;
case StreamTokenizer.TT_EOL:
lines++;
break;
case StreamTokenizer.TT_EOF:
break;
default:
}
}
FileInputStream fstream = new FileInputStream(file1);
BufferedReader br = new BufferedReader(new InputStreamReader(fstream));
String strLine;
while ((strLine = br.readLine()) != null) {
System.out.println(strLine);
}
br.close();
String ask = "Enter Word";
String find = JOptionPane.showInputDialog(ask);
String word = find;
String line = null;
while ((line = lnr.readLine()) != null) {
if (line.indexOf(word) >= 0) {
results.add(lnr.getLineNumber() + line);
}
}
r.close();
total = numbers + words;
lnr.close();
return file1.getName() + " has " + lines + " lines, "
+ total + " words, "
+ characters + " characters. ";
} catch (IOException e) {
display(e.toString(), "Error");
}
return " ";
}
Here's the main class if needed:
import java.io.*;
import java.util.ArrayList;
import javax.swing.*;
public class BasicFile {
File file1;
JFileChooser selection;
File file2 = new File(".", "Backup File");
public BasicFile() {
selection = new JFileChooser(".");
}
public void selectFile() {
int status = selection.showOpenDialog(null);
try {
if (status != JFileChooser.APPROVE_OPTION) {
throw new IOException();
}
file1 = selection.getSelectedFile();
if (!file1.exists()) {
throw new FileNotFoundException();
}
} catch (FileNotFoundException e) {
JOptionPane.showMessageDialog(null, "File Not Found ", "Error", JOptionPane.INFORMATION_MESSAGE);
} catch (IOException e) {
System.exit(0);
}
}
public void backupFile() throws FileNotFoundException {
DataInputStream in = null;
DataOutputStream out = null;
try {
in = new DataInputStream(new FileInputStream(file1));
out = new DataOutputStream(new FileOutputStream(file2));
try {
while (true) {
byte data = in.readByte();
out.writeByte(data);
}
} catch (EOFException e) {
JOptionPane.showMessageDialog(null, "File has been backed up!",
"Backup Complete!", JOptionPane.INFORMATION_MESSAGE);
} catch (IOException e) {
JOptionPane.showMessageDialog(null, "File Not Found ",
"Error", JOptionPane.INFORMATION_MESSAGE);
}
} finally {
try {
in.close();
out.close();
} catch (Exception e) {
display(e.toString(), "Error");
}
}
}
boolean exists() {
return file1.exists();
}
public String toString() {
return file1.getName() + "\n" + file1.getAbsolutePath() + "\n" + file1.length() + " bytes";
}
public String words() {
try {
int words = 0;
int numbers = 0;
int lines = 1;
int characters = 0;
int total = 0;
String c = " ";
FileReader r = new FileReader(file1);
LineNumberReader lnr = new LineNumberReader(r);
StreamTokenizer t = new StreamTokenizer(r);
ArrayList<String> results = new ArrayList<String>();
t.resetSyntax();
t.wordChars('0', '9');
t.wordChars('A', 'Z');
t.wordChars('a', 'z');
t.whitespaceChars(0, ' ');
t.eolIsSignificant(true);
while (t.nextToken() != StreamTokenizer.TT_EOF) {
switch (t.ttype) {
case StreamTokenizer.TT_NUMBER:
numbers++;
break;
case StreamTokenizer.TT_WORD:
characters += t.sval.length();
words++;
break;
case StreamTokenizer.TT_EOL:
lines++;
break;
case StreamTokenizer.TT_EOF:
break;
default:
}
}
FileInputStream fstream = new FileInputStream(file1);
BufferedReader br = new BufferedReader(new InputStreamReader(fstream));
String strLine;
while ((strLine = br.readLine()) != null) {
System.out.println(strLine);
}
br.close();
String ask = "Enter Word";
String find = JOptionPane.showInputDialog(ask);
String word = find;
String line = null;
while ((line = lnr.readLine()) != null) {
if (line.indexOf(word) >= 0) {
results.add(lnr.getLineNumber() + line);
}
}
r.close();
total = numbers + words;
lnr.close();
return file1.getName() + " has " + lines + " lines, "
+ total + " words, "
+ characters + " characters. ";
} catch (IOException e) {
display(e.toString(), "Error");
}
return " ";
}
void display(String msg, String s) {
JOptionPane.showMessageDialog(null, msg, s, JOptionPane.ERROR_MESSAGE);
}
}
You are nearly there.
reinitialize your FileReader and LineNumberReader before your while ((line = lnr.readLine()) != null) loop.
Then your ArrayList will be full of the #String that I think you desire.
what you can do is start counting the lines in the file (starting by 0), then increase by 1 every time a new line is found.. then check if the string you want to find is contained in the line, then print the number of the line where the keyword is found (using the contains() function in Java). I assumed you want to check for both upper and lower case, if you don't want that then simply remove the toLowerCase() ! So, read the file properly in Java:
long lineNumber = 0;
BufferedReader myReader = new BufferedReader(new FileReader("test.txt"));
String line = myReader.readLine();
while(line != null){
lineNumber++;
System.out.println("The line I am now examining is : " + line + " and the line number is : " + lineNumber);
if line.toLowerCase().contains(word.toLowerCase()) {
System.out.println("Line number: " + lineNumber + " contains keyword : " + word);
line = myReader.readLine();
}
Trying to find a word in a large file. File is read line by line. When reading the way redLine exception is thrown. Are there any way around this? You can read it on the floor as a string?
for(String line; (line = fileOut.readLine()) != null; ){
if(line.contains(commandString))
System.out.println(count + ": " + line);
count++;
}
java.lang.OutOfMemoryError:
UDP:
this is all my bad code:
static String file = "files.txt";
static String commandString = "first";
static int count = 1;
public static void main(String[] args) throws IOException
{
try(BufferedReader fileOut = new BufferedReader(new InputStreamReader(new FileInputStream(file), "Cp1251")) ){
for(String line; (line = fileOut.readLine()) != null; ){
if(line.contains(commandString))
System.out.println(count + ": " + line);
count++;
}
System.out.println("before wr close :" + Runtime.getRuntime().freeMemory());
fileOut.close();
}catch(Exception e) {
System.out.println(e);
}
}
Searching for a word, you can read the file bytewise without holding more than a single byte of the file in memory.
Read byte by byte and every time, a byte is equal to the first byte of the searched word, start a second loop and read the following bytes and check if the next byte is equal to the next byte in the word and so on.
To give you an example, I have modified an sample to your needs.
I've omitted on the output of the file, because I don't know, if you want to output all lines or only those which contains your keyword and the latter might be as problematic as reading the code line by line.
static String fileName = "files.txt";
static byte[] searchString = { 'f', 'i', 'r', 's', 't' };
static int count = 0;
static long position = 1;
public static void main(String[] args) throws IOException {
try (FileInputStream file = new FileInputStream(fileName)) {
byte read[] = new byte[1];
outerLoop: while (-1 < file.read(read, 0, 1)) {
position++;
if (read[0] == searchString[0]) {
int matches = 1;
for (int i = 1; i < searchString.length; i++) {
if (-1 > file.read(read, 0, 1)) {
break outerLoop;
}
position++;
if (read[0] == searchString[i]) {
matches++;
} else {
break;
}
}
if (matches == searchString.length) {
System.out.println((++count)+". found at position "+ (position-matches));
}
}
}
file.close();
} catch (Exception e) {
e.printStackTrace();
}
}