java hadoop: FileReader VS InputStreamReader - java

i want use my java class on the hadoop hdfs, now i must rewrite my functions.
the problem is, if i use the InputStreamReader my app read wrong values.
here my code (so it's work, i want use the uncommented code part):
public static GeoTimeDataCenter[] readCentersArrayFromFile(int iteration) {
Properties pro = new Properties();
try {
pro.load(GeoTimeDataHelper.class.getResourceAsStream("/config.properties"));
} catch (Exception e) {
e.printStackTrace();
}
int k = Integer.parseInt(pro.getProperty("k"));
GeoTimeDataCenter[] Centers = new GeoTimeDataCenter[k];
BufferedReader br;
try {
//Path pt=new Path(pro.getProperty("seed.file")+(iteration-1));
//FileSystem fs = FileSystem.get(new Configuration());
//br=new BufferedReader(new InputStreamReader(fs.open(pt)));
br = new BufferedReader(new FileReader(pro.getProperty("seed.file")+(iteration-1)));
for(int i =0; i<Centers.length; i++){
String[] temp = null;
try{
temp = br.readLine().toString().split("\t");
Centers[i] = new GeoTimeDataCenter(Integer.parseInt(temp[0]),new LatLong(Double.parseDouble(temp[1]),Double.parseDouble(temp[2])),Long.parseLong(temp[3]));
}
catch(Exception e) {
temp = Seeding.randomSingleSeed().split("\t");
Centers[i] = new GeoTimeDataCenter(i,new LatLong(Double.parseDouble(temp[0]),Double.parseDouble(temp[1])),DateToLong(temp[2]));
}
}
br.close();
} catch (IOException e) {
e.printStackTrace();
}
return Centers;
}
maybe someone know this problem?
best regards

i have found the problem. i have get a checksum exception. now i delete all .crc files from my input file. in this way i get no checksum exception and the buffered reader work fine (uncommented code part, upstairs).

Related

Reading from a text file in java is returning some garbage value

I'm performing certain commands through command prompt and storing the values in a text file.
wmic logicaldisk where drivetype=3 get deviceid > drive.txt
Now I want to read the string stored in the text file from my java file. When I try to do this:
try {
File file = new File("drive.txt");
FileReader reader = new FileReader(file);
BufferedReader in = new BufferedReader(reader);
int i=0;
while ((string[i] = in.readLine()) != null) {
System.out.println(string[i]);
++i;
}
in.close();
} catch (IOException e) {
e.printStackTrace();
}
I get the output as follows:
ÿþD[]E[]V[]I[]C[]E[]
how to avoid this?
while ((string[i] = in.readLine()) != null) {
System.out.println(string[2]);
}
over there you are missing the i++;
However I would advise you to use this structure: Use a ArrayList instead of an array, since this allows you to have a self-resizing structure, also instead in the while use the method ready(); from the BufferedRead in order to check the end from the document, at the end the for it's just to display the elements in String ArrayList.
ArrayList<String> string = new ArrayList<String>();
try {
File file = new File("drive.txt");
BufferedReader entrada;
entrada = new BufferedReader(new FileReader(file));
entrada.readLine();
while (entrada.ready()) {
string.add(entrada.readLine());
}
entrada.close();
} catch (IOException e) {
e.printStackTrace();
}
for (String elements : string) {
System.out.println(elements);
}
Why do you need a string array here? The size of the array may be wrong? Simply use a string instead of array. I tried this and works fine for me:
try {
String string;
File file = new File("drive.txt");
FileReader reader = new FileReader(file);
BufferedReader in = new BufferedReader(reader);
int i = 0;
while ((string = in.readLine()) != null) {
System.out.println(string);
++i;
}
in.close();
} catch (IOException e) {
e.printStackTrace();
}
If you are using eclipse IDE, change the encoding type. Go to Edit->Set Encoding-> Others->UTF-8.

Reading multiple text file in Java

I have few text files. Each text file contains some path and/or the reference of some other file.
File1
#file#>D:/FilePath/File2.txt
Mod1>/home/admin1/mod1
Mod2>/home/admin1/mod2
File2
Mod3>/home/admin1/mod3
Mod4>/home/admin1/mod4
All I want is, copy all the paths Mod1, Mod2, Mod3, Mod4 in another text file by supplying only File1.txt as input to my java program.
What I have done till now?
public void readTextFile(String fileName){
try {
br = new BufferedReader(new FileReader(new File(fileName)));
String line = br.readLine();
while(line!=null){
if(line.startsWith("#file#>")){
String string[] = line.split(">");
readTextFile(string[1]);
}
else if(line.contains(">")){
String string[] = line.split(">");
svnLinks.put(string[0], string[1]);
}
line=br.readLine();
}
} catch (Exception e) {
e.printStackTrace();
}
}
Currently my code reads the contents of File2.txt only, control does not come back to File1.txt.
Please ask if more inputs are required.
First of all you are jumping to another file without closing the current reader and when you come back you lose the cursor. Read one file first and then write all its contents that match to another file. Close the current reader (Don't close the writer) and then open the next file to read and so on.
Seems pretty simple. You need to write your file once your svnLinks Map is populated, assuming your present code works (haven't seen anything too weird in it).
So, once the Map is populated, you could use something along the lines of:
File newFile = new File("myPath/myNewFile.txt");
// TODO check file can be written
// TODO check file exists or create
FileOutputStream fos = null;
OutputStreamWriter osw = null;
BufferedWriter bw = null;
try {
fos = new FileOutputStream(newFile);
osw = new OutputStreamWriter(fos);
bw = new BufferedWriter(osw);
for (String key: svnLinks.keySet()) {
bw.write(key.concat(" my separator ").concat(svnLinks.get(key)).concat("myNewLine"));
}
}
catch (Throwable t) {
// TODO handle more gracefully
t.printStackTrace();
if (bw != null) {
try {
bw.close();
}
catch (Throwable t) {
t.printStackTrace();
}
}
Here is an non-recursive implementation of your method :
public static void readTextFile(String fileName) throws IOException {
LinkedList<String> list = new LinkedList<String>();
list.add(fileName);
while (!list.isEmpty()) {
BufferedReader br = null;
try {
br = new BufferedReader(new FileReader(new File(list.pop())));
String line;
while ((line = br.readLine()) != null) {
if (line.startsWith("#file#>")) {
String string[] = line.split(">");
list.add(string[1]);
} else if (line.contains(">")) {
String string[] = line.split(">");
svnLinks.put(string[0], string[1]);
}
}
} catch (Exception e) {
e.printStackTrace();
} finally {
br.close();
}
}
}
Just used a LinkedList to maintain the order. I suggest you to add some counter if you to limit the reading of files to a certain number(depth). eg:
while (!list.isEmpty() && readCount < 10 )
This will eliminate the chance of running the code to infinity(in case of circular reference).

How to know the excel file version in java?

I want to read excel files in java. I have some excel files with old format (excel 95) and others in new format (excel 2007). I am currently using poi but it is not able to read the excel files with older format. So what I need is a function that passes the filename which will return a boolean with value true if the format is old format (BIFF5) and false if the format is new (BIFF8). The need of this function is to allow me to use jxl in for older format and poi for newer format.
Here is the code I have:
try
{
// create a new org.apache.poi.poifs.filesystem.Filesystem
POIFSFileSystem poifs = new POIFSFileSystem(fin);
w = new HSSFWorkbook(poifs);
}
catch (IOException e)
{
w = null;
throw e;
}
catch (OutOfMemoryError e) // java.lang.OutOfMemoryError:
{
w = null;
throw e;
}
catch (OldExcelFormatException e) // OldExcelFormatException
{
w = null;
System.out.println("OldExcelFormatException");
translateBIFF5();
}
private void translateBIFF5() throws IOException, CmpException
{
ArrayList<String> row = null;
try
{
jxl_w = Workbook.getWorkbook(excelFile);
}
catch (BiffException e)
{
jxl_w = null;
e.printStackTrace();
}
catch (IOException e)
{
jxl_w = null;
throw e;
}
catch (OutOfMemoryError e) // java.lang.OutOfMemoryError:
{
jxl_w = null;
throw e;
}
if (jxl_w != null)
{
try
{
for (currentSheet = 0; currentSheet < jxl_w.getNumberOfSheets(); currentSheet++)
{
jxl_sheet = jxl_w.getSheet(currentSheet);
. . . . .
I'd recommend trying Andy Khan's JExcel instead of POI. I don't think POI is particularly well designed or documented. I've had great luck with JExcel. Try it.
One way is to call the Windows ASSOC and FTYPE commands, capture the output and parse it to determine the Office version installed.
C:\Users\me>assoc .xls
.xls=Excel.Sheet.8
C:\Users\me>ftype Excel.sheet.8
Excel.sheet.8="C:\Program Files (x86)\Microsoft Office\Office12\EXCEL.EXE" /e
Here a quick example :
import java.io.*;
public class ShowOfficeInstalled {
public static void main(String argv[]) {
try {
Process p = Runtime.getRuntime().exec
(new String [] { "cmd.exe", "/c", "assoc", ".xls"});
BufferedReader input =
new BufferedReader
(new InputStreamReader(p.getInputStream()));
String extensionType = input.readLine();
input.close();
// extract type
if (extensionType == null) {
System.out.println("no office installed ?");
System.exit(1);
}
String fileType[] = extensionType.split("=");
p = Runtime.getRuntime().exec
(new String [] { "cmd.exe", "/c", "ftype", fileType[1]});
input =
new BufferedReader
(new InputStreamReader(p.getInputStream()));
String fileAssociation = input.readLine();
// extract path
String officePath = fileAssociation.split("=")[1];
System.out.println(officePath);
}
catch (Exception err) {
err.printStackTrace();
}
}
}
or
You can search in the registry for the key:
HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\App Paths
This will probably require some work, as evidenced by this question:
read/write to Windows Registry using Java

Manipulate big Textfiles in Java

I was wondering how do you manipulate big Textfiles in Java, if we assume that the Filesize is larger than the memory. I googled that topic and it shows that most people recommend java.niofor such a task.
Unfortunately I haven't found any documentation on how to manipulate the File. For example read every Line, modify it, write it. I tried something like this, but this doesn't work:
FileChannel fileChannel = null;
try {
fileChannel = new RandomAccessFile(file, "rw").getChannel();
ByteBuffer buffer = ByteBuffer.allocate(256);
while (fileChannel.read(buffer) != -1) {
buffer.rewind();
buffer.flip();
String nextLine = buffer.asCharBuffer().toString();
if (replaceBackSlashes) {
nextLine = nextLine.replace("\\\\", "/");
}
if (!(removeEmptyLines && StringUtils.isEmpty(nextLine))) {
buffer.flip();
buffer.asCharBuffer().put(nextLine);
}
buffer.clear();
}
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
if (fileChannel != null) {
try {
fileChannel.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
So what are your recommendations? Also the String nextline, doesn't match anything in my File. Maybe I need to set the encoding?
Line by line. Something like this ...
public static void main(String[] args) throws Exception {
File someFile = new File("someFile.txt");
File temp = File.createTempFile(someFile.getName(), null);
BufferedReader reader = null;
PrintStream writer = null;
try {
reader = new BufferedReader(new FileReader(someFile));
writer = new PrintStream(temp);
String line;
while ((line = reader.readLine())!=null) {
// manipulate line
writer.println(line);
}
}
finally {
if (writer!=null) writer.close();
if (reader!=null) reader.close();
}
if (!someFile.delete()) throw new Exception("Failed to remove " + someFile.getName());
if (!temp.renameTo(someFile)) throw new Exception("Failed to replace " + someFile.getName());
}
Kudos to xagyg for a nice, clean answer! The following just didn't fit into a comment:
If you're running Java 7 already, you can save a lot of boilerplate code by using try-with-resources for the processing loop:
File source = ...
File target = ...
try (BufferedReader in = new BufferedReader(new FileReader(source));
PrintStream out = new PrintStream(target)) {
String line;
while ((line = in.readLine()) != null) {
// manipulate line
out.println(line);
}
}
// no catch or finally clause!
No more of that initalize-to-null-try-catch-finally-close-if-not-null mess, Java will take care of that for you now. Less code, less potential to forget or screw up that crucial call to close().

how do i use java classes in web context?

I have a text file and a class (Reader) that reads the text file and stores each line in a String [].
String name;
String [] lines;
Reader(String name){
this.name = name;
}
public String toString(){
return this.name;
}
public readFile(String filename){
String line = "";
int i = 0;
try{
BufferedReader reader = new BufferedReader(new FileReader(filename));
while(line = reader.readLine()) != null){
lines[i] = line;
i++;
}// while
reader.close();
}
catch(etc...){}
}
I wish to print each array element in table on my jsp page.
Reader r = new Reader("test");
out.print(r.toString());
works and prints 'test' but...
r.readFile("test.txt")
for (int i=0; i < r.lines.length; i++)
out.print(r.lines[i])
does not... However if I run this on the command line its prints the lines [ ] fine
How do I go about doing it in web context?
Try replacing the following:
BufferedReader reader = new BufferedReader(new FileReader(filename));
with
BufferedReader reader = new BufferedReader(new InputStreamReader(this.getClass().getResourceAsStream(filename)));
That should work since the input file is in the same package as Reader.
Update:
I think the problem lies in the TeamData.
readFile("skytest\\data.file")
That's not a valid path to the file. Neither in filesystem, nor in the classpath.
Since, the data.file is in the classpath, you can use getResourceAsStream to load it.
And, since skytest is the root directory (package), "/skytest/data.file" would also be valid here (the leading / means relative package root). Or, since the file lies in the same package as the TeamData, just the file name should be enough "data.file".
So, use of the following:
readFile("data.file")
And change the following:
BufferedReader reader = new BufferedReader(new FileReader(requiredFile));
to
BufferedReader reader = new BufferedReader(new InputStreamReader(this.getClass().getResourceAsStream(requiredFile)));
Also, the following is really a bad practice (that's called swallowing the exception):
catch (IOException ioe) {
//do something about the exception here
return false;
}
Try something like this:
BufferedReader reader = null;
try
{
reader = new BufferedReader(new FileReader(new File("path/to/filename.txt")));
String nextLine = reader.readLine();
while (nextLine != null)
{
System.out.println(nextLine); // do stuff with the line you read in.
nextLine = reader.readLine();
}
reader.close();
}
catch (FileNotFoundException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}

Categories