We Keep Coding

Error or exceptions saving a .txt file (Java)

I run the following piece of Processing (Java) code inside a bigger loop. These lines save a string in a .txt file called kinectDEM.tmp, before doing that, the old file is renamed to kinectDEM1.txt and the new one (kinectDEM.tmp) is renamed to kinectDEM0.txt.
It works fine but sometimes it get stuck and the kinectDEM1.txt file disappears, the code still work but doesn't save the .txt files. No error message appears.
Is there something wrong saving .txt files in that way?
Here's the code:
import java.io.File;
import SimpleOpenNI.*;
import java.util.*;
SimpleOpenNI kinect;
List<int[]> previousKinectValues = new LinkedList<int[]>();
int numPreviousToConsider = 60;
void setup()
{
size(640, 480);
kinect = new SimpleOpenNI(this);
kinect.enableDepth();
frameRate(60);
}
int precedente = millis();
void draw()
{
kinect.update();
PImage depthImage = kinect.depthImage();
image(depthImage, 0, 0);
int[] newDepthValues = kinect.depthMap();
previousKinectValues.add(newDepthValues);
if (previousKinectValues.size() > numPreviousToConsider) {
previousKinectValues.remove(0);
}
int[] depthValues = average(previousKinectValues);
depthValues = reverse(depthValues);
StringBuilder sb = new StringBuilder();
Deque<Integer> row = new LinkedList<Integer>();
int kinectheight = 770; // kinect distance from the baselevel [mm]
int scaleFactor = 1;
int pixelsPerRow = 640;
int pixelsToSkip = 40;
int rowNum = 0;
for (int i = 0; i < depthValues.length; i++) {
if (i > 0 && i == (rowNum + 1) * pixelsPerRow) {
fillStringBuilder(sb, row);
rowNum++;
sb.append("\n");
row = new LinkedList<Integer>();
}
if (i < ((rowNum+1) * pixelsPerRow) - pixelsToSkip) {
//if (i >= (rowNum * pixelsPerRow) + pixelsToSkip) {
row.addFirst((kinectheight - depthValues[i]) * scaleFactor);
}
}
fillStringBuilder(sb, row);
String kinectDEM = sb.toString();
final String[] txt= new String[1]; //creates a string array of 2 elements
int savingtimestep = 2000; // time step in millisec between each saving
if (millis() > precedente + savingtimestep) {
txt[0] = "ncols 600\nnrows 480\nxllcorner 0\nyllcorner 0\ncellsize 1\nNODATA_value 10\n" +kinectDEM;
saveStrings("kinectDEM0.tmp", txt);
precedente = millis();
// delete the old .txt file, from kinectDEM1 to kinectDEMtrash
File f = new File(sketchPath("kinectDEM1.txt"));
boolean success = f.delete();
// rename the old .txt file, from kinectDEM0 to kinectDEM1
File oldName1 = new File(sketchPath("kinectDEM0.txt"));
File newName1 = new File(sketchPath("kinectDEM1.txt"));
oldName1.renameTo(newName1);
// rename kinectDEM0.tmp file to kinectDEM0.txt
File oldName2 = new File(sketchPath("kinectDEM0.tmp"));
File newName2 = new File(sketchPath("kinectDEM0.txt"));
oldName2.renameTo(newName2);
}
}
void fillStringBuilder(StringBuilder sb, Deque<Integer> row) {
boolean emptyRow = false;
while (!emptyRow) {
Integer val = row.pollFirst();
if (val == null) {
emptyRow = true;
} else {
sb.append(val);
val = row.peekFirst();
if (val != null) {
sb.append(" ");
}
}
}
}
int[] average(List<int[]> previousKinectValues) {
if (previousKinectValues.size() > 0) {
int[] first = previousKinectValues.get(0);
int[] avg = new int[first.length];
for (int[] prev : previousKinectValues) {
for (int i = 0; i < prev.length; i++) {
avg[i] += prev[i];
}
}
int num = previousKinectValues.size();
for (int i = 0; i < avg.length; i++) {
avg[i] /= num;
}
return avg;
}
return new int[0];
}

You can simplify your problem into a much smaller example sketch:
saveStrings("one.txt", new String[]{"ABC"});
File oldName = new File(sketchPath("one.txt"));
File newName = new File(sketchPath("two.txt"));
boolean renamed = oldName.renameTo(newName);
println(renamed);
In the future, please try to narrow your problem down to an MCVE like this.
Anyway, this program saves the text ABC to a file named one.txt. It then tries to rename one.txt to two.txt. This is exactly what you're trying to do, just without all that extra kinect code, which doesn't really have anything to do with your problem.
Please run this little example program and then view your sketch directory (Sketch > Show Sketch Folder). You'll see the two.txt file, and that file will contain the text ABC on a single line. Also notice that the program prints true to the console, indicating that the rename was successful. This is exactly what you'd expect.
Now, change the first line to this:
saveStrings("one.txt", new String[]{"XYZ"});
Run the program again. First notice that it prints out false to the console, indicating that the rename was not successful. Then view the sketch folder, and you'll see two text files: one.txt which contains XYZ and two.txt which contains ABC. This is not what we expect, and this is what's happening in your code as well.
So, what's happening is this:
We run our code, create one.txt containing ABC, then rename it to two.txt. We then run the code again, creating one.txt containing XYZ. We then try to rename that new file to two.txt, but we can't.
From the Java API for the File#renameTo() function, emphasis mine:
Many aspects of the behavior of this method are inherently platform-dependent: The rename operation might not be able to move a file from one filesystem to another, it might not be atomic, and it might not succeed if a file with the destination abstract pathname already exists. The return value should always be checked to make sure that the rename operation was successful.
Note that the Files class defines the move method to move or rename a file in a platform independent manner.
So it looks like the rename step is failing, becuase you can't rename a file to overwrite an existing file. Instead, we can use the Files#move() function, which allows us to specify overwrite options:
Files.move(oldName.toPath(), newName.toPath(), StandardCopyOption.REPLACE_EXISTING);

JAVA: ArrayIndexOutOFBoundException while reading imported txt

I'm a beginner, I have got a txt file that user will imported into java, I'm going to read txt file line be line then set variable base on each line and add them to the current record
public void importTXT() {
JFileChooser fc = new JFileChooser();
fc.setAcceptAllFileFilterUsed(false);
fc.setMultiSelectionEnabled(false);
FileNameExtensionFilter filter = new FileNameExtensionFilter(
"TEXT FILES", "txt", "text");
fc.setFileFilter(filter);
int returnVal = fc.showOpenDialog(CollectionFrame.this);
String[] numstrs = null;
if (returnVal == JFileChooser.APPROVE_OPTION) {
File importedFile = fc.getSelectedFile();
try {
Scanner sc = new Scanner(importedFile);
while (sc.hasNextLine()) {
numstrs = sc.nextLine().split("\\s+"); // split by white
// space
}
} catch (IOException e) {
}
// add new collection
Collection newCollection = new Collection(numstrs[0]);
allRecord.addCollection(newCollection);
// add art consignment information
String consignmentName = numstrs[3];
String description = numstrs[4];
I received a ArrayIndexOutOfBoundsException at the second last line
String consignmentName = numstrs[3];
The content of the text file is like:
Richman’s Estate Collection
5
ART
Water Lilies
A superb piece in great condition
Can someone tell me what's wrong?

EDIT:
you are currently reading all lines and replacing each time the value of numstrs
so when you leave the loop, you just got the value of the last line in it.
i think you want to save ALL lines - see below.
END EDIT
you should use an arraylist.
like this:
ArrayList<String[]> numstrsList = new ArrayList<String[]>();
if (returnVal == JFileChooser.APPROVE_OPTION) {
File importedFile = fc.getSelectedFile();
try {
Scanner sc = new Scanner(importedFile);
while (sc.hasNextLine()) {
numstrsList.add(sc.nextLine().split("\\s+")); // split by white
// space
}
} catch (IOException e) {
}
}
you can catch the values of your arraylist with:
for(int i=0:i<numstrsList.size();i++){
String[] oneLineStrArray = numstrsList.get(index)
//do something
}
you should post the text data, otherwise we cant help you about the OutOfBounds error.
Also, i am wondering how you can instantiate a Collection.

As previous comments already has suggested, the file obviously does not contain the information in the form you expect.
I suggest that you add som error handling to it after the scanner read.
if (numstrs.length < 5){
///TODO: add some handling here, exception or error dialog
}

Here, first go through ArrayIndexOutOfBoundsException
Check numstrs length before you try to access index value(s). You can do
String consignmentName = null, description = null;
if (numstrs.length >= 4) {
consignmentName = numstrs[3];
}
if (numstrs.length >= 5) {
description = numstrs[4];
}
Also, as pointed out by one of the comment - numstrs will always have the values returned by the last line.

How to save an object into a file, so that i can load it later?

I've been trying to save an object, a level in my case, into a file, so that i can later load it again. Basically just a save/load function. I can't get it to work.. I keep getting a file of 5 byte, which seems way to small for what should be in it. I know that it might have something to do with Serializable, but i dont know what. Here is my current code:
(btw, i hard coded the level into the program, because i dont know how to save it properly to a file yet. although that is the goal...)
public class BufferSaveGames {
public void saveGameOutputStream(Level level) throws FileNotFoundException {
ObjectOutputStream output;
try {
output = new ObjectOutputStream((new FileOutputStream("SaveGame.dat")));
output.writeObject(level);
output.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
and the second class Level, which it is loading: (removed code not needed for example)
public class Level implements MouseListener, Serializable {
private Level currentLevel;
private int aantalPoppetjesOpLevel;
private String oplossing;
private int timer;
JPanel[][] levelGrid;
Poppetje[] lijstPoppetjes;
public Level() throws IOException{
levelGrid = new JPanel[5][5];
lijstPoppetjes = new Poppetje[5];
for (int a=0;a<5;a++) {
for (int b=0;b<5;b++) {
levelGrid[a][b] = new JPanel();
levelGrid[a][b].setSize(100,100);
levelGrid[a][b].setVisible(true);
levelGrid[a][b].setLayout(null);
levelGrid[a][b].setOpaque(false);
levelGrid[a][b].addMouseListener(this);
}
}
//bovenste rij
levelGrid[0][0].setLocation(10,10);
levelGrid[0][1].setLocation(112,10);
levelGrid[0][2].setLocation(214,10);
levelGrid[0][3].setLocation(316,10);
levelGrid[0][4].setLocation(418,10);
Poppetje roodPoppetje = new Poppetje("Rood", 4, 4);
Poppetje oranjePoppetje = new Poppetje("Oranje", 0, 4);
Poppetje groenPoppetje = new Poppetje("Groen", 1, 2);
Poppetje paarsPoppetje = new Poppetje("Paars", 2, 1);
Poppetje geelPoppetje = new Poppetje("Geel", 3, 3);
//Poppetje blauwPoppetje = new Poppetje("Blauw");
int tempA = roodPoppetje.getLocatieX(roodPoppetje);
int tempB = roodPoppetje.getLocatieY(roodPoppetje);
levelGrid[tempA][tempB].add(roodPoppetje);
lijstPoppetjes[0] = roodPoppetje;
lijstPoppetjes[0].addMouseListener(this);
tempA = oranjePoppetje.getLocatieX(oranjePoppetje);
tempB = oranjePoppetje.getLocatieY(oranjePoppetje);
levelGrid[tempA][tempB].add(oranjePoppetje);
lijstPoppetjes[1] = oranjePoppetje;
lijstPoppetjes[1].addMouseListener(this);

That could be done with "Serialization/Deserialization". Check out this as a first approach:
http://www.wikihow.com/Serialize-an-Object-in-Java
Also try searching with "serialziation" keyword on your favorite search engine (highly probably google hehe). There are a lot of libraries that do this process in such a high level api. Also there are a lot of libraries that do this with powerful uses; like serialization through databases. Sometimes serialization is done using JSON (as it could be a more universal way to do it). Enjoy with serialization! :-)
edit: also search for JSON, beautiful tool

forgot to put the following line in the OutPutStream code:
Level levelX = new Level();
this solved the problem for the very small save game that it created.

removeEldestEntry overriding

How can I override removeEldestEntry method to saving eldest entry to file? Also how to limit the size of a file like I did it in LinkedHashMap. Here is code:
import java.util.*;
public class level1 {
private static final int max_cache = 50;
private Map cache = new LinkedHashMap(max_cache, .75F, true) {
protected boolean removeEldestEntry(Map.Entry eldest) {
return size() > max_cache;
}
};
public level1() {
for (int i = 1; i < 52; i++) {
String string = String.valueOf(i);
cache.put(string, string);
System.out.println("\rCache size = " + cache.size() +
"\tRecent value = " + i + " \tLast value = " +
cache.get(string) + "\tValues in cache=" +
cache.values());
}
I tried to use FileOutPutSTream :
private Map cache = new LinkedHashMap(max_cache, .75F, true) {
protected boolean removeEldestEntry(Map.Entry eldest) throws IOException {
boolean removed = super.removeEldestEntry(eldest);
if (removed) {
FileOutputStream fos = new FileOutputStream("t.tmp");
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeObject(eldest.getValue());
oos.close();
}
return removed;
}
But I have gained an error
Error(15,27): removeEldestEntry(java.util.Map.Entry) in cannot override removeEldestEntry(java.util.Map.Entry) in java.util.LinkedHashMap; overridden method does not throw java.io.IOException
Without IOExecptio compiler asks to handle IOexception and Filenotfoundexception.
Maybe another way exists? Pls show me example code, I am new in java and just trying to understand the basic principles of 2 level caching. Thx

You first need to make sure your method properly overrides the parent. You can make some small changes to the signature, such as only throwing a more specific checked exception that is a sub-class of a checked exception declared in the parent. In this case, the parent does not declare any checked exception so you can not refine that further and may not throw any checked exceptions. So you will have to handle the IOException locally. There are several ways you can do that, convert it to a RuntimeException of some kind and/or log it.
If you are concerned about the file size, you probably do not want to keep just the last removed entry but many of them - so you should open the file for append.
You need to return true from the method to actually remove the eldest and you need to decide if the element should be removed.
When working with files you should use try/finally to ensure that you close the resource even if there is an exception. This can get a little ugly - sometimes it's nice to have a utility method to do the close so you don't need the extra try/catch.
Generally you should also use some buffering for file I/O which greatly improves performance; in this case use wrap the file stream in a java.io.BufferedOutputStream and provide that to the ObjectOutputStream.
Here is something that may do what you want:
private static final int MAX_ENTRIES_ALLOWED = 100;
private static final long MAX_FILE_SIZE = 1L * 1024 * 1024; // 1 MB
protected boolean removeEldestEntry(Map.Entry eldest) {
if (size() <= MAX_ENTRIES_ALLOWED) {
return false;
}
File objFile = new File("t.tmp");
if (objFile.length() > MAX_FILE_SIZE) {
// Do something here to manage the file size, such as renaming the file
// You won't be able to easily remove an object from the file without a more
// advanced file structure since you are writing arbitrary sized serialized
// objects. You would need to do some kind of tagging of each entry or include
// a record length before each one. Then you would have to scan and rebuild
// a new file. You cannot easily just delete bytes earlier in the file without
// even more advanced structures (like having an index, fixed size records and
// free space lists, or even a database).
}
FileOutputStream fos = null;
try {
fos = new FileOutputStream(objFile, true); // Open for append
ObjectOutputStream oos = new ObjectOutputStream(new BufferedOutputStream(fos));
oos.writeObject(eldest.getValue());
oos.close(); // Close the object stream to flush remaining generated data (if any).
return true;
} catch (IOException e) {
// Log error here or....
throw new RuntimeException(e.getMessage(), e); // Convert to RuntimeException
} finally {
if (fos != null) {
try {
fos.close();
} catch (IOException e2) {
// Log failure - no need to throw though
}
}
}
}

You can't change the method signature when overriding a method. So you need to handle the exception in the overridden method instead of throwing it.
This contains a good explanation on how to use try and catch: http://download.oracle.com/javase/tutorial/essential/exceptions/try.html

Finding file size and last modified of SFTP oldest file using Java

I'm using JSch to get files from an SFTP server, but I'm trying to figure out a way to only get the oldest file, and to make sure that it is not currently being written to. The way I imagine myself doing this is first finding which file in the specified remote folder is oldest. I would then check the file size, wait x seconds (probably about 10, just to be safe) and then check it again. If the file size has not changed, I download the file and process it. However, I have no idea how to do this! If anybody knows how to do this, or knows of something else that supports SFTP that has this built-in (I know Apache Commons does, but only does FTPS), it would be greatly appreciated.
Thanks in advance.

Turns out that this is entirely possible in JSch, the hardest part is simply finding the documentation. Code I used is below, hopefully somebody else will find it helpful! (I'm sure there are optimizations to be made, I know, I know. There are also variables that are defined elsewhere, but hopefully anybody that needs this will be able to figure them out!)
public static String oldestFile() {
Vector list = null;
int currentOldestTime;
int nextTime = 2140000000; //Made very big for future-proofing
ChannelSftp.LsEntry lsEntry = null;
SftpATTRS attrs = null;
String nextName = null;
try {
list = Main.chanSftp.ls("*.xml");
if (list.isEmpty()) {
fileFound = false;
}
else {
lsEntry = (ChannelSftp.LsEntry) list.firstElement();
oldestFile = lsEntry.getFilename();
attrs = lsEntry.getAttrs();
currentOldestTime = attrs.getMTime();
for (Object sftpFile : list) {
lsEntry = (ChannelSftp.LsEntry) sftpFile;
nextName = lsEntry.getFilename();
attrs = lsEntry.getAttrs();
nextTime = attrs.getMTime();
if (nextTime < currentOldestTime) {
oldestFile = nextName;
currentOldestTime = nextTime;
}
}
attrs = chanSftp.lstat(Main.oldestFile);
long size1 = attrs.getSize();
System.out.println("-Ensuring file is not being written to (waiting 1 minute)");
Thread.sleep(60000); //Wait a minute to make sure the file size isn't changing
attrs = chanSftp.lstat(Main.oldestFile);
long size2 = attrs.getSize();
if (size1 == size2) {
System.out.println("-It isn't.");
fileFound = true;
}
else {
System.out.println("-It is.");
fileFound = false;
}
}
} catch (Exception ex) {ex.printStackTrace();}
return Main.oldestFile;
}

You can easily do this using edtFTPj/PRO, which supports SFTP.
Simply get a directory listing, and sort the listing by date. If the oldest date isn't in the last few minutes, you can download.

I don't have a direct answer to your question, but it sounds like you want to do something similar to reliable file transfer. This is part of a larger project in Grid computing that is now apparently organized here. I don't know if it includes security features or if you can add them on, but it is an open source project.

calculate the folder size in remote server just call
the ftpFolderSize(ftpFolderSize,client) directory path, and
pass the object FTPClient as a parameter. It will return the
size of folder.
Works only for FTP.
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPFile;
import org.apache.commons.io.FileUtils;
long dirSize = 0L; //global variable
private long ftpFolderSize(String directoryPath,FTPClient client){
try{
client.changeWorkingDirectory(directoryPath);
FTPFile[] ftpFiles = client.listFiles();
if(client.changeWorkingDirectory(directoryPath))
{
for (FTPFile ftpFile : ftpFiles) {
if(ftpFile.isFile()){
dirSize = dirSize+ftpFile.getSize();// file size is calculated
}
else if(ftpFile.isDirectory())
{
dirSize = dirSize + 4096;//folder minimum size is 4kb
ftpFolderSize(directoryPath+"/"+ftpFile.getName(),client);
}
}
}
}catch (Exception e) {
e.printStackTrace();
}
return dirSize;
}