I am trying to write a data from database to a csv file via java thread. For writing i am making use of OPENCSV jar. The problem i am facing is that sometimes in csv file values get corrupted like shown below in line 1 and 4.
I have no idea as to why this is happening. Values coming from the database are all ok (as can be seen in the logs) but in csv file its not.
[E[EcoUnit 01] [Segment B/1] [2017-12-29 22:13:23.047] [ventilation air humidity] [70.18]
[EcoUnit 01] [Segment B/1] [2017-10-25 22:21:36.583] [ventilation air humidity] [69.65]
[EcoUnit 01] [Segment B/1] [2017-10-25 22:22:36.59] [ventilation air humidity] [69.33]
[EcoUnit 01] [Segment B/017-11-14 12:02:48.013] [ventilation fan] [30]
I would be really grateful if anyone can let me suggest why this is happening.
Code is as follows: -
List<String> values = new ArrayList<String>();
fw = new FileWriter(file);
writer = new CSVWriter(fw);
writer.writeNext(headers);
values.add(doc.getFieldValue("Unit_Label").toString());
values.add(doc.getFieldValue("Segment_Label").toString());
values.add("[" + doc.getFieldValue("datestring").toString() + "]");
values.add(doc.getFieldValue("Item_Label").toString());
values.add(doc.getFieldValue("Value").toString());
writer.writeNext(values.toArray(new String[]{}));
Adding complete code of the function responsible for creating file and writing into it.
public void createAndFillFile(String startDateStr, String endDateStr, int fileNumber,SolrDocumentList results){
try{
String startDateParts[] = startDateStr.split(" ");
String startDate = startDateParts[0];
String endDateParts[] = endDateStr.split(" ");
String endDate = endDateParts[0];
if(fileNumber == 1){
Date date = new Date() ;
SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH-mm-ss") ;
String currentDate = dateFormat.format(date); //This line can be removed and in below line directly can be used
zipFile = currentDate + ".zip";
dir = new File("C:" + File.separator + "EcotronDownloadable" + File.separator + currentDate);
dir.mkdir();
path = dir.getAbsolutePath() + File.separator ;
file = new File(path+ startDate + "_" + endDate + "_" + fileNumber + ".csv");
fw = new FileWriter(file);
writer = new CSVWriter(fw);
writer.writeNext(headers);
}
synchronized(file){
for (SolrDocument doc : results) {
List<String> values = new ArrayList<String>();
Thread.sleep(1);
long fileLength = file.length();
if(fileLength<maxFileSize){
values.add(doc.getFieldValue("Unit_Label").toString());
values.add(doc.getFieldValue("Segment_Label").toString());
values.add("[" + doc.getFieldValue("datestring").toString() + "]");
values.add(doc.getFieldValue("Item_Label").toString());
values.add(doc.getFieldValue("Value").toString());
//log.trace(values);
writer.writeNext(values.toArray(new String[]{}));
}
else{
fw.flush();
fw.close();
// writer.close();
j = j + 1;
file = new File(path + startDate + "_" + endDate + "_" + j + ".csv") ;
fw = new FileWriter(file);
writer = new CSVWriter(fw);
writer.writeNext(headers);
values.add(doc.getFieldValue("Unit_Label").toString());
values.add(doc.getFieldValue("Segment_Label").toString());
values.add("[" + doc.getFieldValue("datestring").toString() + "]");
values.add(doc.getFieldValue("Item_Label").toString());
values.add(doc.getFieldValue("Value").toString());
//log.trace(values);
writer.writeNext(values.toArray(new String[]{}));
}
}
}
// fw.flush();
// fw.close();
// writer.close();
}
catch (Exception e) {
e.printStackTrace();
}
}
``
You might have two FileWriter instances (in different threads) pointing to the same file and writing at same time.
try this:
synchronized (file)
{
List<String> values = new ArrayList<String>();
fw = new FileWriter(file);
writer = new CSVWriter(fw);
writer.writeNext(headers);
values.add(doc.getFieldValue("Unit_Label").toString());
values.add(doc.getFieldValue("Segment_Label").toString());
values.add("[" + doc.getFieldValue("datestring").toString() + "]");
values.add(doc.getFieldValue("Item_Label").toString());
values.add(doc.getFieldValue("Value").toString());
writer.writeNext(values.toArray(new String[]{}));
}
Related
I have two models in my program, Bus and Learner.
Each is stored in a txt file, name Busses.txt and Learners.txt, respectively.
I am experiencing an issue where the method to delete a learner entry works, but the method to delete a bus entry does not, even though the code is practically identical.
Learner delete method:
public void deleteLearner(String ID) {
removeBlankLines("Learners.txt");
File oldFile = new File("Learners.txt");
File tempFile = new File("tempFile.txt");
String removeKey = ID;
String LearnerID;
String nameSurname;
boolean status;
String busOfLearner;
String line;
String lineToKeep;
try {
Scanner scFile = new Scanner(new File("Learners.txt"));
while (scFile.hasNext()) {
line = scFile.nextLine();
Scanner scLine = new Scanner(line).useDelimiter("#");
LearnerID = scLine.next();
nameSurname = scLine.next();
status = scLine.nextBoolean();
if (scLine.hasNext()) {
busOfLearner = scLine.next();
} else {
busOfLearner = "";
}
if (!LearnerID.equalsIgnoreCase(removeKey)) {
lineToKeep = LearnerID + "#" + nameSurname + "#" + status + "#" + busOfLearner + "\n";
FileWriter fWriter = new FileWriter(tempFile,true);
BufferedWriter bWriter = new BufferedWriter(fWriter);
bWriter.write(lineToKeep);
bWriter.close();
fWriter.close();
}
scLine.close();
}
scFile.close();
boolean successfulDelete = oldFile.delete();
File transfer = new File("Learners.txt");
boolean successfulRename = tempFile.renameTo(transfer);
}
catch (Exception e) {
System.out.println("An error has occured deleting a learner record " + e);
}
}
Delete bus method:
public void deleteBus(String removeBusName) {
removeBlankLinesBus("Busses.txt");
File oldFile = new File("Busses.txt");
File newFile = new File("NewBusFile.txt");
String deleteKey = removeBusName;
String currentBusName;
int currentNumSeats;
String currentPickLocation;
String currentDropLocation;
String currentPickTime;
String currentDropTime;
String line;
String lineToKeep;
try {
Scanner scFile = new Scanner(new File("Busses.txt"));
while (scFile.hasNext()) {
line = scFile.nextLine();
Scanner scLine = new Scanner(line).useDelimiter("#");
currentBusName = scLine.next();
currentNumSeats = scLine.nextInt();
currentPickLocation = scLine.next();
currentDropLocation = scLine.next();
currentPickTime = scLine.next();
currentDropTime = scLine.next();
if (!currentBusName.equalsIgnoreCase(deleteKey)) {
lineToKeep = currentBusName + "#" + currentNumSeats + "#" + currentPickLocation + "#" + currentDropLocation + "#" + currentPickTime + "#" + currentDropTime + "\n";
FileWriter fWriter = new FileWriter(newFile,true);
BufferedWriter bWriter = new BufferedWriter(fWriter);
bWriter.write(lineToKeep);
bWriter.close();
fWriter.close();
}
scLine.close();
}
scFile.close();
boolean successfulDelete = oldFile.delete();
File transfer = new File("Busses.txt");
boolean successfulRename = newFile.renameTo(transfer);
}
catch (Exception e) {
System.out.println("An error has occured deleting " + removeBusName + " from the file: " + e);
}
}
Problem:
With the delete bus method, the old file doesn't get deleted and the temporary or new file doesn't get renamed to the original file.
I am very confident that all files, streams, scanners, etc. are closed, as it is exactly the same as I did in the delete learner method, which does work and the files are deleted and renamed in the learner delete method as it should.
Assistance would be greatly appreciated.
EDIT: Implementation of methods:
Learner:
System.out.println(myController.PrintLearnerArr(myController.LoadLearner("Learners.txt")));
String delete = "0210045112055";
myController.deleteLearner(delete);
System.out.println(myController.PrintLearnerArr(myController.LoadLearner("Learners.txt")));
Bus:
System.out.println(myController.PrintBusArr(myController.LoadBus("Busses.txt")));
String deleteKey = "deleteme";
myController.deleteBus(deleteKey);
System.out.println(myController.PrintBusArr(myController.LoadBus("Busses.txt")));
I cannot seem to figure this out. In the method below I'm trying to write a boolean to a file in 2 places, however nothing is actually being written. Any help would be greatly appreciated.
private void renameTables(){
String path = MessengerMain.getInstance().getDataFolder() + File.separator + "v3-0-0 Table Rename.txt";
File f = new File(path);
try(ResultSet rs = conn.getMetaData().getTables(null, null, "%", null); Writer w = new PrintWriter(new FileOutputStream(f, false))){
if (!f.exists()){
f.createNewFile();
w.write("false");
w.flush();
}
List<String> lines = Files.readAllLines(Paths.get(path));
if (lines.get(0).equalsIgnoreCase("false")){
System.out.println("[Messenger] Verifying table names...");
int count = 0;
List<String> tables = new ArrayList<String>();
tables.add("messages");
tables.add("scores");
tables.add("contacts");
while (rs.next()){
String table = rs.getString("TABLE_NAME");
if (tables.contains(table)){
update("ALTER TABLE " + table + " RENAME TO " + ("messenger_" + table) + ";");
count++;
}
}
if (count > 0){
System.out.println("[Messenger] Done. " + count + " table" + (count == 1 ? "" : "s") + " renamed.");
}else{
System.out.println("[Messenger] Done. No tables need to be renamed.");
}
w.write("true");
w.flush();
}
} catch (SQLException | IOException e){
e.printStackTrace();
}
}
Following Elliot Frisch's advice (same results):
private void renameTables(){
String path = MessengerMain.getInstance().getDataFolder() + File.separator + "v3-0-0 Table Rename.txt";
File f = new File(path);
try(ResultSet rs = conn.getMetaData().getTables(null, null, "%", null)){
Writer w = new PrintWriter(new FileOutputStream(f, false));
if (!f.exists()){
f.createNewFile();
w.write("false");
w.close(); //close here
}
List<String> lines = Files.readAllLines(Paths.get(path));
if (lines.get(0).equalsIgnoreCase("false")){
System.out.println("[Messenger] Verifying table names...");
int count = 0;
List<String> tables = new ArrayList<String>();
tables.add("messages");
tables.add("scores");
tables.add("contacts");
while (rs.next()){
String table = rs.getString("TABLE_NAME");
if (tables.contains(table)){
update("ALTER TABLE " + table + " RENAME TO " + ("messenger_" + table) + ";");
count++;
}
}
if (count > 0){
System.out.println("[Messenger] Done. " + count + " table" + (count == 1 ? "" : "s") + " renamed.");
}else{
System.out.println("[Messenger] Done. No tables need to be renamed.");
}
w = new PrintWriter(new FileOutputStream(f, false)); //create a new writer
w.write("true");
w.close(); //close here
}
} catch (SQLException | IOException e){
e.printStackTrace();
}
}
Here is a working full minimal, complete, verifiable example
public static void main(String[] args) {
File f = new File(System.getProperty("user.home"), "temp.txt");
String path = f.getPath();
try (Writer w = new FileWriter(f)) {
w.write("false");
} catch (IOException e) {
e.printStackTrace();
}
try {
List<String> lines = Files.readAllLines(Paths.get(path));
System.out.println(lines);
} catch (IOException e) {
e.printStackTrace();
}
}
Output is (as expected)
[false]
I am uploading image from url on ftp by this code.
Image uploaded successful but when i try to resize uploaded image get error as the follows.
String imageUrl = "http://www.totaldesign.ir/wp-content/uploads/2014/11/hamayesh.jpg";
FTPClient ftpClient = new FTPClient();
ftpClient.connect(ftp_server, ftp_port);
ftpClient.login(ftp_user, ftp_pass);
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
ftpClient.enterLocalPassiveMode();
// APPROACH #2: uploads second file using an OutputStream
URL url = new URL(imageUrl);
//**************select new name for image***********/
//get file extention
File file = new File(imageUrl);
String ext = getFileExtension(file);
//get file name from url
String fileName = imageUrl.substring(imageUrl.lastIndexOf('/') + 1, imageUrl.length());
String fileNameWithoutExtn = fileName.substring(0, fileName.lastIndexOf('.'));
//create new image name for upload
String img_name = "simjin.com_" + fileNameWithoutExtn;
//get current year time for image upload dir
Date date = new Date();
DateFormat yeae = new SimpleDateFormat("yyyy");
String current_year = yeae.format(date);
//create dirs if not exist
ftpClient.changeWorkingDirectory(current_year);
int dirCode = ftpClient.getReplyCode();
if (dirCode == 550) {
//create dir
ftpClient.makeDirectory(current_year);
System.out.println("created folder: " + current_year);
}
//get current month time for image upload dir
DateFormat month = new SimpleDateFormat("MM");
String current_month = month.format(date);
//create dirs if not exist
ftpClient.changeWorkingDirectory("/" + current_year + "/" + current_month);
dirCode = ftpClient.getReplyCode();
if (dirCode == 550) {
//create dir
ftpClient.makeDirectory("/" + current_year + "/" + current_month);
System.out.println("created folder: " + "/" + current_year + "/" + current_month);
}
String uploadDir = "/" + current_year + "/" + current_month;
//rename image file if exist
boolean exists;
String filePath = uploadDir + "/" + img_name + "." + ext;
exists = checkFileExists(filePath);
System.out.println("old file path=> " + exists);
//Rename file if exist
int i = 0;
while (exists) {
i++;
img_name = "simjin.com_" + fileNameWithoutExtn + i;
filePath = uploadDir + "/" + img_name + "." + ext;
exists = checkFileExists(filePath);
//break loop if file dont exist
if (!exists) {
break;
}
}
System.out.println("new file path=> " + filePath);
//set image name in array for return
result[0] = img_name;
//*************end select new name for image**********/
System.out.println("ftpClinet Replay Code=> " + ftpClient.getStatus());
//Start uploading second file
InputStream inputStream = new BufferedInputStream(url.openStream());
OutputStream outputStream = ftpClient.storeFileStream(filePath);
System.out.println("outputStream Status=> " + outputStream);
byte[] bytesIn = new byte[10240];
int read = 0;
while ((read = inputStream.read(bytesIn)) != -1) {
outputStream.write(bytesIn, 0, read);
}
inputStream.close();
outputStream.close();
boolean completed = ftpClient.completePendingCommand();
after success upload. I want to resize image by thumbnailator:
if (completed) {
System.out.println("The file is uploaded successfully.");
String new_img_name = uploadDir + "/" + img_name + "-150x150" + "." + ext;
OutputStream os = ftpClient.storeFileStream(filePath);
Thumbnails.of(image_url).size(150, 150).toOutputStream(os);
}
in this section get this error:
OutputStream cannot be null
Where is my wrong? And how to fix it?
I was trying to move file using move file method .but getting this error
"java.io.IOException: Failed to delete original file"
this is my code.
this is for file move without error
public void moveWithoutError(String fileName){
DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
Date date = new Date();
String ts = dateFormat.format(date);
File source = new File("D:/Inbound/");
File dest = new File("D:/Archive/");
for (File file : source.listFiles()) {
String x = (source + "/" + file.getName());
String y = (dest + "/" + addTimestamp(file.getName(), ts));
if(file.getName().equals(fileName)){
File sourceFile = new File(x);
File destnFile = new File(y);
try{
FileUtils.moveFile(sourceFile, destnFile);
System.out.println("Moved file:"+y);
}catch(Exception e){
System.out.println("Unable to move file:"+y);
System.out.println(e);
e.printStackTrace();
}
break;
}
}
}
This code is written for moving the file with containing error
public void moveWithError(String fileName){
DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
Date date = new Date();
String ts = dateFormat.format(date);
File source = new File("D:/Inbound\\");
File dest = new File("D:\\Error\\"); /
for (File file : source.listFiles()) {
String x = (source + "/" + file.getName());
String y = (dest + "/" + addTimestamp(file.getName(), ts));
if(file.getName().equals(fileName)){
File f1 = new File(x);
if(f1.renameTo(new File(y))){
System.out.println(" moved: " + y);
} else {
System.out.println("unable to move: " + y);
}
break;
}
}
}
This code is used to add timestamp to moved file
public static String addTimestamp(String name, String ts) {
int lastIndexOf = name.lastIndexOf('.');
return (lastIndexOf == -1 ?
name + "_" + ts
:
name.substring(0, lastIndexOf) + "-" + ts +
name.substring(lastIndexOf))
.replaceAll("[\\/:\\*\\?\"<>| ]", "");
}
}
while executing this code exception come
abd exception is
"java.io.IOException: Failed to delete original file 'D:\Inbound\Testdata.xlsx' after copy to 'D:\Archive\Testdata-20140812121814.xlsx'"
Note - this file is nor open neither accessing by other process
Please help :(
Exception
java.io.IOException: Failed to delete original file 'D:\Inbound\Testdata.xlsx' after copy to 'D:\Archive\Testdata-20140812130655.xlsx'
at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:2664)
at CRP.MoveFile.moveWithoutError (MoveFile.java:77)
at CRP.CRPDAO.callMoveFile (CRPDAO.java:562)
at CRP.CRP_FileDependency.main (CRP_FileDependency.java:163)
I have this Method that Lists the Files in a Directory and I need to write the same to a file. Apparently my
System.out.println();
is able to list the files and their Sizes and the Dates they were Modified. But My bufferedWriter Does not write Anything in the File. Here is My Method;
public void walk( String path, int limit ) throws IOException {
File root = new File( path );
File[] list = root.listFiles();
File rep = new File("report.txt");
SimpleDateFormat sdf = new SimpleDateFormat("MMM/dd/yyyy HH:mm:ss");
if (list == null) return;
long size;
BufferedWriter bw = new BufferedWriter(new FileWriter(rep));
for ( File f : list ) {
size = f.length()/1000/1000;
if ( f.isDirectory() ) {
walk( f.getAbsolutePath(), limit );
}
else {
if(size >= limit){
System.out.println( "File:" + f.getAbsoluteFile() + " " + size + "MB Last Modified Date: " + sdf.format(f.lastModified()));
bw.write("File:" + f.getAbsoluteFile() + " " + size + "MB Last Modified Date: " + sdf.format(f.lastModified()) + "\n");
}
}
}
bw.close();
}
What Am I Missing? I need to write the Out to the File report.txt but the file is empty.
I think it's because you're trying to open multiple buffered writers to the same file when it's calling itself through recursion. Try creating your writer outside of this method and pass it in as a parameter.
Example
public void myCallingMethod() throws IOException{
File rep = new File("report.txt");
BufferedWriter bw = new BufferedWriter(new FileWriter(rep));
walk("my/path", 4, bw);
bw.close();
}
Here's the code which might solve your problem.
I tried the same.
public class example {
public static void main(String[] args) throws IOException {
// Directory path here
String path = "C:\\";
SimpleDateFormat sdf = new SimpleDateFormat("MMM/dd/yyyy HH:mm:ss");
File folder = new File(path);
File[] listOfFiles = folder.listFiles();
File file = new File("report.txt");
if (!file.exists()) {
file.createNewFile();
}
FileWriter fw = new FileWriter(file.getAbsoluteFile());
BufferedWriter bw = new BufferedWriter(fw);
long size;
int limit = 2;
for (File f : listOfFiles) {
size = f.length() / 1000 / 1000;
if (size >= limit) {
System.out.println("File:" + f.getAbsoluteFile() + " " + size
+ "MB Last Modified Date: "
+ sdf.format(f.lastModified()));
bw.write("File:" + f.getAbsoluteFile() + " " + size
+ "MB Last Modified Date: "
+ sdf.format(f.lastModified()));
}
}
bw.close();
}
}
try to call the flush method after you called the write methode like:
bw.write("File:" + f.getAbsoluteFile() + " " + size + "MB Last Modified Date: " + sdf.format(f.lastModified()) + "\n");
bw.flush();
The flush method flushes your stream to your file
Define a variable like this
StringBuilder fileData = new StringBuilder();
And replace
bw.write("File:" + f.getAbsoluteFile() + " " + size + "MB Last Modified Date: " + sdf.format(f.lastModified()) + "\n");
bw.flush();
With
fileData.append("File:").append(f.getAbsoluteFile()).append(" ").append(size)
.append("MB Last Modified Date: ").append(sdf.format(f.lastModified()))
.append("\n") ;
And after the for loop write fileData to file
bw.write(fileData.toString());
bw.close();
You need to do a java.io.BufferedWriter#flush() before the close.