Eclipse CSV Download cuts off data - java

I currently have a file download process in my java class, listed below, to take all the data in an SQL table and put it in a CSV file for user download. However, when I download the file, all the data is good except it will cut off at random points (usually around line 20 of the data, given there are at least over 100 lines of data). I want to ask, what if making the cutoff? Is it session time related or is the code just problematic?
public String processFileDownload() {
DataBaseBean ckear = new DataBaseBean();
ckear.clearContens();
FacesContext fc = FacesContext.getCurrentInstance();
ExternalContext ec = fc.getExternalContext();
Map<String, Object> m = fc.getExternalContext().getSessionMap();
dbase = (DbaseBean) m.get("dbaseBean");
message = (MessageBean) m.get("messageBean");
dataBean = (DataBean) m.get("dataBean");
dbmsUser = (DbmsUserBean) m.get("dbmsUserBean");
FileOutputStream fos = null;
String path = fc.getExternalContext().getRealPath("/temp");
String tableName = dbmsUser.getTableName();
String fileNameBase = tableName + ".csv";
java.net.URL check = getClass().getClassLoader().getResource(
"config.properties");
File check2 = new File(check.getPath());
path = check2.getParent();
String fileName = path + "/" + dbmsUser.getUserName() + "_"
+ fileNameBase;
File f = new File(fileName);
try {
f.createNewFile();
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
dbase.connect();
dbase.setQueryType("SELECT");
dbase.executeSQL("select * from " + tableName);
if (dbase.getResultSet() == null) {
FacesContext.getCurrentInstance().addMessage("myForm3:errmess",
new FacesMessage("Table doesn't exist!"));
return "failed";
}
Result result = ResultSupport.toResult(dbase.getResultSet());
downlaodedrows = result.getRowCount();
Object[][] sData = result.getRowsByIndex();
String columnNames[] = result.getColumnNames();
StringBuffer sb = new StringBuffer();
try {
fos = new FileOutputStream(fileName);
for (int i = 0; i < columnNames.length; i++) {
sb.append(columnNames[i].toString() + ",");
}
sb.append("\n");
fos.write(sb.toString().getBytes());
for (int i = 0; i < sData.length; i++) {
sb = new StringBuffer();
for (int j = 0; j < sData[0].length; j++) {
sb.append(sData[i][j].toString() + ",");
}
sb.append("\n");
fos.write(sb.toString().getBytes());
}
fos.flush();
fos.close();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
String mimeType = ec.getMimeType(fileName);
FileInputStream in = null;
byte b;
ec.responseReset();
ec.setResponseContentType(mimeType);
ec.setResponseContentLength((int) f.length());
ec.setResponseHeader("Content-Disposition", "attachment; filename=\""
+ fileNameBase + "\"");
try {
in = new FileInputStream(f);
OutputStream output = ec.getResponseOutputStream();
while (true) {
b = (byte) in.read();
if (b < 0)
break;
output.write(b);
}
} catch (NullPointerException e) {
fc.responseComplete();
return "SUCCESS";
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
try {
in.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
fc.responseComplete();
return "SUCCESS";
}

The problem seems to be that you are simply appending commas between values and it's likely one of the values you are writing contains a delimiter, line separator or quote character, which will "break" the CSV format if not correctly escaped.
It would be way easier and faster to use CSV library for that. uniVocity-parsers comes with pre-built routines to dump your resultset into properly formatted CSV. In your case, you could use this library in the following manner:
ResultSet resultSet = dbase.getResultSet();
// Configure the output format as needed before actually dumping the data:
CsvWriterSettings writerSettings = new CsvWriterSettings(); //many settings here, check the tutorials & examples.
writerSettings.getFormat().setLineSeparator("\n");
writerSettings.setHeaderWritingEnabled(true); // we want the column names to be printed out as well.
// Then create a routines object:
CsvRoutines routines = new CsvRoutines(writerSettings);
// The write() method takes care of everything. The resultSet and any other resources required are closed by the routine.
routines.write(resultSet, new File(fileName), "UTF-8");
Hope this helps
Disclaimer: I'm the author of this library. It's open source and free (Apache 2.0. license)

Related

How can I make a directory in Java?

I am trying to create a new file directory, but the function mkdir() doesn't work, neither mkdirs().
Here's my code:
...
while (leitor.hasNext()){
String [] plv = LerPalavras(tamMem, leitor);
Arrays.sort(plv);
String nomeTemp = "/temp/temp" + contador + ".txt"; // I need to create this directory
try{
escritor = new FileWriter(nomeTemp);
for (int i = 0; i < tamMem; i++) {
escritor.write(plv[i] + " ");
}
escritor.close();
} catch (IOException e){
System.out.println(e.getMessage());
}
contador++;
}
...
Edit: I made the edits and now it's working!
File pastaTemp = new File("/temp/temp");
pastaTemp.mkdirs();
while (leitor.hasNext()){
String [] plv = LerPalavras(tamMem, leitor);
Arrays.sort(plv);
File arqTemp = new File (pastaTemp, contador + ".txt");
try{
escritor = new FileWriter(arqTemp);
for (int i = 0; i < tamMem; i++) {
escritor.write(plv[i] + " ");
}
escritor.close();
} catch (IOException e){
System.out.println(e.getMessage());
}
contador++;
}
Try doing this in two steps. First, call File.mkdirs() to create the entire directory structure, if necessary, then create the file you pass to the FileWriter:
try {
File folder = new File("/temp/temp");
folder.mkdirs();
// then create a file object at this location
File file = new File(folder, contador + ".txt");
escritor = new FileWriter(file);
// the rest of your code
}
catch (Exception e) {
}

How to get SdCard cid number programmatically in Nougat?

In all versions, i am able to get SdCard details like serial no., CID number .. except Nougot version.
I am using the below methods to get the SdCard details. Both are working fine all versions except Nougat. Those paths are returning null in nougat version
public void getCID() {
File input = new File("/sys/class/mmc_host/mmc1");
String cidDirectory = null;
File[] sid = input.listFiles();
for (int i = 0; i < sid.length; i++) {
if (sid[i].toString().contains("mmc1:")) {
cidDirectory = sid[i].toString();
String SID = (String) sid[i].toString().subSequence(cidDirectory.length() - 4, cidDirectory.length());
}
}
try {
BufferedReader CID = new BufferedReader(new FileReader(cidDirectory + "/cid"));
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
String getSDCARDiD() {
try {
File file = new File("/sys/block/mmcblk1");
if (file.exists() && file.isDirectory()) {
memBlk = "mmcblk1";
} else {
//System.out.println("not a directory");
memBlk = "mmcblk0";
}
Process cmd = Runtime.getRuntime().exec("cat /sys/block/" + memBlk + "/device/cid");
BufferedReader br = new BufferedReader(new InputStreamReader(cmd.getInputStream()));
sd_cid = br.readLine();
} catch (IOException e) {
e.printStackTrace();
}
return sd_cid;
}
Why it is giving null i didn't understand.
Thanks in advance
Above 7.0 we have to use StorageVolume.getUuid() on StorageVolume which you get from StorageManager.

How to de- serialize the DoccatModel object in java?

I am trying to serialize a DoccatModel Object in Java (I am using OpenNLP algorithm).
It's able to serialize the object properly and I am writing that to an external file. But, I'm not able to deserialise the object back.
I am trying to use ObjectInputStream to read that file back. It's throwing an error:
"java.io.StreamCorruptedException: invalid stream header: 504B0304"
I just want to know how to de serialize the DoccatModel object back so that I can use it further.
The code is as follows. [Apologies for bad code since its still in development stage]
private static void runDocCat(String textField, String updateField, String tgtField)
{
String dbName;
String tableName;
DoccatModel model = null;
JSONArray allDesc = null;
DataConfig BSC = null;
try
{
BSC = new DataConfig();
}
catch (Exception e1)
{
e1.printStackTrace();
}
try
{
dbName = BSC.getdbName();
tableName = BSC.tablename;
String dataQuery = "SELECT ID, VOC, " + tgtField + " from " + dbName + "." + tableName;
allDesc = BSC.getJSONArray(dataQuery);
File file = new File("voc.train");
BufferedWriter output = new BufferedWriter(new FileWriter(file));
for (int i = 0; i < allDesc.length(); i++)
{
String tgt = allDesc.getJSONObject(i).getString(tgtField);
if (tgt.length() < 3)
tgt = "Unknown";
tgt = tgt.replaceAll(" ", "");
String desc = allDesc.getJSONObject(i).getString(textField);
desc = desc.replaceAll("\\r", " ").replaceAll("\\n", ".");
if (!desc.trim().equalsIgnoreCase("nothing"))
{
DocumentSample currDoc = new DocumentSample(tgt, desc);
output.write(currDoc.toString());
output.newLine();
}
}
output.close();
System.out.println("Training Data Generated!");
ObjectStream<String> lineStream = new PlainTextByLineStream(new FileReader(file));
ObjectStream<DocumentSample> sampleStream = new DocumentSampleStream(lineStream);
model = DocumentCategorizerME.train("en", sampleStream);
System.out.println("Model Data \n\n" + model);
// Write to a file
OutputStream modelOut = null;
try
{
String modelfile = "C:\\VOC_Classification\\model.ser";
modelOut = new BufferedOutputStream(new FileOutputStream(modelfile));
model.serialize(modelOut);
System.out.println("Model Data \n\n" + model);
}
catch (IOException e)
{
e.printStackTrace();
}
finally
{
if (modelOut != null)
try
{
modelOut.close();
}
catch (IOException ex)
{
ex.printStackTrace();
}
}
// try
// {
// FileOutputStream fileOut = new FileOutputStream("modelfile.ser");
// ObjectOutputStream out = new ObjectOutputStream(fileOut);
// out.writeObject(model);
// out.close();
// fileOut.close();
// System.out.printf("Serialized data is saved in modelfile.ser");
// } catch (IOException i)
// {
// i.printStackTrace();
// }
DoccatModel model_SER = null;
try
{
FileInputStream fileIn = new FileInputStream("C:\\VOC_Classification\\model.ser");
ObjectInputStream in = new ObjectInputStream(fileIn);
model_SER = (DoccatModel) in.readObject();
in.close();
fileIn.close();
} catch (IOException e)
{
e.printStackTrace();
}
System.out.println("\nTraining Done!\n");
DocumentCategorizerME myCategorizer = new DocumentCategorizerME(model);
System.out.println(" ---------- Starting categorising process ----------");
System.out.println("\nProcess Running, Please wait...\n");
for (int i = 0; i < allDesc.length(); i++)
{
dataQuery = "SELECT ID, VOC," + tgtField + " from " + dbName + "." + tableName;
allDesc = BSC.getJSONArray(dataQuery);
String ID = allDesc.getJSONObject(i).getString("ID");
String desc = allDesc.getJSONObject(i).getString(textField);
String newdesc = desc.replaceAll("\\n", ".").replaceAll("\\r", " ");
if (!newdesc.trim().equalsIgnoreCase("nothing"))
{
double[] outcomes = myCategorizer.categorize(newdesc);
String category = myCategorizer.getBestCategory(outcomes);
if (!category.equalsIgnoreCase("Unknown"))
{
String updQuery = "UPDATE " + dbName + "." + tableName + " set " + updateField + " = " + "'"
+ category + "'" + " WHERE ID = " + ID;
BSC.executeUpdate(updQuery);
}
}
}
System.out.println(" ---------- Process Completed Successfully ----------");
System.out.println("\nCheck your table \"" + tableName + "\" for results");
System.out.print("\n\nPress 1 to finish : ");
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
int ch = Integer.parseInt(br.readLine());
} catch (Exception e)
{
e.printStackTrace();
}
finally
{
try
{
BSC.connectionClose();
}
catch (SQLException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
The OpenNLP documentation is unclear, but there is a simple way of doing this.
private DoccatModel deSerializeModel(String modelPath) throws
IOException {
FileInputStream fileInputStream = new FileInputStream(modelPath);
DoccatModel model = new DoccatModel(fileInputStream);
return model;
}

splitting of csv file based on column value in java

I want to split csv file into multiple csv files depending on column value.
Structure of csv file: Name,Id,Dept,Course
abc,1,CSE,Btech
fgj,2,EE,Btech
(Rows are not separated by ; at end)
If value of Dept is CSE or ME , write it to file1.csv, if value is ECE or EE write it to file2.csv and so on.
Can I use drools for this purpose? I don't know drools much.
Any help how it can be done?
This is what I have done yet:
public void run() {
String csvFile = "C:/csvFiles/file1.csv";
BufferedReader br = null;
BufferedWriter writer=null,writer2=null;
String line = "";
String cvsSplitBy = ",";
String FileName = "C:/csvFiles/file3.csv";
String FileName2 = "C:/csvFiles/file4.csv";
try {
writer = new BufferedWriter(new FileWriter(FileName));
writer2 = new BufferedWriter(new FileWriter(FileName2));
br = new BufferedReader(new FileReader(csvFile));
while ((line = br.readLine()) != null) {
String[] values=line.split(cvsSplitBy);
if(values[2].equals("CSE"))
{
writer.write(line);
}
else if(values[2].equals("ECE"))
{
writer2.write(line);
}
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (br != null) {
try {
br.close();
writer.flush();
writer.close();
writer2.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
1) First find column index using header row or if header is not present then by index
2) Follow below algorithm which will result map of key value where key is column by which split is performed
global resultMap;
Method add(key,row) {
data = (resultMap.containsKey(key))? resultMap.get(key):new ArrayList<String>();
data.add(row);
resultMap.put(key, data );
}
Method getSplittedMap(List rows) {
for (String currentRow : rows) {
add(key, currentRow);
}
return resultMap;
}
hope this helps.
FileOutputStream f_ECE = new FileOutputStream("provideloaction&filenamehere");
FileOutputStream f_CSE_ME = new FileOutputStream("provideloaction&filenamehere");
FileInputputStream fin = new FileinputStream("provideloaction&filenamehere");
int size = fin.available(); // find the length of file
byte b[] = new byte[size];
fin.read(b);
String s = new String(b); // file copied into string
String s1[] = s.split("\n");
for (int i = 0; i < s1.length; i++) {
String s3[] = s1[i].split(",")
if (s3[2].equals("ECE"))
f_ECE.write(s1.getBytes());
if (s3[2].equals("CSE") || s3.equals("EEE"))
f_CSE_ME.write(payload.getBytes());
}

Multiple Audio files won't concatenate

I'm trying to create a simple recorder which gives 'pause' and 'resume' functionality to the user.
Since Android does not support this directly, I'm creating individual files whenever the user presses 'Pause' and 'Resume' with the suffixes _1, _2, so on.
I use the code below to concatenate them
public void mergeAllAndSave() {
// TODO Auto-generated method stub
Enumeration<FileInputStream> allRecordings;
Vector<FileInputStream> audiofiles = new Vector<FileInputStream>();
for (int i = 1; i < count+1; i++) {
try {
audiofiles.add(new FileInputStream(Environment.getExternalStorageDirectory().getPath() + "/"+ AUDIO_RECORDER_FOLDER + "/" + _filename + "_"+ i + file_exts[currentFormat]));
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
allRecordings = audiofiles.elements();
SequenceInputStream siStream = new SequenceInputStream(allRecordings);
try {
FileOutputStream foStream = new FileOutputStream(Environment.getExternalStorageDirectory().getPath() + "/"+ AUDIO_RECORDER_FOLDER + "/" + _filename + file_exts[currentFormat]);
int temp;
while ((temp = siStream.read() ) != -1) {
foStream.write(temp);
}
foStream.close();
siStream.close();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
The code works fine. It gives me a single file. However it contains the contents of the first file only. Logcat does not show any errors, whatsoever.
Anyone with any ideas what is the mistake I am making?
Thanks.
Answer to this question is here.
PS: I cannot add this as a comment because I do not have sufficient reputation.

Categories