I'm designing an Android application that would rely heavily on natural language processing for its purposes. I selected OpenNLP since it seems to offer what I need to offer, made a few classes to encapsulate tokenization, pos tagging, etc, and tested them out in a standard java setting with no issues.
My problem seems to be with the Android File system. OpenNLP calls for a training file to initialize the data model behind each class. However, the constructors for these classes seem to take in a very specific InputStream, as when I manage to successfully reference these files, I either get an error about the access permissions (I've added permissions for reading and writing from/to external storage), or an error stating that "The profile data stream has an invalid format!"
I'm at a loss, as using the standard input stream methods provided by the Android context class doesn't work as the provided input streams are of an invalid format, and attempting to manually access the files using my own input streams brings up permission problems. I've even tried loading the files at run time from the res folder into another file, and then re loading it using a normal FileInputStream, but this once again brings me to the invalid format problem.
Below is the method used to access the files, and an example method for initializing one of the models (they're all fairly uniform). If anybody has an idea what's going on, or if anybody has gotten OpenNLP to work in the Android environment, a little help would be greatly appreciated!
File Access Method:
protected FileInputStream importIfNotExists(){
FileInputStream input = null;
if(mContext != null){
File file = new File(getDirectory(), getFilePath());
if(file.exists()){ //Create input stream from file.
try {
Log.d("Analysis Tool", "Accessing file");
//Crashes here if it exists
input = new FileInputStream(file);
}
catch (FileNotFoundException e) {
Log.d("Speech Analysis Tool", "File not found: " + getFilePath());
input = null;
}
}
else{ //Import resource file, then get input stream
InputStream stream = null;
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
int sample = 0;
try {
Log.d("Analysis Tool", "Loading raw resource");
stream = mContext.getResources().openRawResource(mResId);
Log.d("Analysis Tool", "Creating file to be written to.");
file.createNewFile();
Log.d("Analysis Tool", "Reading bytes from resource.");
sample = stream.read();
while(sample != -1){
bytes.write(sample);
sample = stream.read();
}
stream.close();
Log.d("Analysis Tool", "Creating file: " + getFilePath());
FileOutputStream output = new FileOutputStream(file, false);
Log.d("Analysis Tool", "Writing bytes to " + getFilePath());
bytes.writeTo(output);
bytes.close();
output.close();
Log.d("Analysis Tool", "Retrieving input stream for new file");
input = new FileInputStream(file);
//the input passed from this is typically of an invalid format
}
catch (IOException e) {
Log.d("Speech Analysis Tool", "IOException with: " + getFilePath());
Log.e("Speech Analysis Tool", e.getLocalizedMessage());
input = null;
}
}
}
return input;
}
Model Initialization:
#Override
protected void initializeTool(FileInputStream input) throws InvalidFormatException, IOException{
if(input == null){
Log.e("Speech Tokenizer", "Input stream for tokenizer is null");
return;
}
TokenizerModel model = getModel(input);
mTokenizer = new TokenizerME(model);
}
getFilePath() simply returns the filename and its file type (like en_token.bin), and getDirectory() has varied with little to no success, but is intended to be the directory on external storage where I'd either access these files, or load them in at run time.
Add this line to your code:
System.setProperty("org.xml.sax.driver","org.xmlpull.v1.sax2.Driver");
Helped me, maybe it'll help you
Related
I want to add some string data to a video file but I don't want the video file to get corrupted. What I want to achieve is :-
1.) Add text to a video file.
2.) Extract the text from that video file.
What I tried is :-
public class VideoData{
public static void main(String[] args) {
//create file object
File file = new File("I:/java/MyFolder/SmallVideo.mp4");
try
{
//create FileInputStream object
FileInputStream fin = new FileInputStream(file);
byte fileContent[] = new byte[(int)file.length()];
fin.read(fileContent);
//create string from byte array
String strFileContent = new String(fileContent);
System.out.println("File content : ");
System.out.println(strFileContent);
File dest=new File("I://java//OtherFolder//SmallVideo.mp4");
BufferedWriter bw=new BufferedWriter(new FileWriter(dest));
bw.write(strFileContent + "\nThis is my Text");
bw.flush();
}
catch(FileNotFoundException e)
{
System.out.println("File not found" + e);
}
catch(IOException ioe)
{
System.out.println("Exception while reading the file " + ioe);
}
}
}
Please Help me to do the above mentioned tasks.
An MP4 file is a container that stores properly encoded video, audio, images and subtitles. It's a binary file with standard format specification which means you cannot simply add any extra data to it. Modifying the data could corrupt the file and the decoders (simply video players) might fail to render it.
Also in your code, you read the binary data from mp4 file and converted it to String. That shouldn't be the case. A video file data must be handled in binary mode, not as text.
I didn't understand your actual goal. If you are looking to store some text in MP4 file, you could consider storing it in the meta data section of the video file. See here for an example by using a third party library.
Steganography is a technique of embedding text in images and videos. I guess that's beyond your scope.
Sample code is below. It will copy the target files and directory from one location to another. What's considered best practice for handling IO Exceptions while coping files across a network?
I used printStackTrace() but feel like this is just a place holder for a better solution. Is logging the answer and should there be another step beyond logging to actually "handle" an error?
Thank you for you feedback.
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
/**
This is a test program to copy a directory(s) & file(s) from one location to another.
*/
public class CopyTest{
public static void main(String[] args) {
//Declarations
String sourcePath = "I:\\MB\\PO";
String destPath = "C:\\testPO\\";
System.out.println("Source path: " + sourcePath);
System.out.println("Destination path: " + destPath);
File source = new File(sourcePath);
File dest = new File(destPath);
//Process
//Call to method copyUsingStream
long start = System.nanoTime(); //start recording how much time the copy takes.
copyUsingStream(source, dest); //method to copy the directory/files.
System.out.println("Time taken to copy the file: "+(System.nanoTime() -start) + " nanoseconds");
} //end main method
/**
The copyUsingStream method is a recursive method to copy folders and files from one location to another.
*/
private static void copyUsingStream(File source, File dest) {
if (!source.isDirectory()){
// If source is a file -> copy it to the new folder
InputStream inStream = null;
OutputStream outStream = null;
try {
inStream = new FileInputStream(source);
outStream = new FileOutputStream(dest);
byte[] buffer = new byte[1024];
int length;
while ((length = inStream.read(buffer)) > 0) {
outStream.write(buffer, 0, length);
}
} catch(IOException ioe) {
ioe.printStackTrace();
} finally {
try{
inStream.close();
outStream.close();
System.out.println("File copied from " + source + " to " + dest + "successfully");
} catch(IOException ioe2) {
ioe2.printStackTrace();
}
}
} else {
//If a directory -> create the directory inside the new destination
//List all contents
if (!dest.exists()) {
dest.mkdir();
System.out.println("Directory copied from " + source + " to " + dest + "successfully");
}
String folder_contents[] = source.list();
for (String file : folder_contents) {
File srcFile = new File(source, file);
File destFile = new File(dest, file);
copyUsingStream(srcFile, destFile);
}
}
} //end method copyUsingStream
} //end class CopyTest
Method without the catches:
private static void copyUsingStream(File source, File dest) throws IOException {
if (!source.isDirectory()){
// If source is a file -> copy it to the new folder
InputStream inStream = null;
OutputStream outStream = null;
try {
inStream = new FileInputStream(source);
outStream = new FileOutputStream(dest);
byte[] buffer = new byte[1024];
int length;
while ((length = inStream.read(buffer)) > 0) {
outStream.write(buffer, 0, length);
}
} finally {
inStream.close();
outStream.close();
System.out.println("File copied from " + source + " to " + dest + "successfully");
}
} else {
//If a directory -> create the directory inside the new destination
//List all contents
if (!dest.exists()) {
dest.mkdir();
System.out.println("Directory copied from " + source + " to " + dest + "successfully");
}
String folder_contents[] = source.list();
for (String file : folder_contents) {
File srcFile = new File(source, file);
File destFile = new File(dest, file);
copyUsingStream(srcFile, destFile);
}
}
} //end method copyUsingStream
That depends highly on your application.
Applications that continue running anyway (e.g. web servers, daemons and batch processors) do usually log such errors in a file together with timestamp, thread ID and possibly other helpful information.
I had very good experience with a combination of two log files.
myapp.log receives only important messages, usually warnings and errors. This file is for the regular user and system operator.
debug.log is for the developer. It provides debug messages from the time before an error occurred, but no messages as long everything works fine. To enable this, a memory buffer is required.
If you are interested in that buffer, you may take a look at http://stefanfrings.de/bfUtilities/index.html . The website is german but the library and it's documentation is english.
On a desktop GUI application, when an error aborts the requested operation, it might be nice to show a short error messages in a popup window, and hide the details (stack trace) in an expandable box. Don't forget to tell the user clearly what operation failed. The exception itself may be clear enough to you developer but regular users expect a less technical text. For example: "Loading weather info from service weather.com failed: Connection failed", followed by the stack trace.
For console applications that stop immediately, I prefer so see the stack trace directly on screen as written by printStackTrace().
As Stefan said, it depends on the application.
A good rule of thumb is: Don’t catch an exception unless you are prepared to take a specific action, beyond printing or logging it, or there is no caller to whom you can propagate it.
If you have a general method for copying a file, that method should not make assumptions about why it’s being called. Its job is to copy a file. It should return only if it succeeds in that task. If it does not succeed, it should throw an exception rather than returning.
So for a general copying method, you would want to add throws IOException to the method signature, and have zero try/catch blocks in the method itself. This lets the callers decide how to handle a failure. A GUI application might display an error dialog. A service might just log the exception and try again later.
You yourself should only catch and log an exception at the highest possible level. A GUI application would log it right before displaying the error dialog. (You might also want to include the text of the stack trace in the dialog, in an expandable “Show details” section.) A service might have a main loop or main execution method, where there is no higher caller to whom the exception can be propagated, so there is nothing to do but log it.
I am trying to send Files in fragments using DatagramPackets in Java (part of an assignemt.) When I am trying to save the incoming File I get access denied error, but I believe that it is not a permissions issue.
Here is the brunt of it:
I let the user sending the file to choose it using FileChooser. And create a new Message object.
//....
File f = content.showFileChooser();
byte type = Byte.parseByte("4");
Message m;
try {
if (mode == 1){
m = new Message(f, content.getServerData().getFragmentSize(), (short) (sentMessages.size()+1), type);
serverThread.send(m);
}
//...
During Message creation the file gets split up into byte arrays, where the size of each array is predetermined by the user. The code is quite lengthy so I am not going to post the chopping process, but this is how I convert the File object into a big byte[] which then gets chopped up
Path p = Paths.get(file.getAbsolutePath());
this.rawData = Files.readAllBytes(p);
After the Message is created and chopped up into byte arrays I send them using DatagramPackets. The other side then uses those to create a new Message object. Once all fragments arrive rawData is extracted from the Message object again. The problem believe lies here:
Message m = receivedMessages.get(msgIndex-1);
byte[] fileData = m.getFile();
if (fileData != null){
System.out.println("All file fragments received.");
content.append("Received a file in" + m.getFragmentCount()+" fragments. Choose directory. " ,1);
//I BELIEVE THIS TO BE THE CRITICAL POINT
String filePath = content.chooseDirectory();
if (filePath == null)
return;
FileOutputStream fos;
try {
fos = new FileOutputStream(filePath);
fos.write(fileData);
fos.close();
} catch (IOException e) {
e.printStackTrace();
}
}
Once all fragments arrive I let the user select a directory using FileChooser with DIRECTORY_ONLY choice mode. As I understand, FileOutputStream requires a full path for the new File. Do I have to send the file name and extension separately or can it be extracted from the received File data?
You are writing directory path to filePath, then try to open that directory with FileOutputStream. No wonder that doesn't work, you have to specify the filename too.
String filename = "myfile.txt"; //Or receive and set this some other way
File outFile = new File(filePath, filename);
fos = new FileOutputStream(outFile);
I don't see you sending/receiving the filename anywhere, though. You'll need to either make it constant or transfer it along with file contents.
I have a BufferedWriter which is being used to write to a file which has just been created in the given directory, however, for some reason it is not writing the text that it reads from another file, here is my code:
private static final String tempFileDir = System.getProperty("user.dir") + "/TempATM.txt";
File tempFile = new File(tempFileDir); //Create temporary file to write new info to
File toRenameTo = new File("VirtualATM.txt"); //filename to rename temp file to
if (!tempFile.exists() && !tempFile.isDirectory()) {
tempFile.createNewFile(); //Create temp file if it doesn't already exist.
}
FileOutputStream fos = new FileOutputStream(tempFile, true); //For writing new balance
Writer bw = new BufferedWriter(new OutputStreamWriter(fos, "UTF8"));//For writing new balance
String newLineRead = null;
FileReader fileReader = new FileReader("VirtualATM.txt");//for reading from file
BufferedReader newBufferedReader = new BufferedReader(fileReader);//for reading from file
while((newLineRead = newBufferedReader.readLine()) != null){
if(!newLineRead.contains(cardNumberStr)){
bw.append(newLineRead); //If the line does not contain user entered card number, write line to new file.
((BufferedWriter) bw).newLine();
}else if(newLineRead.contains(cardNumberStr)){
bw.append(newAccountDetails); //Write updated account details if the line read contains users account number
((BufferedWriter) bw).newLine();
}
}
File toDeleteFile = new File("dirToWriteFile"); //File path to delete the file.
if(!toDeleteFile.delete()){
JOptionPane.showMessageDialog(null, "FATAL ERROR! Could not delete VirtualATM.txt", "Error", JOptionPane.ERROR_MESSAGE); // for if there is an error when deleting file
}
if(!file.renameTo(toRenameTo)){
JOptionPane.showMessageDialog(null, "FATAL ERROR! Could not rename the file to VirtualATM.txt", "Error", JOptionPane.ERROR_MESSAGE);//for if there is an error renaming file
}
Edit:
I am also having trouble deleting and renaming the text file, could any suggest what may be causing this problem, what SecurityExceptions etc. may be preventing Java from deleting and renaming a text file (.txt) on Windows 8.1?
You need to either flush the buffer post writing the data to buffer like
bw.flush();
or close the writer like
bw.close();//handle exception if you are not using AutoCloseable feature.
You must either flush the buffer to the disk after writing the data using:
bw.flush();
or / and if you have finished writing the data, you must always close the writer which will automatically flush the data to the disk before closing using:
bw.close();
Hope this helps. Good luck and have fun programming!
Cheers,
Lofty
From a third party I am retrieving .zip files. I want to unzip these to another folder. To this end I found a method that does exactly that, see code below. It iterates through all files and unzips them to another folder. However, when observing the corresponding compression method I found out that this changes for some files. And for some files it states: "invalid compression method", after which it aborts further unzipping of the zip file.
As the compression method seems to change, I suspect I need to set the compression method to the correct one (however that might be a wrong assumption). So rises my question: how to determine the compression method needed?
The code I am using:
public void unZipIt(String zipFile, String outputFolder){
//create output directory is not exists
File folder = new File(OUTPUT_FOLDER);
if(!folder.exists()){
folder.mkdir();
}
FileInputStream fis = null;
ZipInputStream zipIs = null;
ZipEntry zEntry = null;
try
{
fis = new FileInputStream(zipFile);
zipIs = new ZipInputStream(new BufferedInputStream(fis));
while((zEntry = zipIs.getNextEntry()) != null){
System.out.println(zEntry.getMethod());
try{
byte[] tmp = new byte[4*1024];
FileOutputStream fos = null;
String opFilePath = OUTPUT_FOLDER + "\\" + zEntry.getName();
System.out.println("Extracting file to "+opFilePath);
fos = new FileOutputStream(opFilePath);
int size = 0;
while((size = zipIs.read(tmp)) != -1){
fos.write(tmp, 0 , size);
}
fos.flush();
fos.close();
} catch(IOException e){
System.out.println(e.getMessage());
}
}
zipIs.close();
} catch (FileNotFoundException e) {
System.out.println(e.getMessage());
}
catch(IOException ex){
System.out.println(ex.getMessage());
}
}
Currently I am retrieving the following output:
8
Extracting file to C:\Users\nlmeibe2\Documents\Projects\Output_test\SOPHIS_cptyrisk_tradedata_1192_20140616.csv
8
Extracting file to C:\Users\nlmeibe2\Documents\Projects\Output_test\SOPHIS_cptyrisk_underlying_1192_20140616.csv
0
Extracting file to C:\Users\nlmeibe2\Documents\Projects\Output_test\10052013/
12
Extracting file to C:\Users\nlmeibe2\Documents\Projects\Output_test\MRM_Daily_Position_Report_Package_Level_Underlying_View_EQB_v2_COBDATE_2014-06-16_RUNDATETIME_2014-06-17-04h15.csv
invalid compression method
invalid compression method
Since you only print the exception message and not the stack trace (with line numbers), it is impossible to know exactly where the exception is thrown, but I suppose it is not thrown until you actually try to read from the ZipEntry.
If the numbers in your output is the ZIP method, the last entry you encounter is compressed with method 12 (bzip2), which is not supported by the Java ZIP implementation. PKWare (the maintainers of the ZIP format) regularly add new compression methods to the ZIP specification and there are currently some 12-15 (not sure about the exact number) compression methods specified. Java only supports the methods 0 (stored) and 8 (deflated) and will throw an exception with the message "invalid compression method" if you try to decompress a ZIP file using an unsupported compression method.
Both WinZip and the ZIP functions in Windows may use compression methods not supported by the Java API.
Use zEntry.getMethod() to get the compression method
Returns the compression method of the entry, or -1 if not specified.
It will return an int which will be
public static final int STORED
public static final int DEFLATED
or -1 if it don't know the method.
Docs.