We Keep Coding

When i try to delete redundant file even after closing all the references jvm throws the FileSystemException

I'm creating a video record file (avi) with javacv after creation converting this file to an mp4 file with Jaffree. When i try to delete the now redundant avi file it throws the FileSystemException.
I've already released all the file holding Objects but still the exception is being thrown. Also tried to force the deletion with apache common io FileUtils.forceDelete(sample.getFile()); still having the same Exception.
public class PacketRecorderTest {
private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyyMMdd__hhmmSSS");
private static final int RECORD_LENGTH = 10000;
private static final boolean AUDIO_ENABLED = false;
static String inputFile = "rtsp://184.72.239.149/vod/mp4:BigBuckBunny_115k.mov";
static String outputFile = "C:\\Users\\team3\\Desktop\\User\\Vinay\\javacppffmpeg\\frame.avi";
public static void main(String[] args) throws FrameRecorder.Exception, FrameGrabber.Exception, InterruptedException{
packetRecord(inputFile,outputFile);
jafreeToMp4(outputFile, "C:\\Users\\team3\\Desktop\\User\\Vinay\\javacppffmpeg\\frame.mp4");
try {
Files.delete(Paths.get(outputFile));
} catch (IOException e) {
e.printStackTrace();
}
}
public static void packetRecord(String inputFile, String outputFile) throws FrameGrabber.Exception, FrameRecorder.Exception {
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(inputFile);
grabber.start();
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(
outputFile,
1280,
720);
recorder.start(grabber.getFormatContext());
recorder.setFormat("avi");
recorder.setPixelFormat(AV_PIX_FMT_YUV420P);
recorder.setVideoOption("crf", "22");
recorder.setVideoQuality(0);
recorder.setFrameRate(15);
recorder.start();
avcodec.AVPacket packet;
long t1 = System.currentTimeMillis();
while ((packet = grabber.grabPacket()) != null) {
recorder.recordPacket(packet);
if ((System.currentTimeMillis() - t1) > RECORD_LENGTH) {
break;
}
}
grabber.stop();
recorder.stop();
recorder.release();
grabber.release();
}
public static void jafreeToMp4 (String inputFile, String outputFile) {
Path BIN = Paths.get("C:\\ffmpeg\\shared\\bin");
Path VIDEO_MP4 = Paths.get(inputFile);
Path OUTPUT_MP4 = Paths.get(outputFile);
FFmpegResult result = FFmpeg.atPath(BIN)
.addInput(UrlInput.fromPath(VIDEO_MP4))
.addOutput(UrlOutput.toPath(OUTPUT_MP4)
)
.execute();
}
}
java.nio.file.FileSystemException: C:\Users\team3\Desktop\User\Vinay\javacppffmpeg\frame.avi: The process cannot access the file because it is being used by another process.
at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
at java.nio.file.Files.delete(Files.java:1126)
at playground.PacketRecorderTest.main(PacketRecorderTest.java:39)

I can't tell for sure, as I don't have immediate access to the libraries you're using. However, it looks to me as though you still have open InputStreams and OutputStreams. My suggestions would look something like this, depending on the return type of the helper methods:
try(InputStream in = UrlInput.fromPath(VIDEO_MP4);
OutputStream out = UrlOutput.toPath(OUTPUT_MP4)) {
FFmpegResult result = FFmpeg.atPath(BIN)
.addInput(in)
.addOutput(out)
.execute();
}
This will close the resources after the statement is complete.

Was calling the start() method twice recorder.start(grabber.getFormatContext()); and start(). So may the first call may have left the references untracked.

Android Track progress of Azure CloudBlockBlob upload

How can I print the number of bytes that have been uploaded after calling blob.upload(new FileInputStream(imageFile), imageFile.length()); I want to log something like "100/totalBytes bytes have been uploaded, 224/totalBytes bytes have been uploaded..." So I can create a progress bar of the upload progress.
this is the code:
//AzureBlobLoader extends AsyncTask
public class AzureBlobUploader extends AzureBlobLoader {
private Activity act;
private String userName;
private TaggedImageObject img;
private Fragment histFragment;
public AzureBlobUploader(Fragment f, Activity act, String userName, TaggedImageObject img) {
super();
this.act = act;
this.userName = userName;
this.img = img;
this.histFragment = f;
}
#Override
protected Object doInBackground(Object[] params) {
File imageFile = new File(this.img.getImgPath());
try {
// Define the path to a local file.
final String filePath = imageFile.getPath();
// Create or overwrite the blob with contents from the local file.
String[] imagePathArray = filePath.split("/");
String imageName = imagePathArray[imagePathArray.length-1];
System.out.println("Image Name: " + imageName);
String containerName = userName + "/" + imageName;
System.out.println("Container Name: " + containerName);
CloudBlockBlob blob= this.getContainer().getBlockBlobReference(containerName);
//UPLOAD!
blob.upload(new FileInputStream(imageFile), imageFile.length());
//-----DATABASE-----//
//create client
this.setDBClient(
new MobileServiceClient(
"URL",
this.act.getApplicationContext()
)
);
this.setImageTable(this.getDBClient().getTable(Image.class));
this.setIcavTable(this.getDBClient().getTable(ICAV.class));
//IMG TABLE QUERY
String validImageID = containerName.replace("/", "_");
Log.d("Azure", "Valid Image ID: " + validImageID);
Image img = new Image(validImageID, this.img.getUser(), this.img.getLat(), this.img.getLon());
this.getImageTable().insert(img);
for(String context : this.img.getContextAttributeMap().keySet()){
Map<String,String> attributeValueMap = this.img.getContextAttributeMap().get(context);
for(String attribute : attributeValueMap.keySet()){
String value = attributeValueMap.get(attribute);
ICAV icavRow = new ICAV();
icavRow.setImageID(validImageID);
icavRow.setContextID(context);
icavRow.setAttributeID(attribute);
icavRow.setValue(value);
this.getIcavTable().insert(icavRow);
}
}
} catch (Exception e) {
System.out.println(e.toString());
}
return null;
}
#Override
protected void onProgressUpdate(Object... object) {
super.onProgressUpdate(object);
Log.d("progressUpdate", "progress: "+((Integer)object[0] * 2) + "%");
}
#Override
protected void onPostExecute(Object o) {
// to do
}
}

As you can see the Azure SDK doesn't directly allow for that, but it should be fairly easy to wrap your inputstream in another input stream that can give callbacks for bytes read. Something like that:
public class ListenableInputStream extends InputStream {
private final InputStream wraped;
private final ReadListener listener;
private final long minimumBytesPerCall;
private long bytesRead;
public ListenableInputStream(InputStream wraped, ReadListener listener, int minimumBytesPerCall) {
this.wraped = wraped;
this.listener = listener;
this.minimumBytesPerCall = minimumBytesPerCall;
}
#Override
public int read() throws IOException {
int read = wraped.read();
if (read >= 0) {
bytesRead++;
}
if (bytesRead > minimumBytesPerCall || read == -1) {
listener.onRead(bytesRead);
bytesRead = 0;
}
return read;
}
#Override
public int available() throws IOException {
return wraped.available();
}
#Override
public void close() throws IOException {
wraped.close();
}
#Override
public synchronized void mark(int readlimit) {
wraped.mark(readlimit);
}
#Override
public synchronized void reset() throws IOException {
wraped.reset();
}
#Override
public boolean markSupported() {
return wraped.markSupported();
}
interface ReadListener {
void onRead(long bytes);
}
}
minimumBytesPerCall should be initialised with some sensible number, as you probably don't want to be called on every single byte, maybe every half a megabyte should be good.
And remember that this all gets called on the doInBackground thread, so act accordingly.
edit:
I've edited the class above, there was a small error on computing the bytesRead value.
The official documentation explains your follow-up questions https://developer.android.com/reference/java/io/InputStream.html#read()
Reads the next byte of data from the input stream
So read() reads 1 byte of data (or return -1) if reached the end. So yes, it must be called several several times to read a whole image.
Then the method onRead(long) get's called every time at least minimumBytesPerCall have been read (that's to avoid of calling back for every single byte) and once more at the end of the stream (when it returns -1)
The value passed to onRead(long) is the amount that have been read since the last call. So implementing this on your AsyncTask you would have to accumulate this value and compare with the total size of the file.
Something like the following code inside your asynctask should work fine (assuming the Progress generic parameter is a Long):
private long fileLength;
private long totalBytes;
private final ListenableInputStream.ReadListener readListener = new ListenableInputStream.ReadListener() {
#Override
public void onRead(long bytes) {
totalBytes += bytes;
publishProgress(totalBytes);
}
};
and on inside your upload part you replace with:
FileInputStream fis = new FileInputStream(imageFile);
fileLength = imageFile.length();
ListenableInputStream lis = new ListenableInputStream(fi, readListener, 256 * 1024); // this will call onRead(long) every 256kb
blob.upload(lis, fileLength);
and as a last remark, remember that internally the CloudBlockBlob just caching the file on its own memory for later upload, or doing any other weird stuff that is out of your control. All this code does is check that the complete file was read.
happy coding!

Just another way for your needs, there is a MS blog which introduce about uploading a blob to Azure Storage with progress bar and variable upload block size. That code was written in C#, but it's very simple for reading by Java/Android Developer, I think you can easily rewrite it in Java for Android to compute the uploading processbar ratio to share via some public variables.
Hope it helps.

Unexpected output in concurrent file writing in java web

I wrote a very simple Java web application ,just included some basic function like register , sign in , changing the password and some others.
I don't use database. I just create a file in the app to record the users' information and do the database stuff.
I used JMeter to stressing the web application, especially the register interface.
The JMeter shows that the result of the 1000 thread is right
but when I look into the information.txt , which stores the users' information, it's wrong because it stores 700+ record :
but it should include 1000 record, it must be somewhere wrong
I use the singleton class to do the write/read stuff, and i add a synchronized word to the class, the insert() function which is used by register to record the register information is shown as below: (a part of it)
public class Database {
private static Database database = null;
private static File file = null;
public synchronized static Database getInstance() {
if (database == null) {
database = new Database();
}
return database;
}
private Database() {
String path = this.getClass().getClassLoader().getResource("/")
.getPath() + "information.txt";
file = new File(path);
if (!file.exists()) {
try {
file.createNewFile();
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
public void insert(String account, String password, String username) {
RandomAccessFile infoFile = null;
try {
infoFile = new RandomAccessFile(file, "rw");
String record;
long offset = 0;
while ((record = infoFile.readLine()) != null ) {
offset += record.getBytes().length+2;
}
infoFile.seek(offset);
record = account+"|"+password+"|"+username+"\r\n";
infoFile.write(record.getBytes());
infoFile.close();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (infoFile != null) {
try {
infoFile.close();
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
}
}
the question is why would this happened , the synchronized is thread safe, why i lost so many data and some blank line was inserted into it, what could I do the correct it !

You are synchronizing the getInstance() method, but not the insert() method. This makes the retrieval of the instance of Database thread-safe, but not the write operation.

How can I display list of recently opened files?

Can anyone provide me with the guidelines on how to retrieve a list of recently opened files in a folder?
In my application, I have a folder which contains .epub files.
I would like to display a sorted list of recently read/opened books by the user.

The problem is actually not so trivial.
There are two reasons - history's persistence and IO blocking operations.
In fact to ensure persistence of history we can use some solutions:
database (seems most reasonable but requires most effort)
internal file (should be the easiest way)
shared preferences
So, I used second method. In memory I keep just ArrayList<Uri>, and to save it to file I convert it to List<String>, as android.net.Uri doesn't support serialisation.
In internal memory I save serialised object using ObjectIOStream.
So, see the code:
public class FileHistory {
private static final String FILE_NAME = "file-history-v1";
private static final int HISTORY_SIZE = 20;
#NonNull
private final Context mAppContext;
// This is a executor where I can post any runnable
// and all of them will be executed in one pipeline
// keeping posting order.
#NonNull
private final OneThreadExecutor mExecutor;
#Nullable
private ArrayList<Uri> mInternalFilesHistory;
#NonNull
private MutableLiveData<List<Uri>> mFilesHistory = new MutableLiveData<>();
public FileHistory(#NonNull final Context appContext,
#NonNull final OneThreadExecutor executor) {
this.mAppContext = appContext;
this.mExecutor = executor;
loadHistory();
}
public void addEntry(#NonNull final Uri entry) {
if (mInternalFilesHistory == null) {
// The fileHistory is not ready yet.
// Schedule adding entry as next task of serial executor.
mExecutor.execute(() -> addEntry(entry));
return;
}
// Remove entry if exists and add it as first element.
CollectionUtils.removeFirst(mInternalFilesHistory, uri -> uri.equals(entry));
mInternalFilesHistory.add(0, entry);
if (mInternalFilesHistory.size() > HISTORY_SIZE) {
ArrayList<Uri> trimmed = new ArrayList<>(HISTORY_SIZE + 1);
trimmed.addAll(mInternalFilesHistory.subList(0, HISTORY_SIZE));
mInternalFilesHistory = trimmed;
}
mExecutor.execute(this::rePostHistory);
mExecutor.execute(this::saveToInternalStorage);
}
#NonNull
public MutableLiveData<List<Uri>> getFilesHistory() {
return mFilesHistory;
}
private void loadHistory() {
mExecutor.execute(this::loadFromInternalStorage);
mExecutor.execute(this::rePostHistory);
}
private void rePostHistory() {
if (mInternalFilesHistory != null) {
mFilesHistory.postValue(Collections.unmodifiableList(mInternalFilesHistory));
}
}
#SuppressWarnings("unchecked")
#WorkerThread
private void loadFromInternalStorage() {
try {
FileInputStream fis = mAppContext.openFileInput(FILE_NAME);
ObjectInputStream ois = new ObjectInputStream(fis);
ArrayList<String> entries = (ArrayList<String>) ois.readObject();
List<Uri> mapped = CollectionUtils.map(entries, Uri::parse);
if (mInternalFilesHistory == null) {
mInternalFilesHistory = new ArrayList<>(HISTORY_SIZE + 1);
} else {
mInternalFilesHistory.clear();
}
mInternalFilesHistory.addAll(mapped);
fis.close();
ois.close();
} catch (Exception ex) {
mInternalFilesHistory = new ArrayList<>(HISTORY_SIZE + 1);
}
}
#WorkerThread
private void saveToInternalStorage() {
try {
FileOutputStream fis = mAppContext.openFileOutput(FILE_NAME, Context.MODE_PRIVATE);
ObjectOutputStream oos = new ObjectOutputStream(fis);
if (mInternalFilesHistory == null) {
mInternalFilesHistory = new ArrayList<>();
}
List<String> converted = CollectionUtils.map(mInternalFilesHistory, Uri::toString);
oos.writeObject(converted);
fis.close();
oos.close();
} catch (IOException ignored) {
}
}
}
As you can see, internal storage is use to keep that file. So, there is no need to add any additional permissions.
Synchronisation is ensured by using executor which will execute all request, one by one, so even if IO will be slow order or requests will be saved.
We do not block thread with IO operations, because all operations using IO are on WorkerThread. About the result we will be notified via LiveData from android.arch.
In my opinion this is kind of the simplest solution. If we need to keep stats, dates etc. we can save List<MyHistoryEntry>, as long MyHistoryEntry will be serialisable.
As a better approach I would suggest to use database (easier migration etc.).

Does reading properties files multiple times consume lots memory?

I have a class which reads a properties file. Please see below.
The method readProperties() is called many times when the application is running, does that mean there is a memory issue here?
public class PropertyReader {
private static Properties configKeyValuePairs = null;
private static String configPropertiesFileName = "Config.properties";
static void readProperties() throws FileNotFoundException, IOException {
configKeyValuePairs = new Properties();
InputStream input = ConfigReader.class
.getResourceAsStream(configPropertiesFileName);
configKeyValuePairs.load(input);
input.close();
}
static String getUserName(){
//return user name which is from the properties file.
}
}

Assuming your properties file never changes, you can do the following:
public class MyApplicationConfiguration {
private static Properties configKeyValuePairs = new Properties();
private static String configPropertiesFileName = "Config.properties";
static {
InputStream input = null;
try {
input = MyApplicationConfiguration.class
.getResourceAsStream(configPropertiesFileName);
configKeyValuePairs.load(input);
} catch (IOException e) {
// Deal with not being able to load config, could be a fatal error!
} finally {
if (input != null) {
input.close();
}
}
}
public static String getUsername() {
// ...
}
// Implement getters for other configuration key-value pairs
// DO NOT let configKeyValuePairs be returned to anyone
}

Load the properties object once, and store it a class member.
I find it hard to believe that you will have memory issues because of it.
If you find out that you do, then you can always comeback and rethink it, but don't prematurely optimize a problem that probably doesn't exist.

Yes, there could be a very big memory problem, depending on whether or not there are calling classes that hold a reference to the newly created properties object.
Try something like this:
public class PropertyReader {
private static Properties configKeyValuePairs = null;
private static final String configPropertiesFileName = "Config.properties";
public static void readProperties() throws FileNotFoundException, IOException {
if(null == configKeyValuePairs){
InputStream input;
synchronized(PropertyReader.class){
try{
configKeyValuePairs = new Properties();
input = PropertyReader.class
.getResourceAsStream(configPropertiesFileName);
configKeyValuePairs.load(input);
}finally{
//this can still throw ioexception!
if(null != input){
input.close();
}
}
}
}
}