text file limitation in Android? - java

I am trying to read a wav file into an array and then write it on a text file using android application. To validate that the values on the text file are correct, I compare it with Matlab.
I used the same code in Android and Java, and in both cases the results are different, by different I mean that Android saves only 5946 readings out of 20000 readings. On the other hand, when I run the Java code, I get the full 20000 readings !
I have no idea why I get 5946 readings when I run the Android application, while I get 20000 readings when I run the java code. Is there any limitation on the text file size?
Following is the code I wrote (plz note that I use a library for wav file reading):
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
/////////////////////
Done = (TextView) findViewById(R.id.textView1);
Done.setText("processing");
double[] buffer;
int len;
///Reading the wav file into buffer/////
filePath = "mnt/sdcard";
File filein = new File(filePath, "audio.wav");
try
{
// Open the wav file specified as the first argument
WavFile wavFile = WavFile.openWavFile(filein);
// Display information about the wav file
wavFile.display();
// Get the number of audio channels in the wav file
int numChannels = wavFile.getNumChannels();
// Create a buffer of 100 frames
buffer = new double[20000 * numChannels];
int framesRead;
double min = Double.MAX_VALUE;
double max = Double.MIN_VALUE;
do
{
// Read frames into buffer
framesRead = wavFile.readFrames(buffer, 20000);
// Loop through frames and look for minimum and maximum value
for (int s=0 ; s<framesRead * numChannels ; s++)
{
if (buffer[s] > max) max = buffer[s];
if (buffer[s] < min) min = buffer[s];
}
len=buffer.length;
}
while (framesRead != 0);
// Close the wavFile
wavFile.close();
// Output the minimum and maximum value
System.out.printf("Min: %f, Max: %f\n", min, max);
//////////////Saving the array (buffer) into a text file (before.txt)////////
filePath = "mnt/sdcard";
for (int i=0; i<20000; i++)
{
if(fout==null)
try {
fout=new DataOutputStream(new FileOutputStream(new File(filePath,"before.txt")));
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
String str=Double.toString(buffer[i])+" " ;
//Log.v(str,Double.toString(i));
try {
fout.writeBytes(str);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
catch (Exception e)
{
System.err.println(e);
}
if(fout!=null)
{
try {
fout.flush();
fout.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Done.setText("Done");
}

I found that the size of the text file on the smartphone is around 300 KB, However, when I browse it from the PC it is 100KB !
It seems that the system keeps the same size every time I write on it !
Reboot of the phone solved the issue. (Android MTP support does not show recent files until the device is rebooted).
https://code.google.com/p/android/issues/detail?id=38282

Related

App freezes on including a function( ) that writes file from buffer

I am a beginner in android development and I am stuck. Here, I am recording voice using audiorecorder instead of mediarecorder to use noise cancellation feature and I am writing a PCM file from the buffer to an output file and when I include the function for writing the file from the buffer the app just don't respond. it records and saves the file but freezes.
can somebody tell me what's wrong?
private void writeAudioDataToFile() {
// Write the output audio in byte
bufferSizeInBytes = AudioRecord.getMinBufferSize(
RECORDER_SAMPLERATE,
RECORDER_CHANNELS,
RECORDER_AUDIO_ENCODING
);
String filePath = "/sdcard/voice8K16bitmono.wav";
short sData[] = new short[bufferSizeInBytese/2];
FileOutputStream os = null;
try {
os = new FileOutputStream(filePath);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
while (isRecording) {
// gets the voice output from microphone to byte format
ar.read(sData, 0, bufferSizeInBytese/2);
Log.d("eray","Short wirting to file" + sData.toString());
try {
// // writes the data to file from buffer
// // stores the voice buffer
byte bData[] = short2byte(sData);
os.write(bData, 0, bufferSizeInBytes);
} catch (IOException e) {
e.printStackTrace();
}
}
try {
os.close();
} catch (IOException e) {
e.printStackTrace();
}
}
private byte[] short2byte(short[] sData) {
int shortArrsize = sData.length;
byte[] bytes = new byte[shortArrsize * 2];
for (int i = 0; i < shortArrsize; i++) {
bytes[i * 2] = (byte) (sData[i] & 0x00FF);
bytes[(i * 2) + 1] = (byte) (sData[i] >> 8);
sData[i] = 0;
}
return bytes;
}
I think, you are doing file writing operation in main thread (UI thread), that's why app freezes. Kindly do your file operation in another thread Eg. AsyncTask.
Keep in mind that "you do not do the operations which takes more than 5seconds & freezes app".
Here, just sharing the overview of concept to be added:
Add this innerclass inside your activity file.
static class FileAsyncTask extends AsyncTask{
#Override
protected Object doInBackground(Object[] objects) {
// do your file writing stuff here....
return null;
}
#Override
protected void onProgressUpdate(Object[] values) {
super.onProgressUpdate(values);
// update UI - if the file writig is success / failed
}
}
call this async task in your activity file when file need to be written:
new FileAsyncTask().execute(...);

Java OutOfMemoryError while merge large file parts from chunked files

I have a problem when the user upload large files (> 1 GB) (I'm using flow.js library), it creates hundred of thousand small chunked files (e.g 100KB each) inside temporary directory but failed to merge into single file, due to MemoryOutOfException. This is not happened when the file is under 1 GB. I know it sound tedious and you probably suggest me to increase the XmX in my container-but I want to have another angle besides that.
Here is my code
private void mergeFile(String identifier, int totalFile, String outputFile) throws AppException{
File[] fileDatas = new File[totalFile]; //we know the size of file here and create specific amount of the array
byte fileContents[] = null;
int totalFileSize = 0;
int filePartUploadSize = 0;
int tempFileSize = 0;
//I'm creating array of file and append the length
for (int i = 0; i < totalFile; i++) {
fileDatas[i] = new File(identifier + "." + (i + 1)); //indentifier is the name of the file
totalFileSize += fileDatas[i].length();
}
try {
fileContents = new byte[totalFileSize];
InputStream inStream;
for (int j = 0; j < totalFile; j++) {
inStream = new BufferedInputStream(new FileInputStream(fileDatas[j]));
filePartUploadSize = (int) fileDatas[j].length();
inStream.read(fileContents, tempFileSize, filePartUploadSize);
tempFileSize += filePartUploadSize;
inStream.close();
}
} catch (FileNotFoundException ex) {
throw new AppException(AppExceptionCode.FILE_NOT_FOUND);
} catch (IOException ex) {
throw new AppException(AppExceptionCode.ERROR_ON_MERGE_FILE);
} finally {
write(fileContents, outputFile);
for (int l = 0; l < totalFile; l++) {
fileDatas[l].delete();
}
}
}
Please show the "inefficient" of this method, once again... only large files that cannot be merge using this method, smaller one ( < 1 GB) no problem at all....
I appreciate if you do not suggest me to increase the heap memory instead show me the fundamental error of this method... thanks...
Thanks
It's unnecessary to allocate the entire file size in memory by declaring a byte array of the entire size. Building the concatenated file in memory in general is totally unnecessary.
Just open up an outputstream for your target file, and then for each file that you are combining to make it, just read each one as an input stream and write the bytes to outputstream, closing each one as you finish. Then when you're done with them all, close the output file. Total memory use will be a few thousand bytes for the buffer.
Also, don't do I/O operations in finally block (except closing and stuff).
Here is a rough example you can play with.
ArrayList<File> files = new ArrayList<>();// put your files here
File output = new File("yourfilename");
BufferedOutputStream boss = null;
try
{
boss = new BufferedOutputStream(new FileOutputStream(output));
for (File file : files)
{
BufferedInputStream bis = null;
try
{
bis = new BufferedInputStream(new FileInputStream(file));
boolean done = false;
while (!done)
{
int data = bis.read();
boss.write(data);
done = data < 0;
}
}
catch (Exception e)
{
//do error handling stuff, log it maybe?
}
finally
{
try
{
bis.close();//do this in a try catch just in case
}
catch (Exception e)
{
//handle this
}
}
}
} catch (Exception e)
{
//handle this
}
finally
{
try
{
boss.close();
}
catch (Exception e) {
//handle this
}
}
... show me the fundamental error of this method
The implementation flaw is that you are creating a byte array (fileContents) whose size is the total file size. If the total file size is too big, that will cause an OOME. Inevitably.
Solution - don't do that! Instead "stream" the file by reading from the "chunk" files and writing to the final file using a modest sized buffer.
There are other problems with your code too. For instance, it could leak file descriptors because you are not ensure that inStream is closed under all circumstances. Read up on the "try-with-resources" construct.

multi threaded downloader in java

OK I am using the following function to create multiple threads to download a file. You can see the functions takes link, starting byte, ending byte and the path to download the file as argument. I call this function 2 times to create two threads to download the required file.
For example, if the file is of 100 bytes I do the following
thread-1 --> DownloadFile("http://localhost/file.zip", 0, 50, "output.zip");
thread-2 --> DownloadFile("http://localhost/file.zip", 50, 100, "output.zip");
But you know what happens, only a few bytes don't get downloaded and my progress bar gets stuck at 99%. That's the problem!!!
Why it gets stuck at 99%? In words why some bytes are being lost? I could see the total number of bytes in the downloaded variable.
Here is the function
public void DownloadFile(final String link, final long start,final long end, final String path){
new Thread(new Runnable(){
public void run(){
try {
URL url = new URL(link);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestProperty("Range", "bytes="+start+"-"+end);
BufferedInputStream bis = new BufferedInputStream(conn.getInputStream());
RandomAccessFile raf = new RandomAccessFile(path,"rw");
raf.seek(start);
int i=0;
byte bytes[] = new byte[1024];
while((i = bis.read(bytes))!=-1){
raf.write(bytes, 0, i);
downloaded = downloaded+i;
int perc = (int) ((downloaded*100)/FileSize);
progress.setValue(perc);
percentLabel.setText(Long.toString(downloaded)+" out of "+FileSize);
}
if(FileSize==downloaded){
progress.setValue(100);
JOptionPane.showMessageDialog(null, "Download Success! ");
progress.setValue(0);
downloaded=0;
downBtn.setText("Download");
}
bis.close();
raf.close();
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}).start();
}
Thanks in anticipation.
RandomAccessFile is not thread safe.
raf.seek(begin) fails, see the documentation of RandomAccessFile.seek()
Sets the file-pointer offset, measured from the beginning of this
file, at which the next read or write occurs. The offset may be set
beyond the end of the file. Setting the offset beyond the end of the
file does not change the file length. The file length will change only
by writing after the offset has been set beyond the end of the file.
You may download parts of file into separate files then merge them.
Are you sure that parallel downloads are faster?

java playing sounds in order

Hi I have following java programme that play some sounds.I want to play sounds in order for example after ending of sound1 i want to play sound2 and then sound3 the following is my java code and function of playing sound .
private void playsound(String file)
{
try {
crit = AudioSystem.getClip();
AudioInputStream inputStream1 = AudioSystem.getAudioInputStream(this.getClass().getResource(file));
crit.open(inputStream1);
//if(!crit.isOpen())
{
crit.start();
}
} catch (Exception ex) {
System.out.println(ex.getMessage());
}
}
and calling it as following
playsound("/sounds/filesound1.au");
playsound("/sounds/filesound2.au");
playsound("/sounds/filesound3.au");
the programme is plying sound in parallel which I don't want.I want to play in order
Regards
I got the following code from somewhere that I can't remember right now but it plays the music consequently:
public static void play(ArrayList<String> files){
byte[] buffer = new byte[4096];
for (String filePath : files) {
File file = new File(filePath);
try {
AudioInputStream is = AudioSystem.getAudioInputStream(file);
AudioFormat format = is.getFormat();
SourceDataLine line = AudioSystem.getSourceDataLine(format);
line.open(format);
line.start();
while (is.available() > 0) {
int len = is.read(buffer);
line.write(buffer, 0, len);
}
line.drain();
line.close();
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
The reason this plays the files consequently and not all at the same time is because write blocks until the requested amount of data has been written. This applies even if the requested amount of data to write is greater than the data line's buffer size.
Make sure to include drain() from the code above. drain() waits for the buffer to empty before it close()s.

Split audio file (mp3, wav, wma) into 1s chunks

I am getting the FFT back on an audio file, but it doesn't take into account the time in the song that that frequency occurred. I first tried getting the length of the file, and then spreading the FFT results equally over the track length, but that might be wrong and not give the correct frequencies back. So now I am trying to get the file split up into 1 second chunks and then return the frequency for that second alone, and then I will store that in a database to save it.
But I have no clue on how to save it, all other threads I have found and research I have done only shows how to break into x amount of parts, not per second as in a song. Is there a way to do this?
Sorry if this is a trivial topic, but I am very new to Java and programming, so this is quite a struggle for me.
Thanks in advance
Here is my code so far:
File file = new File(FILENAME);
float durationInSeconds = 0;
Tag tag;
java.util.logging.Logger.getLogger("org.jaudiotagger").setLevel(Level.OFF);
AudioFile audioFile;
try {
audioFile = AudioFileIO.read(file);
System.out.println("Track length = " + audioFile.getAudioHeader().getTrackLength());
durationInSeconds = audioFile.getAudioHeader().getTrackLength();
} catch (CannotReadException | TagException | ReadOnlyFileException
| InvalidAudioFrameException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
try {
convert();
System.out.println(((durationInSeconds)/endResult.length)*1000);
for(int i = 0; i < endResult.length; i++) {
Thread.sleep((long) (((durationInSeconds)/endResult.length)*1000));
System.out.println(endResult[i]);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

Categories