Android Programming: Looping Voice Recognition - java

I'm looking to make a custom Android app to help out a physically disabled person. I need to heavily rely on voice recognition. The idea would be:
the app (via voice recognition) accepts his speech
the app parses his speech and executes his commands ("email dad", "text fred", "what time is it?", etc, plus other things I'll be adding to control his tv, lights, etc)
after execution, the app waits for his next command (loop back to #1)
I have #1 & #2 working fine, but I can't figure out a good method for #3. I can't leave Google's voice recognition view running indefinitely because it could be hours before the next command. But the 'trigger' does have to be based on sound/voice. Has to be completely hands free.
Ideally, the app would just listen for a sound, and if that sound is above a certain pre-programmed decibel level, I'd start voice recognition. Is it possible to constantly loop just listening for a noise, then react if it's 'so' loud?
Any ideas?
Thanks

So the idea is that you want voice recognition to be ongoing. It is very hard to do that. But the way that I accomplished this is by using:
try {
Thread.sleep(4500);
mSpeechRecognizer.startListening(mSpeechRecognizerIntent);
} catch (InterruptedException e) {
// It depends on your app logic what to do with InterruptedException
// You can process it or rethrow or restore interrupted flag
}
After the onReady (so if the user doesn't say something around 4.5 seconds it resets and at results. This worked great for me.

CMUSphinx is a great solution for this:
You can easily listen continuously, a voice detection API is provided
You can increase accuracy for a custom set of commands
You can adapt the model to the user voice thus increasing accuracy significantly
For more details on using CMUSphinx on Android see
http://cmusphinx.sourceforge.net/2011/05/building-pocketsphinx-on-android/

Related

Java JSyn Loopstation

I'm trying to program a loopstation program in JSyn inspired by something like this. The idea is, that I can record to Wav-files and play them from the program. That worked well until I attempted to do this for multiple files simultaneously.
How do I approach this?
I cannot create multiple synthesizer engines otherwise I get an error, so I have created a class with my line out and my synthesizer. But if the audio only plays while the synthesizer sleeps, how can I play from multiple files at once?
System.out.println("queue the sample");
samplePlayer.dataQueue.queue(sample);
System.out.println("queued the sample");
// Wait until the sample has finished playing.
do {
mySynth.sleepFor(1); //synth has to sleep while sample is being played
} while (samplePlayer.dataQueue.hasMore()); //this seems to always return true -> synth never wakes up & the program crashes
This is adapted from the examples included in the JSyn library. I have based most of my own coding on the JSyn Programmer's Guide
This is what the GUI looks like (programmed in Java Swing). It responds to the mouse and the numpad. This works.
The constructor of my output class. This contains the synthesizer and line out.
public OutputMix() {
filePath = sampleMachineFrame.filePath; //string containing path to location for sample files
mySynth = JSyn.createSynthesizer();
myLineOut = new LineOut();
mySynth.add(myLineOut);
recorder = new RecordMic[10]; //one recorder for each button (my own class)
player = new PlayFromWav[10]; //one player for each button (my own class)
}
The recording works absolutely fine. I can even start overlapping recordings (i.e. record to two files at once) and play them with an external program.
But when I try to play them back the synthesizer never wakes up and I am also struggling to imagine how I would play multiple files at once.
Thanks for your help :)
In general, you do not want to sleep in your program unless you are sequencing events. Just queue samples in response to Button events and let them run.
If the files are small enough then you can just load them into memory. Then you can queue multiple files any time you want. You might need to increase the heap size of your Java virtual machine.
Here is an example: https://github.com/philburk/jsyn/blob/master/tests/com/jsyn/examples/PlaySample.java
If the samples are too big then you will have to stream them off the disk using multiple threads, which is more difficult.
You can make all the samples the same size. Then they will stay in phase. Or you can trigger them at specific beats using timestamps.

Syncing Two Metronomes In Java using Bluetooth for Android

I have a bit of a unique issue. I'm collaborating with several other Computer Science Majors at my university on an Android Metronome app that allows two users with the app to sync metronomes. Right now, I can send the desired beats per minute of the metronome from one phone to the other and both phones will start to play their respective metronomes. Due to the latency of Bluetooth, the metronomes are only in sync about 20% of the time.
So, here's where the problem is. We're trying to make it so both of the metronomes will start at the exact same time. One way I've thought of doing this is once the first user presses the send button on their phone, a time stamp will be created. This time stamp will be exactly two seconds after the user presses the send button. The time stamp will then be sent to the second phone and the phone will utilize this time stamp to start it's metronome at the same exact time as the first phone. I've tried accomplishing this by putting UTC time in a while loop and then constantly checking the time, to no avail. Does anyone have any ideas as to how we can go about implementing this? I couldn't find any similar problem on StackOverflow, or any other website for that matter.
It's a bit late probably for your project, but if interested. This is a bit long, it's more like a case study.
If your project was about syncing two device clocks remotely, then this won't help you, but if it was about syncing the metronomes, then this may be a start.
First, getting millisecond accurate timings and callbacks in android using java is next to impossible, since android is not a real time system ( All sleep methods and timer might not execute on the exact millisecond you're expecting ). You may want to go with NDK for triggering your beat events, because native threads in android can achieve that, but if you're happy with your single metronome implementation, then that's good enough to read the rest of this .
Full disclosure :
I am the author of JAM the app referenced in the answer I just published a few days ago.
I ran into the same problem when developing the app.
And whether this answer will help you or not, depending on your situation.
and if you're going down the "clock sync" route ( Which I do plan on exploring to improve my app down the road with more features ) but the solution that I found did not involves time sync.
I will mention my approach and then state what I found as advantages/disadvantages for each of the methods .
Instead of syncing clocks ( A very difficult problem, especially for what the app was supposed to do , and that is trigger ticks on multiple devices at the "same" time ) I opted for a subscribe/publish model, where multiple devices can "subscribe" to a host device via bluetooth, and the host device controls and "publishes" the metronome beats.
most times sending a few hundred bytes ( Enough information about each beat ) via bluetooth takes less than 2ms , it could spike up to 10 and sometimes even 30, but that rarely seems to happen .
That approach took care of the syncing problem, now I can send beat events from one device to another in 2ms , and if some sort of delay happened, it will self correct once the interference is gone, because the other event comes in time.
However that approach does require constant connection, you can't start and separate the devices later on, but it's much easier to implement.
Now the interesting part, the problem I never thought of when starting this, and that was the Android 10 millisecond audio path latency problem. I'm not affiliated with superpowered, but I did use their engine in my app to deal with that problem .
To summarize :
Metronome sync over bluetooth in Android has two issues :
1 - triggering beat events simultaneously ( or within acceptable latency )
2 - The beat event should trigger sound events with minimal delay after the beat event is received
#1 is related to the original question, #2 I thought was an important addition to the original problem this question is asking.
I would use (S)NTP to check the clock drift to a public time server. After that I would just send the interval and a start time and let the devices calculate the interval individually. So you just need to sync changes and relay on the clocks of the individual devices. Together with the now known clock drift you can fix this error.

Executing code after MediaPlayer is done playing?

I was wondering I'm working on an Android app where MediaPlayer/TextToSpeech is used to tell the user something and then a Speech Recognition object opens up after the MediaPlayer/TextToSpeech finishes and another is played after they answer the question with the recognition. Problem is I'm not 100% sure of a way, or good way, to delay the Speech Recognition from opening right after MediaPlayer is starts, or waiting till mediaplayer is done before speech recognition opens up. I see a few options based off my own knowledge/Google -
use the methods isPlaying(MediaPlayer)/isSpeaking(TextToSpeech) and putting them in some kind of loop to check until they aren't true anymore, then opening the speech recognition.
Call Thread.sleep(). The problem I see with this is I'm not sure exactly how long user is going to speak so it could still get interrupted unless I put it at a high time which just slows the app down.
I've research and played around with the Handler class a bit, but haven't fully comprehended how to do this...anyone help?
If you want to do something when a MediaPlayer finishes, you can set a OnCompletionListener with setOnCompletionListener() to receive a callback when the media playback is complete.

What is the best (most efficient) way to show notifications when the app is closed?

My app is basically is a schedule app, where I'd like to show notifications x minutes before an (user-set) event occurs.
I read the documentation, but it only covers showing a notification at the time the notification code is being executed (which I guess is the only way).
My guess is that if I want to have notifications show up even when my app is closed, I need to somehow make the app run in the background and constantly check for upcoming events, calculate the time left, and show a notification when the time left equals the time before the event the user chose to be notified at.
I read this question (+answers): How to get android notifications when app was closed?.
There is a pretty detailed answer (using services) I could simply implement, but the other answer claims this approach is "crappy".
The second answer also suggest the usage of AlarmManager which, after reading the doc, doesn't sound that bad (executing code at a specific time).
However, since I'm targeting API19 it's "inexact" (according to the doc).
So what exactly is the most efficient (and the right) way to do this?
You need to implement alarm manager with service. Set the time user chose and show notification at that time. The sample shows a Toast, you can use notification/sound/dialog, anything.
Check out this sample.

Java: How to implement listener to provide realtime audio stream position?

I'm working on an application to synchronize with realtime audio playback. I would like to define a listener with a specific timestep, and use the listener to get updates every time the audio being played has advanced by a timestep. Implementing this functionality must be possible (existence proof being time counter of audio player apps) but the architecture I have in mind might not be possible.
Ideally, I would listen to the audio stream on the SPEAKER or HEADPHONE Target Port of the sound card. The existing LineListener/LineEvent functionality only supports START/STOP/OPEN/CLOSE events, and I can't subclass and augment the Target Port implementation (or SourceDataLine or TargetDataLine).
How can I do this? Do I need to have a Thread constantly polling the getMicrosecondPosition() or getLongFramePosition() methods and fire my own event at the appropriate time? That sounds very inefficient. If I used a Timer to trigger the poll at regular time periods, then I'm better off than just using the system timers between the available START and STOP events that are supported on Port/DataLines. I would call the getMicrosecondPosition() method to synchronize, but I think that is likely to result in occaisional errors due to drift between the system Timer and real time audio. I'd expect to see repeats or skips of the counter (i.e. 1:23:003, 1:23:004, 1:23:006, 1:23:007... or 1:23:003, 1:23:004, 1:23:004, 1:23:005..).
I'm looking at the javazoom mp3 player source to see how they did it but no luck figuring it out so far. Any help/tips will be appreciated.

Categories