I'm trying to program a loopstation program in JSyn inspired by something like this. The idea is, that I can record to Wav-files and play them from the program. That worked well until I attempted to do this for multiple files simultaneously.
How do I approach this?
I cannot create multiple synthesizer engines otherwise I get an error, so I have created a class with my line out and my synthesizer. But if the audio only plays while the synthesizer sleeps, how can I play from multiple files at once?
System.out.println("queue the sample");
samplePlayer.dataQueue.queue(sample);
System.out.println("queued the sample");
// Wait until the sample has finished playing.
do {
mySynth.sleepFor(1); //synth has to sleep while sample is being played
} while (samplePlayer.dataQueue.hasMore()); //this seems to always return true -> synth never wakes up & the program crashes
This is adapted from the examples included in the JSyn library. I have based most of my own coding on the JSyn Programmer's Guide
This is what the GUI looks like (programmed in Java Swing). It responds to the mouse and the numpad. This works.
The constructor of my output class. This contains the synthesizer and line out.
public OutputMix() {
filePath = sampleMachineFrame.filePath; //string containing path to location for sample files
mySynth = JSyn.createSynthesizer();
myLineOut = new LineOut();
mySynth.add(myLineOut);
recorder = new RecordMic[10]; //one recorder for each button (my own class)
player = new PlayFromWav[10]; //one player for each button (my own class)
}
The recording works absolutely fine. I can even start overlapping recordings (i.e. record to two files at once) and play them with an external program.
But when I try to play them back the synthesizer never wakes up and I am also struggling to imagine how I would play multiple files at once.
Thanks for your help :)
In general, you do not want to sleep in your program unless you are sequencing events. Just queue samples in response to Button events and let them run.
If the files are small enough then you can just load them into memory. Then you can queue multiple files any time you want. You might need to increase the heap size of your Java virtual machine.
Here is an example: https://github.com/philburk/jsyn/blob/master/tests/com/jsyn/examples/PlaySample.java
If the samples are too big then you will have to stream them off the disk using multiple threads, which is more difficult.
You can make all the samples the same size. Then they will stay in phase. Or you can trigger them at specific beats using timestamps.
Related
I'm having some issues playing mp3 files using AudioClip with JavaFX.
my code is like this:
// define String path to mp3 file
myFilePath = "path/to/file.mp3";
// Create an AudioClip Object from path
audioToPlay = new AudioClip(Objects.requireNonNull(getClass().getResource(audioToPlay)).toExternalForm());
// play the AudioClip
audioToPlay.play();
The audio files I am playing are all mp3 files and they are all fairly short (< 5 seconds). About 80-90% of the files are playing without any issue but the rest aren't playing at all, or are only playing for a very short time (less than a second so it sounds like a click).
I have no idea why the 10-20% aren't working; they play perfectly fine using my media player and from my browser.
Is the issue because I'm using AudioClip? Should I be using something else to play my short audio files?
Edit: myFilePath is defined as a String and audioToPlay is defined as an AudioClip object.
AudioClip is appropriate for short files that can be held in memory. This issue might be related to how or when you are calling the play() method. Does this occur when calling the play method via a button click, where the GUI stays open? Some folks don't realize playback is a daemon thread. Act of playback won't prevent a program from closing if the program triggers the sound and then completes and exits before the sound finishes playing. Also FWIW, once loaded into memory, there's no need to keep reloading AudioClip over and over.
Thought I'd add this: it seems to me that sometimes Java doesn't gracefully handle the first audio call in a session, that there is often a stutter. Due to this, I often play a "silent" sound at the start of a program, in order to get things going. Just mentioning, in case the problem occurs when the program is first getting going. If you use a GUI (with button to make sound) then you can test if an additional call repeats the problem or not, and thus help verify whether it's the sound file or the circumstance.
How can you ensure the transition between two pieces of audio is seamless?
In a JavaFX application, I am using the javafx.scene.media.MediaPlayer to play an intro-piece, which is proceeded by the main/looping-piece. The media is played just fine, but the issue is the transition and loop.
Here is what I am currently doing:
private static void foo(final Media intro, final Media loop) {
final MediaPlayer introPlayer = new MediaPlayer(intro);
introPlayer.play();
final MediaPlayer loopPlayer = new MediaPlayer(loop);
loopPlayer.pause(); // An attempt to load the media so it will be ready to be played.
introPlayer.setOnEndOfMedia(loopPlayer::play());
loopPlayer.setOnEndOfMedia(() -> loopPlayer.seek(Duration.ZERO));
//loopPlayer.setCycleCount(Integer.MAX_VALUE); // Similar to the above line, but there is still a delay between loops.
}
The MediaPlayer::pause does help some, but there is a very noticeable delay between the end of the intro media and the start of the looping media. Furthermore, there is another noticeable delay between the end of the looping media and the repeat.
I additionally tried using javafx.scene.media.AudioClip, since it supposedly has less overhead than a javafx.scene.media.MediaPlayer. I wrote my own listener to tell when the track ended (and to immediately thereafter start the looping piece), but I still saw a similar delay.
Here were some similar posts I found, but did not provide a solution to the problem:
JavaFX MediaPlayer playing background music loop with small intro music
This one is definitely relevant (coincidentally, it is almost the
anniversary of that post), but I am already using a .wav formatted
media file and still experience a delay.
JavaFX AudioClip.play()
Which is similar to what I tried with the Audioclip, exept I used a
scheduled executor to time when to replay the audio. (Where I still
experienced a delay).
And, as a final note, I have tested my audio in Audacity, where they transitioned and looped seamlessly.
What are some recommended solutions for these types of problems?
Edit:
Added an addendum to the code-block, mentioning MediaPlayer::setCycleCount(Integer)
I realize it's been a while since you posted. Have you found an answer? I am wondering if you loaded loopPlayer before playing introPlayer, if that would help.
If MediaPlayer's "listener" is just kind of sluggish, maybe switching to using Java's SourceDataLine with a LineListener used to trigger the looping cue would work more seamlessly? (I would use a Clip for the looping playback.)
Last suggestion, I have an audio library AudioCue that could work for this. The library includes an AudioCueListener that can trigger an event (such as starting another AudioCue playing) when a cue ends. But limitations of the library would require that you hold your music in memory, and that the source files be .wav's.
The AudioClip Javadocs state that an AudioClip represents a segment of audio that can be played with minimal latency and are usable immediately. However, it also states
Media objects are however better suited for long-playing sounds. This is primarily because AudioClip stores in memory the raw, uncompressed audio data for the entire sound, which can be quite large for long audio clips. A MediaPlayer will only have enough decompressed audio data pre-rolled in memory to play for a short amount of time so it is much more memory efficient for long clips, especially if they are compressed.
Depending on the length of the looping media, an AudioClip might be better suited for you. Instead of needing a ScheduledExecutorService to replay the audio, you can use AudioClip.setCycleCount(AudioClip.INDEFINITE) to loop forever.
Using this info, I believe your best bet is to use a MediaPlayer for the intro and then utilize MediaPlayer#setOnEndOfMedia to call the looping AudioClip; possibly having a small delay between the intro and looping transition but seamless after that.
I'm looking to write a simple MIDI-driven audio sequencer using Javasound.
I have multiple samples (one for each MIDI pitch) that are loaded into memory as a (globally accessible) Map<MidiPitch,AudioInputStream>.
A custom subclass of javax.sound.midi.Receiver responds to incoming MIDI events as follows:
If the event is a note-on, a Clip is obtained and played as follows:
Clip clip = AudioSystem.getClip();
clip.open(lookupAIS(pitch));
clip.start();
The clip is then added to a globally accessible Map<MidiPitch,List<Clip>>, representing started clips, i.e. clips for which start() has been called as above, but for which a note-off event has not yet been received.
If the event is a note-off, the corresponding list of started clips is obtained from the above map. The clip at the head of the list is removed, and stop() and close() are called on it.
The above Receiveris connected to MidiSystem.getSequencer() in the usual way, then the following called:
sequencer.setSequence(MidiSystem.getSequence(new File(myMidFile)))
sequencer.open()
sequencer.start()
Thread.sleep(aLongTime())
sequencer.stop()
sequencer.close()
The above works when the driving MIDI sequence is at a slow tempo, but at higher tempos, notes simply hang (even for sequences containing a very small number of notes).
My understanding is that clip.start() is run within a separate thread behind the scenes by the Javasound API.
Can anyone suggest why this might be happening? Is it perhaps a synchronization issue?
EDIT: By 'hang', I mean that some notes are stuck, despite the fact that log output reports that the corresponding 'stop' method has been called.
EDIT2: It looks as if the hanging first happens when a given note is played for the second time. This happens even if the MIDI sequence is monophonic, i.e. the previous note has stopped.
Your method of loading the Clip for each play is going to be a considerable source of variable latency. Every time you call this, the file is read anew and will not start playing until it the entire file has finished loading.
I recommend pre-loading all the clips and holding them in memory. When the note-on is called, set the clip cursor to zero and then play:
clip[mapIndex].setFramePosition(0);
clip[mapIndex].start();
These clips should have already been opened. I'm putting them in an array and using "mapIndex" as a plausible way of selecting the correct clip that might work with the mapping you've already set up.
You probably won't need to "stop" or "close" the clips until the entire sequence has finished, unless the clips are rather long and are designed to be stopped while in progress, or if they are being played as loops.
This should improve things considerably. I can't say if it will fix everything. The cpu is probably doing some thread multiplexing, and it is plausible that occasionally, in your current code, the clip.close is being called on one thread before the clip has finished loading on the other.
I'm looking to make a custom Android app to help out a physically disabled person. I need to heavily rely on voice recognition. The idea would be:
the app (via voice recognition) accepts his speech
the app parses his speech and executes his commands ("email dad", "text fred", "what time is it?", etc, plus other things I'll be adding to control his tv, lights, etc)
after execution, the app waits for his next command (loop back to #1)
I have #1 & #2 working fine, but I can't figure out a good method for #3. I can't leave Google's voice recognition view running indefinitely because it could be hours before the next command. But the 'trigger' does have to be based on sound/voice. Has to be completely hands free.
Ideally, the app would just listen for a sound, and if that sound is above a certain pre-programmed decibel level, I'd start voice recognition. Is it possible to constantly loop just listening for a noise, then react if it's 'so' loud?
Any ideas?
Thanks
So the idea is that you want voice recognition to be ongoing. It is very hard to do that. But the way that I accomplished this is by using:
try {
Thread.sleep(4500);
mSpeechRecognizer.startListening(mSpeechRecognizerIntent);
} catch (InterruptedException e) {
// It depends on your app logic what to do with InterruptedException
// You can process it or rethrow or restore interrupted flag
}
After the onReady (so if the user doesn't say something around 4.5 seconds it resets and at results. This worked great for me.
CMUSphinx is a great solution for this:
You can easily listen continuously, a voice detection API is provided
You can increase accuracy for a custom set of commands
You can adapt the model to the user voice thus increasing accuracy significantly
For more details on using CMUSphinx on Android see
http://cmusphinx.sourceforge.net/2011/05/building-pocketsphinx-on-android/
Let's say I have a number of sounds (imagine piano/guitar notes).
I want to play each sound after a given interval, for example 200 miliseconds.
But I want to let the previous sound "ring out".
Although the approach below works ok for longer delays (700-1000 ms) it's not too precise.
For short delays, sometimes the sounds "bunch up" and play in rapid succession.
What I tried (sort of pseudo code):
for (Clip clip: clipList){
clip.start();
Thread.sleep(500);
}
My guess this has something to do with thread scheduling by JVM/OS...
Any ideas?
EDIT:
As advised in the comments I tried with a timer too:
final Timer timer = new Timer();
TimerTask timerTask = new TimerTask() {
#Override
public void run() {
try{
Clip clip = clipList.remove();
clip.start();
}catch (NoSuchElementException e) {
timer.cancel();
}
}
};
timer.schedule(timerTask, 0, delay);
I still get the same behaviour when I have 7-8 sounds and 500 ms delay.
Having the notes "ring out" is a great thing, and something that has driven my researches and trials with Java Sound as well. I'm not sure I can help specifically, but I can share an experience or two.
I made a clip player that allows one to retrigger the clip before it has finished playing (allowing the first to play out while another is started), and at different sample rates. This is done by giving the clip multiple cursors, and having them increment through the internally-stored sample data independently. (The playbacks that are at higher or lower speeds use linear interpolation to derive audio values when the cursor lands in between two samples.) The output is funneled into a single SourceDataLine.
Curiously, when this clip player is invoked multiple times, as a stand-alone program, there is a bunching up that occurs that is similar to what you describe. However, I also wrote an audio mixer that is capable of playing back both a number of these clips AND a sourceDataLine .wav file AND some live FM synth sounds, mixing them all into a SINGLE SourceDataLine output, and the timing on this is pretty darn good!
It really baffles me, as the clip part (and the triggers to the clips) are virtually the same code. One key difference might be that the AudioMixer I wrote is set to run continuously, so some of possible timing problems might be coming from when code is run from bytecode versus from memory. The HotSpot compiler will run code a few times as bytecode before committing to placing it in memory where it will run more quickly, and this would account for some timing problems.
I found a good article on timing issues you might want to check out:
http://quod.lib.umich.edu/cgi/p/pod/dod-idx?c=icmc;idno=bbp2372.2007.131
Basically, java doesn't offer "real time" guarantees, and this is the big challenge to achieving low latency performance. Sources of variance include: the issue of when HotSpot or whatever decides to run from bytecode or from memory, garbage collection, vm thread switching.
Having clips that "play out" would be a key component to doing branching music, allowing one to zig instead of zag at a moment in a game, for example, and the music or sfx would remain "seamless." That is part of what I'm envisioning. Of course, it would also be nice if there was a DAW where one could take a track and designate it be saved as multiple audio tiles (that overlap/playout), rather than making one export each tile individually. But that is getting ahead of the game...Was this a general direction you are also thinking about? Or do you have another application in mind? (Just curious.)