We Keep Coding

Playing multiple samples in Javasound

I'm looking to write a simple MIDI-driven audio sequencer using Javasound.
I have multiple samples (one for each MIDI pitch) that are loaded into memory as a (globally accessible) Map<MidiPitch,AudioInputStream>.
A custom subclass of javax.sound.midi.Receiver responds to incoming MIDI events as follows:
If the event is a note-on, a Clip is obtained and played as follows:
Clip clip = AudioSystem.getClip();
clip.open(lookupAIS(pitch));
clip.start();
The clip is then added to a globally accessible Map<MidiPitch,List<Clip>>, representing started clips, i.e. clips for which start() has been called as above, but for which a note-off event has not yet been received.
If the event is a note-off, the corresponding list of started clips is obtained from the above map. The clip at the head of the list is removed, and stop() and close() are called on it.
The above Receiveris connected to MidiSystem.getSequencer() in the usual way, then the following called:
sequencer.setSequence(MidiSystem.getSequence(new File(myMidFile)))
sequencer.open()
sequencer.start()
Thread.sleep(aLongTime())
sequencer.stop()
sequencer.close()
The above works when the driving MIDI sequence is at a slow tempo, but at higher tempos, notes simply hang (even for sequences containing a very small number of notes).
My understanding is that clip.start() is run within a separate thread behind the scenes by the Javasound API.
Can anyone suggest why this might be happening? Is it perhaps a synchronization issue?
EDIT: By 'hang', I mean that some notes are stuck, despite the fact that log output reports that the corresponding 'stop' method has been called.
EDIT2: It looks as if the hanging first happens when a given note is played for the second time. This happens even if the MIDI sequence is monophonic, i.e. the previous note has stopped.

Your method of loading the Clip for each play is going to be a considerable source of variable latency. Every time you call this, the file is read anew and will not start playing until it the entire file has finished loading.
I recommend pre-loading all the clips and holding them in memory. When the note-on is called, set the clip cursor to zero and then play:
clip[mapIndex].setFramePosition(0);
clip[mapIndex].start();
These clips should have already been opened. I'm putting them in an array and using "mapIndex" as a plausible way of selecting the correct clip that might work with the mapping you've already set up.
You probably won't need to "stop" or "close" the clips until the entire sequence has finished, unless the clips are rather long and are designed to be stopped while in progress, or if they are being played as loops.
This should improve things considerably. I can't say if it will fix everything. The cpu is probably doing some thread multiplexing, and it is plausible that occasionally, in your current code, the clip.close is being called on one thread before the clip has finished loading on the other.

Clip sometimes does not play

I have a Sound class that contains a method that, when called, plays a sound using a Clip object (in this case, clip).
public static void play() {
clip.stop(); // The purpose of the first three lines
clip.flush(); // is to restart the Clip object so it
clip.setFramePosition(0); // can be played multiple times.
clip.start();
}
The instantiation of the Clip object occurs in a separate static method which is called prior to this method, which is why the above method can be declared static.
Another class that implements KeyListener contains the following code:
public void keyPressed(KeyEvent e) {
Sound.play(); // Sound is the class that implements the previous method.
}
Therefore, my code should be playing the sound associated with clip everytime a key is pressed. However, if I press a key quickly and repeatedly, the sound will sometimes not play. This is especially noticeable after a while (It seems as though the problem gets worse after each key press).
Why does this happen, and how can I circumvent this problem?

I have had the same kind of problems in the past, and something which worked for me is adding a line listener for whenever a line has finished, and closing it.
The code underneath is a stripped down version of what I use:
music = AudioSystem.getClip();
AudioInputStream ais = AudioSystem.getAudioInputStream(Sound.class.getResource("/sounds" + filename));
music.open(ais);
music.addLineListener(new LineListener(){
public void update(LineEvent e){
if(e.getType() == LineEvent.Type.STOP){
e.getLine().close();
}
}
});
music.start();
When you create the clip, just add in the line listener. When you reset the clip using your play function, it should play properly. I hope this works for you!

I came across this post via a reference in a similar question.
Yes, the number of Clips playing at once can contribute to the lag, but I don't know how to predict how much of an effect it will be, as this depends on how the JVM interacts with the OS.
You might be able to aid the situation by making the buffer size of the Clip smaller. SourceDataLine and Clip both seem to only allow interactions with incoming requests at buffer boundaries (I'm not sure this is a 100% accurate statement). Specifying the Clip's buffer size is a bit circuitous, as it requires obtaining the data as a PCM array. But if you wish to give it a try, the API is here.
A good way to reduce the number of lines is to use a sound library like TinySound. Many of the programmers at Java-gaming.org have used this library with success.
Another alternative that I'm promoting is AudioCue, a class I recently wrote for concurrent Clip playback. The license is BCD, source is provided, so feel free to examine, tinker with, and make use of the code. AudioCue doesn't reduce the number of output lines (yet). But if you are doing the common thing of managing multiple copies of the same cue, it could help in this regard as all concurrent instances are mixed down to a single output. Basic principle: the file is loaded into an array and played back via cursors that iterate over the array and merge their output into a SourceDataLine. This setup also allows implementation of smooth, real-time volume, panning and pitch fading.

Why is it necessary to close an audio Clip?

public void playClick(String file)
{
try {
Clip clip = AudioSystem.getClip ();
AudioInputStream ais = AudioSystem.getAudioInputStream (new File(file));
clip.open (ais);
clip.start ();
}
catch (Exception e)
{
System.out.println(e);
}
}
I've seen a question which is kind of like this one but I guess I'll be a bit more specific.
So the code above is a method I put in a separate class to be invoked when the user clicks on a button. It seems to work fine without the .close() but I'm still pretty sure there's got to be some pretty good reason why I should use close() even though the file I'm playing is very small?
The sound played is only a .wav file which is a very short beep sound, of very small size.
Also if there are some flaws in the code above that you could point out, please do so.
If this happens to be a completely duplicated question, I'll apologize then and I'm hoping you can provide the link to it.

You need to close the clip to free any resources used by it. If you don't you will leak memory and eventually crash with an OutOfMemoryException.

From Sun's documentation for Line:
Closes the line, indicating that any system resources in use
by the line can be released. If this operation succeeds,
the line is marked closed and a CLOSE event is dispatched
to the line's listeners.

Yes it's a duplicate, see here: Closing Streams in Java
Mainly: You can run out of resources, like file-pointers, memory etc.
While researching i found out, that Java 7 introduced some syntactical sugar to make this "easier":
http://docs.oracle.com/javase/7/docs/technotes/guides/language/try-with-resources.html

playing a sequence of Clips with a delay between them

Let's say I have a number of sounds (imagine piano/guitar notes).
I want to play each sound after a given interval, for example 200 miliseconds.
But I want to let the previous sound "ring out".
Although the approach below works ok for longer delays (700-1000 ms) it's not too precise.
For short delays, sometimes the sounds "bunch up" and play in rapid succession.
What I tried (sort of pseudo code):
for (Clip clip: clipList){
clip.start();
Thread.sleep(500);
}
My guess this has something to do with thread scheduling by JVM/OS...
Any ideas?
EDIT:
As advised in the comments I tried with a timer too:
final Timer timer = new Timer();
TimerTask timerTask = new TimerTask() {
#Override
public void run() {
try{
Clip clip = clipList.remove();
clip.start();
}catch (NoSuchElementException e) {
timer.cancel();
}
}
};
timer.schedule(timerTask, 0, delay);
I still get the same behaviour when I have 7-8 sounds and 500 ms delay.

Having the notes "ring out" is a great thing, and something that has driven my researches and trials with Java Sound as well. I'm not sure I can help specifically, but I can share an experience or two.
I made a clip player that allows one to retrigger the clip before it has finished playing (allowing the first to play out while another is started), and at different sample rates. This is done by giving the clip multiple cursors, and having them increment through the internally-stored sample data independently. (The playbacks that are at higher or lower speeds use linear interpolation to derive audio values when the cursor lands in between two samples.) The output is funneled into a single SourceDataLine.
Curiously, when this clip player is invoked multiple times, as a stand-alone program, there is a bunching up that occurs that is similar to what you describe. However, I also wrote an audio mixer that is capable of playing back both a number of these clips AND a sourceDataLine .wav file AND some live FM synth sounds, mixing them all into a SINGLE SourceDataLine output, and the timing on this is pretty darn good!
It really baffles me, as the clip part (and the triggers to the clips) are virtually the same code. One key difference might be that the AudioMixer I wrote is set to run continuously, so some of possible timing problems might be coming from when code is run from bytecode versus from memory. The HotSpot compiler will run code a few times as bytecode before committing to placing it in memory where it will run more quickly, and this would account for some timing problems.
I found a good article on timing issues you might want to check out:
http://quod.lib.umich.edu/cgi/p/pod/dod-idx?c=icmc;idno=bbp2372.2007.131
Basically, java doesn't offer "real time" guarantees, and this is the big challenge to achieving low latency performance. Sources of variance include: the issue of when HotSpot or whatever decides to run from bytecode or from memory, garbage collection, vm thread switching.
Having clips that "play out" would be a key component to doing branching music, allowing one to zig instead of zag at a moment in a game, for example, and the music or sfx would remain "seamless." That is part of what I'm envisioning. Of course, it would also be nice if there was a DAW where one could take a track and designate it be saved as multiple audio tiles (that overlap/playout), rather than making one export each tile individually. But that is getting ahead of the game...Was this a general direction you are also thinking about? Or do you have another application in mind? (Just curious.)

Audio clip getting stuck

I am seeing some strange behaviour with Clip instances in Java.
The purpose of the class I am working on is to keep count of the number of Clip instances containing the same sound sample (indexed by URI.) When the application requests to play a clip and there are already three or more clips from the same source already playing, the following steps are performed:
Sort the currently-playing clips by a weighted sum of PAN and framePosition.
Select the clip with the highest value as the one to be stopped and re-started.
Re-start the clip (the following method):
void restart(Clip clip, float gain, float pan) {
clip.stop();
clip.flush();
pan = Math.max(-1f, Math.min(pan, 1f));
((FloatControl) clip.getControl(FloatControl.Type.MASTER_GAIN))
.setValue(gain);
((FloatControl) clip.getControl(FloatControl.Type.PAN))
.setValue(pan);
clip.setFramePosition(0);
clip.start();
}
Strange behaviour occurs if this method is called many times in quick succession (e.g. 20 times within 1ms):
The clip plays
The clip fires a START event to signal that it has started playing
The clip never fires a STOP event.
Subsequent calls to stop and start have no effect (but do not throw exceptions.)
getFramePosition always returns 0, even when the clip is audible (for the last time.)
Any idea what could be causing this?
I don't think it's a threading issue (at least not in my code.) Only one thread is calling the public methods of my class (and they are all synchronized anyway)
Might be related to this bug.

The calls to DataLine.start and DataLine.stop are already synchronized on the DataLine's mixer inside AbstractDataLine.
I strongly suspect that somewehere down the call stack (below implStart()/implStop() of whatever DataLine incarnation you got, very likely inside the native nStart/nstop) at least one asynchronous call is made, thus resulting in the race condition you observe.
It would be impossible for you to work around this kind of problem using synchronized or any other Java construct without more intimate knowledge of the native implementation that is called.
A viable, immediate workaround could be to close the old clip and open a new instance instead of rewinding the old one. Not optimal, but it may well do the trick pending a deeper investigation.
In order to be able to perform the aforementioned deeper investigation, one would have to know what platform you are on, as well as a confirmation of the actual (implementation) class names of your Clip and Mixer instances.
UPDATE
In parallel, please use introspection to set com.sun.media.sound.Printer.trace = true (or provide your own implementation of com.sun.media.sound.Printer in the CLASSPATH.)
Essentially DirectClip.open() spawns a thread which accesses several volatile variables (of particular interest being doIO) in a non-threadsafe manner, which could potentially result in the main playback loop hanging.
You can confirm (or infirm) this (in conjunction with the Printer traces) by forcing a thread dump at the time of the apparent hang, and inspecting the playback thread state/stack trace (or use a debugger.)
If doIO etc. access turns out not to be the problem then continuing to dig at the native implementations is still the thing to do; if doIO etc. access does turn out to be the problem then again, there is no easy fix (you can try to use introspection to grab DirectClip.thread and signal it periodically in case it stalls because of doIO -- again, to be confirmed.)