I am working on an application for college music majors. A feature i am considering is slowing down music playback without changing its pitch. I have seen this done in commercial software, but cannot find any libraries or open source apps that do anything like this.
Are there libraries out there?
How could this be done from scratch from various file formats?
Note: I am working in java but am not oppossed to changing languages.
Timestretching is quite hard. The more you slow down or speed up the sound the more artifacts you get. If you want to know what they sound like listen to "The Rockafeller Skank" by Fat Boy Slim. There are a lot of ways to do it that all have their own strengths and weaknesses. The math can get really complex. That's why there are so many proprietary algorithms.
This page explains things a bit clearer than I can and links to the Dirac library.
http://www.dspdimension.com/admin/time-pitch-overview/
I found this link for java code to do pitch shifting/timestretching
http://www.adetorres.com/keychanger/KeyChangerReadme.html
I use soundstretch to speed up podcasts which is works quite well, haven't tried it on music though.
This site explains how it's done in the physical world:
http://www.wendycarlos.com/other/Eltro-1967/index.html
I don't know how you would emulate that in software though... I'll keep looking
One way to do it would be to double the sampling rate without changing the sampling rate of your source. (Low quality example, but easy to implement. Note: You can also decrease the sampling rate as well).
Check out any math related to phase vocoders.
Another common method is to create an array of fft bins that store data for scheduled intervals of your sound. Then you can choose how quickly to iterate through the bins, and you can re-synthesize that audio data for as long as you choose thus enabling you to stretch out one short segment of your sound for as long as you like.
audacity does it out of the box and it's free. THere are several plug ins for mp3 players as well that are free. Apparently it's pretty easy to do with an mp3 since it's already coded in the frequency domain.
Related
I am creating an Android application for musicians because I am in a band and see that I needed something of this sort, but could not find one that could do what I wanted, and was wondering how would someone be able to take input from a microphone. Turn that input into a file, and compare it with all of prerecorded sound files in a database to determine the note(s), or chord(s) that are being played. I'm not having trouble with getting the input, but I'm stumped on how one would be able to compare one sound file to another in terms of frequency or something of the kind. I haven't yet been able to find an answer that could really be used to help with the problem, nor have I been able to find a Java library that handles sound comparison. I know this is a extremely hard task to accomplish, but I also know it can be done and would like to have a go at it. If anyone could offer advice, a link to a library that could do such a thing, or even if someone already has done it and could show me exactly how to do it I would be extremely grateful. Thank you for your time, and any feedback is appreciated!
I recommend you check out the music-g API put out by Google. It's an open-source library written in Java that you can integrate with an Android app. It provides sound similarity metrics.
http://code.google.com/p/musicg/
While I don't think we can close as a duplicate (as it isn't, technically), please do a search before posting. What you are asking isn't specific to your platform, so much as the kinds of algorithms you need to implement.
From my post here: How can I do real-time pitch detection in .Net?
See these references: http://cnx.org/content/m11714/latest/
http://www.gamedev.net/community/forums/topic.asp?topic_id=506592&whichpage=1
Line 48 in Spectrum.cpp in the Audacity source code seems to be close to what you want. They also reference an IEEE paper by Tolonen and Karjalainen.
Basically, you need to start with some FFT, but it is much more complicated than that. I think you will find that the near-impossibility of this task (especially for a whole band, a non-clear audio input source, etc.) will make this project not worth it. Psychoacoustics, particularly with distorted guitars, will make this very difficult.
There are tons of really solid posts on this topic here: https://stackoverflow.com/search?q=pitch+detection
I am trying to build a program in Java, which would be able to output sound to different output lines on my sound card.
How is it possible to do in Java?
I'm not sure what background you have in working with sound (it might be bigger than mine) but what I think you have is a number of lines that you want to direct to a number of ports? http://download.oracle.com/javase/7/docs/api/javax/sound/sampled/package-summary.html - This might be an API for this, which I don't know much about (I've seen a program using something like that with VoIP).
By the way if yr working on Windows, I believe that some information about your soundcard is stored in the registry, and you might have to update the registry from the program to gain access to it (not sure).
I hope I helped somehow :)
Check out the Java sound programmer's guide.
To get this to work I had to reconfigure my sound card to expose its ports as separate "outs"; otherwise the sound card would route the sound automatically depending on whether a pair of headphones were plugged in. (On Windows.)
You can use Jack (Jack Audio Connection Kit) to route sound to your soundcard. It is a little complicated, it doesn't work with pulseaudio, but you can make it work.
JnaJack provides an interface from Java to the Jack API via JNA. It's really quite easy.
I wrote it up in more detail here.
I would have thought that this would be an easy thing to do, but no amount of googling around has turned up any solutions.
I have written an application for a client that runs in full screen and allows the user to page through educational books in order to help teach kids to read. So far so good. Some phrases from the displayed materials are read back to the user and, again, so far so good. Normally, these sounds are read at the system volume. (ie, at whatever volume any other system sound would be played.)
The client, however, now wants the user to be able to adjust the volume in program. I have the UI and processing end of that working, but I'm having a hard time adjusting the volume of the clips being played in a meaningful way. Right now I have a funky setup that involves reading the original gain of the clip and then adjusting that up or down for each clip. While this does adjust the volume, it does not seem to do it relative to the system volume -- all of the sounds are much quieter than the system volume.
So, my question really is: how do you suggest controlling sound volume within a program? All of my research has turned up nothing meaningful, which implies to me that it's kind of hard or even impossible, but that just doesn't seem right.
Oh, details: I'm reading in WAVs as AudioInputStreams and playing them as java.sound.sampled.Clip. I'm controlling the gain using FloatControl.Type.MASTER_GAIN. (FloatControl.Type.VOLUME is not supported.) I'm stuck using Java 1.4.
You might also look to see if there is an older version of the Java Media Framework (JMF) that will support Java 1.4... it might provide you with a more rich environment for working with audio.
Other than that, you say you are stuck in 1.4... you might see if there is any way you can get upgraded to Java 5 or 6, though you may want to test either of these first on your local machine to see if they actually help any.
I guess one last suggestion is to find some Java game development sites and post your question there; I would think they would have figured out ways around it... though you could also get a lot of 'upgrade your jvm' responses there too. :-)
Good luck.
This is an OS-specific thing to do, and you'll have to use either JNI or J/Invoke or JNIWrapper or jna or...
I am currently developing an application that helps the user to tune his guitar and generate guitar effects. This is in real-time. I've been looking through java applications that could give an idea to generate guitar effects such as overdrive and delay but I couldn't find any. Also a source on creating a waveform in real time is needed. Your comments would be so much help, thanks in advance.
John says:
First, forget Java ... Secondly, you will be interfacing with the hardware ... Java does not support this kind of thing.
Jeez, that's kinda harsh - you should have told Sun that this wasn't possible before they published the API for this: http://java.sun.com/products/java-media/sound/. There's lots done with sound in Java, and I've never had an issue with latency or buffers, even on somewhat decrepit hardware.
Good examples # http://www.jsresources.org/examples/index.html
Good help # http://java.sun.com/products/java-media/sound/list.html
... having said that, John's comments on learning DSP & waveform analysis are on the $$$.
Have fun - Dave
This open source project maybe a good reference for you. There's a function that construct the waveform
http://code.google.com/p/musicg/
Regarding the feasability of low latency sound processing: Have a look at this article about Harmonicon, a java soft-synth. It's a sample playing synth implemented entirely in java, using the Metronome GC, which has upper latency guaranties in < 2 ms running on a realtime OS.
Regarding wave form generation/dsp, check out the example in this question, Java generating sound, a very simple wave form generation example.
First, forget Java. Java is a managed run-time which does garbage collection. When this happens you will hear shuttering because you wanna keep your sound buffer small to minimize latency,
Secondly, you will be interfacing with the hardware i.e. sound card, Java does not support this kind of thing and so you'll either have to write some hardware abstraction in JNI or find an existing solution, but there's a problem with that to. It's unlikely that you'll get the real-time performance from the Java platform.
What you wanna do is that you wanna go with C++ for this, and you will wanna learn more about partial-differentiation, DSP, sound synthesis and waveform analysis. This is quite a lot to take on but it should give you a good sense of direction if you start reading up on relevant research...
I am trying to make an application in which one component captures the screen of the user (for screen casting). I am aware that there are two options to achieve the same using a Java applet (please correct me if I am wrong). First is to use the java applet to take screen shots continuously and convert it into a video and upload it as a video file. And second is to create a java vnc server and record it as a .fbs file and play it using a player like: http://www.wizhelp.com/flashlight-vnc/index.html
I would like to know the best solution in terms of video quality, file size, cross-platform compatibility (windows and mac), firewall problems and finally ease of implementation.
I am very new to Java. Please tell me whats the best solution for my problem. Also, is it easy enough for me to program it on my own or should I get it developed via a freelancer. I have tons of programming experience (5+ years in LAMP) but none in Java.
Thank you very much.
I agree that this is pretty hard. I implemented those two solutions (VNC and onboard screen capture) plus a third (capture from an external VGA source via an Epiphan grabber) for a former employer. I had the best bandwidth-to-quality ratio with VNC, but I got higher framerate with VGA capture. In all three cases, I reduced the frames + capture times to PNGs and sequenced them in a QuickTime reference movie. Then I made flattened video (MPEG4 or SWF) of the results. In my case, I then synchronized the screen video with a DV stream.
In the end the technology worked (see a sample of the output) but our business model failed.
From what I know, the older versions of applet had security restrictions that may not allow for screen capture. Instead, a java application may be feasible.
Regarding the build-it-yourself vs the fire-a-coder, it depends on how you value your time compared to what you can find on a freelancer site.
I think you can find someone from India/Romania/Poland/Other countries that can make it for an affordable price
Given your Java knowledge and the difficulty of the task, have you considered taking an alternative approach? For example, how about a native VNC server for the end-user, which is just a small download and then they click "Run." And that native server is programmed to capture the screen and send it straight to your web server, which has a client like vnc2swf or other means of converting the VNC stream to a video or .fbs file? Does all that make sense?
Admittedly, without Java, you have to prepare one executable program per platform you want to support, however, I don't know. That still sounds easier to me. Consider Copilot.com. They are doing VNC but they still use small native apps for each platform.
Sorry but this seems the kind of job that requires a lot of experience. Even if you find code snippets all around the net to fix this and that, the overall result may be way worse than simply hiring an experienced Java programmer.