Related
I am building an online web application which offers clients to test their language skills.
The application consists of series of different test the clients can freely do. One of these tests is one where the client sees a sentence on the web-page and then is required to speak the sentence to a microphone. The idea would be to record the audio stream, send it to server and save it there to a file.
Is this possible using PHP, Java, or something similar to them?
Thank you for any help :)
I know you asked for PHP, I'm guessing that's your language of choice, but have you considered using html5 to record your audio? Note: browser support is still lacking but it might be worth looking into depending on your requirements.
Check out this tutorial; it uses the GETUSERMEDIA API to access you camera and microphone hardware.
Update: Here is another good tutorial on the subject.
The only realistic solutions for solving the problem are with flash, as suggested by another answer, or a java applet. In a year or two that will hopefully change and you'll be able to use javascript and HTML5, but this is now.
I would say this is a problem better and more easily solved with flash, but for java, You need to write a java applet to do the recording. Here are some hints to get you started:
http://docs.oracle.com/javase/tutorial/deployment/applet/
http://www.java2s.com/Tutorial/Java/0120__Development/CapturingAudiowithJavaSoundAPI.htm
http://www.jsresources.org/
You do not need java speech API or JMF, just java sound. You might WANT JMF to compress the audio file to make uploading go faster, but you can get this working without. Note that you may need to "sign" your applet in order to record audio in a browser.
If you are averse to flash, you could also use haxe to write your flash code.
Not 100% what you're looking for but I think it'll at least give you some ideas.
How to record streaming audio to the server
It is possible to solve this problem using Flash and Java both. You can use Flash component to record sound and send it in chunks to server side which can be implemented using java.
Pros to create sound recorder in Flash are many.
Flash is widely installed plugin in Browser
Java Applet gives warning before running it which many times turns off users.
Numerically checked, most media applications are created using Flash thereby providing you nice support.
Java Applets are used where thick client has to fit into browser and must also be secure. You will still find many real time treading applications are written in java applets.
Here are couple of tutorials to create Sound Recorder in Flash
http://active.tutsplus.com/tutorials/actionscript/create-a-useful-audio-recorder-app-in-actionscript-3/
Adobe record sound and save
This collected stream of sound can be stored in chunks on client side and can be synced with backend server(any server side language shall be fine.).
Implementing your server in Java has advantage. If tomorrow need arises, you can stream the same sound using Red5 server which is implemented in Java.
See more here: http://www.red5.org/
There is an audio recording SDK for this which is written in PHP, Flash and HTML5. http://recordmp3online.com
I am actually the author of that SDK (full disclosure)
Update:
Since the media side of JFX has been open sourced, I've looked into this myself and it is indeed possible, but requires changing and rebuilding the JFX source (both Java and C parts.) The process is described here for anyone that wants to have a go - I add MKV support in that example, but it should be very similar for other plugins.
The remainder of the question is thus mainly historical, but I'll leave it here for reference.
Background
I've been using VLCJ thus far for playing video in my application. It works, but if possible I'd like to see if I can achieve a similar level of support for common codecs by migrating to JavaFX and saving myself a lot of hassle with multiple VMs and suchlike that VLCJ needs to play multiple videos reliably. I won't go into it here but see my answer to this question if you're interested in the details. There's also the issue of cross-platform compatibility, it works on Mac and Linux ok but I haven't worked out how to get it to show on Mac yet (I believe there's some security in place to prevent one process gaining access to another's native components, but again that's beyond the scope of this question.)
It boils down to the fact that while it works, it's a lot of maintenance and hassle working with multiple VMs and bridging them stably if there's another solution that would be easier. VLC does have a pretty legendary level of support for playing pretty much anything which is why I've gone with it thus far, and I'd be interested to see if I can get a similar result in JavaFX - or at least if it can provide the means for doing so in a cross platform manner.
Research
JavaFX 2.0 supports video - great! But at the moment the official line is it supports "FLV containing VP6 video and MP3 audio". Is there a way to extend this to add in support for more codecs? There's no hard codec that I'd like to support, it's more a case of as many as I can so I'm looking for an extensible method to go about the above.
I wondered if it would play video for codecs installed natively on the machine and that it just doesn't advertise itself as such (because that functionality obviously is machine dependant and not cross-platform.) But no dice, I've tried a number of common formats and it really does refuse to play anything other than what it states.
From looking at JavaFX 1.3 it also supports other platform dependant codecs depending on where it's installed. Is there a way to get this behaviour with JavaFX 2? Or is it planned at all for a subsequent release? I haven't been able to find any information on it on the roadmap or any comment from Oracle about it.
Only thing I could find from searching extensively is here which implies that it may be possible but no-one seems to know how. I'd also be interested to know if it's based on GStreamer why all the formats supported by GStreamer aren't included by default either?
In terms of playing DVDs with JavaFX I've got absolutely nowhere, so I'm assuming that's just a no-go at the moment. If anyone does have any ideas or information though, I'm all ears.
Other approaches
One approach which I was half wondering may be possible is crowbarring the JMC jar out of the old JavaFX as described here and trying to get that working alongside JavaFX 2. I don't suppose anyone has had any luck with that approach or something similar?
All things failing, if anyone has any information or links on if / when support for additional codecs will be supported out of the box, then I'd be interested to hear that also. Or if anyone has any contact details for someone at Oracle I could ask that would also be appreciated! I've been longing for decent video support in Java for some time, and I guess what this boils down to is trying to figure out if JavaFX is the answer to this, or just another half hearted attempt that will never play more than what it does at the moment! I'm hoping it's not the latter, but I've yet to see much to show that's the case.
Believe me, I feel and know your frustration. I have pondered this for a while, but I had to use un-straight means of solving my issues.
There are many ways around this, each with limitations but depends on what works for you:
Docs say WebView works with HTML5, which plays videos supported on the platform (Though sadly not flash). If using a webview to play video works for you, you can try this out. You can even draw over it with other nodes.
Portable VLC Player! If maybe you're developing some sort of projector/director app and you want fullscreen video, you can have portable VLC player play the video in fullscreen in one screen with it's controls in the other. Used this solution and it works quite well for mac and windows. :)
Only thing is you can't draw nodes on the video as it's an external app, with just the illusion of fullscreen video of your app.
If you ever need to utilize the power of flash within your javafx 2.0 application, then use a swt-based browser(or something Like the DJ Project if you're a Swinger) as they support all features of your native browser.
I've now managed to compile MKV support into JavaFX successfully, and it does take some, but not a great deal of effort on the native layer also. See here for the discussion surrounding it, and here for the result submitted as a patch / JIRA ticket.
I've written a much more comprehensive guide on the process here which may be of interest to anyone else looking to go down this route.
What follows is my brief investigation before I actually seriously looked at compiling other media support in, though I'll leave it here for reference.
Now that JFX8 has been released and is completely open source, I've spent a bit of time looking at how this could be done, and whether it could be done without patching the JFX source. Unfortunately the answer to that latter point is an almost definite no, at least not without horrible bytecode manipulation hacks. I may look into this more practically at a later date, but I'll document what I've worked out so far from the source available.
The magic starts from the Media constructor, which is ultimately where the MediaException pops out from (with the MEDIA_UNSUPPORTED flag if you try to play an unsupported format.) From there it creates the Locator, whose constructor ensures that the URL is one that's supported. It's init() method is then called in a separate thread, which performs some sanity checking on the URL string, reads the file, then proceeds to try to work out what the format is.
The relevant code for this part of the method is thus:
if (scheme.equals("file") || scheme.equals("jar")) {
InputStream stream = getInputStream(uri);
stream.close();
isConnected = true;
contentType = MediaUtils.filenameToContentType(uriString); // We need to provide at least something
}
if (isConnected) {
// Check whether content may be played.
// For WAV use file signature, since it can detect audio format
// and we can fail sooner, then doing it at runtime.
// This is important for AudioClip.
if (MediaUtils.CONTENT_TYPE_WAV.equals(contentType)) {
contentType = getContentTypeFromFileSignature(uri);
if (!MediaManager.canPlayContentType(contentType)) {
isMediaSupported = false;
}
} else {
if (contentType == null || !MediaManager.canPlayContentType(contentType)) {
// Try content based on file name.
contentType = MediaUtils.filenameToContentType(uriString);
if (Locator.DEFAULT_CONTENT_TYPE.equals(contentType)) {
// Try content based on file signature.
contentType = getContentTypeFromFileSignature(uri);
}
if (!MediaManager.canPlayContentType(contentType)) {
isMediaSupported = false;
}
}
}
// Break as connection has been made and media type checked.
break;
}
From this we can see a first "dumb" attempt is made to grab the file content based on its name (this is what MediaUtils.filenameToContentType() does.) There's then some special cases for checking for different types of wav file, but if that fails then we fall back on a cleverer check which looks at the actual file signature. Both these checks are in MediaUtils. This latter check is much more extensive, and looks at the first few bytes of the file to see if it can work out a format that way. If it can't, then it bails out and throws the exception that then pops out as our dreaded MEDIA_UNSUPPORTED flag.
If the type is identified correctly though, there's still another hurdle to go through - it has to be supported by the current platform. Some platforms are loaded dynamically depending on the environment, however the GSTPlatform always exists, thus we would need to put any additional (universal) formats here. This is relatively simple, a CONTENT_TYPES array exists which just holds the array of supported formats.
Unfortunately cloning the JavaFX repo seems to be failing for me at the moment, otherwise I'd attempt to put some of this in practice. But in lieu of the above, what actually needs to happen to add support for further formats? It actually doesn't seem hugely difficult.
In MediaUtils, support needs to be added to the filenameToContentType() method to handle the new file extension. This is trivial.
In the same class, support needs to be added to the fileSignatureToContentType() method to work out the file type based on its signature. This is a tad more complex, but still not too bad. This may even be optional, since the current code only seems to use this as a fallback if the format isn't identified correctly (or at all) from the file extension. A comprehensive list of file signatures for different formats can be found here which should help with this task.
In GSTPlatform, the new content type needs to be added to the list of supported content types.
On the Java side of things, this appears to be all that's necessary to get it to accept the content type and at least attempt to pass it down to the native Gstreamer layer.
However, I'm no expert in GStreamer, so while I'm aware there's many more formats that it can handle and play that JavaFX currently refuses, I'm unsure as to how exactly they've removed this capacity. They've definitely done it in the Java layer above, but they may have also done it on the native GStreamer level - at this point I'm unsure.
I assume they've made some changes to GStreamer for JFX8 - but at the present time they're not listed on the relevant project page, so it's quite hard to work out exactly what they've changed for this version.
The next step would be to grab the JFX8 source, build with the above proposed changes for a new content type, and then see what errors (if any) occur on the native level, then take it from there.
The API design does not appear to have support for rolling your own codecs. Pretty much all of the classes are final (e.g. VideoTrack, Media, MediaPlayer etc). I assume that the actual video decoding is done with internal classes at present, meaning there is no way to override them.
There is a plan to Open Source JavaFX 2.0, I suspect as we approach the release of JDK8. Hopefully when they do this we can see how they resolve their codecs from the Media(String source) constructor and see if we can hook into this somehow.
And now, Javafx2.1 finally supports mp4 H.264 so you should now be good to go without the above posted stunts. :)
Current open feature requests for this in the JavaFX bug tracking system:
JDK-8091656 Wishlist for more media format support
JDK-8091755 Media should support InputStream
Read the linked feature requests and the associated comments on them to understand their current status (or lack thereof ;-) for the JavaFX distribution version that you are using.
Note, for the InputStream based Media API, one of the later comments by a JavaFX developer is "I propose we consider this for JDK 10", so I guess it may be a possibility in the future...
Also note, if you are not sure if JavaFX currently has in-built support for a given encoding type or not, a comprehensive overview of supported media encodings and media container types is provided in the javadoc for the javafx.media package (just ensure that you review the version of the javadoc which matches your version of JavaFX).
Those who may be interested in other solutions to at least get a video to play from JavaFX, even if it is a media type not natively supported by JavaFX and you don't want to hack the native JavaFX media support just to get your video to play, can also see my answer to the related question:
Playing h265 HEVC in a JavaFX client
When I run FreeTTS examples, I get this error:
LINE UNAVAILABLE: Format is pcm_signed 16000.0 Hz 16 bits 1 channel big endian
In this post, Freetts problem in Java
someone claims it's a known Linux/Java sound bug and has a workaround,
linking to http://forums.sun.com/thread.jspa?threadID=5189363 .
But this link does not work anymore since Oracle screwed it.
Archive.org seems not to have this page archived.
Does anyone have the workaround / patch for FreeTTS?
Thanks,
Ondra
Linux's ALSA is one large, complex API. OpenJDK and Sun's JDK seem to use it differently. Most modern Linux distributions also use PulseAudio, which virtualizes ALSA so that all audio goes through PulseAudio for software mixing before going to ALSA for playback.
When nothing is accessing the sound card, and Java is the only user, it tends to work. However when something else has the sound card open, Java apps quickly break with both your error and "javax.sound.sampled.LineUnavailableException: Audio Device Unavailable".
One potential workaround is to enumerate all mixers in the system with AudioSystem.getMixerInfo(), then try to open the line with AudioSystem.getSourceDataLine(format, mixerInfo) for the mixers you want. Some will work better than others. In particular the "Java Sound Audio Engine" and the "default [default]" mixers, if they exist, tend to work.
The only solution if you don't want to modify the FreeTTS source code though, is to install pulse-java. This registers a special PulseAudio sound provider, which bypasses the ALSA virtualization and goes directly to PulseAudio. Ubuntu installs this as part of its OpenJDK package.
Someone should really patch Java Sound to play with ALSA in a friendlier way. For one ALSA device names should be prefixed with plug: to get ALSA to convert sound formats and sample rates on the fly. And the other rules for safe ALSA subset should also be followed.
Hmm, I had better luck googling after asking the question, so...:
http://workorhobby.blogspot.com/2011/02/java-audio-freetts-line-unavailable.html
A big thanks to the author.
Update: Actually, this is not a nice workaround since it will keep FreeTTS on hold until the line is free.
FWIU, the mentioned patch had better solution - not demanding exclusive access or such.
Update: I've compiled a FreeTTS troubleshooting page.
A program based on FreeTTS, the free text-to-speech engine for Java, was getting occasional errors
"LINE UNAVAILABLE: Format is ..."
Turns out there is no Java Exception or other mechanism to detect this error that occurs inside the FreeTTS library. All you get is the message on System.out, so there is no good way to react programatically.
Workaround: Configure the FreeTTS audio player to attempt accessing the audio device more than once until it succeeds. In this example, a short delay of 0.1 seconds is used to not miss an opportunity to grab the audio device; we keep trying for 30 seconds:
System.setProperty("com.sun.speech.freetts.audio.AudioPlayer.openFailDelayMs", "100");
System.setProperty("com.sun.speech.freetts.audio.AudioPlayer.totalOpenFailDelayMs", "30000");
If the audio device is permanently used by another program, there is of course no way to get access. Under Linux, this command will display the ID of the process that is currently holding the audio device, so you can then try to get rid of the offending program:
/sbin/fuser /dev/dsp
Regarding the link screwed by Oracle - given that older SO answer you refer mentions horrendous java linux sound bug that is still not fixed and suggests to check the third post, it is likely that lost thread was migrated to:
https://forums.oracle.com/forums/thread.jspa?threadID=2206163
above thread starts with reference to JMF Bug 4352921 at http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4352921_
third post in the thread says "Yes, according to the API docs it's a "catastrophic" error from which no player can return safely. Which is why it's so strange to be able to trigger it so easily..."
workaround to the problem discussed is described in the sixth post as follows:
I stumbled upon a blog post that suggested that Java needs to be told to use the OSS libraries as it's not yet up to ALSA. The command to use was "padsp" which forces the application to use OSS. So if I call "padsp jmstudio", it now plays and mixes the audio just fine. I tried it with my application also, which prefetches a number of players in the same JVM, and they all prefetched perfectly. So it would seem, for now, JMF applications on Linux may need to be called through padsp.
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
i am currently working on a web application that needs to accept video uploaded by users in any format (.avi, .mov, etc.) and convert them to flv for playing in a flash-based player.
Since the site is OpenCms-based, the best solution would be a ready-made plugin for OpenCms that allowed to upload and play videos doing the transcode operation in background, but just a set of Java classes to do the transcode would be great and then i could make the uploading form and playback part on my own.
There's a great open source tool call FFmpeg that I use to transcode my videos. I use PHP making shell calls to make it come to life, but I can't imagine that it would be too hard to get it to play nice with Java. (Maybe this could be a good starting point for you.)
I feed my installation 30+ gig batches on a weekly basis and it always comes out as quality material. The only tricky part for me has been getting it compiled to handle a wide variety of video formats. On the bright side, this has provided me with heavy lifting I need.
You can encode video in Java using Xuggler, which is a Java API that natively uses FFmpeg's C code behind the scenes.
You basically have two choices if you want to host, transcode and stream flv files (and don't want to buy a video transcoding application): you can call out to FFMpeg/MEncoder or you can use an external Web service. You could also sidestep the problem completely by allowing them to embed YouTube videos on your site.
If you go the 'local FFMpeg route' I would suggest simply using ProcessBuilder and constructing a command-line to execute FFMpeg. That way you get full control over what gets executed, you avoid JNI, which is an absolute nightmare to work with, and you keep OS-specific code out of your app. You can find FFMPeg with all the bells and whistles for pretty much any platform. There's a good chance it's already on your server.
The nice thing about the 'Local FFMPeg' route is that you don't have to pay for any extra hosting, and everything is running locally, although your hosting admin might start complaining if you're using a crazy amount of disk and CPU. There are some other StackOverflow questions that talk about some of the gotchas using FFMpeg to create flvs that you can actually play in the flash player.
The Web service route is nice because there is less setup involved. I have not used Hey!Watch but it looks promising. PandaStream is easy to set up and it works well, plus you get all your videos on S3 with no additional effort.
This can be slightly tangential, but I have found Panda Stream to be a very useful solution to all kinds of video encoding problems.
All you have to do is to upload the video file to an Amazon EC2 instance running Panda and it will encode the video to your desired formats and quality and will issue a callback to your application server with the details when it's done. You can then use the bundled Flash Video player or your own player to play the videos on your site.
It's a very scalable (thanks to Amazon EC2 & S3), cost-effective and customisable solution compared to rolling your own.
Highly recommended.
Update:
The architecture of Panda is as follows:
(source: pandastream.com)
Page displays Panda's upload form in an iframe or popup
Video upload with AJAX progress bar
API callback when encoding is complete
Video streamed to user
There is an open source library used by MPlayer, called mencoder, wich supports FLV, as well as a lot of other codecs.
There is a Java GUI you could see how was made
This could help too.
I don't seem to be able to find any example not called from the console, so it may not be usefull for you. :S
Edit
Also take a look at this question.
You could try using an online service like HeyWatch to convert your video. Never used it but they claim
"transparent upload, send videos
transparently from your website"
Not a java solution, but you wouldn't have to worry about what OS your web application is on.
If OS wasn't an issue I agree with the answer theBadDawg gave. I don't know of and have had not any luck finding a pure java solution.
Encoding files in one format to another takes a lot of development time to get right, which is why there is so little in terms of decoders/encoders that are able to accomplish those feats. The ones that are well known and used the most are ffmpeg and mencoder.
What you may want to look into is to see if the platform you are running on (Windows/Mac OS X/Other unix) has an underlying set of API calls you can use that is able to decode the files, and re-encode them. Windows has DirectShow and Mac OS X has Quicktime. Not sure if you can access those API's using Java though.
FFMpeg does have a Java wrapper available: FFMPEG Java, and there is also FOBS which has a JNI available for their C++ wrapper around ffmpeg. The last one that I found jFFmpeg, however there are some posts that I found with Google suggesting that the project may not be alive any longer.
Your best bet would be either mencoder from mplayer and or ffmpeg. Ffmpeg can be installed as a separate binary and then called from other code using the default "shell" commands. If you are however not able to execute commands you may need to look at using an online conversion website like Mark Robinson suggested.
FFMpeg is the best when it comes to video transcoding.
You can use java wrappers for ffmpeg -
http://fmj-sf.net/ffmpeg-java/getting_started.php
http://sourceforge.net/projects/jffmpeg/
If you want to do it with java, you can do it very easily using Xuggle.
They have a great website explaining how to do everything
the documentation is here:
http://build.xuggle.com/view/Stable/job/xuggler_jdk5_stable/javadoc/java/api/index.html
and an excellent tutorial telling you how to do what you want is here:
http: //blog.xuggle.com/2009/06/05/introduction-to-xuggler-mediatools/
They provide an easy way to do what you want in the first tutorial, which is simple trans-coding.
I've found that it works alright for encoding to flv. What it does behind the scenes is use ffmpeg, so anything that will trip up ffmpeg will also fail with xuggle.
The relevant sample java code is:
// create a media reader
IMediaReader reader = ToolFactory.makeReader("videofile.flv");
// add a viewer to the reader, to see the decoded media
reader.addListener(ToolFactory.makeWriter("output.mov", reader));
// read and decode packets from the source file and
// and dispatch decoded audio and video to the writer
while (reader.readPacket() == null)
;
Which I got from
http ://wiki.xuggle.com/MediaTool_Introduction
If you want some fully working clojure code... here it is :)
(import '(com.xuggle.mediatool ToolFactory))
(import '(com.xuggle.mediatool IMediaDebugListener IMediaDebugListener$Event))
(defn readerRecurse
"calls .readPacket until there's nothing left to do2"
[reader]
(if (not (nil? (.readPacket reader))) ; here .readPacket actually does the processing as a side-effect.
true ; it returns null when it has MORE ro process, and signals an error when done...
(recur reader)))
(defn convert
"takes video and converts it to a new type of video"
[videoInput videoOutput]
(let [reader (ToolFactory/makeReader videoInput)]
(doto reader
(.addListener (ToolFactory/makeWriter videoOutput reader))
(.addListener (ToolFactory/makeDebugListener (into-array [IMediaDebugListener$Event/META_DATA]))))
(readerRecurse reader)))
now all you have to do is something like:
(convert "/path/to/some_file.stupid_extention" "/path/to/awesome.flv")
and you're done!
You might also be interested in hearing that we've now released Panda as a hosted service as well, which makes the setup and scaling easier :)
http://pandastream.com
yea, ffmpeg is the best for this work...We use ffmpeg to convert video for a long time and it works with all video formats..numerous options are there..
I am trying to make an application in which one component captures the screen of the user (for screen casting). I am aware that there are two options to achieve the same using a Java applet (please correct me if I am wrong). First is to use the java applet to take screen shots continuously and convert it into a video and upload it as a video file. And second is to create a java vnc server and record it as a .fbs file and play it using a player like: http://www.wizhelp.com/flashlight-vnc/index.html
I would like to know the best solution in terms of video quality, file size, cross-platform compatibility (windows and mac), firewall problems and finally ease of implementation.
I am very new to Java. Please tell me whats the best solution for my problem. Also, is it easy enough for me to program it on my own or should I get it developed via a freelancer. I have tons of programming experience (5+ years in LAMP) but none in Java.
Thank you very much.
I agree that this is pretty hard. I implemented those two solutions (VNC and onboard screen capture) plus a third (capture from an external VGA source via an Epiphan grabber) for a former employer. I had the best bandwidth-to-quality ratio with VNC, but I got higher framerate with VGA capture. In all three cases, I reduced the frames + capture times to PNGs and sequenced them in a QuickTime reference movie. Then I made flattened video (MPEG4 or SWF) of the results. In my case, I then synchronized the screen video with a DV stream.
In the end the technology worked (see a sample of the output) but our business model failed.
From what I know, the older versions of applet had security restrictions that may not allow for screen capture. Instead, a java application may be feasible.
Regarding the build-it-yourself vs the fire-a-coder, it depends on how you value your time compared to what you can find on a freelancer site.
I think you can find someone from India/Romania/Poland/Other countries that can make it for an affordable price
Given your Java knowledge and the difficulty of the task, have you considered taking an alternative approach? For example, how about a native VNC server for the end-user, which is just a small download and then they click "Run." And that native server is programmed to capture the screen and send it straight to your web server, which has a client like vnc2swf or other means of converting the VNC stream to a video or .fbs file? Does all that make sense?
Admittedly, without Java, you have to prepare one executable program per platform you want to support, however, I don't know. That still sounds easier to me. Consider Copilot.com. They are doing VNC but they still use small native apps for each platform.
Sorry but this seems the kind of job that requires a lot of experience. Even if you find code snippets all around the net to fix this and that, the overall result may be way worse than simply hiring an experienced Java programmer.