I would like to know how to upload a file to Amazon S3 with 'Pause and Resume' support? (Via a web browser).
Are there any sample web applications available? Any programming language / framework is fine.
Thanks in advance.
SOLUTION
I implemented the following app. Github Link.
It is based on the sample app and gem from Condominios.
All credit to https://github.com/cotag/ for a great gem and work.
FEATURES:
- Pause / Resume support ~ 5MB chunks
- Large File Upload
- Progress Bar
- No Java Applet / No Flash
- Registration system via devise
That one is a big one. I have been looking for a clean answer for that for a very long time. I even built somethings but it always comes down to using a medium for your application. I think the best solution I have found is using this. It really is a very simple idea and the great part is it only uses a small amount of flash to use.
S3 Heroku Flash Uploader
Github Source Code
However the down side is your not going to be able to upload anything successfully that is over 512 MB there is some sort of cashing fall out after that point. Loose track or something. I think the only other solution that I can think of is to build a Java Application that would handling the uploading to the server. At least then you have a more stable connection and don't have to worry about the problems with the browser.
You should build it using the multipart upload API. Here's the link for Java:
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/dev/mpListPartsJavaAPI.html
The idea would be to initiate a multi-part upload, start uploading parts (whose size would be based on the client's transfer rate) and whenever the user pauses the upload, stop uploading parts. You won't have byte by byte pause granularity, but I suspect the user would not notice that.
I implemented the following app. Github Link:
https://github.com/interpegasus/condo_example
It is based on the sample app and gem from:
http://cotag.github.com/Condominios/
All credit to https://github.com/cotag/ for a great gem and work.
FEATURES:
Pause / Resume support ~ 5MB chunks
Large File Upload
Progress Bar
No Java Applet / No Flash
Registration system via devise
Evaporate js is a pretty good plugin for this specific task.
Have a look here:
https://github.com/TTLabs/EvaporateJS
You'll need a client something like this: https://github.com/23/resumable.js
And a server that:
Writes the chunks somewhere (locally or to S3)
Uploads the fully-assembled file to S3.
You are not going to be able to do it straight from the browser to S3.
Update: S3 supports CORS now. http://aws.amazon.com/about-aws/whats-new/2012/08/31/amazon-s3-announces-cross-origin-resource-sharing-CORS-support/
Have you tried S3Fox firefox plugin ?
It has both front end and backend implmentation with S3
https://medium.com/#selvakumar.ponnusamy/resumable-file-upload-with-s3-ce039cbc8865
Related
I know the topic is not an easy one, but I am looking for a Java class to send an HLS stream from the server to the client.
I have files being generated greater and greater:
out.m3u8
out0.ts
out1.ts
out2.ts
out3.ts
out4.ts
out5.ts
out6.ts
This is generated using ffmpeg from an original source:
ffmpeg -i http://sourceurl.com:9981/stream/channel/1232131 out.m3u8
I can play it using VLC.
Somehow, I need to stream this live to the clients.
At this point, I do not really care about different bit rates, i just want live streaming to work, in mobile browsers and on desktop browsers.
I found this class:
https://github.com/Red5/red5-hls-plugin/blob/master/plugin/src/main/java/org/red5/stream/http/servlet/PlayList.java
Which might be doing something like that.
I have pulled in hls.js into my application in hopes of using it for desktops.
HLS should however work IOS devices without hls.js right now.
How should one serve HLS content from the server? It's very difficult to find any good and simple example to do that.
Anyone knows of the steps needed to do that ?
I've looked into Wowza and Red5 just a little bit, but unsure what they can provide for me at this stage and seems to be overly complicated to setup just to serve some files. But please explain to me why that's not the case.
The H in HLS stands for HTTP. The point of streaming tech such as HLS DASH HDS smooth streaming, etc is that no special server is necessary. Just plain HTTP. you can use something like nginx, or any HTTP server class/library available for Java or any other language.
I have been working on an android project in which I have to download/upload few files via HTTP. I was wondering if there is a way to have resumable downloads/uploads for the files. As in, if my file is being downloaded or uploaded and there is a subtle internet choke for very minimal time (this sometimes corrupts the file and the process is stopped and next time it starts from 0 ) the downloading/uploading is paused and once the internet is back again on my device, the downloading/uploading starts from the same point where it was stopped at so that the file does not get corrupted and the process does not start from 0.
Is there any way to achieve this functionality in android/Java ? Please do let me know. Thanks in advance.
Html itself doesn't provide such ability to load file in chunks. FileUpload is simple object which works with file as whole and so sends it from scratch. To fulfill your requirements you need more sophisticated client/server relations. Java Applet is good candidate to do so on the client side and server side is trivial. However you need to implement some protocol (like handshake, start to send file, continue from some location, validation) and this is not an easy task. Even most commonly protocols (for example ftp) don't provide such ability. And even when you create all this stuff it will be compatible only with itself. Is it really worth all the efforts? Common answer is - no. That's the reason why we don't see such approach in the wild.
I have a problem I've been dealing with lately. My application asks its users to upload videos, to be shared with a private community. They are teaching videos, which are not always optimized for web quality to start with. The problem is, many of the videos are huge, way over the 50 megs I've seen in another question. In one case, a video was over a gig, and the only solution I had was to take the client's video from box.net, upload it to the video server via FTP, then associate it with the client's account by updating the database manually. Obviously, we don't want to deal with videos this way, we need it to all be handled automatically.
I've considered using either the box.net or dropbox API to facilitate large uploads, but would rather not go that way if I don't have to. We're using PHP for the main logic of the site, though I'm comfortable with many other languages, especially Python, but including Java, C++, or Perl. If I have to dedicate a whole server or server instance to handling the uploads, I will.
I'd rather do the client-side using native browser JavaScript, instead of Flash or other proprietary tech.
What is the final answer to uploading huge files though the web, by handling the server response in PHP or any other language?
It is possible to raise the limits in Apache and PHP to handle files of this size. The basic HTTP upload mechanism does not offer progressive information, however, so I would usually consider this acceptable only for LAN-type connections.
The normal alternative is to locate a Flash or Javascript uploader widget. These have the bonus that they can display progressive information and will integrate well with a PHP-based website.
For php http://php.net/manual/en/features.file-upload.php
Note the ini files changes in the first comment.
Edit: Assuming you are running into timeout issues.
I find it hard to find some conclusive information on this. I have a dedicated server in a datacenter with Debian 5.0. I have an iPhone/iPad app which uses a JAVA EE (Glassfish 2.1) backend, and I am in the process of implementing video into the App. This includes live streaming and video's are longer than 10 minutes I need HTTP Live Streaming.
What is the best open-source/free solution to implement? This is only a pilot project, so I don't want to subscribe to any paid service. I have currently nothing in place yet for the live streaming, so am flexible to adapt any system (server or client side).
I came across:
Darwin (but am not sure that project is alive, as there is not a lot of info)
Red5 (but cannot find conclusive if this would allow an easy implementation of HTTP live streaming)
FFMPEG
Regarding the video's, I would ideally like to upload a 720p version to the server (for iPad) and then convert automatic (either on the fly when requested or prepared when the file is uploaded) to the required formats for iPhone/iTouch and low bandwidth. For live streaming I would like to be able to provide the content in about 30 seconds from it streaming into the server.
I am not envisaging high demands (e.g. a lot of simultaneous requests, and if so (e.g. live event) on one stream which should be able to be dealt with using HTTP-live streaming, it only needs encoding and segmenting once).
In the )not so near) future android will probably be made part of the App as well.
Any hints/tutorial/suggestions/advice would be really appreciated.
Wowza is pretty good for live streaming to iOS (as well as flash)
It isn't free though.
Latest development version of VLC supports HTTP live streaming.
You'll have to build from source as this has been added to the git repository not so long ago.
http://wiki.videolan.org/Documentation:Streaming_HowTo/Streaming_for_the_iPhone
I am now using the Xuggler framework, which is Java based. Seems to do exactly the job I am looking for, although no build in segmented etc. is available. Instead I try now to write one myself which at the same time integrates exactly with my system
Refer to Apple's http live streaming document and best practices.
https://developer.apple.com/streaming/
This should be a good point to get started.
What is the source of the live video? The iPhone only supports playback of H.264 baseline profile level 3 or mpeg-4 video with aac audio. The iPhone itself encodes video to these specs, but most other encoders don't (including many Android phones). If your video is not encoded to this spec, you'll first have to transcode. FFMpeg (with libx264) will do this nicely. Then you'll need to generate the dynamic .m3u8 playlist file. Wowza will do this for you out of the box, and will accept an rtmp stream from FFmpeg (but is not free). I don't believe that red5 supports Apple http streaming. There are free servers that claim to, but I've never used them. Take a look at http://erlyvideo.org/. Otherwise, you can do it yourself fairly simply. FFmpeg will output an mpeg-ts stream. All that the playlist generator needs to do, then, is cut this into 188-byte-aligned chunks, and return a playlist containing the last n. You can even use an http byte offset module to make the playlist reference a single file. Read Apple's http streaming docs at https://developer.apple.com/streaming/
This has been discussed before here. Using Java, I have developed my web services on Tomcat for a media library. I want to add a functionality for streaming media while dynamically transcoding them as appropriate to mobile clients. There are few questions I am pondering over :
How exactly to stream the files (both audio and video) ? I am coming across many streaming servers - but I want something to be done on my code from Tomcat itself. Do I need to install one more server, i.e , the streaming server - and then redirect streaming requests to that server from Tomcat ?
Is it really a good idea to dynamically transcode ? Static transcoding means we have to replicate the same file in 'N' formats - something which is space consuming and I dont want. So is there a way out ?
Is it possible to stream the data "as it is transcoded"...that is, I dont want to start streaming when the transcoding has finished (as it introduces latency) - rather I want to stream the transcoded data bytes as they are produced. I apologize if this is an absurd requirement...I have no experience of either transcoding or streaming.
Other alternatives like ffmpeg, Xuggler and other technologies mentioned here - are they a better approach for getting the job done ?
I dont want to use any proprietary / cost based alternative to achieve this goal, and I also want this to work in production environments. Hope to get some help here...
Thanks a lot !
Red5 is another possible solution. Its open source and is essentially Tomcat with some added features. I don't know how far back in time the split from the Tomcat codebase occurred but the basics are all there (and the source - so you can patch what's missing).
Xuggler is a lib 'front end' for ffmpeg and plays nicely with Red5. If you intend to do lots of transcoding you'll probably run into this code along the way.
Between these two projects you can change A/V format and stream various media.
Unless you really need to roll your own I'd reccomend an OSS project with good community support.
For your questions:
1.) This is the standard space vs. performace tradeoff. You see the same thing in generating hash tables and other computationally expensive operations. If space is a larger issue than processor time, then dynamic transcoding is your only way out.
2.) Yes, you can stream during the transcode process. VLC http://www.videolan.org/vlc/ does this.
3.) I'd really look into VLC if I were you.