Struggling with Android MediaCodec, I'm looking for a straight forward process to change the resolution of a video file in Android.
For now I'm trying a single thread transcoding method that makes all the work step by step so I can understand it well, and at high level it looks as follows:
public void TranscodeVideo()
{
// Extract
MediaTrack[] tracks = ExtractTracks(InputPath);
// Decode
MediaTrack videoTrack = tracks.Where(o => o.IsVideo).FirstOrDefault();
MediaTrack rawVideoTrack = DecodeTrack(videoTrack);
// Edit?
// ResizeVideoTrack(rawVideoTrack);
// Encode
MediaFormat newFormat = MediaHelper.CreateVideoOutputFormat(videoTrack.Format);
MediaTrack encodeVideodTrack = EncodeTrack(rawVideoTrack , newFormat);
// Muxe
encodeVideodTrack.Index = videoTrack.Index;
tracks[Array.IndexOf(tracks, videoTrack)] = encodeVideodTrack;
MuxeTracks(OutputPath, tracks);
}
Extraction works fine, returning a track with audio only and a track with video only. Muxing works fine combining again two previous tracks. Decoding works but I don't know how to check it, the raw frames on the track weight much more than the originals so I assume that it's right.
Problem
The encoder input buffer size is smaller than the raw frames size, and also related to the encoding configured format, so I assume that I need to resize the frames in some way but I don't find anything useful. I'm correct on this? I'm missing something? What is the way to go resizing Raw video frames? Any help? :S
PD
Maybe you will notice that I'm using C# (Xamarin.Android) for more fun. But the underlaying API is of course Java.
I'm using ByteBuffers, not Surfaces because it seems easier. I will be the next step using surfaces, any advice is welcome.
I know that the single thread process is highly inefficient, but makes it simple. It will be another next step to connect the decoder output buffer to the encoder input buffer.
I digged through PhilLab, Grafika and Bigflake examples but nothing seems to be very useful for me.
Avoiding to use ffmpeg on Android.
Thank you everyone for your time.
Going off of the comment above to implement libVLC
Add this to your app root's build.gradle
allprojects {
repositories {
...
maven {
url 'https://jitpack.io'
}
}
}
Add this to your dependent app's build.gradle
dependancies {
...
implementation 'com.github.masterwok:libvlc-android-sdk:3.0.13'
}
Here is an example of loading an RTSP stream as an activity
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.camera_stream_layout);
// Get URL
this.rtspUrl = getIntent().getExtras().getString(RTSP_URL);
Log.d(TAG, "Playing back " + rtspUrl);
this.mSurface = findViewById(R.id.camera_surface);
this.holder = this.mSurface.getHolder();
ArrayList<String> options = new ArrayList<>();
options.add("-vvv"); // verbosity
//Add vlc transcoder options here
this.libvlc = new LibVLC(getApplicationContext(), options);
this.holder.setKeepScreenOn(true);
//this.holder.setFixedSize();
// Create media player
this.mMediaPlayer = new MediaPlayer(this.libvlc);
this.mMediaPlayer.setEventListener(this.mPlayerListener);
// Set up video output
final IVLCVout vout = this.mMediaPlayer.getVLCVout();
vout.setVideoView(this.mSurface);
//Set size of video to fit app screen
DisplayMetrics displayMetrics = new DisplayMetrics();
getWindowManager().getDefaultDisplay().getMetrics(displayMetrics);
ViewGroup.LayoutParams videoParams = this.mSurface.getLayoutParams();
videoParams.width = displayMetrics.widthPixels;
videoParams.height = displayMetrics.heightPixels;
vout.setWindowSize(videoParams.width, videoParams.height);
vout.addCallback(this);
vout.attachViews();
final Media m = new Media(this.libvlc, Uri.parse(this.rtspUrl));
//Use this to add transcoder options m.addOption("vlc transcode options here");
this.mMediaPlayer.setMedia(m);
this.mMediaPlayer.play();
}
Here is the documentation of vlc transcoder options
https://wiki.videolan.org/Documentation:Streaming_HowTo_New/
You are right, the input buffer size of the encoder is smaller because it expects input to be of the specified dimensions. The encoder only, like the name suggests, encodes.
I read your question as more of a "why" than a "how" question so i'll only point you to where you'll find the "why's"
The decoded frame is a YUV image (is suggest to quickly skim through the wikipedia article), usually NV21 if i'm not mistaken but might be different from device to device. To do this i suggest you use a library as every every plane of the image needs to be scaled down differently and it usually takes care of filtering.Check out libYUV. If you are interested in the actual resizing algorithms check out this and for implementations this.
If you are not required to handle the decoding and encoding with bytebuffers, i suggest to use a surface as you already mentioned. It has multiple benefits over decoding to bytebuffers.
More memory efficient as there is no copy between the native buffer and app allocated buffer, the native buffers are simply geting swapped from and to the surface.
If you plan to render the frame, be it for resizing or displaying, it can be done by the devices graphic processor. On how to do that check out BigFlakes DecodeEditEncode test.
In hope this answers some of your questions.
I'm Using Android Universal Image loader in my news app i'm currently working on. I'm using NewsAPI to fetch news. After parsing the JSON from the website. I populate a list view using a custom adapter and universal image loader is used to get the images in background. But for some pictures UIL doesn't support scheme(protocol) by default [null].You should implement this support yourself (BaseImageDownloader.getStreamFromOtherSource(...)) exception is displayed.
I think in some images the URI is not in the format of Library expecting.
So my question is how do i fill the image's position which gave me the error from a picture that i have in the Drawable Folder. (404 picture indicating the image could not be downloaded)
i found where i should put the code.
protected InputStream getStreamFromOtherSource(String imageUri, Object extra) throws IOException {
throw new UnsupportedOperationException(String.format("UIL doesn't support scheme(protocol) by default [%s]. You should implement this support yourself (BaseImageDownloader.getStreamFromOtherSource(...))", new Object[]{imageUri}));
}
I need to know what kind of code i should put here to put a 404 image from my drawable file if the image is not downloadable.
Bro.. you can use Piccasso image library for android. It can solve your issue.
Sample code:
if(!image.equals("default")){
Picasso.with(Profile.this).load(image).networkPolicy(NetworkPolicy.OFFLINE)
.placeholder(R.drawable.default).into(mDisplayImage, new Callback() {
#Override
public void onSuccess() {
}
#Override
public void onError() {
Picasso.with(Profile.this).load(image).placeholder(R.drawable.default).into(mDisplayImage);
}
});
}
Here it can replace the default (i.e) 404 found image while Error condition
And small suggesstion that, you can use Easy News Library for simple retrieval of news from NewsAPI. Here we don't need to parse the JSON. It gives the data by single method call
Relevant code:
YouTubeThumbnailView first_video = (YouTubeThumbnailView) findViewById(R.id.main_video);
first_video.initialize(Config.YOUTUBE_API, new YouTubeThumbnailView.OnInitializedListener() {
#Override
public void onInitializationSuccess(YouTubeThumbnailView youTubeThumbnailView, YouTubeThumbnailLoader youTubeThumbnailLoader) {
final String video = getResources().getString(R.string.principal_funcoes);
youTubeThumbnailLoader.setVideo(video);
}
The time it takes for the method 'youTubeThumbnailLoader.setVideo(String s)" to work is absurd.
It takes 30+ seconds for a thumbnail to show up, with full cabled connection (100mbps).
It's definitely impossible for a user to wait more than 2 seconds for the thumbnail to show up, and have a completely blank screen while he waits for that to happen.
What can I do to load the video thumbnail any faster, or at least make YouTubeThumbnailView show a loading image while it fetches the thumbnail?
You can use direct youtube api urls for getting youtube thumbnails and they are way to faster too. here are urls you can try.
Start:-- It is only give you default size thumbnails.
https://ytimg.googleusercontent.com/vi/<insert-youtube-video-id-here>/default.jpg
For the high quality version of the thumbnail use a url similar to this:
https://ytimg.googleusercontent.com/vi/<insert-youtube-video-id-here>/hqdefault.jpg
There is also a medium quality version of the thumbnail, using a url similar to the HQ:
https://ytimg.googleusercontent.com/vi/<insert-youtube-video-id-here>/mqdefault.jpg
For the standard definition version of the thumbnail, use a url similar to this:
https://ytimg.googleusercontent.com/vi/<insert-youtube-video-id-here>/sddefault.jpg
For the maximum resolution version of the thumbnail use a url similar to this:
https://ytimg.googleusercontent.com/vi/<insert-youtube-video-id-here>/maxresdefault.jpg
It helps me to develop youtube application, Hope it also help you.
I'm making this netflix style app in which images are loaded into different categories. Let's say Dog videos (has 15 images), Cat videos (15 images), etc... All the images are loaded from a URL, it kind of takes a while for all to load. I was wondering if there was anything I could do to speed up the process? Or maybe show an empty container then fill it as the images load (that would be cool).
This is what I have done:
I have multiple async calls in one Activity, (1 async call per category)
JSONTask1 dogTask = new JSONTask1();
JSONTask2 catTask = new JSONTask2();
JSONTask3 pigTask = new JSONTask3();
JSONTask4 horseTask = new JSONTask4();
dogTask.execute("");
catTask.execute("");
pigTask.execute("");
horseTask.execute("");
I have all of those in a row in my actual code. Thanks.
I would use the "proxy pattern". Basically, you need to create a class that contains the minimal informations required for the display. In which, you have a preview image.
When ever you load everything you start by showing the preview content, ie : a loading gif for everypicture with the title of the movie or whatever. and basically the proxy would have a "loadImage" method that would make an ajax call or async call and the photos would load one by one. Plus, to make the loading easier, make sure the photos are not oversized.
You can see Picasso answers , in picasso i suggest you this way :
Picasso.with(getApplicationContext()).load("your url").placeholder(R.drawable.your_place_holder).error(R.drawable.showing_when_error_occured)
.into(imageView, new Callback() {
#Override
public void onSuccess() {
}
#Override
public void onError() {
}
});
Also another suggestion from me : convert your thumb images to base64 format in backend, then firstly retrieve your thumbs and show them. Then start an async task and change images when successfull.
Like whatsapp. In whatsapp you have thumb images they have so low resolution and super fast. When you click image if you have internet connection they load actual thumb images, and click again they load larger image.
Picasso website :http://square.github.io/picasso/
Load them asynchronously with Picasso, you can even show a placeholder image until the real one is loaded
I'm doing a website for a family member's wedding. A feature they requested was a photo section where all the guests could go after the wedding and upload their snaps. I said this was a stellar idea and I went off to build it.
Well there's just the one problem: logistics. Upload speeds are slow and photos from modern cameras are huge (2-5+Megs).
I will only need ~800px wide images and some of them might require rotating so ideally I'm looking about using a client-side editor to do three things:
Let users pick multiple files
Let them rotate some images so they're the right way up
Resize them and then upload
And in my dream world, it'd be free and open source. Any ideas?
Just a reminder: this is something the guests have to use. Some of them will be pretty computer savvy but others will be almost completely illiterate. Installing desktop apps isn't really an option. And I assume 98% of them have Flash and Java installed.
Edit: I'd prefer a Flash/Java option over SilverLight, not least because it has a smaller install rate at the moment, but also because I'm on Linux and I'd like to test it =)
The most common solution for this is a java applet, although most of them are not free. Examples:
http://www.jumploader.com/
http://www.aurigma.com/Products/ImageUploader/OnlineDemo.aspx
http://www.javaatwork.com/java-upload-applet/details.html
JUpload, mentioned by ScArcher2
I have had good luck with Gallery. It is free, open source, and has all the features you mentioned.
It will allow your users to upload photos without any intervention from you.
Another option could be to allow people to upload their photos to whatever service they're used to using (flickr, google, smugmug, or any other), and just accept a username for that service, or a URL for the folder.
Then you can have your application grab a copy of those pictures to store locally with a consistent interface.
I've used swfupload quite a bit. It's pretty awesome: http://www.swfupload.org/
If you are doing this with Flash and using Flickr, then I would check out the AS3 Flickr library:
http://code.google.com/p/as3flickrlib/
which has support for uploading images.
Upload requires authentication. The library also contains a Flex based control for handling this:
http://www.mikechambers.com/blog/2008/08/12/flex-based-flickr-api-authorization-control/
(the rest of the library is ActionScript 3 and can be used in Flex or Flash.
Probably the easiest solution is to just have the images uploaded to Flickr, edited in Picnik (built into Flickr now), and then loaded onto the users site using either the Flickr RSS feeds or APIs:
http://www.flickr.com/help/picnik/
http://www.flickr.com/services/api/
hope that helps...
mike chambers
mesh#adobe.com
I'd use an applet. You could do the resizing of the pictures and rotating on the client side.
It looks like JUpload may do this for you.
Picasa is a pretty great/free photo management app. It let's you do some pretty impressive editing, and has upload capabilities, though I can't remember if it will upload to anywhere, or just certain popular sites (like Flickr).
You could use Silverlight or Flash or some custom plugin to allow managed uploads, where you can display a progress bar for example. There isn't much you can do about upload speeds but you can at least show them progress while it's going on.
I don't know of any canned upload programs you can use but it shouldn't be too hard to make one (unless you don't know Flash or Silverlight).
How about using PhotoShop Online It allows you to edit photos with a web based editor and offers 2GB of storage. I've not used it myself so don't know if it allows for multiple users to access the same account though
Out of curiosity, on what web stack is this to run? LAMP? 2k3+IIS? etc etc? Many of the open source solutions out there are cross-platform but others are not...
Is E-mailing the photo in an available option?
Most people who want to share photos probably already know how to send photos in email. And most email clients has already solved the problems of file uploading.
Just setup one gmail/whatevermail account and have your website poll the inbox.
It's something like what TwitPic does for twitter but your requirements seem to be more simpler than that.
Personally most users don't understand DPI and their images even trimmed down end up larger than the php.ini for most hosting companies allow.
I'm not sure how much control you want to give them or how you want the public side to behave.
I'd suggest using a dropbox FTP application such as http://etonica.com/dropbox/index.html (tango dropbox) It's free to your clients and you only have to pay for your version so you can set up the FTP information and secure it.
I'd have them download something link paint.net (which is FREE) have them edit the photos to the proper size and then just drag and drop them to this application. it's easy and doesn't require php.ini to be modified.
You could also use something like slideshowpro's director application.
I completly agree with zigdon, allow different sites, but only pick up photos from the web. I you still want to allow uploads, and put a cap on size.
Now, if you want to throw yourself into something big, I would suggest putting a cap on size, and then using JQuery (or other library) to work with the images.
Just my 2 cents
You could also have them email the pictures to picasa. Picasa web has a feature where you can send images to a "secret" email that will post them to a picasa account. Set up a picasa account, distribute the "secret" email, and wait for all the pictures to show up.
Going the Flickr route is easy and will work well.
If you want to go more advanced, I'd recommend snipshot or picknik (Flickr uses it). Both are free to use and have APIs to use.
I am currently required to implement the similar requirement as Oli.
I believe Facebook.com use java applet of some sort, and it work pretty well but I am not sure if the applet available as OSS. I am going to look into JUpload suggested by ScArcher2.
If you guy no of any other good applet please keep it coming.
I'd highly suggest using FileBrowser by Lussomo. It's as easy as 'drag and drop' :D
I've used it for my game development team where we had a raw dump of over 200 concept art images, and we simply extracted FileBrowser to a PHP-enabled webserver and dumped the images in appropriate directories (1 per album), and ran the thumbnailing script. It handles cropping of the images, and optimizing their size for you. So much better than using something like Menalto Gallery where you have to upload them through an awkward upload interface.
Try this out
http://www.lunarvis.com/products/tinymcefilebrowserwithupload.php
Depends on the web server. If you can use servlets, try this :
// UploadServlet.java : Proof of Concept - Mike Smith March 2006
// Accept a file from the client, assume it is an image, rescale it and save it to disk for later display
import javax.servlet.http.*;
import javax.imageio.*;
import java.io.*;
import java.util.*;
import java.sql.*;
import org.apache.commons.fileupload.*;
import org.apache.commons.fileupload.disk.*;
import org.apache.commons.fileupload.servlet.*;
import java.awt.image.*;
import java.awt.*;
public class UploadServlet extends HttpServlet {
public static void printHeader(PrintWriter pw) {
pw.println("<HEAD><TITLE>Upload Servlet</TITLE><HEAD>");
pw.println("<BODY>");
}
public static void printTrailer(PrintWriter pw) {
pw.println("<img src=\"../images/poweredby.png\" align=left>");
pw.println("<img src=\"../images/tomcat-power.gif\" align=right>");
pw.println("</BODY></HTML>");
}
public void init() { // Servlet init() : called when the servlet is LOADED (not when invoked)
}
public void service(HttpServletRequest req, HttpServletResponse res) throws IOException {
DiskFileItemFactory dfifact;
ServletFileUpload sfu;
java.util.List items;
Iterator it;
FileItem fi;
String field, filename, contype;
boolean inmem, ismulti;
long sz;
BufferedImage img;
int width, height, nwidth, nheight, pixels;
double scaling;
final int MAXPIXELS = 350 * 350;
res.setContentType("text/html");
PrintWriter pw = res.getWriter();
printHeader(pw);
ismulti = FileUpload.isMultipartContent(req);
if (ismulti) {
pw.println("Great! Multipart detected");
dfifact = new DiskFileItemFactory(999999, new File("/tmp"));
sfu = new ServletFileUpload(dfifact);
try {
items = sfu.parseRequest(req);
} catch (FileUploadException e) {
pw.println("Failed to parse file, error [" + e + "]");
printTrailer(pw);
pw.close();
return;
}
it = items.iterator();
while (it.hasNext()) {
fi = (FileItem) it.next();
if (fi.isFormField()) {
pw.println("Form field [" + fi.getFieldName() + "] value [" + fi.getString() + "]");
}
else { // Its an upload
field = fi.getFieldName();
filename = fi.getName();
contype = fi.getContentType();
inmem = fi.isInMemory();
sz = fi.getSize();
pw.println("Upload field=" + field + " file=" + filename + " content=" + contype + " inmem=" + inmem
+ " size=" + sz);
InputStream istream = fi.getInputStream();
img = ImageIO.read(istream);
nwidth = width = img.getWidth();
nheight = height = img.getHeight();
pixels = width * height;
if (pixels > MAXPIXELS) {
scaling = Math.sqrt((double) MAXPIXELS / (double) pixels);
nheight = (int) ((double) height * scaling);
nwidth = (int) ((double) width * scaling);
}
BufferedImage output = new BufferedImage(nwidth, nheight, BufferedImage.TYPE_3BYTE_BGR);
Graphics2D g = output.createGraphics();
g.setRenderingHint(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BILINEAR);
g.drawImage(img, 0, 0, nwidth, nheight, null);
ImageIO.write(output, "jpeg", new File("/var/tomcat/webapps/pioneer/demo.jpg"));
istream.close();
}
}
}
else
pw.println("Bugger! Multipart not detected");
printTrailer(pw);
pw.close();
}
public void destroy() {
}
}
GIMP (http://www.gimp.org/) is a good tool for doing resize and is open source.