I am developing an app that play an RTSP stream from a camera using VLC, but I have a situation where I need to get frame of the stream without playing it.Kindly suggest the proper way of doing it. kindly recommend the best Library or SDK to achieve this.
I have used MediaMetadataRetriever, but so far no luck. I have also used FFmpegMediaMetadataRetriever but the app crashes.
D/apitrace: apitrace: warning: caught signal 6
D/apitrace: call flush from exceptionCallback
A/libc: Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 5035 (pool-3-thread-1), pid 4832
I have used below code.
ExecutorService executor = Executors.newSingleThreadExecutor();
Handler handler = new Handler(Looper.getMainLooper());
executor.execute(()->{
Bitmap bitmap;
try {
if (mmr == null){
mmr = new FFmpegMediaMetadataRetriever();
}
mmr.setDataSource(url);
bitmap = mmr.getFrameAtTime();
Log.d("BITMAP_IS_DONE",bitmap.toString());
mmr.release();
handler.post(()->{
thumbnailArrays[1].setImageBitmap(bitmap);
});
} catch (IllegalArgumentException ie) {
ie.printStackTrace();
}
});
Related
My problem changed a bit, please take a look at the EDIT 2 below
I am learning how to work with Services and audio recording on Android.
I want to create an app, that will do nothing except starting a service: after requesting permissions (RECORD_AUDIO, INTERNET) the app only calls startService() from the MainActivity.
The service will then record audio and stream it to given IP address. I took my inspiration for the audio streaming server and client from the answers to this question. I am testing the app on Android 6.0 in the Android Studio Emulator so far.
This is the onStartCommand() method of my service. I create a new thread where I wait until the user has clicked "Allow" in the permissions request dialog. When I receive the permissions I initialize the AudioRecord and then I read from it in a while-loop and send the data away.
#Override
public int onStartCommand(Intent intent, int flags, int startId) {
Thread streamThread = new Thread(new Runnable() {
#Override
public void run() {
waitForPermissions();
try {
DatagramSocket socket = new DatagramSocket();
byte[] buffer = new byte[minBufSize];
DatagramPacket packet;
final InetAddress destination = InetAddress.getByName(address);
recorder = new AudioRecord(MediaRecorder.AudioSource.MIC, sampleRate, channelConfig, audioFormat, minBufSize * 10);
recorder.startRecording();
while (status == true) {
minBufSize = recorder.read(buffer, 0, buffer.length);
packet = new DatagramPacket(buffer, buffer.length, destination, port);
socket.send(packet);
}
} catch (...) {...}
}
});
streamThread.start();
return START_STICKY;
}
In onDestroy() I set the flag status to false so that the loop in the thread created in onStartCommand() terminates. Then I release the recorder so that it can be initialized and used again.
#Override
public void onDestroy(){
status = false;
recorder.stop();
recorder.release();
super.onDestroy();
}
This app works works only partially:
When I launch the listening server on the computer and the app in the emulator, I can hear the audio normally.
When I minimize the app (tap Home button), the audio stream is still OK.
My problem is that when I kill the app (swipe it right in the running apps screen), the audio stops. In the Logcat I can see that the service has been restarted and runs as usual (initializes the AudioRecord and everything, no exceptions thrown), only I can't hear anything. When I run the app again (and therefore call the startService() again), I see an exception in the Logcat telling me that the AudioRecord start failed with an error code -38 (meaning that the microphone is in use by the previous instance of the service).
What am I doing wrong here? Why is the service running, but audio not streaming when I kill the app?
Many thanks for your answers.
EDIT:
I know this approach won't work on newer Android versions. This app is just for private purposes and will be run on one of my old phones, either with Android 5.1.1 or 6.0.
After I read the article in this answer I moved what was previously in onDestroy to onTaskRemoved. However, nothing changed.
The Service is apparently still sending some packets after it is restarted. I looked at them in Wireshark and the packets are still leaving the device and the Server is still receiving them. The payload of the packets is nonzero, unique for every packet.
EDIT 2:
I changed the buffer size to AudioRecord.getMinBufferSize(sampleRate, channelConfig, audioFormat) * 10 in the Android app and AudioRecord.getMinBufferSize(sampleRate, channelConfig, audioFormat) * 20 in the Server application. Now when I swipe the app, the Service restarts and then I can hear some audio. But the sound is very chunky after the restart and I don't know why. It also gets chunky when I uninstall the app, keep the server running and then install and run the app again.
So the problem is probably on the Server side. What could be the problem with the server? Am I working correctly with the SourceDataLine? The Server code looks like this (taken from here):
...
static int sampleRate = 8000;
static int bufferLen = minBufSize * 20;
public static void main(String args[]) throws Exception {
DatagramSocket serverSocket = new DatagramSocket(port);
byte[] receiveData = new byte[bufferLen];
format = new AudioFormat(sampleRate, 16, 1, true, false);
dataLineInfo = new DataLine.Info(SourceDataLine.class, format);
sourceDataLine = (SourceDataLine) AudioSystem.getLine(dataLineInfo);
sourceDataLine.open(format);
sourceDataLine.start();
FloatControl volumeControl = (FloatControl) sourceDataLine.getControl(FloatControl.Type.MASTER_GAIN);
volumeControl.setValue(volumeControl.getMaximum());
while (status == true) {
DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length);
serverSocket.receive(receivePacket);
toSpeaker(receivePacket.getData(), receivePacket.getLength());
}
sourceDataLine.drain();
sourceDataLine.close();
}
public static void toSpeaker(byte soundbytes[], int length) {
try {
sourceDataLine.write(soundbytes, 0, length);
} catch (...) {...}
}
...
The sample rate of the server is identical to the Android app. Format is AudioFormat.ENCODING_PCM_16BIT.
Read this article: What exactly happens to running services when you swipe an android app from recent app list ?
In Your case the service stops working, in recreation because of the start sticky it seems you have a problem that recording fails which I don't know what it is. In the sense of improving your service use IntentService or a foreground service to solve the problem.
BTW, if you want to use your app in android 8 or higher you probably has to use ForeGrounService only since IntentService will be killed by system after several minutes if app is in background. Read this: Background Execution Limits
So after testing all the solutions you might be agree with me on using a foreground service: Android Foreground Service Example
I'm trying to play music in my app and while the media player is loading I want to allow the user to keep on using the app, but the app gets stuck for few seconds every time I start loading the media player, and when the media player is finished loading, only then the app returns to normal and starts working again, sometimes it's not just freezing, it also shows popup menu from the OS that prompts the user to quit the app.
I couldn't find any solution in Google or YouTube, anyone knows what's wrong with my code?
final Handler handler = new Handler();
Runnable runnable = new Runnable() {
#Override
public void run() {
try {
String STREAM_URL = #####; // here i put the URL of the song
mediaPlayer = new MediaPlayer();
mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
try {
mediaPlayer.setDataSource(STREAM_URL);
mediaPlayer.prepare();
} catch (IOException e) {
e.printStackTrace();
}
} catch (NullPointerException e) {
Log.d(TAG, "run: NullPointerException = " + e.getMessage());
FirebaseCrash.log("Tag = " + TAG + "run: NullPointerException = " + e.getMessage());
}
}
}
};
handler.post(runnable);
Even though you are creating a Handler, the creation of the MediaPlayer still happens on the main UI thread. You should call prepareAsync or use an AsyncTask or some other means to avoid calling prepare on the main thread.
From the documentation for Handler:
When you create a new Handler, it is bound to the thread / message
queue of the thread that is creating it
If you are streaming music from the network, preparing the media for playback is especially going to take a while. One option may be to call prepareAsync instead of calling prepare. In that case, you should set the OnPreparedListener, and in that callback call start.
I'm developing a stream video app in Android with MediaPlayer. The problem is that I need to show the current bitrate, but I haven't found any valid suggestions on how to do get it?
Here is how I'm setting the video url to play:
mediaPlayer = new MediaPlayer();
try {
mediaPlayer.setDataSource(VIDEO_PATH);
mediaPlayer.prepare();
mediaPlayer.init();
} catch (IOException e) {
e.printStackTrace();
}
I don't know if the only way to get that working is using ExoPlayer (which I've read it may be possible)
Any suggestions?
Thanks!
Apparently you cannot do this with MediaPlayer but you can use MediaMetadataRetriever, which is available since API level 10, i.e., quite a while ago.
int getBitRate(String url) {
final MediaMetadataRetriever mmr = new MediaMetadataRetriever();
try {
mmr.setDataSource(url, Collections.EMPTY_MAP);
return Integer.parseInt(mmr.extractMetadata(MediaMetadataRetriever.METADATA_KEY_BITRATE));
} catch (NumberFormatException e) {
return 0;
} finally {
mmr.release();
}
}
The disadvantage this can have is that you will make an extra HTTP request for getting the metadata (only an RTT if you are streaming from an URI; if you are reading from an file descriptor it could be more serious). Hopefully no big deal.
I'm trying to better understand the behavior of threads in my android app. For some reason, when I use while(true) in one of my worker threads, code within that thread's run method that exists sequentially BEFORE the while(true) loop never executes. To be clear, I'm not sure if the code(toast messages) actually isn't executing or if the way the thread synchronization is handled by the Android OS is causing my Toast messages not to display. This behavior appears to be some sort of blocking but I can't figure out why this happens.
My app uses 3 threads: the UI thread(default/main thread in an Android app), a thread to infinitely read data from the device's USB port during runtime, and a thread to process this data via messages from the USB-read thread. The problem seems to occur in my USBController class. When I comment out my infinite while loop, all of the Toast messages before the start of the loop display just fine. When I don't comment out my while(true), NO TOAST MESSAGES EVER DISPLAY! I'm pretty confused by this, I think i'm misunderstanding something fundamental about thread handling by the Android OS. Even if a while loop were to cause blocking, which i don't think it since it resides in a worker thread, why wouldn't the toast messages that occur before the while loop be triggered? Is this a synchronization issue? Am I misusing Android's Handler-Looper system?
Code below. Note: I've included the relevant portion of the main activity and the entirety of the USBController class. My implementation of this class relies heavily on the USB to Serial library found here mik3y/usb-serial-for-android. I don't think it's necessary, but i've included the class that contains my third thread, SensorDataBuffer, that receives messages from the thread UsbController.
UsbController.java
public class UsbController extends Thread{
...
#Override
public void run() {
android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_DEFAULT); //sets thread to default queing priority
Looper.prepare();
Toast.makeText(mContext.getApplicationContext(), "Hello from UsbController's run method!", Toast.LENGTH_SHORT).show();
// **********************USB otg*******************************
//Obtain permission to use Android device's USB intent
PendingIntent mPermissionIntent;
mPermissionIntent = PendingIntent.getBroadcast(mContext, 0, new Intent(ACTION_USB_PERMISSION), 0);
// Find all available drivers from attached devices.
ProbeTable customTable = new ProbeTable();
customTable.addProduct(0x03EB, 0x2044, CdcAcmSerialDriver.class);
UsbManager manager = (UsbManager) mContext.getSystemService(Context.USB_SERVICE);
UsbSerialProber prober = new UsbSerialProber(customTable);
List<UsbSerialDriver> availableDrivers = prober.findAllDrivers(manager);
if (availableDrivers.isEmpty()) {
Toast.makeText(mContext.getApplicationContext(), "No available USB drivers found",Toast.LENGTH_SHORT).show(); // Toast message for debugging
}
else { // open connection to first avail. driver
UsbSerialDriver driver = availableDrivers.get(0);
Toast.makeText(mContext.getApplicationContext(), "Driver found",Toast.LENGTH_SHORT).show(); // Toast message for debugging
UsbDeviceConnection connection = manager.openDevice(driver.getDevice());
Toast.makeText(mContext.getApplicationContext(), "Device Driver Opened",Toast.LENGTH_SHORT).show(); // Toast message for debugging
if (connection == null) { // You probably need to call UsbManager.requestPermission(driver.getDevice(), ..)
Toast.makeText(mContext.getApplicationContext(),"Connection to device not allowed, need permissions",Toast.LENGTH_LONG).show();
manager.requestPermission(driver.getDevice(),mPermissionIntent); //conn test
if (manager.hasPermission(driver.getDevice())==true){
Toast.makeText(mContext.getApplicationContext(),"Permissions granted",Toast.LENGTH_SHORT).show();
}
}
else { // Read some data! Most have just one port (port 0).
List<UsbSerialPort> myPortList = driver.getPorts();
UsbSerialPort port = myPortList.get(0);
Toast.makeText(mContext.getApplicationContext(),"USB OTG Connection Established",Toast.LENGTH_SHORT).show();
try {
port.open(connection);
port.setParameters(9600, 8, UsbSerialPort.STOPBITS_1, UsbSerialPort.PARITY_NONE); // sets baud rate,databits, stopbits, & parity
port.setDTR(true); //necessary to make Arduino Micro begin running it's program
Toast.makeText(mContext.getApplicationContext(),"port opened, parameters set, DTR set",Toast.LENGTH_SHORT).show();
byte buffer[] = new byte[16];
String incompPacket = "";
Toast.makeText(mContext.getApplicationContext(), "hi again!"), Toast.LENGTH_LONG).show();
while (true){ //continuous loop to read data
numBytesRead = port.read(buffer, 100);
arduinoData = new String(buffer, "US-ASCII");
String raw = arduinoData.substring(0, numBytesRead);
if (numBytesRead > 0) {
...
}
}
} catch (IOException e) {
Toast.makeText(mContext, e.getMessage(), Toast.LENGTH_SHORT).show();
}
}
}
Looper.loop();
}
}
MainActivity.java
...
#Override
protected void onCreate(Bundle savedInstanceState) {
//Multi-threading
//Create thread to handle incoming data from USB Controller thread
SensorDataBuffer pressureDataBuffer = new SensorDataBuffer(MainActivity.this);
Thread bufferThread = new Thread(pressureDataBuffer);
bufferThread.start();
//Create USB Serial Worker thread which will continuously receive data
UsbController serialDataLink = new UsbController(PlayFrets.this);
Thread sensorMonitorThread = new Thread(serialDataLink);
sensorMonitorThread.start();
//Toast.makeText(this, "USB Controller thread started", Toast.LENGTH_SHORT).show();
//Build GUI
super.onCreate(savedInstanceState);
requestWindowFeature(Window.FEATURE_NO_TITLE); //Removes action bar from display
getWindow().setFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN, WindowManager.LayoutParams.FLAG_FULLSCREEN); //Removes status bar from display
//Create AsyncTask to load the note files. A splash screen will be displayed while task is executing
new AsyncTask_NoteFileLoader(this).execute();
}
...
SensorDataBuffer.java
public class SensorDataBuffer extends Thread{
//Handler subclass which accepts messages one by one in
//the main activitiy's FIFO message que called a "Looper"
//The worker thread, sensorMonitor, runs UsbController in parallel
//with the UI thread and continuously formats and sends pressure sensor
//values read from the microcontroller to the Handler which updates the
//corresponding pressure state logic variables in the UI thread.
public void run(){
android.os.Process.setThreadPriority(Process.THREAD_PRIORITY_URGENT_AUDIO); //TODO:priority was previously more favorable, test this to ensure UI doesn't lag
Looper.prepare(); //create MessageQue to receive messages from USB Controller thread
UsbController.setHandler(bufferHandler);
bufferHandler = new Handler(Looper.myLooper()) {
//do stuff
};
Looper.loop();
}
}
How about using HandlerThreads, Handlers and Runnables instead? Makes your code a lot cleaner and easier to maintain.
In your onCreate() just create a couple of them:
HandlerThread usbThread = new HandlerThread("USBController");
usbThread.start();
usbHandler = new Handler(usbThread.getLooper());
HandlerThread sensorThread = new HandlerThread("SensorDataBuffer");
sensorThread.start();
sensorHandler = new Handler(sensorThread.getLooper());
Then you create your Runnables and post them to the Handlers
usbHandler.post(new Runnable(){
run(){
//....
numBytesRead = port.read(buffer, 100);
if (numBytesRead > 0) {
sensorHandler.post(new Runnable(){run(){//doSomething}});
}
//....
if(isStillRunning)
usbHandler.post(this);
}
});
You can let the runnable post itself and it will run forever. From within you can post runnables to other handlers (like the Main Thread Handler) to show your Toasts.
Im quite new to android and i have searched about this for quite a while. I would like to build an application that is something like a decibel meter. In realtime it shows the sound level. It there is much noise in the room, there will be something indicating that, if its quiet something will indicate that!.
I don't have any idea at all how to do this. Could anyone explain what the basics of the microphone-sound-level application? If its possible, maybe provide some code?
Thanks!
You can use MediaRecorder.getMaxAmplitude().
if (ActivityCompat.checkSelfPermission(this, Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
ActivityCompat.requestPermissions(this, new String[]{Manifest.permission.RECORD_AUDIO},
RECORD_AUDIO);
}
Get the noise level using the MediaRecorder,
mRecorder = new MediaRecorder();
mRecorder.setAudioSource(MediaRecorder.AudioSource.MIC);
mRecorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP);
mRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB);
mRecorder.setOutputFile("/dev/null");
try {
mRecorder.prepare();
} catch (IllegalStateException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
mRecorder.start();
Start the MediaRecorder,
private Runnable mSleepTask = new Runnable() {
public void run() {
//Log.i("Noise", "runnable mSleepTask");
mSensor.start();
if (!mWakeLock.isHeld()) {
mWakeLock.acquire();
}
//Noise monitoring start
// Runnable(mPollTask) will execute after POLL_INTERVAL
mHandler.postDelayed(mPollTask, POLL_INTERVAL);
}
};
Create Runnable Thread to check the noise level frequently,
private Runnable mPollTask = new Runnable() {
public void run() {
double amp = mSensor.getAmplitude();
//Log.i("Noise", "runnable mPollTask");
// Runnable(mPollTask) will again execute after POLL_INTERVAL
mHandler.postDelayed(mPollTask, POLL_INTERVAL);
}
};
Convert the Amplitude to decibel using the following formula,
return 20 * Math.log10(mRecorder.getMaxAmplitude() / 2700.0);
Monitoring the Voice and Alert for the Louder Noise.
// Create runnable thread to Monitor Voice
private Runnable mPollTask = new Runnable() {
public void run() {
double amp = mSensor.getAmplitude();
//Log.i("Noise", "runnable mPollTask");
updateDisplay("Monitoring Voice...", amp);
if ((amp > mThreshold)) {
callForHelp(amp);
//Log.i("Noise", "==== onCreate ===");
}
// Runnable(mPollTask) will again execute after POLL_INTERVAL
mHandler.postDelayed(mPollTask, POLL_INTERVAL);
}
};
This question has been addressed generally for Java, and the required classes are available in Android.
The basic idea is to sample the data line for the microphone, and calculate the level from the returned buffer.
How to calculate the level/amplitude/db of audio signal in java?
You can also have a look at the Visualizer class which does FFT frequency analysis, however the permissions for microphone may not be consistent across various devices. You may also have to connect it to the Equalizer class to access the mic.
https://developer.android.com/reference/android/media/audiofx/Visualizer.html
There's a great app in the marketplace called Audalyzer
http://code.google.com/p/moonblink/wiki/Audalyzer
Also check this discussion..
Android: sample microphone without recording to get live amplitude/level?