We Keep Coding

Android webRTC video call inside a Background Service

Forgive me if this question was already asked, I couldn't find an answer for my case.
So, I have an Android app with Voice & Video call feature. I used webRTC for this.
I was able to make both Voice and Video call working perfectly inside an Activity, but now I want to keep the call running while the user exit the CallActivity and go back to the ChatActivity (to send a file/link/photo for example).
I managed to make the Voice call run perfectly inside a Background Service, but video call won't work as expected.
The remote video won't be displayed even though the audio from the video track is playing.
here is my Background Service code :
#Override
public void onAddStream(MediaStream mediaStream) {
if (mediaStream.videoTracks.size() > Constants.ONE || mediaStream.audioTracks.size() > Constants.ONE) {
return;
}
//check for video track, means this is a video call
if (!isAudioCall && mediaStream.videoTracks.size() > Constants.ZERO) {
remoteVideoTrack = mediaStream.videoTracks.get(Constants.ZERO);
CallActivityNew.remoteVideoTrack = remoteVideoTrack;
try {
localAudioTrack.setEnabled(true);
//Now ask the UI to display the video track
sendOrderToActivity(Constants.START_REMOTE_VIDEO, null);
} catch (Exception ignored) {}
} else if (mediaStream.audioTracks.size() > Constants.ZERO) {
//Means this is a Voice call, only audio tracks available
remoteAudioTrack = mediaStream.audioTracks.get(Constants.ZERO);
try {
localAudioTrack.setEnabled(true);
remoteAudioTrack.setEnabled(true);
} catch (Exception ignored) {}
}
}
and below my CallActivity code :
case Constants.START_REMOTE_VIDEO: {
if (remoteVideoView == null) {
remoteVideoView = findViewById(R.id.remote_gl_surface_view);
}
remoteVideoView.init(eglBaseContext, null);
remoteVideoView.setEnableHardwareScaler(true);
remoteVideoView.setMirror(true);
remoteVideoView.setScalingType(RendererCommon.ScalingType.SCALE_ASPECT_FIT);
remoteVideoView.setZOrderMediaOverlay(true);
//Apply video track to the Surface View in order to display it
remoteVideoTrack.addSink(remoteVideoView);
//now enable local video track
Handler handler = new Handler();
handler.postDelayed(new Runnable() {
#Override
public void run() {
//now enable local video track
remoteVideoTrack.setEnabled(true);
}
}, Constants.TIME_THREE_HUNDRED_MILLIS);
setSpeakerphoneOn(false);
break;
}
I am sending orders from Service to Activity, the "case Constants.START_REMOTE_VIDEO" work after receiving the order from Service.
I don't see where the problem, why am I only hearing sound but the remote video won't start display !!
Thank you in advance for helping.

After testing for long hours, I found that my code works just fine, I just forget to change the view visibility from "GONE" to "VISIBLE".
Yeah that was the solution, i swear xD

Handling Advertisements in Native app using Selenium Java

I am automating Android Native Gaming App, our app renders ads from different ad networks. After submitting each game you may see static ad and video ad or you may not see an ad at all. If video ad is found it may vary from 30 secs to 1 min.
Important thing, when i use Appium inspector to spy various video ad screens shall be identified only by Class(android.webkit.WebView, android.widget.VideoView, android.widget.Button, android.view.View, android.widget.Image & android.widget.ImageView.). After Ad complete playing we need to click on device back button and play next game.
Can you suggest any good approach to automate this kind of app? Any sample code is much appreciated.

Option I: you have is to ask your developers to create a version of the application without ads.
Advantages - No ads.
Disadvantages - You will not be testing the exact same code as you plan to release.
You can disable only fullscreen ad.
I think there is no best way to do this. Stable automate checks or checking the exact same code as you plan to release.
Option II: is to catch if the ad is visible, and press back button.
For example (example fo Android):
protected boolean checkAdvert(AppiumDriver<WebElement> driver, int timeout) {
By adTree = By.xpath("//hierarchy/android.widget.FrameLayout[1]/android.widget.LinearLayout[1]/android.widget.FrameLayout[1]/android.widget.RelativeLayout[1]");
Map<String, Object> adParams = new HashMap<>();
//trying to wait for the ad to come up and then click the Expense button
driver.manage().timeouts().implicitlyWait(0, TimeUnit.SECONDS);
driver.context("NATIVE_APP");
FluentWait<WebDriver> await = new FluentWait<WebDriver> (driver)
.withTimeout(timeout, TimeUnit.SECONDS)
.pollingEvery(500, TimeUnit.MILLISECONDS)
.ignoring(NoSuchElementException.class);
try {
await.until (ExpectedConditions.visibilityOf(driver.findElement(adTree)));
// go BACK to eliminate the popup
adParams.clear();
adParams.put("keySequence", "BACK");
driver.executeScript("mobile:presskey", adParams);
System.out.println("Press the back button to get out of ad");
return true;
} catch (Exception t) {
System.out.println("no ad showed up");
return false;
}
}
and use this in page object class:
public void addExp(String desc, String amount) {
do {
try {
driver.context("WEBVIEW");
driver.findElement(expDesc).sendKeys(desc);
driver.findElement(expAmnt).sendKeys(amount);
adClick = false;
} catch (NoSuchElementException ne) {
adClick = checkAdvert(driver, 1);
if (!adClick) throw ne;
}
} while (adClick);
}
But you must remember that ads may be different, you can try to find a universal selector. But I think it will hard to cover all cases.
adTree = By.xpath("//hierarchy/android.widget.FrameLayout[1]/android.widget.LinearLayout[1]/android.widget.FrameLayout[1]/android.widget.RelativeLayout[1]");

How to know if a specific app comes to foreground?

Check if an app, for example, Instagram is started by user.
Note: My app is targeting lollipop and above versions in android

Yeah the only way you can do it is through the Accessibility Service. Look at this page to understand how to create it. https://developer.android.com/training/accessibility/service.html They will also need to enable the service via the services -> accessibility screen.
AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED you can probably interrogate the package in front to figure out if Instigram is on top.
You definitely don't want to use getRunningTasks since the function was modified in Android 5.0+

I figured out that I can do this by using usage access feature.
#TargetApi(Build.VERSION_CODES.LOLLIPOP)
public static String getForegroundProcess(Context context) {
String topPackageName = null;
UsageStatsManager usage = (UsageStatsManager) context.getSystemService(Context.USAGE_STATS_SERVICE);
long time = System.currentTimeMillis();
List<UsageStats> stats = usage.queryUsageStats(UsageStatsManager.INTERVAL_DAILY, time - 1000*1000, time);
if (stats != null) {
SortedMap<Long, UsageStats> runningTask = new TreeMap<Long,UsageStats>();
for (UsageStats usageStats : stats) {
runningTask.put(usageStats.getLastTimeUsed(), usageStats);
}
if (runningTask.isEmpty()) {
return null;
}
topPackageName = runningTask.get(runningTask.lastKey()).getPackageName();
}
if(topPackageName==null) {
Intent intent = new Intent(Settings.ACTION_USAGE_ACCESS_SETTINGS);
context.startActivity(intent);
}
return topPackageName;
}
Now continuously check if the desired app is in the foreground.
String fg=getForegroundProcess(getApplicationContext());
if(fg != null && fg.contains("com.instagram.android")){
//Instagram is in foreground
}else {
}
I continuously run the above code with a job service.Which is available for
lollipop and above.

Changing Flash setting of Android Camera 2 at runtime

Basically, what I am trying to do is change the CONTROL_AE_MODE by button click in the app. The user can use AUTO flash(ON_AUTO_FLASH), turn if ON(ON_ALWAYS_FLASH), or OFF(CONTROL_AE_MODE_OFF).
In this example: https://github.com/googlesamples/android-Camera2Basic/blob/master/Application/src/main/java/com/example/android/camera2basic/Camera2BasicFragment.java
Line 818, they set the flash once:
// Use the same AE and AF modes as the preview.
captureBuilder.set(CaptureRequest.CONTROL_AF_MODE,
CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE);
setAutoFlash(captureBuilder);
// Orientation
int rotation = activity.getWindowManager().getDefaultDisplay().getRotation();
captureBuilder.set(CaptureRequest.JPEG_ORIENTATION, ORIENTATIONS.get(rotation));
CameraCaptureSession.CaptureCallback CaptureCallback
= new CameraCaptureSession.CaptureCallback() {
#Override
public void onCaptureCompleted(#NonNull CameraCaptureSession session,
#NonNull CaptureRequest request,
#NonNull TotalCaptureResult result) {
showToast("Saved: " + mFile);
Log.d(TAG, mFile.toString());
unlockFocus();
}
};
mCaptureSession.stopRepeating();
mCaptureSession.capture(captureBuilder.build(), CaptureCallback, null);
And then builds the CaptureSession at line 840.
Is there a way to change the CONTROL_AE_MODE after the preview is made?
I have tried remaking the session, which kinda worked:
if(flashMode == CameraView.CAMERA_FLASH_ON){
Log.e("CAMERA 2", "FLASH ON");
mPreviewCaptureRequestBuilder.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_ON_ALWAYS_FLASH);
}else if(flashMode == CameraView.CAMERA_FLASH_OFF){
Log.e("CAMERA 2", "FLASH OFF");
mPreviewCaptureRequestBuilder.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_OFF);
}else if(flashMode == CameraView.CAMERA_FLASH_AUTO){
Log.e("CAMERA 2", "FLASH AUTO");
mPreviewCaptureRequestBuilder.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_ON_AUTO_FLASH);
}
mFlashMode = flashMode;
if (mCameraCaptureSession != null) {
mCameraCaptureSession.close();
mCameraCaptureSession = null;
}
createCameraPreviewSession();
For some reason, CONTROL_AE_MODE_OFF would turn the whole preview black.
I tried looking in the docs for methods to update but haven't found anything.
Any tutorials or docs is much appreciated.

As mentioned by #cyborg86pl when switching flash modes you should not switch CONTROL_AE_MODE . Instead you can switch between FLASH_MODE´s. Here is a working example for my case:
when (currentFlashState) {
FlashState.AUTO -> {
previewRequestBuilder.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_ON_AUTO_FLASH)
}
FlashState.ON -> {
previewRequestBuilder.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_ON)
previewRequestBuilder.set(CaptureRequest.FLASH_MODE, CameraMetadata.FLASH_MODE_TORCH)
}
FlashState.OFF -> {
previewRequestBuilder.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_ON)
previewRequestBuilder.set(CaptureRequest.FLASH_MODE, CaptureRequest.FLASH_MODE_OFF)
}
}
previewRequest = previewRequestBuilder.build()
captureSession.setRepeatingRequest(previewRequest, captureCallback, backgroundHandler)

I don't know why your preview turn black, but you don't need to close capture session manually. From .close() method's docs:
Using createCaptureSession(List , CameraCaptureSession.StateCallback,
Handler) directly without closing is the recommended approach for
quickly switching to a new session, since unchanged target outputs can
be reused more efficiently.
So you can reuse existing CaptureRequest.Builder, set your changed value, build new PreviewRequest and just start new session with this new request, like this:
try {
// Change some capture settings
mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_ON);
// Build new request (we can't just edit existing one, as it is immutable)
mPreviewRequest = mPreviewRequestBuilder.build();
// Set new repeating request with our changed one
mCaptureSession.setRepeatingRequest(mPreviewRequest, mCaptureCallback, mBackgroundHandler);
} catch (CameraAccessException e) {
e.printStackTrace();
}
It will be much faster (almost without any visible freeze of preview).

What you want is disabling flash, not auto-exposure (AE), thus you want to use CONTROL_AE_MODE_ON rather than CONTROL_AE_MODE_OFF.
As mentioned in the documentation:
CONTROL_AE_MODE_ON
The camera device's autoexposure routine is active, with no flash control.

playing video using jmf

I am trying to play a video file using JMF but it gives me No Media Player found exception.
Here is my code, can anyone tell me what I am doing wrong here?
public class MediaPanel extends JPanel {
public MediaPanel(URL mediaURL) {
setLayout(new BorderLayout());
try {
Manager.setHint(Manager.LIGHTWEIGHT_RENDERER, true);
Player mediaPlayer = Manager.createRealizedPlayer(mediaURL);
Component video = mediaPlayer.getVisualComponent();
Component controls = mediaPlayer.getControlPanelComponent();
if (video != null)
add(video, BorderLayout.CENTER);
if (controls != null)
add(controls, BorderLayout.SOUTH);
mediaPlayer.start();
} catch (NoPlayerException noPlayerException) {
System.err.println("No media player found");
} // end catch
catch (CannotRealizeException cannotRealizeException) {
System.err.println("Could not realize media player");
} // end catch
catch (IOException iOException) {
System.err.println("Error reading from the source");
}
}
}
public class MediaTest {
public static void main(String args[]) {
// create a file chooser
JFileChooser fileChooser = new JFileChooser();
// show open file dialog
int result = fileChooser.showOpenDialog(null);
if (result == JFileChooser.APPROVE_OPTION) // user chose a file
{
URL mediaURL = null;
Player mediaPlayer = null;
try {
// get the file as URL
mediaURL = fileChooser.getSelectedFile().toURL();
} catch (MalformedURLException malformedURLException) {
System.err.println("Could not create URL for the file");
}
if (mediaURL != null) {
JFrame mediaTest = new JFrame("Media Tester");
mediaTest.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
MediaPanel mediaPanel = new MediaPanel(mediaURL);
mediaTest.add(mediaPanel);
mediaTest.setSize(300, 300);
mediaTest.setVisible(true);
}
}
}
}
The exception that I am getting is No media player found

What kind of video are you trying to play? JMF is a pretty old library and won't be able to play most of modern video formats, only a few old ones (i am not even sure which ones).
Actually, if I am right, to play something specific you will have to write/add your own video-encoders into JMF or at least download and use existing ones, which are usually outdated.
If you really want to have something like tunable video player that could play any modern video there are two options (in my opinion):
Use vlcj library to embed VLC video player into your Java-application
USe JavaFX media player
I am offering only those two because I have dig through tons of libraries some time ago and there were nothing else even close to these two. Plus most of other libraries are outdated as well as JMF itself and these two are getting frequent updates and are supported with lots of users so those two are the best choice.
In case you don't mind embedding Java FX player into your application - that might be your choice.
On the other hand - vlcj is stable and easily integrated into Swing applications (its not like its hard with Java FX, but vlcj might be better for some cases).
Anyway, it is your call what to choose.