Gaps in audio when connecting to a Bluetooth device - java
I am using SSML, so my app can speak. The app itself works perfectly fine on my phone BUT when I connect my phone with a device over Bluetooth, there is mostly a gap or a delay. Either at the beginning or in the middle of the speech.
So for instance, when the audio is Hello John, I am your assistant. How can I help you?, the output could be sistant. How can I help you?. Sometimes the sentences are fluent but sometimes there are these gaps.
This is how I play the audio file:
String myFile = context.getFilesDir() + "/output.mp3";
mMediaPlayer.reset();
mMediaPlayer.setDataSource(myFile);
mMediaPlayer.prepare();
mMediaPlayer.start();
And this is the entire class of it:
public class Tts {
public Context context;
private final MediaPlayer mMediaPlayer;
public Tts(Context context, MediaPlayer mMediaPlayer) {
this.context = context;
this.mMediaPlayer = mMediaPlayer;
}
#SuppressLint({"NewApi", "ResourceType", "UseCompatLoadingForColorStateLists"})
public void say(String text) throws Exception {
InputStream stream = context.getResources().openRawResource(R.raw.credential); // R.raw.credential is credential.json
GoogleCredentials credentials = GoogleCredentials.fromStream(stream);
TextToSpeechSettings textToSpeechSettings =
TextToSpeechSettings.newBuilder()
.setCredentialsProvider(
FixedCredentialsProvider.create(credentials)
).build();
// Instantiates a client
try (TextToSpeechClient textToSpeechClient = TextToSpeechClient.create(textToSpeechSettings)) {
// Replace {name} with target
SharedPreferences sharedPreferences = context.getSharedPreferences("target", Context.MODE_PRIVATE);
String target = sharedPreferences.getString("target", null);
text = (target != null) ? text.replace("{name}", target) : text.replace("null", "");
// Set the text input to be synthesized
String myString = "<speak><prosody pitch=\"low\">" + text + "</prosody></speak>";
SynthesisInput input = SynthesisInput.newBuilder().setSsml(myString).build();
// Build the voice request, select the language code ("en-US") and the ssml voice gender
// ("neutral")
VoiceSelectionParams voice =
VoiceSelectionParams.newBuilder()
.setName("de-DE-Wavenet-E")
.setLanguageCode("de-DE")
.setSsmlGender(SsmlVoiceGender.MALE)
.build();
// Select the type of audio file you want returned
AudioConfig audioConfig =
AudioConfig.newBuilder().setAudioEncoding(AudioEncoding.MP3).build();
// Perform the text-to-speech request on the text input with the selected voice parameters and
// audio file type
SynthesizeSpeechResponse response = textToSpeechClient.synthesizeSpeech(input, voice, audioConfig);
// Get the audio contents from the response
ByteString audioContents = response.getAudioContent();
// Write the response to the output file.
try (FileOutputStream out = new FileOutputStream(context.getFilesDir() + "/output.mp3")) {
out.write(audioContents.toByteArray());
}
String myFile = context.getFilesDir() + "/output.mp3";
mMediaPlayer.setAudioAttributes(new AudioAttributes.Builder().setContentType(AudioAttributes.CONTENT_TYPE_MUSIC).build());
mMediaPlayer.reset();
mMediaPlayer.setDataSource(myFile);
mMediaPlayer.prepare();
mMediaPlayer.setOnPreparedListener(mediaPlayer -> mMediaPlayer.start());
}
}
}
The distance cannot be the reason, since my phone is right next to the device.
Google's SSML needs an internet connection. So I am not quite sure if the gap is because of Bluetooth or internet connection.
So I am trying to close the gap, no matter what the reason is. The audio should be played, when it is prepared and ready to be played.
What I tried
This is what I have tried but I don't hear a difference:
mMediaPlayer.setAudioAttributes(new AudioAttributes.Builder().setContentType(AudioAttributes.CONTENT_TYPE_SPEECH).build());
Instead of mMediaPlayer.prepare(), I also tried it with mMediaPlayer.prepareAsync() but then the audio will not be played (or at least I can't hear it).
Invoking start() in a listener:
mMediaPlayer.setOnPreparedListener(mediaPlayer -> {
mMediaPlayer.start();
});
Unfortunately, the gap is sometimes still there.
Here is my proposed solution. Check out the // *** comments in the code to see what I changed in respect to your code from the question.
Also take it with a grain of salt, because I have no way of testing that right now.
Nevertheless - as far as I can tell - that is all you can do using the MediaPlayer API. If that still doesn't work right for your BlueTooth device, you should try a different BlueTooth device and if that doesn't help either, maybe you can switch the whole thing to use the AudioTrack API instead of MediaPlayer, which gives you a low latency setting and you could use the audio data directly from the response instead of writing it to a file and reading it from there again.
public class Tts {
public Context context;
private final MediaPlayer mMediaPlayer;
public Tts(Context context, MediaPlayer mMediaPlayer) {
this.context = context;
this.mMediaPlayer = mMediaPlayer;
}
#SuppressLint({"NewApi", "ResourceType", "UseCompatLoadingForColorStateLists"})
public void say(String text) throws Exception {
InputStream stream = context.getResources().openRawResource(R.raw.credential); // R.raw.credential is credential.json
GoogleCredentials credentials = GoogleCredentials.fromStream(stream);
TextToSpeechSettings textToSpeechSettings =
TextToSpeechSettings.newBuilder()
.setCredentialsProvider(
FixedCredentialsProvider.create(credentials)
).build();
// Instantiates a client
try (TextToSpeechClient textToSpeechClient = TextToSpeechClient.create(textToSpeechSettings)) {
// Replace {name} with target
SharedPreferences sharedPreferences = context.getSharedPreferences("target", Context.MODE_PRIVATE);
String target = sharedPreferences.getString("target", null);
text = text.replace("{name}", (target != null) ? target : ""); // *** bug fixed
// Set the text input to be synthesized
String myString = "<speak><prosody pitch=\"low\">" + text + "</prosody></speak>";
SynthesisInput input = SynthesisInput.newBuilder().setSsml(myString).build();
// Build the voice request, select the language code ("en-US") and the ssml voice gender
// ("neutral")
VoiceSelectionParams voice =
VoiceSelectionParams.newBuilder()
.setName("de-DE-Wavenet-E")
.setLanguageCode("de-DE")
.setSsmlGender(SsmlVoiceGender.MALE)
.build();
// Select the type of audio file you want returned
AudioConfig audioConfig =
AudioConfig.newBuilder().setAudioEncoding(AudioEncoding.MP3).build();
// Perform the text-to-speech request on the text input with the selected voice parameters and
// audio file type
SynthesizeSpeechResponse response = textToSpeechClient.synthesizeSpeech(input, voice, audioConfig);
// Get the audio contents from the response
ByteString audioContents = response.getAudioContent();
// Write the response to the output file.
try (FileOutputStream out = new FileOutputStream(context.getFilesDir() + "/output.mp3")) {
out.write(audioContents.toByteArray());
}
String myFile = context.getFilesDir() + "/output.mp3";
mMediaPlayer.reset();
mMediaPlayer.setDataSource(myFile);
mMediaPlayer.setAudioAttributes(new AudioAttributes.Builder() // *** moved here (should be done before prepare and very likely AFTER reset)
.setContentType(AudioAttributes.CONTENT_TYPE_SPEECH) // *** changed to speech
.setUsage(AudioAttributes.USAGE_ASSISTANT) // *** added
.setFlags(AudioAttributes.FLAG_AUDIBILITY_ENFORCED) // *** added
.build());
mMediaPlayer.prepare();
// *** following line changed since handler was defined AFTER prepare and
// *** the prepare call isn't asynchronous, thus the handler would never be called.
mMediaPlayer.start();
}
}
}
Hope that get's you going!
Related
Stream audio from aws s3 to Discord in java
I am trying to make a discord bot that plays custom sounds, i put the sounds in a aws s3 bucket and i can retrieve them but i dont know how to stream them to discord, i can stream audio files saved locally just fine, to stream local files i use lavaplayer. This is how i get the file from the s3 bucket: fullObject = s3Client.getObject(new GetObjectRequest("bucket-name", audioName)); System.out.println("Content-Type: " + fullObject.getObjectMetadata().getContentType()); S3ObjectInputStream s3is = fullObject.getObjectContent(); This i how i play the local files with lavaplayer: String toPlay = "SoundBoard" + File.separator + event.getArgs(); MessageChannel channel = event.getChannel(); AudioChannel myChannel = event.getMember().getVoiceState().getChannel(); AudioManager audioManager = event.getGuild().getAudioManager(); AudioPlayerManager playerManager = new DefaultAudioPlayerManager(); AudioPlayer player = playerManager.createPlayer(); AudioPlayerSendHandler audioPlayerSendHandler = new AudioPlayerSendHandler(player); audioManager.setSendingHandler(audioPlayerSendHandler); audioManager.openAudioConnection(myChannel); TrackScheduler trackScheduler = new TrackScheduler(player); player.addListener(trackScheduler); playerManager.registerSourceManager(new LocalAudioSourceManager()); playerManager.loadItem(toPlay, new AudioLoadResultHandler() { #Override public void trackLoaded(AudioTrack track) { trackScheduler.addQueue(track); } #Override public void noMatches() { channel.sendMessage("audio not found").queue(); trackScheduler.addQueue(null); } #Override public void loadFailed(FriendlyException throwable) { System.out.println("error " + throwable.getMessage()); } }); player.playTrack(trackScheduler.getTrack()); So is there a way to stream the files directly with lavaplayer or in another way? (im trying to avoid saving the audio to a file then playing it and then deleting it)
Play the audio without saving it in a file
I am using the Google API Text-To-Speech and I would like to simply hear "Hello World". This is what I have so far: /** Demonstrates using the Text-to-Speech API. */ #RequiresApi(api = Build.VERSION_CODES.KITKAT) public void hello() throws Exception { InputStream stream = getResources().openRawResource(R.raw.credential); // R.raw.credential is credential.json GoogleCredentials credentials = GoogleCredentials.fromStream(stream); TextToSpeechSettings textToSpeechSettings = TextToSpeechSettings.newBuilder() .setCredentialsProvider( FixedCredentialsProvider.create(credentials) ).build() ; // Instantiates a client try (TextToSpeechClient textToSpeechClient = TextToSpeechClient.create(textToSpeechSettings)) { // Set the text input to be synthesized SynthesisInput input = SynthesisInput.newBuilder().setText("Hello, World!").build(); // Build the voice request, select the language code ("en-US") and the ssml voice gender // ("neutral") VoiceSelectionParams voice = VoiceSelectionParams.newBuilder() .setLanguageCode("en-US") .setSsmlGender(SsmlVoiceGender.NEUTRAL) .build(); // Select the type of audio file you want returned AudioConfig audioConfig = AudioConfig.newBuilder().setAudioEncoding(AudioEncoding.MP3).build(); // Perform the text-to-speech request on the text input with the selected voice parameters and // audio file type SynthesizeSpeechResponse response = textToSpeechClient.synthesizeSpeech(input, voice, audioConfig); // Get the audio contents from the response ByteString audioContents = response.getAudioContent(); // Write the response to the output file. try (OutputStream out = new FileOutputStream("output.mp3")) { out.write(audioContents.toByteArray()); System.out.println("Audio content written to file \"output.mp3\""); } } } I get the error: java.io.FileNotFoundException: output.mp3 (Read-only file system) Most of the codes I have copied from Google's documentation but I don't even want to save that audio in a file. The text "Hello, World!" should simply be played without being saved first. Is this possible?
Error: Could not execute method for android:onClick
I have found this code on a Google documentation page (Android Studio changed it a bit automatically): #RequiresApi(api = Build.VERSION_CODES.KITKAT) public static void ssmlToAudio(String ssmlText, String outFile) throws Exception { // Instantiates a client try (TextToSpeechClient textToSpeechClient = TextToSpeechClient.create()) { // Set the ssml text input to synthesize SynthesisInput input = SynthesisInput.newBuilder().setSsml(ssmlText).build(); // Build the voice request, select the language code ("en-US") and // the ssml voice gender ("male") VoiceSelectionParams voice = VoiceSelectionParams.newBuilder() .setLanguageCode("en-US") .setSsmlGender(SsmlVoiceGender.MALE) .build(); // Select the audio file type AudioConfig audioConfig = AudioConfig.newBuilder().setAudioEncoding(AudioEncoding.MP3).build(); // Perform the text-to-speech request on the text input with the selected voice parameters and // audio file type SynthesizeSpeechResponse response = textToSpeechClient.synthesizeSpeech(input, voice, audioConfig); // Get the audio contents from the response ByteString audioContents = response.getAudioContent(); // Write the response to the output file try (OutputStream out = new FileOutputStream(outFile)) { out.write(audioContents.toByteArray()); System.out.println("Audio content written to file " + outFile); } } } I would like to run this method on a click event. So this is what I have tried so far: #RequiresApi(api = Build.VERSION_CODES.LOLLIPOP) public void onClick(View view) throws Exception { ssmlToAudio("Hello", "test"); } But if I run my app and click on a button, I'll get this error: java.lang.IllegalStateException: Could not execute method for android:onClick What am I doing wrong?
You have to implement the onClickListener in your activity and then override the onClick method.
How can I delete a pre-existing image from storage before re-downloading using DownloadManager?
I am writing code for an Android app using Eclipse that is supposed to download an image from a URL (which is generated by the app, elsewhere in the code, using GPS information), then attach the newly downloaded image to an e-mail to be sent. I am able to, in general, accomplish this without much issue. My problem is this: I only want one image downloaded by the app to be present in the device's external storage at any given time. Deleting the image after the email intent does not work, because because the app doesn't always call onStop or onDestroy when switching to another app to send the email. Time-sensitive deleting of the image will not work either, because I cannot assume that the user will send only one email from the app per hour. I want to give the user the freedom of sending as many of these emails (with one newly downloaded image, each) as they wish. My current method (which works MOST of the time) is this: in the downloadFile method, simply check for the file's existence (I call it sensorMap.png), then delete it if it exists, before downloading a new one. This SHOULD ensure that there may be only one sensorMap.png image in external storage at any given time (EDIT: it does do this), and that when it comes time to attach the image to the email intent, there will be exactly one image ready to go. Instead, I see that sometimes a second sensorMap image is sometimes being downloaded into storage (i.e. "sensorMap-1.png"), OR the image cannot be attached to the email due to a "File size: 0 bytes" error, OR the image cannot be attached due to a "File does not exist" error. I am unsure what the difference between the latter two problems is. EDIT: Upon manually examining the contents of the directory I created, it seems that, as intended, I end up with only one image titled "sensorMap.png" at a time; it remains in the directory after the app closes, as expected. However, I still occasionally get the "File size: 0 bytes" message or the "File does not exist." message with no attached image, even though I see that the image DOES exist upon looking in directory afterwards. Other times, everything works just fine. It's rather bewildering. In addition, there is an issue of the button which sends the email becoming unresponsive occasionally. Most of the time, it prompts the user to select an email client, as intended, but occasionally the button will LOOK as if clicked, but do nothing. When this happens, the logcat does not sense that the button was even clicked (I inserted a println statement to test it). I am unsure of why my delete-before-download is not working flawlessly; the basic idea, at least, appears to be logically sound. Here is the code pertaining to my issue: Code used to download file (in MainCountActivity.java): //Function to download image given URL. Will use to attach image file to email. public void downloadFile(String uRl) { //delete existing file first so that only one sensorMap image exists in memory //at any given time. File file = new File(Environment.getExternalStorageDirectory()+"/SensorLocationImages"); File checkFile = new File(Environment.getExternalStorageDirectory()+"/SensorLocationImages/sensorMap.png"); if(checkFile.exists()) { //debugging: System.out.println("About to delete file!"); //deleteFiles(Environment.getExternalStorageDirectory()+"/SensorLocationImages"); checkFile.delete(); } DownloadManager mgr = (DownloadManager) getActivity().getSystemService(Context.DOWNLOAD_SERVICE); Uri downloadUri = Uri.parse(uRl); DownloadManager.Request request = new DownloadManager.Request( downloadUri); request.setAllowedNetworkTypes( DownloadManager.Request.NETWORK_WIFI | DownloadManager.Request.NETWORK_MOBILE) .setAllowedOverRoaming(false).setTitle("Sensor Location Map") .setDescription("Pinpointed is the location from which the log file was sent.") .setDestinationInExternalPublicDir("/SensorLocationImages", "sensorMap.png"); mgr.enqueue(request); } public Activity getActivity() //I wasn't sure if this would work, but it did. Or at least appears to. { return this; } Method to send email (in MainCountActivity.java): public void sendEmail(String toAddress, String ccAddress, String bccAddress, String subject, String body, String attachmentMimeType) throws Exception{ try { Intent emailIntent = new Intent(Intent.ACTION_SEND_MULTIPLE); emailIntent.setType(attachmentMimeType); //new String sToAddress[] = { toAddress }; String sCCAddress[] = { ccAddress}; String sBCCAddress[] = { bccAddress }; emailIntent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK); emailIntent.putExtra(Intent.EXTRA_EMAIL, sToAddress); emailIntent.putExtra(android.content.Intent.EXTRA_CC, sCCAddress); emailIntent.putExtra(android.content.Intent.EXTRA_BCC, sBCCAddress); emailIntent.putExtra(Intent.EXTRA_SUBJECT, subject); emailIntent.putExtra(Intent.EXTRA_TEXT, body); //get URI of logfile File tempFile = new File(Environment.getExternalStorageDirectory () + MainCountActivity.dirPath); Uri uri = Uri.fromFile(tempFile); //create URI arraylist and add first URI ArrayList<Uri> uris = new ArrayList<Uri>(); uris.add(uri); //get URI of map image and add to arraylist //make sure it is there to attach File file = new File(Environment.getExternalStorageDirectory()+"/SensorLocationImages"); do { downloadFile(getMapLink()); //createDirectoryAndSaveFile(getBitmapFromURL(getMapLink()), "sensorMap.png"); } while (!file.exists()); uris.add(Uri.fromFile(new File(Environment .getExternalStorageDirectory() + "/SensorLocationImages/sensorMap.png"))); //+ "/sdcard/SensorLocationImages/sensorMap.png"))); emailIntent.putParcelableArrayListExtra(Intent.EXTRA_STREAM, uris); startActivity(emailIntent); } catch(Exception ex) { ex.printStackTrace(); throw ex; } } OnClick method, for my occasional button issue (In MaincountActivity.java): public void onClick(View v){ switch(v.getId()) { case R.id.textView1: { break; } case R.id.Reset: { //allowCounting will let the program know when to let it to count or not, depending if Start or Stop button are pressed. logCount=0; mCounter.setText("Total: 0"); mToggle.setChecked(false); break; } /* case R.id.toggleButton: { break; }*/ case R.id.SendEmail: { //for debugging purposes: System.out.println("Email button being clicked!"); LocationManager locationManager = (LocationManager) getSystemService(LOCATION_SERVICE); if (locationManager.isProviderEnabled(LocationManager.GPS_PROVIDER)) { Toast.makeText(this, "GPS is enabled in your device", Toast.LENGTH_SHORT).show(); try { sendEmail("","","","Sensor Log Info",getEmailBody(),"multipart/mixed"); } catch (Exception e) { e.printStackTrace(); } } else { showGPSAlertForEmail(); } break; } } Basically, I really want to know why my delete-then-download method has not worked every time. Logcat errors have provided no insight. Thank you for your time.
How can I intercept the audio stream on an android device?
Let's suppose that we have the following scenario: something is playing on an android device (an mp3 par example, but it could be anything that use the audio part of an android device). From an application (android application :) ), I would like to intercept the audio stream to analyze it, to record it, etc. From this application (let's say "the analyzer") I don't want to start an mp3 or something, all I want is to have access to the audio stream of android. Any advice is appreciated, it could a Java or C++ solution.
http://developer.android.com/reference/android/media/MediaRecorder.html public class AudioRecorder { final MediaRecorder recorder = new MediaRecorder(); final String path; /** * Creates a new audio recording at the given path (relative to root of SD * card). */ public AudioRecorder(String path) { this.path = sanitizePath(path); } private String sanitizePath(String path) { if (!path.startsWith("/")) { path = "/" + path; } if (!path.contains(".")) { path += ".3gp"; } return Environment.getExternalStorageDirectory().getAbsolutePath() + path; } /** * Starts a new recording. */ public void start() throws IOException { String state = android.os.Environment.getExternalStorageState(); if (!state.equals(android.os.Environment.MEDIA_MOUNTED)) { throw new IOException("SD Card is not mounted. It is " + state + "."); } // make sure the directory we plan to store the recording in exists File directory = new File(path).getParentFile(); if (!directory.exists() && !directory.mkdirs()) { throw new IOException("Path to file could not be created."); } recorder.setAudioSource(MediaRecorder.AudioSource.MIC); recorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP); recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB); recorder.setOutputFile(path); recorder.prepare(); recorder.start(); } /** * Stops a recording that has been previously started. */ public void stop() throws IOException { recorder.stop(); recorder.release(); } }
Consider using the AudioPlaybackCapture API that was introduced in Android 10 if you want to get the audio stream for a particular app.