We Keep Coding

GCP Speech to text - Java API not working

I have a sample .webm file recorded using MediaRecorder in Chrome Browser. When I use Google speech java client to get transcription for the video, it returns empty transcription. Here is what my code looks like
SpeechSettings settings = null;
Path path = Paths.get("D:\\scrap\\gcp_test.webm");
byte[] content = null;
try {
content = Files.readAllBytes(path);
settings = SpeechSettings.newBuilder().setCredentialsProvider(credentialsProvider).build();
} catch (IOException e1) {
throw new IllegalStateException(e1);
}
try (SpeechClient speech = SpeechClient.create(settings)) {
// Builds the request for remote FLAC file
RecognitionConfig config = RecognitionConfig.newBuilder()
.setEncoding(AudioEncoding.LINEAR16)
.setLanguageCode("en-US")
.setUseEnhanced(true)
.setModel("video")
.setEnableAutomaticPunctuation(true)
.setSampleRateHertz(48000)
.build();
RecognitionAudio audio = RecognitionAudio.newBuilder().setContent(ByteString.copyFrom(content)).build();
// RecognitionAudio audio = RecognitionAudio.newBuilder().setUri("gs://xxxx/gcp_test.webm") .build();
// Use blocking call for getting audio transcript
RecognizeResponse response = speech.recognize(config, audio);
List<SpeechRecognitionResult> results = response.getResultsList();
for (SpeechRecognitionResult result : results) {
SpeechRecognitionAlternative alternative = result.getAlternativesList().get(0);
System.out.printf("Transcription: %s%n", alternative.getTranscript());
}
} catch (Exception e) {
e.printStackTrace();
System.err.println(e.getMessage());
}
If, I use the same file and visit https://cloud.google.com/speech-to-text/ and upload file in the demo section. It seems to work fine and shows transcription. I am clueless about whats going wrong here. I verified the request sent by demo and here it what looks like
I am sending the exact set of parameters, but that didn't work. Tried uploading file to Cloud storage but that too gave same result (no transcription).

After going through error and trials (and looking at the javascript samples), I could solve the issue. The serialized version of audio should be in FLAC format. I was sending the video file(webm) as is to Google Cloud. The demo on the site extracts audio stream using Javascript Audio API and then sents the data in base64 format to make it work.
Here are the steps that I executed to get the output.
Used FFMPEG to extract audio stream into FLAC format from webm.
ffmpeg -i sample.webm -vn -acodec flac sample.flac
The extracted file should be made available using either Storage cloud or send as ByteString.
Set the appropriate model while calling the speech API (for english language video model works, while for french language command_and_search). I don't have any logical reason for this. I realised it after trial and error with demo on Google cloud site.

I got results with flac encoded file.
Sample code results words with timestamp,
public class SpeechToTextSample {
public static void main(String... args) throws Exception {
try (SpeechClient speechClient = SpeechClient.create()) {
String gcsUriFlac = "gs://yourfile.flac";
RecognitionConfig config =
RecognitionConfig.newBuilder()
.setEncoding(AudioEncoding.FLAC)
.setEnableWordTimeOffsets(true)
.setLanguageCode("en-US")
.build();
RecognitionAudio audio = RecognitionAudio.newBuilder().setUri(gcsUriFlac).build(); //for large files
OperationFuture<LongRunningRecognizeResponse, LongRunningRecognizeMetadata> response = speechClient.longRunningRecognizeAsync(config, audio);
while (!response.isDone()) {
System.out.println("Waiting for response...");
Thread.sleep(1000);
}
// Performs speech recognition on the audio file
List<SpeechRecognitionResult> results = response.get().getResultsList();
for (SpeechRecognitionResult result : results) {
SpeechRecognitionAlternative alternative = result.getAlternativesList().get(0);
System.out.printf("Transcription: %s%n", alternative.getTranscript());
for (WordInfo wordInfo : alternative.getWordsList()) {
System.out.println(wordInfo.getWord());
System.out.printf(
"\t%s.%s sec - %s.%s sec\n",
wordInfo.getStartTime().getSeconds(),
wordInfo.getStartTime().getNanos() / 100000000,
wordInfo.getEndTime().getSeconds(),
wordInfo.getEndTime().getNanos() / 100000000);
}
}
}
}
}
GCP supports different languages, I have used "en-US" for my example.
Please refer following link document to know language list.

Java picam preview

I am currently trying to incorporate a temporary stream in my card panel layed out like this.
Layout
Where it says scan your QRCode im trying to get a stream from the picam. Here is the issue.
I dont know how i can buffer that into the Java application
I got the command i want --> "raspistill -w 200 -h 200 -q 100 -t 5", but i just dont know how this would work. This is the first time im dealing with any video stream.
As for the second part. I need that "preview" to take a picture whenever it is able to grab the QRCode. I checked both apis for the raspberry pi camera but im still lost as in for direction. I also need this buffered so i can instantly parse it into my decodeQRCode method. What component do i need to accomplish this?

I decided to go with sarxos webcam api.
class VideoFeed extends Thread {
public void run() {
webcam.open();
boolean bool = true;
while (bool) {
try {
BufferedImage image = webcam.getImage();
var = BackEnd.refund(image,type[0]);
lblCamera.setIcon(new ImageIcon(image));
if (var[0] != null) {
bool = false;
webcam.close();
btnScan.doClick();
} else {
Thread.sleep(10);
}
} catch (InterruptedException ex) {
System.out.println("Error: " + ex);
}
}
}
}
started it by doing
new VideoFeed().start();
With this i opted for a more general driver.

JavaFX MediaPlayer highly inaccurate seeking

I'm using 320 kbps roughly 1 hour long MP3 files. The project I'm working on would seek in a collection of music inside an MP3 file so that it can shuffle the songs. I would give the timestamps to the program and it would seek to the song. It would work if JavaFX's seek method wasn't highly inaccurate.
After using MediaPlayer.seek(duration) The MediaPlayer.getCurrentTime() returns the duration we seeked to as expected. However if we listen to the mp3 file(either without seeking or in an external mp3 player) we realize that the time reported and reality is very different, sometimes even seconds.
For example MediaPlayer.seek(Duration.millis(2000)) results seeking to 0 seconds. A 2 second failure rate is not acceptable.
With WAV it seems to work. Though it does not with MP3.
The two workarounds I think so far are possible:
Writing an MP3 Decoder and Player which doesn't have the bug
Using uncompressed WAV files
Does anyone know anything better?
If anyone needs the source code there isn't much more in it:
public static void main(String[] args) {
MediaPlayer player = null;
JFXPanel fxPanel = new JFXPanel(); //To initialize JavaFX
try {
String url = new File("E:\\Music\\test.mp3").toURI().toURL().toExternalForm();
player = new MediaPlayer(new Media(url));
System.out.println("File loaded!");
} catch (MalformedURLException e) {
//e.printStackTrace();
System.out.println("Error with filename!");
System.exit(0);
}
player.play();
System.out.println("Playing!");
while (true)
{
Scanner reader = new Scanner(System.in);
String string = reader.nextLine();
if (string.equals("Exit")) System.exit(0);
else if (string.equals("Seek"))
{
player.seek(Duration.millis(2000)); //this seeks to the beggining of the file
player.pause();
try {
Thread.sleep(100); //player.getCurrentTime() doesn't update immediately
} catch (InterruptedException e) { }
System.out.println("Time: " + player.getCurrentTime().toMillis() + " / " + player.getTotalDuration().toMillis());
player.play();
}
}
}

I would recommend using the javazoom library. It is an open source java library that already has this stuff written without errors(At least none that I found).
Source
http://www.javazoom.net/index.shtml

Place your call to the seek method off the UI thread or your UI will hang.
new Thread(() ->player.seek(Duration.millis(2000))).start();

how to get bit rate , sampling rate and no. of channels of a audio file in android

I need your help for the below query:
Query:
Is there any way of getting following info of an audio file.
Sample rate, Channel, Bitrate of an audio file.
For extracting bitrate, "MediaMetadataRetriever" API is available (METADATA_KEY_BITRATE).
Please suggest if it can be done using any android API.
Found this below API, But its use is actually in different.
http://developer.android.com/reference/android/medi/AudioTrack.html
I want to extract these using Android API programmactically :
Sampling rate, Quantization, Channel of an input audio file.
Please help on this.
Thanks in advance.

This can be done using MeiaExtractor like this:
MediaExtractor mex = new MediaExtractor();
try {
mex.setDataSource(path);// the adresss location of the sound on sdcard.
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
MediaFormat mf = mex.getTrackFormat(0);
int bitRate = mf.getInteger(MediaFormat.KEY_BIT_RATE);
int sampleRate = mf.getInteger(MediaFormat.KEY_SAMPLE_RATE);
int channelCount = mf.getInteger(MediaFormat.KEY_CHANNEL_COUNT);

Use MediaPlayer.getTrackInfo() during playback (after METADATE_UPDATE event come to onInfo callback) to obtain MediaFormat object by invoke getFormat for audio stream track. And then from MediaFormat you can get:
BIT_RATE
CHANNEL_COUNT
SAMPLE_RATE

How do you Edit Video ID3v2 Tags in Java

I have been doing some research on ID3V2 tags and Video Formats such as MP4 and WMV. The two top libraries for editing ID3V2 tags seem to be:
Entagged and Jaudiotagger
Both of these support only audio formats. ( They support M4A and WMA but not MP4 and WMV ) I guess first off why is this? Then are they any alternatives.

It appears JID3 will do the trick. It doesn't have any restrictions on extension.
http://jid3.blinkenlights.org/
Now hopefully someone finds this open-source project a designer!
Here is an example of using it with several different file formats:
public class JITExample {
private static MediaFile audioFile;
public static void main(String... megaThrustersAreGo) {
//File file = new File("/home/rhigdon/Desktop/project-voltron/test-files/video.mp4");
//File file = new File("/home/rhigdon/Desktop/project-voltron/test-files/movGetOutTheWay_iPhone_Cellular_1.3gp");
File file = new File("/home/rhigdon/Desktop/project-voltron/test-files/movGetOutTheWay_HD_WMV_720p_1.wmv");
//Entagged Soltuion
audioFile = new MP3File(file);
try {
ID3V2_3_0Tag tag = new ID3V2_3_0Tag();
tag.setArtist("Ryan Higdon");
tag.setAlbum("Ryan's Funky Beats");
audioFile.setID3Tag(tag);
audioFile.sync();
for (ID3Tag eachTag : audioFile.getTags()) {
System.out.println(eachTag.toString());
}
} catch (ID3Exception e) {
e.printStackTrace();
System.out.println("something bad happened");
}
}
}

According to the introduction page here http://www.id3.org/Introduction, ogg, wma and aac uses their own formats separated from ID3v2.
Another library for editing ID3v2 and playing mp3:s is JLayer. It doesn't need JMF and it is available for both J2SE and J2ME.