I have some trouble extracting raw AMR audio frames from a .3gp file. I followed the link: "http://android.amberfog.com/?p=181" but instead of "mdat" box type I've got "moov". I read somewhere that "moov" box and "mdat" box location differ from device to device. Does anyone knows how to correctly skip the .3gp headers and extract raw AMR data? Below is a code snippet:
public AMRFile convert3GPDataToAmr(String rawAmrFilePath) throws IOException {
if (_3gpFile == null) {
return null;
}
System.out.println("3GP file length: "+_3gpFile.getRawFile().length());
//FileInputStream is = new FileInputStream(_3gpFile.getRawFile());
ByteArrayInputStream bis = new ByteArrayInputStream(FileUtils.readFileToByteArray(_3gpFile.getRawFile()));
// read FileTypeHeader
System.out.println("Available1: "+bis.available());
FileTypeBox ftypHeader = new FileTypeBox(bis);
System.out.println("Available2: "+bis.available());
String header = ftypHeader.getHeaderAsString();
if(!FileTypeBox.HEADER_TYPE.equalsIgnoreCase(header)){
throw new IOException("File is not 3GP. ftyp header missing.");
}
// You can check if it is correct here
// read MediaDataHeader
MediaDataBox mdatHeader = new MediaDataBox(bis);
System.out.println("Available3: "+bis.available());
header = mdatHeader.getHeaderAsString();
if(!MediaDataBox.HEADER_TYPE.equalsIgnoreCase(header)){
//here is THE PROBLEM!!!!! - header is "moov" instead of "mdat" !!!!!!!!!!!!!
throw new IOException("File is not 3GP. mdat header missing.");
}
// You can check if it is correct here
int rawAmrDataLength = mdatHeader.getDataLength();
System.out.println("RAW Amr length: "+bis.available());
int fullAmrDataLength = AMR_MAGIC_HEADER.length + rawAmrDataLength;
byte[] amrData = new byte[fullAmrDataLength];
System.arraycopy(AMR_MAGIC_HEADER, 0, amrData, 0, AMR_MAGIC_HEADER.length);
bis.read(amrData, AMR_MAGIC_HEADER.length, rawAmrDataLength);
bis.close();
//create raw amr file
File rawAmrFile = new File(rawAmrFilePath);
FileOutputStream fos = new FileOutputStream(rawAmrFile);
AMRFile amrFile = null;
try {
fos.write(amrData);
} catch (Exception e) {
Log.e(getClass().getName(), e.getMessage(), e);
}
finally{
fos.close();
amrFile = new AMRFile(rawAmrFile);
}
System.out.println("AMR file length: "+amrFile.getRawFile().length());
return amrFile;
}
I used some HEX Viewer tool to look into my .3gp file and I saw that mdat box wasn't in the place where the algorithm has looked for, so I decided to read from the stream until I find the mdat box. Now the extraction works ok. I modified MediaDataBox a little bit:
public MediaDataBox(FileInputStream is) throws IOException {
super(is);
//check the mdat header - if not read until mdat if found
long last32Int = 0;
long curr32Int = 0;
long temp;
while (is.available()>=8) {
temp = curr32Int = readUint32(is);
//test like this to avoid low memory issues
if((HEADER_TYPE.charAt(0) == (byte)(temp >>> 24)) &&
(HEADER_TYPE.charAt(1) == (byte)(temp >>> 16)) &&
(HEADER_TYPE.charAt(2) == (byte)(temp >>> 8)) &&
(HEADER_TYPE.charAt(3) == (byte)temp)){
size = last32Int;
type = curr32Int;
boxSize = 8;
break;
}
last32Int = curr32Int;
}
}
and super class:
public _3GPBox(FileInputStream is) throws IOException{
size = readUint32(is);
boxSize += 4;
type = readUint32(is);
boxSize += 4;
}
Related
I'm trying to Gzip a file for output in Play Framework 2.2.1, with Java.
This is not an asset, it is not a static file. For instance, it can be a user avatar which the user uploads. For example, a PNG image.
I've searched a for this and found only how to GZIP strings and that the Play Framework does automatic Gzipping for public assets, which this is not.
Some code I've tried:
public static Result userAvatar(long userId) throws IOException {
UserAvatar avatar = UserAvatar.get(userId);
InputStream avatarStream;
Long version;
// Use the default avatar.
if (avatar == null) {
avatarStream = Play.current().resourceAsStream("public/images/noavatar.png").get();
version = 0L;
} else {
avatarStream = new ByteArrayInputStream(avatar.avatar);
version = avatar.version;
}
byte[] byteArray = IOUtils.toByteArray(avatarStream);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(byteArray.length);
OutputStream gzip = new GZIPOutputStream(outputStream);
int len;
while ((len = avatarStream.read(byteArray)) > 0) {
gzip.write(byteArray, 0, len);
}
avatarStream.close();
gzip.close();
// The client has the correct image cached if the ETag matches
String eTag = request().getHeader("If-None-Match");
if (eTag != null && eTag.equals(version.toString())) {
return status(NOT_MODIFIED, "Not modified");
}
response().setContentType("image/png");
response().setHeader(ETAG, version.toString());
return Results.ok(outputStream.toByteArray());
}
This did not work and google is only returning answers for gzipping strings. Can anyone help?
EDIT: Does not work in this case means the result was 0 bytes.
len is bound to always be 0.
In this line:
byte[] byteArray = IOUtils.toByteArray(avatarStream);
you read all of avatarStream - it is now empty, 0 bytes left.
And in this line:
while ((len = avatarStream.read(byteArray)) > 0) {
you check how much you can still read of it - which is 0 bytes.
Replace
int len;
while ((len = avatarStream.read(byteArray)) > 0) {
gzip.write(byteArray, 0, len);
}
by just
gzip.write(byteArray);
Is there a way to check if an attachment is already present in the PDF document while creating the document (not after the document is saved to disk)? While parsing a XML to PDF I came across over multiple attachments which have the same content (Base64 String from XML > byte[]) and the same name. Currently the attachments are added multiple times, but I want to check if an attachment (with the same content or name) already exists (PdfWriter API?) and if YES, only a new Annotation should be created to the existing attachment.
NOTE: the check should happen while creating the PDF, not with a PdfReader and an existing PDF
EDIT:
Thanks to #Bruno Lowagie I got it working:
protected HashMap<String, PdfFileSpecification> cache = new HashMap<>();
private final byte[] BUFFER = new byte[1024];
public PdfFileSpecification getPdfFileSpecification(final PdfWriter pdfWriter, final String name, final byte[] data) throws IOException {
String hash = createMD5Hash(data);
PdfFileSpecification pdfFileSpecification = cache.get(hash);
if (pdfFileSpecification == null) {
pdfFileSpecification = PdfFileSpecification.fileEmbedded(pdfWriter, null, name, data);
cache.put(hash, pdfFileSpecification);
return pdfFileSpecification;
}
System.out.println(String.format("Name: %s Hash: %s", name, hash));
return pdfFileSpecification;
}
private String createMD5Hash(final byte[] data) {
MessageDigest messageDigest;
try {
messageDigest = MessageDigest.getInstance("MD5");
} catch (NoSuchAlgorithmException e) {
return null;
}
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(data);
try {
int i;
while ((i = byteArrayInputStream.read(BUFFER)) != -1) {
messageDigest.update(BUFFER, 0, i);
}
byteArrayInputStream.close();
} catch (IOException e) {
return null;
}
byte[] mdbytes = messageDigest.digest();
StringBuffer sb = new StringBuffer();
for (int i = 0; i < mdbytes.length; i++) {
sb.append(Integer.toString((mdbytes[i] & 0xff) + 0x100, 16).substring(1));
}
return sb.toString();
}
So every time I have to deal with a new attachment I do it like this:
PdfFileSpecification fs = getPdfFileSpecification(pdfWriter, name, data)
PdfAnnotation an = PdfAnnotation.createFileAttachment(pdfWriter, rectangle, name, fs);
Allow me to take your code and introduce some pseudo code to show you how I would do this:
protected Map<String, PdfFileSpecification> cache =
new HashMap<String, PdfFileSpecification>();
public void cellLayout(final PdfPCell pdfPCell, final Rectangle rectangle, final PdfContentByte[] pdfContentBytes) {
String hasheddata = createHash(attachment);
PdfFileSpecification fs = cache.get(hasheddata);
if (fs == null) {
fs = PdfFileSpecification.fileEmbedded(writer, null, displayname, attachment);
cache.put(hasheddata, fs);
}
PdfAnnotation an = PdfAnnotation.createFileAttachment(writer, rectangle, displayname, fs);
writer.addAnnotation(an);
}
This code won't compile because I left out some parts that aren't relevant to the problem. I only kept the stuff that explains the concept of creating the cache for the file specifications.
I create a hash of the attachment bytes to save memory. You will have to implement the createHash() method using the hashing algorithm of your choice. Before I create a new FileSpecification that will write bytes to the PdfWriter, I check if I can't reuse an already existing file specification. If one exists, I reuse it in an annotation. If it doesn't exist I create a new file specification.
May somebody help me to know how can I do in java what I do in ruby with the code below.
The ruby code below uses unpack('H*')[0] to stores the complete binary file content in variable "var" in ASCII format.
IO.foreach(ARGV[0]){ |l|
var = l.unpack('H*')[0]
} if File.exists?(ARGV[0])
Update:
Hi Aru. I've tested the way you say in the form below
byte[] bytes = Files.readAllBytes(testFile.toPath());
str = new String(bytes,StandardCharsets.UTF_8);
System.out.println(str);
But when I print the content of variable "str", the printout shows only little squares, like is not decoding the content. I'd like to store in "str" the content of binary file in ASCII format.
Update #2:
Hello Aru, I'm trying to store in array of bytes all the binary file's content but I don't know how to do it. It worked
with "FileUtils.readFileToByteArray(myFile);" but this is an external library, is there a built in option to do it?
File myFile = new File("./Binaryfile");
byte[] binary = FileUtils.readFileToByteArray(myFile); //I have issues here to store in array of bytes all binary content
String hexString = DatatypeConverter.printHexBinary(binary);
System.out.println(hexString);
Update #3:
Hello ursa and Aru, Thanks for your help. I've tried both of your solutions and works so fine, but seeing Files.readAllBytes() documentation
it says that is not intended to handle big files and the binary file I want to analyse is more than 2GB :(. I see an option with your solutions, read
chunk by chunk. The chunks inside the binary are separated by the sequence FF65, so is there a way to tweak your codes to only process one chunk at a
time based on the chunk separator? If not, maybe with some external library.
Update #4:
Hello, I'm trying to modify your code since I'd like to read variable size chunks based of
value of "Var".
How can I set an offset to read the next chunk in your code?
I mean,
- in first iteration read the first 1024,
- In this step Var=500
- in 2d iteration read the next 1024 bytes, beginning from 1024 - Var = 1024-500 = 524
- In this step Var=712
- in 3rd iteration read the next 1024 bytes, beginning from 1548 - Var = 1548-712 = 836
- and so on
is there a method something like read(number of bytes, offset)?
You can use commons-codec Hex class + commons-io FileUtils class:
byte[] binary = FileUtils.readFileToByteArray(new File("/Users/user/file.bin");
String hexEncoded = Hex.encodeHex(binary);
But if you just want to read content of TEXT file you can use:
String content = FileUtils.readFileToString(new File("/Users/user/file.txt", "ISO-8859-1");
With JRE 7 you can use standard classes:
public static void main(String[] args) throws Exception {
Path path = Paths.get("path/to/file");
byte[] data = Files.readAllBytes(path);
char[] hexArray = "0123456789ABCDEF".toCharArray();
char[] hexChars = new char[data.length * 2];
for ( int j = 0; j < data.length; j++ ) {
int v = data[j] & 0xFF;
hexChars[j * 2] = hexArray[v >>> 4];
hexChars[j * 2 + 1] = hexArray[v & 0x0F];
}
System.out.println(new String(hexChars));
}
This should do what you want:
try {
File inputFile = new File("someFile");
byte inputBytes[] = Files.readAllBytes(inputFile.toPath());
String hexCode = DatatypeConverter.printHexBinary(inputBytes);
System.out.println(hexCode);
} catch (IOException e) {
System.err.println("Couldn't read file: " + e);
}
If you don't want to read the entire file at once, you can do so as well. You'll need an InputStream of some sort.
File inputFile = new File("C:\\Windows\\explorer.exe");
try (InputStream input = new FileInputStream(inputFile)) {
byte inputBytes[] = new byte[1024];
int readBytes;
// Read until all bytes were read
while ((readBytes = input.read(inputBytes)) != -1) {
System.out.printf("%4d bytes were read.\n", readBytes);
System.out.println(DatatypeConverter.printHexBinary(inputBytes));
}
} catch (FileNotFoundException ex) {
System.err.println("Couldn't read file: " + ex);
} catch (IOException ex) {
System.err.println("Error while reading file: " + ex);
}
I have uploaded a video file .mp4(18MB) into gridfs . and trying to read it from java code .here are some points i am unable to move further
1) i can able to retrieve the whole video into byte array and able to play
2) for first Nbytes means starting from first chunk to n no of chunks also i can able to play using directly querying from fs.chunks ... as below and giving to servletOutputstream ..
DBCollection a= db.getCollection("fs.chunks");DBCursor cur1=a.find().limit(10);
System.out.println(cur1);
byte[] destination2 =new byte[2621440];
int length2 = 0;
while(cur1.hasNext()) {
byte[] b2 = (byte[]) cur1.next().get("data");
System.arraycopy(b2, 0, destination2, length2, b2.length);
length2 += b2.length;
System.out.println("##########");
System.out.println(destination2.length);
}
3) I was stuck here, while reading from middle of the chunks , means after skip(n) chunks in the find() operation , unable to play the video by windows media player.saying unable to codec and etc error.. am i trying in a right way ?
DBCollection a= db.getCollection("fs.chunks");
DBCursor cur1=a.find(new BasicDBObject("n",new BasicDBObject("$gt",9))).limit(10);
System.out.println(cur1);
byte[] destination2 =new byte[2621440];
int length2 = 0;
while(cur1.hasNext()) {
byte[] b2 = (byte[]) cur1.next().get("data");
System.arraycopy(b2, 0, destination2, length2, b2.length);
length2 += b2.length;
System.out.println("##########");
System.out.println(destination2.length);
}
...........
public void showVideos(Model model,HttpServletResponse response) throws IOException {............response.setHeader("Content-Type", "video/quicktime");
response.setHeader("Content-Disposition", "inline; filename=\"" + filename + "\"");//byte[] bytearray =destination2
//response.s
ServletOutputStream out = response.getOutputStream();
System.out.println("hello");
int n=0;
//while(is.read(bytes, 0, 4096) != -1)
{
System.out.println(n++);
out.write(bytearray);
}
please suggest me for retrieving the part of the video file and play it from grid fs?
I'd use the GridFS classes for this purpose. Pseudo code below. myFS points to the bucket and findOne looks for the id of the file.
GridFS myFS = null;
if (bucket.isPresent()) {
myFS = new GridFS(m.getDb(), bucket.get());
} else {
myFS = new GridFS(m.getDb());
}
return Optional.fromNullable(myFS.findOne(id));
I'm trying to write a simple RTF document pretty much from scratch in Java, and I'm trying to embed JPEGs in the document. Here's an example of a JPEG (a 2x2-pixel JPEG consisting of three white pixels and a black pixel in the upper left, if you're curious) embedded in an RTF document (generated by WordPad, which converted the JPEG to WMF):
{\pict\wmetafile8\picw53\pich53\picwgoal30\pichgoal30
0100090000036e00000000004500000000000400000003010800050000000b0200000000050000
000c0202000200030000001e000400000007010400040000000701040045000000410b2000cc00
020002000000000002000200000000002800000002000000020000000100040000000000000000
000000000000000000000000000000000000000000ffffff00fefefe0000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000000000
0000001202af0801010000040000002701ffff030000000000
}
I've been reading the RTF specification, and it looks like you can specify that the image is a JPEG, but since WordPad always converts images to WMF, I can't see an example of an embedded JPEG. So I may also end up needing to transcode from JPEG to WMF or something....
But basically, I'm looking for how to generate the binary or hexadecimal (Spec, p.148: "These pictures can be in hexadecimal (the default) or binary format.") form of a JPEG given a file URL.
Thanks!
EDIT: I have the stream stuff working all right, I think, but still don't understand exactly how to encode it, because whatever I'm doing, it's not RTF-readable. E.g., the above picture instead comes out as:
ffd8ffe00104a464946011106006000ffdb0430211211222222223533333644357677767789b988a877adaabcccc79efdcebcccffdb04312223336336c878ccccccccccccccccccccccccccccccccccccccccccccccccccffc0011802023122021113111ffc401f001511111100000000123456789abffc40b5100213324355440017d123041151221314161351617227114328191a182342b1c11552d1f024336272829a161718191a25262728292a3435363738393a434445464748494a535455565758595a636465666768696a737475767778797a838485868788898a92939495969798999aa2a3a4a5a6a7a8a9aab2b3b4b5b6b7b8b9bac2c3c4c5c6c7c8c9cad2d3d4d5d6d7d8d9dae1e2e3e4e5e6e7e8e9eaf1f2f3f4f5f6f7f8f9faffc401f103111111111000000123456789abffc40b51102124434754401277012311452131612415176171132232818144291a1b1c19233352f0156272d1a162434e125f11718191a262728292a35363738393a434445464748494a535455565758595a636465666768696a737475767778797a82838485868788898a92939495969798999aa2a3a4a5a6a7a8a9aab2b3b4b5b6b7b8b9bac2c3c4c5c6c7c8c9cad2d3d4d5d6d7d8d9dae2e3e4e5e6e7e8e9eaf2f3f4f5f6f7f8f9faffda0c31021131103f0fdecf09f84f4af178574cd0b42d334fd1744d16d22bd3f4fb0b74b6b5bb78902450c512091c688aaaa8a0500014514507ffd9
This PHP library would do the trick, so I'm trying to port the relevant portion to Java. Here is is:
$imageData = file_get_contents($this->_file);
$size = filesize($this->_file);
$hexString = '';
for ($i = 0; $i < $size; $i++) {
$hex = dechex(ord($imageData{$i}));
if (strlen($hex) == 1) {
$hex = '0' . $hex;
}
$hexString .= $hex;
}
return $hexString;
But I don't know what the Java analogue to dechex(ord($imageData{$i})) is. :( I got only as far as the Integer.toHexString() function, which takes care of the dechex part....
Thanks all. :)
Given a file URL for any file you can get the corresponding bytes by doing (exception handling omitted for brevity)...
int BUF_SIZE = 512;
URL fileURL = new URL("http://www.somewhere.com/someurl.jpg");
InputStream inputStream = fileURL.openStream();
byte [] smallBuffer = new byte[BUF_SIZE];
ByteArrayOutputStream largeBuffer = new ByteArrayOutputStream();
int numRead = BUF_SIZE;
while(numRead == BUF_SIZE) {
numRead = inputStream.read(smallBuffer,0,BUF_SIZE);
if(numRead > 0) {
largeBuffer.write(smallBuffer,0,BUF_SIZE);
}
}
byte [] bytes = largeBuffer.toByteArray();
I'm looking at your PHP snippet now and realizing that RTF is a bizarre specification! It looks like each byte of the image is encoded as 2 hex digits (which doubles the size of the image for no apparent reason). The the entire thing is stored in raw ASCII encoding. So, you'll want to do...
StringBuilder hexStringBuilder = new StringBuilder(bytes.length * 2);
for(byte imageByte : bytes) {
String hexByteString = Integer.toHexString(0x000000FF & (int)imageByte);
if(hexByteString .size() == 1) {
hexByteString = "0" + hexByteString ;
}
hexStringBuilder.append(hexByteString);
}
String hexString = hexStringBuilder.toString();
byte [] hexBytes = hexString.getBytes("UTF-8"); //Could also use US-ASCII
EDIT: Updated code sample to pad 0's on the hex bytes
EDIT: negative bytes were getting logically right shifted when converted to ints >_<
https://joseluisbz.wordpress.com/2013/07/26/exploring-a-wmf-file-0x000900/
Maybe help you this:
String HexRTFBytes = "Representations text of bytes from Image RTF File";
String Destiny = "The path of the output File";
FileOutputStream wmf;
try {
wmf = new FileOutputStream(Destiny);
HexRTFBytes = HexRTFBytes.replaceAll("\n", ""); //Erase New Lines
HexRTFBytes = HexRTFBytes.replaceAll(" ", ""); //Erase Blank spaces
int NumBytesWrite = HexRTFBytes.length();
int WMFBytes = NumBytesWrite/2;//One byte is represented by 2 characters
byte[] ByteWrite = new byte[WMFBytes];
for (int i = 0; i < WMFBytes; i++){
se = HexRTFBytes.substring(i*2,i*2+2);
int Entero = Integer.parseInt(se,16);
ByteWrite[i] = (byte)Entero;
}
wmf.write(ByteWrite);
wmf.close();
}
catch (FileNotFoundException fnfe)
{System.out.println(fnfe.toString());}
catch (NumberFormatException fnfe)
{System.out.println(fnfe.toString());}
catch (EOFException eofe)
{System.out.println(eofe.toString());}
catch (IOException ioe)
{System.out.println(ioe.toString());}
This code take the representation in one string, and result is stored in a file.
https://joseluisbz.wordpress.com/2011/06/22/script-de-clases-rtf-para-jsp-y-php/
Now if you want to obtain the representation of the image file, you can use this:
private void ByteStreamImageString(byte[] ByteStream) {
this.Format = 0;
this.High = 0;
this.Wide = 0;
this.HexImageString = "Error";
if (ByteStream[0]== (byte)137 && ByteStream[1]== (byte)80 && ByteStream[2]== (byte)78){
this.Format = PNG; //PNG
this.High = this.Byte2PosInt(ByteStream[22],ByteStream[23]);
this.Wide = this.Byte2PosInt(ByteStream[18],ByteStream[19]);
}
if (ByteStream[0]== (byte)255 && ByteStream[1]== (byte)216
&& ByteStream[2]== (byte)255 && ByteStream[3]== (byte)224){
this.Format = JPG; //JPG
int PosJPG = 2;
while (PosJPG < ByteStream.length){
String M = String.format("%02X%02X", ByteStream[PosJPG+0],ByteStream[PosJPG+1]);
if (M.equals("FFC0") || M.equals("FFC1") || M.equals("FFC2") || M.equals("FFC3")){
this.High = this.Byte2PosInt(ByteStream[PosJPG+5],ByteStream[PosJPG+6]);
this.Wide = this.Byte2PosInt(ByteStream[PosJPG+7],ByteStream[PosJPG+8]);
}
if (M.equals("FFDA")) {
break;
}
PosJPG = PosJPG+2+this.Byte2PosInt(ByteStream[PosJPG+2],ByteStream[PosJPG+3]);
}
}
if (this.Format > 0) {
this.HexImageString = "";
int Salto = 0;
for (int i=0;i < ByteStream.length; i++){
Salto++;
this.HexImageString += String.format("%02x", ByteStream[i]);
if (Salto==64){
this.HexImageString += "\n"; //To make readable
Salto = 0;
}
}
}
}