I have uploaded a video file .mp4(18MB) into gridfs . and trying to read it from java code .here are some points i am unable to move further
1) i can able to retrieve the whole video into byte array and able to play
2) for first Nbytes means starting from first chunk to n no of chunks also i can able to play using directly querying from fs.chunks ... as below and giving to servletOutputstream ..
DBCollection a= db.getCollection("fs.chunks");DBCursor cur1=a.find().limit(10);
System.out.println(cur1);
byte[] destination2 =new byte[2621440];
int length2 = 0;
while(cur1.hasNext()) {
byte[] b2 = (byte[]) cur1.next().get("data");
System.arraycopy(b2, 0, destination2, length2, b2.length);
length2 += b2.length;
System.out.println("##########");
System.out.println(destination2.length);
}
3) I was stuck here, while reading from middle of the chunks , means after skip(n) chunks in the find() operation , unable to play the video by windows media player.saying unable to codec and etc error.. am i trying in a right way ?
DBCollection a= db.getCollection("fs.chunks");
DBCursor cur1=a.find(new BasicDBObject("n",new BasicDBObject("$gt",9))).limit(10);
System.out.println(cur1);
byte[] destination2 =new byte[2621440];
int length2 = 0;
while(cur1.hasNext()) {
byte[] b2 = (byte[]) cur1.next().get("data");
System.arraycopy(b2, 0, destination2, length2, b2.length);
length2 += b2.length;
System.out.println("##########");
System.out.println(destination2.length);
}
...........
public void showVideos(Model model,HttpServletResponse response) throws IOException {............response.setHeader("Content-Type", "video/quicktime");
response.setHeader("Content-Disposition", "inline; filename=\"" + filename + "\"");//byte[] bytearray =destination2
//response.s
ServletOutputStream out = response.getOutputStream();
System.out.println("hello");
int n=0;
//while(is.read(bytes, 0, 4096) != -1)
{
System.out.println(n++);
out.write(bytearray);
}
please suggest me for retrieving the part of the video file and play it from grid fs?
I'd use the GridFS classes for this purpose. Pseudo code below. myFS points to the bucket and findOne looks for the id of the file.
GridFS myFS = null;
if (bucket.isPresent()) {
myFS = new GridFS(m.getDb(), bucket.get());
} else {
myFS = new GridFS(m.getDb());
}
return Optional.fromNullable(myFS.findOne(id));
Related
I'm trying to Gzip a file for output in Play Framework 2.2.1, with Java.
This is not an asset, it is not a static file. For instance, it can be a user avatar which the user uploads. For example, a PNG image.
I've searched a for this and found only how to GZIP strings and that the Play Framework does automatic Gzipping for public assets, which this is not.
Some code I've tried:
public static Result userAvatar(long userId) throws IOException {
UserAvatar avatar = UserAvatar.get(userId);
InputStream avatarStream;
Long version;
// Use the default avatar.
if (avatar == null) {
avatarStream = Play.current().resourceAsStream("public/images/noavatar.png").get();
version = 0L;
} else {
avatarStream = new ByteArrayInputStream(avatar.avatar);
version = avatar.version;
}
byte[] byteArray = IOUtils.toByteArray(avatarStream);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(byteArray.length);
OutputStream gzip = new GZIPOutputStream(outputStream);
int len;
while ((len = avatarStream.read(byteArray)) > 0) {
gzip.write(byteArray, 0, len);
}
avatarStream.close();
gzip.close();
// The client has the correct image cached if the ETag matches
String eTag = request().getHeader("If-None-Match");
if (eTag != null && eTag.equals(version.toString())) {
return status(NOT_MODIFIED, "Not modified");
}
response().setContentType("image/png");
response().setHeader(ETAG, version.toString());
return Results.ok(outputStream.toByteArray());
}
This did not work and google is only returning answers for gzipping strings. Can anyone help?
EDIT: Does not work in this case means the result was 0 bytes.
len is bound to always be 0.
In this line:
byte[] byteArray = IOUtils.toByteArray(avatarStream);
you read all of avatarStream - it is now empty, 0 bytes left.
And in this line:
while ((len = avatarStream.read(byteArray)) > 0) {
you check how much you can still read of it - which is 0 bytes.
Replace
int len;
while ((len = avatarStream.read(byteArray)) > 0) {
gzip.write(byteArray, 0, len);
}
by just
gzip.write(byteArray);
May somebody help me to know how can I do in java what I do in ruby with the code below.
The ruby code below uses unpack('H*')[0] to stores the complete binary file content in variable "var" in ASCII format.
IO.foreach(ARGV[0]){ |l|
var = l.unpack('H*')[0]
} if File.exists?(ARGV[0])
Update:
Hi Aru. I've tested the way you say in the form below
byte[] bytes = Files.readAllBytes(testFile.toPath());
str = new String(bytes,StandardCharsets.UTF_8);
System.out.println(str);
But when I print the content of variable "str", the printout shows only little squares, like is not decoding the content. I'd like to store in "str" the content of binary file in ASCII format.
Update #2:
Hello Aru, I'm trying to store in array of bytes all the binary file's content but I don't know how to do it. It worked
with "FileUtils.readFileToByteArray(myFile);" but this is an external library, is there a built in option to do it?
File myFile = new File("./Binaryfile");
byte[] binary = FileUtils.readFileToByteArray(myFile); //I have issues here to store in array of bytes all binary content
String hexString = DatatypeConverter.printHexBinary(binary);
System.out.println(hexString);
Update #3:
Hello ursa and Aru, Thanks for your help. I've tried both of your solutions and works so fine, but seeing Files.readAllBytes() documentation
it says that is not intended to handle big files and the binary file I want to analyse is more than 2GB :(. I see an option with your solutions, read
chunk by chunk. The chunks inside the binary are separated by the sequence FF65, so is there a way to tweak your codes to only process one chunk at a
time based on the chunk separator? If not, maybe with some external library.
Update #4:
Hello, I'm trying to modify your code since I'd like to read variable size chunks based of
value of "Var".
How can I set an offset to read the next chunk in your code?
I mean,
- in first iteration read the first 1024,
- In this step Var=500
- in 2d iteration read the next 1024 bytes, beginning from 1024 - Var = 1024-500 = 524
- In this step Var=712
- in 3rd iteration read the next 1024 bytes, beginning from 1548 - Var = 1548-712 = 836
- and so on
is there a method something like read(number of bytes, offset)?
You can use commons-codec Hex class + commons-io FileUtils class:
byte[] binary = FileUtils.readFileToByteArray(new File("/Users/user/file.bin");
String hexEncoded = Hex.encodeHex(binary);
But if you just want to read content of TEXT file you can use:
String content = FileUtils.readFileToString(new File("/Users/user/file.txt", "ISO-8859-1");
With JRE 7 you can use standard classes:
public static void main(String[] args) throws Exception {
Path path = Paths.get("path/to/file");
byte[] data = Files.readAllBytes(path);
char[] hexArray = "0123456789ABCDEF".toCharArray();
char[] hexChars = new char[data.length * 2];
for ( int j = 0; j < data.length; j++ ) {
int v = data[j] & 0xFF;
hexChars[j * 2] = hexArray[v >>> 4];
hexChars[j * 2 + 1] = hexArray[v & 0x0F];
}
System.out.println(new String(hexChars));
}
This should do what you want:
try {
File inputFile = new File("someFile");
byte inputBytes[] = Files.readAllBytes(inputFile.toPath());
String hexCode = DatatypeConverter.printHexBinary(inputBytes);
System.out.println(hexCode);
} catch (IOException e) {
System.err.println("Couldn't read file: " + e);
}
If you don't want to read the entire file at once, you can do so as well. You'll need an InputStream of some sort.
File inputFile = new File("C:\\Windows\\explorer.exe");
try (InputStream input = new FileInputStream(inputFile)) {
byte inputBytes[] = new byte[1024];
int readBytes;
// Read until all bytes were read
while ((readBytes = input.read(inputBytes)) != -1) {
System.out.printf("%4d bytes were read.\n", readBytes);
System.out.println(DatatypeConverter.printHexBinary(inputBytes));
}
} catch (FileNotFoundException ex) {
System.err.println("Couldn't read file: " + ex);
} catch (IOException ex) {
System.err.println("Error while reading file: " + ex);
}
Is it possible to use the AngelList API to get the entire list of companies (startups) inside the AngelList website without knowing the ID of all of them?
Or, is there a way to get all of the company IDs?
I'm trying a JSON parser with random URL, because the AngelList URLs are randomly used for location, market and other.
I would like to obtain all the AngelList startups and companies in a txt file
for (int h = 1612; h<=1885; h++){
do {
// change the URL as per the requirement and also paginating it
String libURL = "https://api.angel.co/1/tags/" + h + "/startups?page="
+ i;
InputStream in = URI.create(libURL).toURL().openStream();
// writing each page into a seperate file
FileOutputStream fout = new FileOutputStream(
"/Users/Fabio/Desktop/FilesAngellist/file" + i + ".txt");
byte data[] = new byte[1024];
int count;
while ((count = in.read(data, 0, 1024)) != -1) {
fout.write(data, 0, count);
}// end of while
if (i == 1) {
DownloadJobJson db = new DownloadJobJson(); // code to pull
// "last_page" value
// from json file
pagenumber = db.DownloadJobJson1();
}// end of if
i = i + 1;
} while (i <= pagenumber);// end of do-while()
}
This is the code of the JSON downloader from AngelList URL
Solved using linkedin j project
I have some data that I have saved into a file using Matlab. I have saved this data in Matlab as follows:
fwrite(fid,numImg2,'integer*4');
fwrite(fid,y,'integer*4');
fwrite(fid,imgName,'char*1');
fwrite(fid,a,'integer*4');
fwrite(fid,img.imageData,'double');
I read this data back into Matlab using the following code
fread(fid,1,'integer*4');// Returns numImg2
fread(fid,1,'integer*4');// Returns y which is the number of cha rectors in the image name, i use in the following line to read the image name, say for example if the image name is 1.jpg, then using the following will return the image name
fread(fid,5,'char*1');
fread(fid,1);
etc...
I want to be able to read this data on an android phone. This is the code I have at the moment.
DataInputStream ds = new DataInputStream(new FileInputStream(imageFile));
//String line;
// Read the first byte to find out how many images are stored in the file.
int x = 0;
byte numberOfImages;
int numImages = 0;
while(x<1)
{
numberOfImages = ds.readByte();
numImages = (int)numberOfImages;
x++;
}
int lengthName = 0;
String imgName = "";
for(int y=1; y<=numImages; y++)
{
lengthName = ds.readInt();
byte[] nameBuffer = new byte[lengthName];
char[] name = new char[lengthName];
for(int z = 1; z<=5;z++)
{
nameBuffer[z-1] = ds.readByte();
//name[z-1] = ds.readChar();
}
imgName = new String(nameBuffer);
//imgName = name.toString();
}
text.append(imgName);
I cannot seem to retrieve the image name as a string from the binary file data. Any help is much appreciated.
I'm not sure it will work but anyway:
byte[] nameBuffer = new byte[lengthName];
if(ds.read(nameBuffer) != lengthName) {
// error handling here
return;
}
imgName = new String(nameBuffer, "ISO-8859-1");
I'm trying to write a simple RTF document pretty much from scratch in Java, and I'm trying to embed JPEGs in the document. Here's an example of a JPEG (a 2x2-pixel JPEG consisting of three white pixels and a black pixel in the upper left, if you're curious) embedded in an RTF document (generated by WordPad, which converted the JPEG to WMF):
{\pict\wmetafile8\picw53\pich53\picwgoal30\pichgoal30
0100090000036e00000000004500000000000400000003010800050000000b0200000000050000
000c0202000200030000001e000400000007010400040000000701040045000000410b2000cc00
020002000000000002000200000000002800000002000000020000000100040000000000000000
000000000000000000000000000000000000000000ffffff00fefefe0000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000000000
0000001202af0801010000040000002701ffff030000000000
}
I've been reading the RTF specification, and it looks like you can specify that the image is a JPEG, but since WordPad always converts images to WMF, I can't see an example of an embedded JPEG. So I may also end up needing to transcode from JPEG to WMF or something....
But basically, I'm looking for how to generate the binary or hexadecimal (Spec, p.148: "These pictures can be in hexadecimal (the default) or binary format.") form of a JPEG given a file URL.
Thanks!
EDIT: I have the stream stuff working all right, I think, but still don't understand exactly how to encode it, because whatever I'm doing, it's not RTF-readable. E.g., the above picture instead comes out as:
ffd8ffe00104a464946011106006000ffdb0430211211222222223533333644357677767789b988a877adaabcccc79efdcebcccffdb04312223336336c878ccccccccccccccccccccccccccccccccccccccccccccccccccffc0011802023122021113111ffc401f001511111100000000123456789abffc40b5100213324355440017d123041151221314161351617227114328191a182342b1c11552d1f024336272829a161718191a25262728292a3435363738393a434445464748494a535455565758595a636465666768696a737475767778797a838485868788898a92939495969798999aa2a3a4a5a6a7a8a9aab2b3b4b5b6b7b8b9bac2c3c4c5c6c7c8c9cad2d3d4d5d6d7d8d9dae1e2e3e4e5e6e7e8e9eaf1f2f3f4f5f6f7f8f9faffc401f103111111111000000123456789abffc40b51102124434754401277012311452131612415176171132232818144291a1b1c19233352f0156272d1a162434e125f11718191a262728292a35363738393a434445464748494a535455565758595a636465666768696a737475767778797a82838485868788898a92939495969798999aa2a3a4a5a6a7a8a9aab2b3b4b5b6b7b8b9bac2c3c4c5c6c7c8c9cad2d3d4d5d6d7d8d9dae2e3e4e5e6e7e8e9eaf2f3f4f5f6f7f8f9faffda0c31021131103f0fdecf09f84f4af178574cd0b42d334fd1744d16d22bd3f4fb0b74b6b5bb78902450c512091c688aaaa8a0500014514507ffd9
This PHP library would do the trick, so I'm trying to port the relevant portion to Java. Here is is:
$imageData = file_get_contents($this->_file);
$size = filesize($this->_file);
$hexString = '';
for ($i = 0; $i < $size; $i++) {
$hex = dechex(ord($imageData{$i}));
if (strlen($hex) == 1) {
$hex = '0' . $hex;
}
$hexString .= $hex;
}
return $hexString;
But I don't know what the Java analogue to dechex(ord($imageData{$i})) is. :( I got only as far as the Integer.toHexString() function, which takes care of the dechex part....
Thanks all. :)
Given a file URL for any file you can get the corresponding bytes by doing (exception handling omitted for brevity)...
int BUF_SIZE = 512;
URL fileURL = new URL("http://www.somewhere.com/someurl.jpg");
InputStream inputStream = fileURL.openStream();
byte [] smallBuffer = new byte[BUF_SIZE];
ByteArrayOutputStream largeBuffer = new ByteArrayOutputStream();
int numRead = BUF_SIZE;
while(numRead == BUF_SIZE) {
numRead = inputStream.read(smallBuffer,0,BUF_SIZE);
if(numRead > 0) {
largeBuffer.write(smallBuffer,0,BUF_SIZE);
}
}
byte [] bytes = largeBuffer.toByteArray();
I'm looking at your PHP snippet now and realizing that RTF is a bizarre specification! It looks like each byte of the image is encoded as 2 hex digits (which doubles the size of the image for no apparent reason). The the entire thing is stored in raw ASCII encoding. So, you'll want to do...
StringBuilder hexStringBuilder = new StringBuilder(bytes.length * 2);
for(byte imageByte : bytes) {
String hexByteString = Integer.toHexString(0x000000FF & (int)imageByte);
if(hexByteString .size() == 1) {
hexByteString = "0" + hexByteString ;
}
hexStringBuilder.append(hexByteString);
}
String hexString = hexStringBuilder.toString();
byte [] hexBytes = hexString.getBytes("UTF-8"); //Could also use US-ASCII
EDIT: Updated code sample to pad 0's on the hex bytes
EDIT: negative bytes were getting logically right shifted when converted to ints >_<
https://joseluisbz.wordpress.com/2013/07/26/exploring-a-wmf-file-0x000900/
Maybe help you this:
String HexRTFBytes = "Representations text of bytes from Image RTF File";
String Destiny = "The path of the output File";
FileOutputStream wmf;
try {
wmf = new FileOutputStream(Destiny);
HexRTFBytes = HexRTFBytes.replaceAll("\n", ""); //Erase New Lines
HexRTFBytes = HexRTFBytes.replaceAll(" ", ""); //Erase Blank spaces
int NumBytesWrite = HexRTFBytes.length();
int WMFBytes = NumBytesWrite/2;//One byte is represented by 2 characters
byte[] ByteWrite = new byte[WMFBytes];
for (int i = 0; i < WMFBytes; i++){
se = HexRTFBytes.substring(i*2,i*2+2);
int Entero = Integer.parseInt(se,16);
ByteWrite[i] = (byte)Entero;
}
wmf.write(ByteWrite);
wmf.close();
}
catch (FileNotFoundException fnfe)
{System.out.println(fnfe.toString());}
catch (NumberFormatException fnfe)
{System.out.println(fnfe.toString());}
catch (EOFException eofe)
{System.out.println(eofe.toString());}
catch (IOException ioe)
{System.out.println(ioe.toString());}
This code take the representation in one string, and result is stored in a file.
https://joseluisbz.wordpress.com/2011/06/22/script-de-clases-rtf-para-jsp-y-php/
Now if you want to obtain the representation of the image file, you can use this:
private void ByteStreamImageString(byte[] ByteStream) {
this.Format = 0;
this.High = 0;
this.Wide = 0;
this.HexImageString = "Error";
if (ByteStream[0]== (byte)137 && ByteStream[1]== (byte)80 && ByteStream[2]== (byte)78){
this.Format = PNG; //PNG
this.High = this.Byte2PosInt(ByteStream[22],ByteStream[23]);
this.Wide = this.Byte2PosInt(ByteStream[18],ByteStream[19]);
}
if (ByteStream[0]== (byte)255 && ByteStream[1]== (byte)216
&& ByteStream[2]== (byte)255 && ByteStream[3]== (byte)224){
this.Format = JPG; //JPG
int PosJPG = 2;
while (PosJPG < ByteStream.length){
String M = String.format("%02X%02X", ByteStream[PosJPG+0],ByteStream[PosJPG+1]);
if (M.equals("FFC0") || M.equals("FFC1") || M.equals("FFC2") || M.equals("FFC3")){
this.High = this.Byte2PosInt(ByteStream[PosJPG+5],ByteStream[PosJPG+6]);
this.Wide = this.Byte2PosInt(ByteStream[PosJPG+7],ByteStream[PosJPG+8]);
}
if (M.equals("FFDA")) {
break;
}
PosJPG = PosJPG+2+this.Byte2PosInt(ByteStream[PosJPG+2],ByteStream[PosJPG+3]);
}
}
if (this.Format > 0) {
this.HexImageString = "";
int Salto = 0;
for (int i=0;i < ByteStream.length; i++){
Salto++;
this.HexImageString += String.format("%02x", ByteStream[i]);
if (Salto==64){
this.HexImageString += "\n"; //To make readable
Salto = 0;
}
}
}
}