how to convert File object to Netty ByteBuffer - java

Is there a way I can get netty ByteBuf from a java File object? I am trying to use Custom Snappy class to validate checksum. Below is my code where I'd like to get File object in bytebuf variable.
import io.netty.buffer.ByteBuf;
File file = new File("boot.zip");
ByteBuf byteBuf = Unpooled.buffer(128);
byteBuf.writeBytes(new FileInputStream(file));
int expectedChecksum = Snappy.calculateChecksum(byteBuf);
logger.info("checksum = "+expectedChecksum);
boolean checksumValid = true;
try {
CustomSnappy.validateChecksum(expectedChecksum, byteBuf);
} catch(DecompressionException e) {
checksumValid = false;
e.printStackTrace();
logger.error("DecompressionException in checksum calculation " + e);
} catch(Exception e) {
checksumValid = false;
e.printStackTrace();
logger.error("Exception in checksum calculation " + e);
}
I used Unpooled.buffer(128) but it gives me No signature of method :io.netty.buffer.UnpooledHeapByteBuf.writeBytes() is applicable for argument types: (java.io.FileInputStream) values: [java.io.FileInputStream#28b6520b]

You can created a MappedByteBuf from a FileChannel and wrap it in ByteBuffer via Unpooled.wrappedBuffer(..).

Here is an example of how to "create a MappedByteBuf from a FileChannel and wrap it in ByteBuffer via Unpooled.wrappedBuffer(..)":
try {
File file = new File(filename);
FileInputStream fileInputStream = new FileInputStream(file);
FileChannel fileChannel = fileInputStream.getChannel();
MappedByteBuffer mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, 0, file.length());
ByteBuf byteBuf = Unpooled.wrappedBuffer(mappedByteBuffer);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}

You can use the method ByteBuf#writeBytes(InputStream). This should work, but I did not test it.
ByteBuf buffer = // see http://stackoverflow.com/a/15088238/834
buffer.writeBytes(new FileInputStream(file), file.length());
See http://netty.io/4.0/api/io/netty/buffer/ByteBuf.html#writeBytes(java.io.InputStream,%20int)
The documentation shows how to create a new ByteBuf: http://netty.io/4.0/api/io/netty/buffer/Unpooled.html

If you are using JDK 1.7, the class of java.nio.file.Files could make it easy with amount of static methods.
ByteBuf buffer = Unpooled.copiedBuffer(Files.readAllBytes(new File(this.getClass().getResource("/requests/overPackageRequest").getFile()).toPath()));

Related

A java program that creates WAV files [duplicate]

With Java:
I have a byte[] that represents a file.
How do I write this to a file (ie. C:\myfile.pdf)
I know it's done with InputStream, but I can't seem to work it out.
Use Apache Commons IO
FileUtils.writeByteArrayToFile(new File("pathname"), myByteArray)
Or, if you insist on making work for yourself...
try (FileOutputStream fos = new FileOutputStream("pathname")) {
fos.write(myByteArray);
//fos.close(); There is no more need for this line since you had created the instance of "fos" inside the try. And this will automatically close the OutputStream
}
Without any libraries:
try (FileOutputStream stream = new FileOutputStream(path)) {
stream.write(bytes);
}
With Google Guava:
Files.write(bytes, new File(path));
With Apache Commons:
FileUtils.writeByteArrayToFile(new File(path), bytes);
All of these strategies require that you catch an IOException at some point too.
Another solution using java.nio.file:
byte[] bytes = ...;
Path path = Paths.get("C:\\myfile.pdf");
Files.write(path, bytes);
Also since Java 7, one line with java.nio.file.Files:
Files.write(new File(filePath).toPath(), data);
Where data is your byte[] and filePath is a String. You can also add multiple file open options with the StandardOpenOptions class. Add throws or surround with try/catch.
From Java 7 onward you can use the try-with-resources statement to avoid leaking resources and make your code easier to read. More on that here.
To write your byteArray to a file you would do:
try (FileOutputStream fos = new FileOutputStream("fullPathToFile")) {
fos.write(byteArray);
} catch (IOException ioe) {
ioe.printStackTrace();
}
Try an OutputStream or more specifically FileOutputStream
Basic example:
String fileName = "file.test";
BufferedOutputStream bs = null;
try {
FileOutputStream fs = new FileOutputStream(new File(fileName));
bs = new BufferedOutputStream(fs);
bs.write(byte_array);
bs.close();
bs = null;
} catch (Exception e) {
e.printStackTrace()
}
if (bs != null) try { bs.close(); } catch (Exception e) {}
File f = new File(fileName);
byte[] fileContent = msg.getByteSequenceContent();
Path path = Paths.get(f.getAbsolutePath());
try {
Files.write(path, fileContent);
} catch (IOException ex) {
Logger.getLogger(Agent2.class.getName()).log(Level.SEVERE, null, ex);
}
////////////////////////// 1] File to Byte [] ///////////////////
Path path = Paths.get(p);
byte[] data = null;
try {
data = Files.readAllBytes(path);
} catch (IOException ex) {
Logger.getLogger(Agent1.class.getName()).log(Level.SEVERE, null, ex);
}
/////////////////////// 2] Byte [] to File ///////////////////////////
File f = new File(fileName);
byte[] fileContent = msg.getByteSequenceContent();
Path path = Paths.get(f.getAbsolutePath());
try {
Files.write(path, fileContent);
} catch (IOException ex) {
Logger.getLogger(Agent2.class.getName()).log(Level.SEVERE, null, ex);
}
I know it's done with InputStream
Actually, you'd be writing to a file output...
This is a program where we are reading and printing array of bytes offset and length using String Builder and Writing the array of bytes offset length to the new file.
`Enter code here
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
//*This is a program where we are reading and printing array of bytes offset and length using StringBuilder and Writing the array of bytes offset length to the new file*//
public class ReadandWriteAByte {
public void readandWriteBytesToFile(){
File file = new File("count.char"); //(abcdefghijk)
File bfile = new File("bytefile.txt");//(New File)
byte[] b;
FileInputStream fis = null;
FileOutputStream fos = null;
try{
fis = new FileInputStream (file);
fos = new FileOutputStream (bfile);
b = new byte [1024];
int i;
StringBuilder sb = new StringBuilder();
while ((i = fis.read(b))!=-1){
sb.append(new String(b,5,5));
fos.write(b, 2, 5);
}
System.out.println(sb.toString());
}catch (IOException e) {
e.printStackTrace();
}finally {
try {
if(fis != null);
fis.close(); //This helps to close the stream
}catch (IOException e){
e.printStackTrace();
}
}
}
public static void main (String args[]){
ReadandWriteAByte rb = new ReadandWriteAByte();
rb.readandWriteBytesToFile();
}
}
O/P in console : fghij
O/P in new file :cdefg
You can try Cactoos:
new LengthOf(new TeeInput(array, new File("a.txt"))).value();
More details: http://www.yegor256.com/2017/06/22/object-oriented-input-output-in-cactoos.html

Reading a binary file into byte array

I need to read a binary file and save each byte into a byte array. I've read other stackoverflow posts on this topic, but cannot figure out why mine does not work. Here is what I have:
String fileOne = "file1.bin";
byte[] byteArray = new byte[1000];
try{
FileInputStream fileIS = new FileInputStream(fileOne);
ObjectInputStream is = new ObjectInputStream(fileIS);
is.read(byteArray);
is.close();
for(int i =0; i < byteArray.length; i++){
System.out.println(byteArray[i]);
}
}
catch (FileNotFoundException e){
e.toString();
System.exit(0);
}
catch (IOException io){
io.toString();
System.exit(0);
}
Here's a way to read the contents of a file into a byte array. FileInputStream is all you need – leave ObjectInputStream out of it (unless you are explicitly dealing with data that was created from an ObjectOutputStream, but that doesn't seem to be the case since you are calling println() on each byte).
public static void main(String[] args) {
String filename = "file1.bin";
try (FileInputStream fis = new FileInputStream(filename)) {
byte[] bytes = fis.readAllBytes();
for (byte b : bytes) {
System.out.print(b);
}
} catch (Exception e) {
e.printStackTrace();
}
}
A few things here:
omit using ObjectInputStream – not needed for reading byte data, and won't work unless the data was created by the corresponding output stream. From the Javadoc: "An ObjectInputStream deserializes primitive data and objects previously written using an ObjectOutputStream. "
use try with resources – it will close the associated stream for you
catch Exception – in the code you posted, you will only see info if FileNotFoundException or IOException is thrown. For anything else, your code doesn't handle them or print out any info.

How do Java deflation/inflation streams work?

I'm trying to compress and decompress some bytes of data with the Deflater and InflaterOutputStream.
The problem is that compression seems to work (I'm not
sure since the compressed data is always the same even if I have random test data).
But the decompression return nothing at all.
What am I doing wrong?
My console output:
Test data: D8A8E00821608F227AE473774E177216
Compressed data: 789C
Decompressed data:
My program:
SecureRandom random = new SecureRandom();
byte[] testdata = new byte[16];
random.nextBytes(testdata);
System.out.println("Test data: " + DatatypeConverter.printHexBinary(testdata));
byte[] compressed = null;
try (ByteArrayOutputStream buffer = new ByteArrayOutputStream())
{
try (DeflaterOutputStream stream = new DeflaterOutputStream(buffer))
{
stream.write(testdata);
stream.flush();
compressed = buffer.toByteArray();
System.out.println("Compressed data: " + DatatypeConverter.printHexBinary(compressed));
}
}
catch (IOException e)
{
System.out.println("IOException during compression.");
}
byte[] decompressed = null;
try (ByteArrayOutputStream buffer = new ByteArrayOutputStream())
{
try (InflaterOutputStream stream = new InflaterOutputStream(buffer))
{
stream.write(compressed);
stream.flush();
decompressed = buffer.toByteArray();
System.out.println("Decompressed data: " + DatatypeConverter.printHexBinary(decompressed));
}
}
catch (IOException e)
{
System.out.println("IOException during decompression.");
}
The problem is that you're only flushing the stream - which doesn't necessarily mean there's no more data to come, which can affect the decompression.
If you change both of your flush() calls to close(), you'll see you get the appropriate data back... or as you're using a try-with-resources statement, just let that close the inner stream, and wait until after that to call toByteArray:
try (ByteArrayOutputStream buffer = new ByteArrayOutputStream())
{
try (DeflaterOutputStream stream = new DeflaterOutputStream(buffer))
{
stream.write(testdata);
}
compressed = buffer.toByteArray();
System.out.println("Compressed data: " + Arrays.toString(compressed));
}
catch (IOException e)
{
System.out.println("IOException during compression.");
}
(Ditto when decompressing.)

java.util.zip.ZipInputStream nulls partway through decompress

zipped xml string which has been retrieved from a web response given by production server.
String is good as same methodology in .net gives valid result. (The data is a list of car makes.)
In Android however the ZipInputStream read produces a buffer of the right size (4895) but it has null data from position 2661 onwards. The last car decompressed correctly is 'MG'
The method does not error.
Can anybody see what is wrong?
thanks
private byte[] decompressZip(String zipText) throws IOException{
try {
byte[] zipBytes = MakeBytes(zipText);
byte[] zipData;
ByteArrayInputStream b = new ByteArrayInputStream(zipBytes);
BufferedInputStream buf = new BufferedInputStream(b);
//ZipInputStream zin = new ZipInputStream(b); doesn't matter both constuctors have the fault.
ZipInputStream zin = new ZipInputStream(buf);
ZipEntry entry;
if((entry=zin.getNextEntry())!=null)
{
int entrySize=(int)entry.getSize();
zipData = new byte[entrySize];
zin.read(zipData, 0, entrySize);
return zipData;
}
} catch (Exception e) {
// TODO Auto-generated catch block
String sError = e.getMessage();
}
return null;
}

Java: How do I convert InputStream to GZIPInputStream?

I have a method like
public void put(#Nonnull final InputStream inputStream, #Nonnull final String uniqueId) throws PersistenceException {
// a.) create gzip of inputStream
final GZIPInputStream zipInputStream;
try {
zipInputStream = new GZIPInputStream(inputStream);
} catch (IOException e) {
e.printStackTrace();
throw new PersistenceException("Persistence Service could not received input stream to persist for " + uniqueId);
}
I wan to convert the inputStream into zipInputStream, what is the way to do that?
The above method is incorrect and throws Exception as "Not a Zip Format"
converting Java Streams to me are really confusing and I do not make them right
The GZIPInputStream is to be used to decompress an incoming InputStream. To compress an incoming InputStream using GZIP, you basically need to write it to a GZIPOutputStream.
You can get a new InputStream out of it if you use ByteArrayOutputStream to write gzipped content to a byte[] and ByteArrayInputStream to turn a byte[] into an InputStream.
So, basically:
public void put(#Nonnull final InputStream inputStream, #Nonnull final String uniqueId) throws PersistenceException {
final InputStream zipInputStream;
try {
ByteArrayOutputStream bytesOutput = new ByteArrayOutputStream();
GZIPOutputStream gzipOutput = new GZIPOutputStream(bytesOutput);
try {
byte[] buffer = new byte[10240];
for (int length = 0; (length = inputStream.read(buffer)) != -1;) {
gzipOutput.write(buffer, 0, length);
}
} finally {
try { inputStream.close(); } catch (IOException ignore) {}
try { gzipOutput.close(); } catch (IOException ignore) {}
}
zipInputStream = new ByteArrayInputStream(bytesOutput.toByteArray());
} catch (IOException e) {
e.printStackTrace();
throw new PersistenceException("Persistence Service could not received input stream to persist for " + uniqueId);
}
// ...
You can if necessary replace the ByteArrayOutputStream/ByteArrayInputStream by a FileOuputStream/FileInputStream on a temporary file as created by File#createTempFile(), especially if those streams can contain large data which might overflow machine's available memory when used concurrently.
GZIPInputStream is for reading gzip-encoding content.
If your goal is to take a regular input stream and compress it in the GZIP format, then you need to write those bytes to a GZIPOutputStream.
See also this answer to a related question.

Categories