I have a problem with the time it requires for .gz files to get uncompressed using the following code:
import java.io.*;
import java.util.zip.*;
public class UnGunzipClass{
public static boolean ungunzip(String compressedFile, String decompressedFile){
try{
// in
FileInputStream fileIn = new FileInputStream(compressedFile);
GZIPInputStream gZipIn = new GZIPInputStream(fileIn);
BufferedInputStream in = new BufferedInputStream(gZipIn);
// out
FileOutputStream fileOut = new FileOutputStream(decompressedFile);
BufferedOutputStream out = new BufferedOutputStream(fileOut);
int n = 0;
int len = 1024*1024*1024;
byte[] buffer = new byte[len];
while((n = in.read(buffer,0,len)) > 0){
out.write(buffer,0,n);
}
gZipIn.close();
fileOut.close();
return true;
} catch (IOException e){
e.printStackTrace();
return false;
}
}
}
Note: files are up to 100MB, but it is still taking me tens of minutes to run, so I am trying to get something faster. Speed is good :)
You created your BufferedInputStream from the GZIPInputStream, for performance you would do that in the reverse order. Also, I suggest you shrink your buffer size (and use your buffered streams). Finally, I would use a try-with-resources Statement and that might look something like
public static boolean ungunzip(String compressedFile,
String decompressedFile) {
final int BUFFER_SIZE = 32768;
try (InputStream in = new GZIPInputStream(new BufferedInputStream(
new FileInputStream(compressedFile)), BUFFER_SIZE);
OutputStream out = new BufferedOutputStream(
new FileOutputStream(decompressedFile), BUFFER_SIZE)) {
int n = 0;
byte[] buffer = new byte[BUFFER_SIZE];
while ((n = in.read(buffer, 0, BUFFER_SIZE)) > 0) {
out.write(buffer, 0, n);
}
return true;
} catch (IOException e) {
e.printStackTrace();
}
return false;
}
Related
I am working on reading a file and write same file, but the problem is the downloaded file is 2kb larger than input original file.
Some piece of code
#Override
public void run() {
try {
BufferedInputStream bis;
ArrayList<byte[]> al =new ArrayList<byte[]>();
File file = new File(Environment.getExternalStorageDirectory(), "test.mp3");
byte[] bytes = new byte[2048];
bis = new BufferedInputStream(new FileInputStream(file));
OutputStream os = socket.getOutputStream();
int read ;
int fileSize = (int) file.length();
int readlen=1024;
while (fileSize>0) {
if(fileSize<1024){
readlen=fileSize;
System.out.println("Hello.........");
}
bytes=new byte[readlen];
read = bis.read(bytes, 0, readlen);
fileSize-=read;
al.add(bytes);
}
ObjectOutputStream out1 = new ObjectOutputStream(new FileOutputStream(Environment.getExternalStorageDirectory()+"/newfile.mp3"));
for(int ii=1;ii<al.size();ii++){
out1.write(al.get(ii));
// out1.flush();
}
out1.close();
File file1 = new File(Environment.getExternalStorageDirectory(), "newfile.mp3");
Don't use an ObjectOutputStream. Just use the FileOutputStream, or a BufferedOutputStream wrapped around it.
The correct way to copy streams in Java is as follows:
byte[] buffer = new byte[8192]; // or more, or even less, anything > 0
int count;
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
out.close();
Note that you don't need a buffer the size of the input, and you don't need to read the entire input before writing any of the output.
Wish I had $1 for every time I've posted this.
I think you should use ByteArrayOutputStream not an ObjectOutputStream.
I belive this is not a raw code, but the parts of the code, placed in different procedures, otherwise it is meaningless.
For example, in case you want to stream some data from a file, process this data, and then write the data to another file.
BufferedInputStream bis = null;
ByteArrayOutputStream al = new ByteArrayOutputStream();
FileOutputStream out1 = null;
byte[] bytes;
try {
File file = new File("testfrom.mp3");
bis = new BufferedInputStream(new FileInputStream(file));
int fileSize = (int) file.length();
int readLen = 1024;
bytes = new byte[readLen];
while (fileSize > 0) {
if (fileSize < readLen) {
readLen = fileSize;
}
bis.read(bytes, 0, readLen);
al.write(bytes, 0, readLen);
fileSize -= readLen;
}
bis.close();
} catch (IOException e){
e.printStackTrace();
}
//proceed the data from al here
//...
//finish to proceed
try {
out1 = new FileOutputStream("testto.mp3");
al.writeTo(out1);
out1.close();
} catch (IOException e){
e.printStackTrace();
}
Don't forget to use try-catch directives where it needed
http://codeinventions.blogspot.ru/2014/08/creating-file-from-bytearrayoutputstrea.html
I have a file that has been compressed (zip) by a .net application using system.io.compression.gzipstream library.
I tried to decompress (unzip) it using java.util.zip.GZIPInputStream in java.
it threw:
java.io.EOFException
at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:246)
at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:237)
at java.util.zip.GZIPInputStream.readUInt(GZIPInputStream.java:229)
at java.util.zip.GZIPInputStream.readTrailer(GZIPInputStream.java:197)
at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:92)
at java.io.FilterInputStream.read(FilterInputStream.java:90)
at juwdemoproj.FileWriterTest.gunzipIt(FileWriterTest.java:353)
at juwdemoproj.FileWriterTest.main(FileWriterTest.java:51)
My sample code:
public static void gunzipIt() {
byte[] buffer = new byte[1024];
String zipFilePath = "C:\\Juw\\JR\\file\\output\\020030214112016.zip";
String destDirectory = "C:\\Juw\\JR\\file\\output\\targetUnzip";
try {
GZIPInputStream gzis =
new GZIPInputStream(new FileInputStream(zipFilePath));
FileOutputStream out = new FileOutputStream(destDirectory);
int len;
while ((len = gzis.read(buffer)) > 0) {
out.write(buffer, 0, len);
}
gzis.close();
out.close();
System.out.println("Done");
} catch (IOException ex) {
ex.printStackTrace();
}
}
Question:
- Is it possible to unzip the file in java, when the file was compressed by a .net application?
- Any zip library that can achieve the above requirement?
Thanks
I use this code and it solve my issue.
public void gunzipIt(){
byte[] buffer = new byte[1024];
boolean isValid = true;
try{
GZIPInputStream gzis =
new GZIPInputStream(new FileInputStream(INPUT_GZIP_FILE));
FileOutputStream out =
new FileOutputStream(OUTPUT_FILE);
while (isValid) {
int len;
try{
len = gzis.read(buffer);
}catch(Exception ex){
len = 0;
isValid = false;
}
if (len > 0) {
out.write(buffer, 0, len);
}else{
isValid = false;
}
}
gzis.close();
out.close();
System.out.println("Done");
}catch(IOException ex){
ex.printStackTrace();
}
}
at the end of the file there is a empty string, so it throwing error. i get this code from my colleague and it works.
I know that there's a way of converting a file to byte array in chunks, here's a sample code:
InputStream inputStream = new FileInputStream(videoFile);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] b = new byte[1024];
int bytesRead =0;
while ((bytesRead = inputStream.read(b)) != -1)
{
bos.write(b, 0, bytesRead);
}
I'm looking for the opposite: a way of converting a byte array into a file in chunks. I didn't find any example of doing it in chunks.
You just have to use either the write(byte[]) or write(byte[],int,int) methods from the FileOutputStream class.
byte[] to file:
FileOutputStream fop = null; File file;
try {
file = new File(filePath);
fop = new FileOutputStream(file, true);
fop.write(chunk);
fop.flush();
fop.close();
System.out.println("Done");
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (fop != null) {
fop.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
Try this for file to byte[]:
InputStream is = new FileInputStream(file);
int length = (int) file.length();
int take = 262144;//size of your chunk
byte[] bytes = new byte[take];
int offset=0;
int a = 0;
do {
a = is.read(bytes, 0, take);
offset += a;
//And you can add here each chunk created in to a list, etc, etc.
//encode to base 64 this is extra :)
String str = Base64.encodeToString(bytes, Base64.DEFAULT);
} while (offset < length);=
is.close();
is=null;
Consider generalizing the problem.
This method copies data in chunks:
public static <T extends OutputStream> T copy(InputStream in, T out)
throws IOException {
byte[] buffer = new byte[1024];
for (int r = in.read(buffer); r != -1; r = in.read(buffer)) {
out.write(buffer, 0, r);
}
return out;
}
This can then be used in both reading to and from byte arrays:
try (InputStream in = new FileInputStream("original.txt");
OutputStream out = new FileOutputStream("copy.txt")) {
byte[] contents = copy(in, new ByteArrayOutputStream()).toByteArray();
copy(new ByteArrayInputStream(contents), out);
}
An example would be a simple image.
I have tried so many things and it just refuses to work despite making a whole lot of sense.
What I've done so far is I'm able to grab 25 pictures and add them to
/sdcard/app name/sub/dir/filename.jpg
They all appear there according to the DDMS but they always have a filesize of 0.
I'm guessing it's probably because of my input stream?
Here's my function that handles the downloading and saving.
public void DownloadPages()
{
for (int fileC = 0; fileC < pageAmount; fileC++)
{
URL url;
String path = "/sdcard/Appname/sub/dir/";
File file = new File(path, fileC + ".jpg");
int size=0;
byte[] buffer=null;
try{
url = new URL("http://images.bluegartr.com/bucket/gallery/56ca6f9f2ef43ab7349c0e6511edb6d6.png");
InputStream in = url.openStream();
size = in.available();
buffer = new byte[size];
in.read(buffer);
in.close();
}catch(Exception e){
}
if (!new File(path).exists())
new File(path).mkdirs();
FileOutputStream out;
try{
out = new FileOutputStream(file);
out.write(buffer);
out.flush();
out.close();
}catch(Exception e){
}
}
}
It just keeps giving me 25 files in that directory but all of their file sizes are zero. I have no idea why. This is practically the same code I've used in a java program.
PS...
If you're gonna give me a solution... I've already tried code like this. It doesn't work.
try{
url = new URL(urlString);
in = new BufferedInputStream(url.openStream());
fout = new FileOutputStream(filename);
byte data[] = new byte[1024];
int count;
System.out.println("Now downloading File: " + filename.substring(0, filename.lastIndexOf(".")));
while ((count = in.read(data, 0, 1024)) != -1){
fout.write(data, 0, count);
}
}finally{
System.out.println("Download complete.");
if (in != null)
in.close();
if (fout != null)
fout.close();
}
}
Here's an image of what my directories look like
http://oi48.tinypic.com/2cpcprm.jpg
A bit change to your second option, try it as following way,
byte data[] = new byte[1024];
long total = 0;
int count;
while ( ( count = input.read(data)) != -1 )
{
total += count;
output.write( data,0,count );
}
This one is different in while statement while ((count = in.read(data, 0, 1024)) != -1)
Using Guava something like this should work:
String fileUrl = "xxx";
File file = null;
InputStream in;
FileOutputStream out;
try {
Uri url = new URI(fileUrl);
in = url.openStream();
out = new FileOutputStream(file)
ByteStreams.copy(in, out);
}
catch (IOException e) {
System.out.println(e.toString());
}
finally {
in.close();
out.flush();
out.close();
}
I am trying to get file content in bytes in Android application. I have get the file in SD card now want to get the selected file in bytes. I googled but no such success. Please help
Below is the code to get files with extension. Through this i get files and show in spinner. On file selection I want to get file in bytes.
private List<String> getListOfFiles(String path) {
File files = new File(path);
FileFilter filter = new FileFilter() {
private final List<String> exts = Arrays.asList("jpeg", "jpg", "png", "bmp", "gif","mp3");
public boolean accept(File pathname) {
String ext;
String path = pathname.getPath();
ext = path.substring(path.lastIndexOf(".") + 1);
return exts.contains(ext);
}
};
final File [] filesFound = files.listFiles(filter);
List<String> list = new ArrayList<String>();
if (filesFound != null && filesFound.length > 0) {
for (File file : filesFound) {
list.add(file.getName());
}
}
return list;
}
here it's a simple:
File file = new File(path);
int size = (int) file.length();
byte[] bytes = new byte[size];
try {
BufferedInputStream buf = new BufferedInputStream(new FileInputStream(file));
buf.read(bytes, 0, bytes.length);
buf.close();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Add permission in manifest.xml:
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
The easiest solution today is to used Apache common io :
http://commons.apache.org/proper/commons-io/javadocs/api-release/org/apache/commons/io/FileUtils.html#readFileToByteArray(java.io.File)
byte bytes[] = FileUtils.readFileToByteArray(photoFile)
The only drawback is to add this dependency in your build.gradle app :
implementation 'commons-io:commons-io:2.5'
+ 1562 Methods count
Since the accepted BufferedInputStream#read isn't guaranteed to read everything, rather than keeping track of the buffer sizes myself, I used this approach:
byte bytes[] = new byte[(int) file.length()];
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(file));
DataInputStream dis = new DataInputStream(bis);
dis.readFully(bytes);
Blocks until a full read is complete, and doesn't require extra imports.
Here is a solution that guarantees entire file will be read, that requires no libraries and is efficient:
byte[] fullyReadFileToBytes(File f) throws IOException {
int size = (int) f.length();
byte bytes[] = new byte[size];
byte tmpBuff[] = new byte[size];
FileInputStream fis= new FileInputStream(f);;
try {
int read = fis.read(bytes, 0, size);
if (read < size) {
int remain = size - read;
while (remain > 0) {
read = fis.read(tmpBuff, 0, remain);
System.arraycopy(tmpBuff, 0, bytes, size - remain, read);
remain -= read;
}
}
} catch (IOException e){
throw e;
} finally {
fis.close();
}
return bytes;
}
NOTE: it assumes file size is less than MAX_INT bytes, you can add handling for that if you want.
If you want to use a the openFileInput method from a Context for this, you can use the following code.
This will create a BufferArrayOutputStream and append each byte as it's read from the file to it.
/**
* <p>
* Creates a InputStream for a file using the specified Context
* and returns the Bytes read from the file.
* </p>
*
* #param context The context to use.
* #param file The file to read from.
* #return The array of bytes read from the file, or null if no file was found.
*/
public static byte[] read(Context context, String file) throws IOException {
byte[] ret = null;
if (context != null) {
try {
InputStream inputStream = context.openFileInput(file);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
int nextByte = inputStream.read();
while (nextByte != -1) {
outputStream.write(nextByte);
nextByte = inputStream.read();
}
ret = outputStream.toByteArray();
} catch (FileNotFoundException ignored) { }
}
return ret;
}
In Kotlin you can simply use:
File(path).readBytes()
You can also do it this way:
byte[] getBytes (File file)
{
FileInputStream input = null;
if (file.exists()) try
{
input = new FileInputStream (file);
int len = (int) file.length();
byte[] data = new byte[len];
int count, total = 0;
while ((count = input.read (data, total, len - total)) > 0) total += count;
return data;
}
catch (Exception ex)
{
ex.printStackTrace();
}
finally
{
if (input != null) try
{
input.close();
}
catch (Exception ex)
{
ex.printStackTrace();
}
}
return null;
}
A simple InputStream will do
byte[] fileToBytes(File file){
byte[] bytes = new byte[0];
try(FileInputStream inputStream = new FileInputStream(file)) {
bytes = new byte[inputStream.available()];
//noinspection ResultOfMethodCallIgnored
inputStream.read(bytes);
} catch (IOException e) {
e.printStackTrace();
}
return bytes;
}
Following is the working solution to read the entire file in chunks and its efficient solution to read the large files using a scanner class.
try {
FileInputStream fiStream = new FileInputStream(inputFile_name);
Scanner sc = null;
try {
sc = new Scanner(fiStream);
while (sc.hasNextLine()) {
String line = sc.nextLine();
byte[] buf = line.getBytes();
}
} finally {
if (fiStream != null) {
fiStream.close();
}
if (sc != null) {
sc.close();
}
}
}catch (Exception e){
Log.e(TAG, "Exception: " + e.toString());
}
To read a file in bytes, often used to read binary files, such as pictures, sounds, images, etc.
Use the method below.
public static byte[] readFileByBytes(File file) {
byte[] tempBuf = new byte[100];
int byteRead;
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
try {
BufferedInputStream bufferedInputStream = new BufferedInputStream(new FileInputStream(file));
while ((byteRead = bufferedInputStream.read(tempBuf)) != -1) {
byteArrayOutputStream.write(tempBuf, 0, byteRead);
}
bufferedInputStream.close();
return byteArrayOutputStream.toByteArray();
} catch (Exception e) {
e.printStackTrace();
return null;
}
}