We Keep Coding

Azure, Java: Read and Unzip a file which is saved in Azure Storage (Blobs) and encoded by Avro

I have a file in Azure Storage which is zipped and then encoded by Avro as Blob. I read it and decode it as you see in the following code:
public static int decodeAvroFile(String avroFile) throws Exception {
GenericDatumReader<Object> reader=new GenericDatumReader<Object>();
org.apache.avro.file.FileReader<Object> fileReader= DataFileReader.openReader(new File(avroFile),reader);
ByteArrayOutputStream os = new ByteArrayOutputStream();
try {
Schema schema=fileReader.getSchema();
DatumWriter<Object> writer=new GenericDatumWriter<Object>(schema);
JsonEncoder encoder = EncoderFactory.get().jsonEncoder(schema, os);
for (Object datum : fileReader) writer.write(datum,encoder);
encoder.flush();
SaveJasonInDB saveJson = new SaveJasonInDB();
saveJson.zipJsonParser(os, avroFile + "-decodedFile");
}
finally {
fileReader.close();
}
return 0;
}
After that, I have to pars it to get the content in zip file via JSON:
public void zipJsonParser(ByteArrayOutputStream os,String jsonFile) throws ParseException, IOException {
JSONParser parser = new JSONParser();
String st = os.toString();
JSONObject json = (JSONObject) parser.parse(st);
System.out.println(json.get("EnqueuedTimeUtc"));
JSONObject body = (JSONObject) json.get("Body");
createZipFile((String) body.get("bytes"), jsonFile);
}
Finaly I want to unzip the zip file:
public void unzip(String zipFilePath, String destDirectory) throws IOException {
File destDir = new File(destDirectory);
if (!destDir.exists()) {
destDir.mkdir();
}
ZipInputStream zipIn = new ZipInputStream(new FileInputStream(zipFilePath));
ZipEntry entry = zipIn.getNextEntry();
// iterates over entries in the zip file
while (entry != null) {
String filePath = destDirectory + File.separator + entry.getName();
if (!entry.isDirectory()) {
// if the entry is a file, extracts it
extractFile(zipIn, filePath);
} else {
// if the entry is a directory, make the directory
File dir = new File(filePath);
dir.mkdir();
}
zipIn.closeEntry();
entry = zipIn.getNextEntry();
}
zipIn.close();
}
private void extractFile(ZipInputStream zipIn, String filePath) throws IOException {
BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream(filePath));
byte[] bytesIn = new byte[BUFFER_SIZE];
int read = 0;
while ((read = zipIn.read(bytesIn)) != -1) {
bos.write(bytesIn, 0, read);
}
bos.close();
}
Everything is ok when I try the last part of code to unzip a normal zip file. But in this example, we have to decode Avro file and extract the zip file from a JSON element, it doesn't work. You see a part of JSON here:
{"EnqueuedTimeUtc":"2017-11-28T09:45:55.4560000Z","Properties":{},"SystemProperties":{"connectionDeviceId":"firstSAM","contentEncoding":"","connectionDeviceGenerationId":"636451410108166633","contentType":"application/json","connectionAuthMethod":"{\"scope\":\"hub\",\"type\":\"sas\",\"issuer\":\"iothub\",\"acceptingIpFilterRule\":null}","enqueuedTime":"2017-11-28T09:45:55.4560000Z"},"Body":{"bytes":"PK\u0003\u0004\n\u0000\u0000\u0000\u0000\u0000SK|K\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u000
Could you please help me??

Loading text file gives an error with path

I want to read a text file. For this I am giving a path of the file but its not getting read.
Giving error like : ClassLoader referenced unknown path: /data/app/com.kiranaapp-1/lib/arm
I have saved the text file in helper folder of an app.
public void ReadFile() {
try {
BufferedReader in = new BufferedReader(new FileReader("E:/siddhiwork/KiranaCustomerApp/app/src/main/java/com/kiranacustomerapp/helper/itemNames.txt"));
String str;
List<String> list = new ArrayList<String>();
while ((str = in.readLine()) != null) {
list.add(str);
}
String[] stringArr = list.toArray(new String[0]);
}
catch (FileNotFoundException e)
{
System.out.print(e);
}
catch (IOException e)
{
System.out.print(e);
}
}
As I debug to see if file is getting read and strings are stored in an array,
but nothing happens.
Help please , Thank you..
Edit :
My attempt to get strings in list, not getting any value in itemList
public class StartupActivity extends AppCompatActivity {
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
List<String> itemList = new ArrayList<>();
itemList = readRawTextFile(StartupActivity.this);
}
public static List<String> readRawTextFile(Context context) {
String sText = null;
List<String> stringList;
try{
InputStream is = context.getResources().openRawResource(R.raw.item_names);
//Use one of the above as per your file existing folder
int size = is.available();
byte[] buffer = new byte[size];
is.read(buffer);
is.close();
sText = new String(buffer, "UTF-8");
stringList = new ArrayList<String>(Arrays.asList(sText.split(" ")));
System.out.print(stringList);
} catch (IOException ex) {
ex.printStackTrace();
return null;
}
return stringList;
}
}

You should not give a file path to the computer path. Store file either in assets folder or in raw folder then fetch from there in android.
public String loadTextFromFile() {
String sText = null;
try {
//If your file is in assets folder
InputStream is = getAssets().open("file_name.txt");
//If your file is in raw folder
InputStream is = getResources().openRawResource(R.raw.file_name);
//Use one of the above as per your file existing folder
int size = is.available();
byte[] buffer = new byte[size];
is.read(buffer);
is.close();
sText = new String(buffer, "UTF-8");
} catch (IOException ex) {
ex.printStackTrace();
return null;
}
return sText;
}
To split text with "," format:
String[] sTextArray = sText.replace("\"", "").split(",");
List<String> stringList = new ArrayList<String>(Arrays.asList(sTextArray));

First of all, Put the file in raw directory under res directory.
Now try below code to read file,
public static String readRawTextFile(Context ctx, int resId) {
InputStream inputStream = ctx.getResources().openRawResource(resId);
InputStreamReader inputreader = new InputStreamReader(inputStream);
BufferedReader buffreader = new BufferedReader(inputreader);
String line;
StringBuilder text = new StringBuilder();
ArrayList<String> lineList = new ArrayList<>();
try {
while (( line = buffreader.readLine()) != null) {
text.append(line);
lineList.add(line);
text.append('\n');
}
} catch (IOException e) {
return null;
}
// Use your arraylist here, since its filled up.
return text.toString();
}

If the file is generated dynamically in cache, you can
File file = getCacheDir() + "FOLDER_PATH_WITH_FILENAME";
Otherwise, save the file in assets folder inside main directory.
main
-----> java
-----> res
-----> assets
-----> AndroidManifest.xml
then, get file using:
InputStream inputStream = getAssets().open("FILE_NAME");

How to write a database to a text file in android

I am working on a Spying application for my college project purpose. For that i have logged the Calls, Location and SMS of the device and stored them in a database. Now i want to export the contents of the database to a text file.. I tried the below code.
private void readAndWriteCallsData() {
File dataBaseFile = getDatabasePath("DATABASE");
File callDataFile = new File(Environment.getDataDirectory()+"/data/com.example.myapp/databases/"+"DATABASE");
try {
BufferedReader dbFileReader = new BufferedReader(new FileReader(callDataFile));
String eachLine;
while((eachLine = dbFileReader.readLine()) != null)
{
Callslog.append(eachLine);
Callslog.append("\n");
}
} catch (IOException e) {
e.printStackTrace();
}
}
But that is not working... Please help me...

You can encode the database file from binary stream to character stream by Base64, then decode the text when nessesary.
First find a Base64 library. You can use http://sourceforge.net/projects/iharder/files/base64/. There's only one file, "Base64.java".
Code example:
private void readAndWriteCallsData() {
File callDataFile = new File(Environment.getDataDirectory()+"/data/com.example.myapp/databases/"+"DATABASE");
try {
FileInputStream fis = new FileInputStream(callDataFile);
try{
byte[] buf = new byte[512];
int len;
while((len = fis.read(buf)) > 0){
String text = Base64.encodeBytes(buf, 0, len); // encode binary to text
Callslog.append(text);
Callslog.append("\n");
}
}finally{
fis.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
To revert it, code like following:
private void revertCallsData() {
File encodedCallDataFile; // get reference to the encoded text file
try {
BufferedReader br = new BufferedReader(new FileReader(encodedCallDataFile));
try{
String line;
while((line = br.readLine()) != null){
byte[] bin = Base64.decode(line); // decode each line to binary, you can get the original database file
}
}finally{
br.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}

ok guys after a lot of hit and trial i finally found the solution, here is the code, i saved the functionality in a button.
final String SAMPLE_DB_NAME = "MyDBName.db";//database name
save.setOnClickListener(new OnClickListener() {
#Override
public void onClick(View v) {
// TODO Auto-generated method stub
File sd = Environment.getExternalStorageDirectory();
File data = Environment.getDataDirectory();
FileChannel source=null;
FileChannel destination=null;
String currentDBPath = "/data/"+ "your package name" +"/databases/"+SAMPLE_DB_NAME;
String backupDBPath = SAMPLE_DB_NAME;
File currentDB = new File(data, currentDBPath);
File backupDB = new File(sd, backupDBPath);
try {
source = new FileInputStream(currentDB).getChannel();
destination = new FileOutputStream(backupDB).getChannel();
destination.transferFrom(source, 0, source.size());
source.close();
destination.close();
Toast.makeText(getApplicationContext(),"Your database has been exported",
Toast.LENGTH_LONG).show();
} catch(IOException e) {
e.printStackTrace();
}
}
});
the database will be saved in /storage/emulated/0/

I would recommend to export into a structered file format such as JSON or CSV. Here is my JSON exporter method. Maybe it helps
private static final String LOG_FOLDER = "/ExportFolder";
private static final String FILE_NAME = "export_file.json";
public static void exportMeasurementsJSON(Handler mHandler) {
sendToastMessage("Export to JSON started", mHandler);
File folder = new File(Environment.getExternalStorageDirectory()
+ LOG_FOLDER);
if (!folder.exists())
folder.mkdir();
final String filename = folder.toString() + "/"
+ getLogFileName(".json");
try {
FileWriter fw = new FileWriter(filename, false /* append */);
// get the db
SomeDateSource db = PIApplication.getDB();
// Google Gson for serializing Java Objects into JSON
Gson mGson = new GsonBuilder().create();
Cursor c = db.getAllRows();
if (c != null) {
while (c.moveToNext()) {
fw.append(mGson.toJson(new DBEntry(c
.getString(1), c.getString(2), c
.getDouble(3), c.getLong(4))));
fw.append('\n');
}
c.close();
}
fw.close();
sendToastMessage("Export finished", mHandler);
} catch (Exception e) {
sendToastMessage("Something went wrong", mHandler);
e.printStackTrace();
}
}
If you're interested I can also add my CSV exporter.

Your question is not that clear (Are you trying to copy the file to an alternative location or export the actual data from it?)
If you only wish to copy the file, you can copy the db file using the following method:
public static void copyFile(String sourceFileFullPath, String destFileFullPath) throws IOException
{
String copyFileCommand = "dd if=" + sourceFileFullPath + " of=" + destFileFullPath;
Runtime.getRuntime().exec(copyFileCommand);
}
Simply call that method with your database file path (/data/data/package_name/databases/database_name) as sourceFileFullPath and your target file path as destFileFullPath. You can than use tools such as SQLite Expert to view the content of the database on your PC/Laptop.
If your intention is to export the data from the database and store it in a text file (a CSV file or anything similar), then you should not read the database file content, and instead use the SQLiteDatabase class to query each table contents into a Cursor and iterate it to write each cursor row into a text file.

You could export the entire db into your sdcard folder and then use SQLite manager to open and see it's content.
A Example is available here: http://www.techrepublic.com/blog/software-engineer/export-sqlite-data-from-your-android-device/

Here is the complete method for writing the Database in the SD Card:
/**
* Copy the app db file into the sd card
*/
private void backupDatabase(Context context) throws IOException {
//Open your local db as the input stream
String inFileName = "/data/data/yourappPackageName/databases/yourDBName.db";
// OR use- context.getFilesDir().getPath()+"/databases/yourDBName.db";//
File dbFile = new File(inFileName);
FileInputStream fis = new FileInputStream(dbFile);
String outFileName = Environment.getExternalStorageDirectory()+"/"+SQLiteDataHelper.DB_NAME;
//Open the empty db as the output stream
OutputStream output = new FileOutputStream(outFileName);
//transfer bytes from the inputfile to the outputfile
byte[] buffer = new byte[1024];
int length;
while ((length = fis.read(buffer))>0){
output.write(buffer, 0, length);
}
//Close the streams
output.flush();
output.close();
fis.close();
}
Hope it will help you.

One way to do this (I assume its a long procedure, easy one though), if you know the database and get all the tables and retrieve info from those tables. Since, we are talking about sqlite DBs, I assume it will be small.
SELECT * FROM dbname.sqlite_master WHERE type='table';

FileNotFoundException while copying data to SD Card - Android

Following is the code which I use to copy a folder containing a txt file. The folder resides in assets folder of my application. While I copy , I get File not found exception in the line out = new FileOutputStream(newFileName);
I get this working perfectly, when I save this to /data/data folder; ie; internal memory. I have checked the SD card state and it shows mounted.
public class CpyAsset extends Activity {
/** Called when the activity is first created. */
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
copyFileOrDir("edu1");//directory name in assets
}
File sdCard = Environment.getExternalStorageDirectory();
private void copyFileOrDir(String path) {
AssetManager assetManager = this.getAssets();
String assets[] = null;
try {
assets = assetManager.list(path);
if (assets.length == 0) {
copyFile(path);
} else {
File dir = new File (sdCard.getAbsolutePath());
if (!dir.exists()){
System.out.println("Created directory"+sdCard.getAbsolutePath());
boolean result = dir.mkdir();
System.out.println("Result of directory creation"+result);
}
for (int i = 0; i < assets.length; ++i) {
copyFileOrDir(path + "/" + assets[i]);
}
}
} catch (IOException ex) {
System.out.println("Exception in copyFileOrDir"+ex);
}
}
private void copyFile(String filename) {
AssetManager assetManager = this.getAssets();
InputStream in = null;
OutputStream out = null;
try {
in = assetManager.open(filename);
String newFileName = sdCard.getAbsolutePath() + "/"+filename;
out = new FileOutputStream(newFileName);
byte[] buffer = new byte[1024];
int read;
while ((read = in.read(buffer)) != -1) {
out.write(buffer, 0, read);
}
in.close();
in = null;
out.flush();
out.close();
out = null;
} catch (Exception e) {
System.out.println("Exception in copyFile"+e);
}
}
}
Exception
01-01 06:13:34.783: INFO/System.out(11334): Exception in copyFilejava.io.FileNotFoundException: /mnt/sdcard/edu1/anees.txt: open failed: ENOENT (No such file or directory)
The folder(and the content) I try to copy is in assets/edu1/abc.txt
Can someone please let me know what causes this as I cannot find any obvious reasons for the same? Any help is much appreciated.

You are always trying to create the external storage root dir in this part:
File dir = new File (sdCard.getAbsolutePath());
if (!dir.exists()){
System.out.println("Created directory"+sdCard.getAbsolutePath());
boolean result = dir.mkdir();
System.out.println("Result of directory creation"+result);
}
so you are not creating the folder edu1/ and the creation of the file anees.txt in that folder will fail.

in your code you check if the sdcard path is exist while you should check for your path which result in the dir "edu1" is never created try use this instead
File dir = new File (sdCard.getAbsolutePath()+"/"+path);

Try it out this way.......
File f = new File("/sdcard/assets/edu1/abc.txt");
FileWriter fw = new FileWriter(f);
BufferedWriter bw = new BufferedWriter(fw);

Android - Reading text File from Application subclass

I have a method in a subclass of Application that needs to read information from a text file. I have used this method in a subclass of Activity and it worked fine.
InputStream is = this.getResources().openRawResource(R.raw.elements);
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
String str = reader.readLine();
However this gives a null pointer exception if I use this code in the Application subclass.
the text file is in the res/raw folder.

private String readTxt() {
InputStream inputStream = getResources().openRawResource(R.raw.elements);
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
int i;
try {
i = inputStream.read();
while (i != -1) {
byteArrayOutputStream.write(i);
i = inputStream.read();
}
inputStream.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return byteArrayOutputStream.toString();
}
Works Perfectly, Source : Display text file in /res/raw