We Keep Coding

Why can't my phone app open a file it stored in a Room Database?

Problem: My Android phone app can open various file types stored in an Android Room pre-populated SQLite database but it cannot open files the app itself has added to the pre-populated database (except it can open .txt files). I believe the issue is probably with how the I coded the copying and conversion of a selected file to byte[] data. The app is java based, and I have done this in Java before in a desktop app, so I just can't seem to find the issue. Maybe it is a permission issue, I'm just not sure and someone standing outside looking in may see what I can't.
What I have tried: Since the app can open various existing pre-populated files successfully from the DB, I've concentrated on and stepped through methods writing files to the DB. I'm not receiving any errors. I suspect it may just be minor issue since I can't seem to see it.
What I'm trying to do: I'm trying to emulate the desktop version of this app into a Android phone version. I know it's not recommended or common practice to populate files to a DB, but this app needs to be able to read and write files to the DB supporting it. This will be a full range of file types like the desktop version (e.g., pics, docs, audio, video, etc.). However, as I stated above, .txt files seem to have no issue. The user can select files stored on their phone into a table that captures the fileName and filePath to a TableRow in a TableLayout. Below are methods involved. The plan is to refactor functionality once I get it working:
Capturing the full path and filename for each row - Uses the captured filepath to convert to a byte[] to store the data. The filename and file byte data are stored in a Files table, example, Files(fileName, fileData(byte[])). Each file is added to an ArrayList<Files> which the method returns
public static List<Files> captureNoteFiles(TableLayout table){
List<Files> noteFiles = new ArrayList<>();
int i = table.getChildCount();
if(i>1){
for (int itr = 1; itr<i; itr++) { // iterating through indexes
TableRow tr = (TableRow) table.getChildAt(itr);
TextView tv = (TextView) tr.getChildAt(1); // 1 is the file path position
File f = new File(tv.getText().toString());
String n = f.getName();
try {
FileInputStream fis = new FileInputStream(f.getPath());
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
for (int read; (read = fis.read(buf)) != -1; ) {
bos.write(buf, 0, read);
}
fis.close();
noteFiles.add(new Files(0, n, bos.toByteArray()));
} catch (Exception e) {
e.printStackTrace();
Log.d("Input File", e.toString());
}
}
}
return noteFiles;
}
Iteration of the ArrayList - The ArrayList<Files> is iterated and populated to the Files table and an ID capture to associate those files with a particular note of reference.
public static void addNewNoteFiles(int noteID, List<Files> nf){
if(nf.size()>0) {
for (Files f : nf) {
long id = rdb.getFilesDao().addFile(f);
rdb.getFilesByNoteDao().insert(new FilesByNote(noteID, (int) id));
}
}
}
Files Entity
#Entity(tableName = "Files")
public class Files implements Parcelable {
#PrimaryKey(autoGenerate = true)
#ColumnInfo(name = "FileID")
private int fileID;
#ColumnInfo(name = "FileName")
private String fileName;
#TypeConverters(FileTypeConverter.class)
#ColumnInfo(name = "FileData", typeAffinity = ColumnInfo.TEXT)
private byte[] fileData;
#SuppressWarnings(RoomWarnings.CURSOR_MISMATCH)
public Files(int fileID, String fileName, byte[] fileData){
this.fileID = fileID;
this.fileName = fileName;
this.fileData = fileData;
}
}

First you are assuming that an insert works as per :-
long id = rdb.getFilesDao().addFile(f);
rdb.getFilesByNoteDao().insert(new FilesByNote(noteID, (int) id));
What if the row isn't inserted? and returns an id of -1?
So I'd suggest adding getters to the Files class such as :-
public int getFileID() {
return fileID;
}
public String getFileName() {
return fileName;
}
public byte[] getFileData() {
return fileData;
}
and then add the following to FilesDao :-
#Query("SELECT coalesce(length(FileData)) FROM Files WHERE FileID=:fileId")
abstract long getFilesDataLength(long fileId);
and then amending the addNewNoteFiles to be :-
public static void addNewNoteFiles(int noteID, List<Files> nf){
final String TAG = "ADDNEWNOTE";
if(nf.size()>0) {
for (Files f : nf) {
long id = rdb.getFilesDao().addFile(f);
if (id > 0) {
long lengthOfFileData = rdb.getFilesDao().getFilesDataLength(id);
Log.d(TAG,
"Inserted File = " + f.getFileName() +
" DataLength = " + f.getFileData().length +
" ID = " + f.getFileID() +
" Length of Stored Data = " + lengthOfFileData);
if (f.getFileData().length != lengthOfFileData) {
Log.d(TAG,"WARNING FileData length MISMATCH for File = " + f.getFileName() + "\n\t Expected " + f.getFileData().length + " Found " + lengthOfFileData);
}
rdb.getFilesByNoteDao().insert(new FilesByNote(noteID, (int) id));
} else {
Log.d(TAG,"NOT INSERTED File = " + f.getFileName());
}
}
}
}
Run and check the log. Are all the files inserted? Do the lengths match? Are the lengths as expected (if all 0 lengths, or some, then obviously something is amiss when building the ByteArrayOutputStream)
You may wish to add similar for inserting the FilesByNote i.e. have the insert Dao return a long (it returns the rowid) and check if the value is > 0.
You may wonder what rowid is. Well it's a normally hidden column, perhaps hidden as it would appear that FilesByNotes is an associative table mapping(associating) Note(s) with Files and as such has a composite primary key NoteId and FileId which is not an alias of the rowid, so rowid will be hidden as such. However, the value will be auto-generated or -1 if no row is inserted.
ALL tables, with the exception of tables defined with WITHOUT ROWID, have a rowid column. Room does not allow thee definition of WITHOUT ROWID tables.
You wouldn't be concerned about the value if it's greater than 0, just that it is greater than 0 and thus a row was inserted.
The above may help to determine any issues encountered when inserting the data. If there are none found then the issue is else where.

Java: Syntax error on token "else"

Okay, I've looked at quite a few stack overflow questions and quite a few blogs, and still cannot find the answer to this. No, it doesn't look like it is missing braces, has extra semicolons, or any other typo that I am aware of. Why is the 'else' showing an error?
// check if myid property is set and create it if not
String subtopicMyId = subtopicNode.hasProperty("myid") ? subtopicNode.getProperty("myid").getString() : "";
if (subtopicMyId.equals("")) {
// generate new myid and check it against the list of existing IDs until a unique one is generated
do {
// generate new myid
Object topicmyidobj = new Object();
subtopicMyId = topicmyidobj.toString().split("#")[1].toUpperCase();
} while ( subtopicMyId.equals("") || existingMyIds.contains(subtopicMyId) );
// set myid on this node
subtopicNode.setProperty("myid", subtopicMyId);
subtopicNode.setProperty("parentid", topicMyId);
subtopicNode.save();
// add new myid to list of existsing IDs so that it doesn't get reused
existingMyIds.add(subtopicMyId);
} else {
// if subtopic has myid already
// compare the parentid to the parent myid
String subtopicParentId = subtopicNode.getProperty("parentid").getString();
if (!subtopicParentId.equals(topicMyId)) {
// they don't match
String subtopicNodePath = subtopicNode.getPath();
String topicNodePath = topicNode.getPath();
// find path to topic node that has matching myid to this subtopic's parentid
// loop through parent nodes
NodeIterator reorgTopicsIter = compNode.getNodes();
while (reorgTopicsIter.hasNext()) {
// loop through parent objects to find a matching myid for parentid
Node reorgTopicNode = (Node)reorgTopicsIter.next();
// get the myid property from this node, if it exists, and compare the parentid to it
String reorgTopicMyId = reorgTopicNode.hasProperty("myid") ? reorgTopicNode.getProperty("myid").getString() : "";
if (!reorgTopicMyId.equals("")) {
// parent myid exists and is not blank
if (reorgTopicMyId.equals(subtopicParentId)) {
// parentid does match parent myid
String reorgTopicNodePath = reorgTopicNode.getPath();
// determine how many parent objects there are
int reorgTopicSubtopics = 0;
NodeIterator reorgSubtopicsIter = reorgTopicNode.getNodes();
while (reorgSubtopicsIter.hasNext()) {
Node reorgSubtopicNode = (Node)reorgSubtopicsIter.next();
reorgTopicSubtopics++;
}
// set source to this child object
String source = subtopicNode.getPath();
// set destination to matching parent object with new child object appended
String destination = reorgTopicNodePath + "/subtopic-" + (reorgTopicSubtopics + 1);
// create session for move and perform move
Session session = resourceResolver.adaptTo(Session.class);
session.move(source, destination);
session.save();
} else {
// parentid does not match parent myid.
// nothing we need to do here;
// it just moves on to check next parent myid.
}
} else {
// parent myid does not exist or is blank
}
} else {
// no more parent objects to loop through, so we need to check if a match was found
// if no match was found, then parent was deleted or no longer exists, so we need to remove this child
}
} else {
// parentid does match parent myid
}
}
Here is the error in the console:
An error occurred at line: 145 in the jsp file: /apps/covidien/components/content/utilities/faq-node-process/faq-node-process.jsp
Syntax error on token "else", delete this token
142: subtopicNode.save();
143: // add new myid to list of existsing IDs so that it doesn't get reused
144: existingMyIds.add(subtopicMyId);
145: } else {
146: // if subtopic has myid already
147: // compare the parentid to the parent myid
148: String subtopicParentId = subtopicNode.getProperty("parentid").getString();

the number of if and else statement is not same. you have 4 if blocks but 5 else block.

five else with four if, not match.
At least number of if should more than else.

else can't be used after while. There's nothing in that else block anyway, so it should just be deleted.
There's a lot of redundant code here. My rewrite:
// check if myid property is set and create it if not
String subtopicMyId = subtopicNode.getProperty("myid").getString();
if (subtopicMyId == null) {
// generate new myid and check it against the list of existing IDs until a unique one is generated
do {
// generate new myid
Object topicmyidobj = new Object();
subtopicMyId = topicmyidobj.toString().split("#")[1].toUpperCase();
} while ( subtopicMyId.equals("") || existingMyIds.contains(subtopicMyId) );
// set myid on this node
subtopicNode.setProperty("myid", subtopicMyId);
subtopicNode.setProperty("parentid", topicMyId);
subtopicNode.save();
// add new myid to list of existsing IDs so that it doesn't get reused
existingMyIds.add(subtopicMyId);
} else {
// if subtopic has myid already
// compare the parentid to the parent myid
String subtopicParentId = subtopicNode.getProperty("parentid").getString();
if (!subtopicParentId.equals(topicMyId)) {
// they don't match
String subtopicNodePath = subtopicNode.getPath();
String topicNodePath = topicNode.getPath();
// find path to topic node that has matching myid to this subtopic's parentid
// loop through parent nodes
NodeIterator reorgTopicsIter = compNode.getNodes();
while (reorgTopicsIter.hasNext()) {
// loop through parent objects to find a matching myid for parentid
Node reorgTopicNode = (Node)reorgTopicsIter.next();
// get the myid property from this node, if it exists, and compare the parentid to it
String reorgTopicMyId = reorgTopicNode.getProperty("myid").getString();
if (reorgTopicMyId != null && reorgTopicMyId.equals(subtopicParentId)) {
// parentid does match parent myid
String reorgTopicNodePath = reorgTopicNode.getPath();
// determine how many parent objects there are
int reorgTopicSubtopics = 0;
NodeIterator reorgSubtopicsIter = reorgTopicNode.getNodes();
while (reorgSubtopicsIter.hasNext()) {
Node reorgSubtopicNode = (Node)reorgSubtopicsIter.next();
reorgTopicSubtopics++;
}
// set source to this child object
String source = subtopicNode.getPath();
// set destination to matching parent object with new child object appended
String destination = reorgTopicNodePath + "/subtopic-" + (reorgTopicSubtopics + 1);
// create session for move and perform move
Session session = resourceResolver.adaptTo(Session.class);
session.move(source, destination);
session.save();
} else {
// parentid does not match parent myid.
// nothing we need to do here;
// it just moves on to check next parent myid.
}
}
} else {
// parentid does match parent myid
}
}

Java ExecutorService Runnable doesn't update value

I'm using Java to download HTML contents of websites whose URLs are stored in a database. I'd like to put their HTML into database, too.
I'm using Jsoup for this purpose:
public String downloadHTML(String byLink) {
String htmlInPage = "";
try {
Document doc = Jsoup.connect(byLink).get();
htmlInPage = doc.html();
} catch (org.jsoup.UnsupportedMimeTypeException e) {
// process this and some other exceptions
}
return htmlInPage;
}
I'd like to download websites concurrently and use this function:
public void downloadURL(int websiteId, String url,
String categoryName, ExecutorService executorService) {
executorService.submit((Runnable) () -> {
String htmlInPage = downloadHTML(url);
System.out.println("Category: " + categoryName + " " + websiteId + " " + url);
String insertQuery =
"INSERT INTO html_data (website_id, html_contents) VALUES (?,?)";
dbUtils.query(insertQuery, websiteId, htmlInPage);
});
}
dbUtils is my class based on Apache Commons DbUtils. Details are here: http://pastebin.com/iAKXchbQ
And I'm using everything mentioned above in a such way: (List<Object[]> details are explained on pastebin, too)
public static void main(String[] args) {
DbUtils dbUtils = new DbUtils("host", "db", "driver", "user", "pass");
List<String> categoriesList =
Arrays.asList("weapons", "planes", "cooking", "manga");
String sql = "SELECT lw.id, lw.website_url, category_name " +
"FROM list_of_websites AS lw JOIN list_of_categories AS lc " +
"ON lw.category_id = lc.id " +
"where category_name = ? ";
ExecutorService executorService = Executors.newFixedThreadPool(10);
for (String category : categoriesList) {
List<Object[]> sitesInCategory = dbUtils.select(sql, category );
for (Object[] entry : sitesInCategory) {
int websiteId = (int) entry[0];
String url = (String) entry[1];
String categoryName = (String) entry[2];
downloadURL(websiteId, url, categoryName, executorService);
}
}
executorService.shutdown();
}
I'm not sure if this solution is correct but it works. Now I want to modify code to save HTML not from all websites in my database, but only their fixed ammount in each category.
For example, download and save HTML of 50 websites from the "weapons" category, 50 from "planes", etc. I don't think it's necessary to use sql for this purpose: if we select 50 sites per category, it doesn't mean we save them all, because of possibly incorrect syntax and connection problems.
I've tryed to create separate class implementing Runnable with fields: counter and maxWebsitesPerCategory, but these variables aren't updated. Another idea was to create field Map<String,Integer> sitesInCategory instead of counter, put each category as a key there and increment its value until it reaches maxWebsitesPerCategory, but it didn't work, too. Please, help me!
P.S: I'll also be grateful for any recommendations connected with my realization of concurrent downloading (I haven't worked with concurrency in Java before and this is my first attempt)

How about this?
for (String category : categoriesList) {
dbUtils.select(sql, category).stream()
.limit(50)
.forEach(entry -> {
int websiteId = (int) entry[0];
String url = (String) entry[1];
String categoryName = (String) entry[2];
downloadURL(websiteId, url, categoryName, executorService);
});
}
sitesInCategory has been replaced with a stream of at most 50 elements, then your code is run on each entry.
EDIT
In regard to comments. I've gone ahead and restructured a bit, you can modify/implement the content of the methods I've suggested.
public void werk(Queue<Object[]> q, ExecutorService executorService) {
executorService.submit(() -> {
try {
Object[] o = q.remove();
try {
String html = downloadHTML(o); // this takes one of your object arrays and returns the text of an html page
insertIntoDB(html); // this is the code in the latter half of your downloadURL method
}catch (/*narrow exception type indicating download failure*/Exception e) {
werk(q, executorService);
}
}catch (NoSuchElementException e) {}
});
}
^^^ This method does most of the work.
for (String category : categoriesList) {
Queue<Object[]> q = new ConcurrentLinkedQueue<>(dbUtils.select(sql, category));
IntStream.range(0, 50).forEach(i -> werk(q, executorService));
}
^^^ this is the for loop in your main
Now each category tries to download 50 pages, upon failure of downloading a page it moves on and tries to download another page. In this way, you will either download 50 pages or have attempted to download all pages in the category.

Java Storing Multiple Rows From Select Queries

This is actually a re-do of an older question of mine that I have completely redone because my old question seemed to confuse people.
I have written a Java program that Queries a database and is intended to retrieve several rows of data. I have previously written the program in Informix-4GL and I am using a sql cursor to loop through the database and store each row into a "dynamic row of record". I understand there are no row of records in Java so I have ended up with the following code.
public class Main {
// DB CONNECT VARIABLE ===========================
static Connection gv_conn = null;
// PREPARED STATEMENT VARIABLES ==================
static PreparedStatement users_sel = null;
static ResultSet users_curs = null;
static PreparedStatement uinfo_sel = null;
static ResultSet uinfo_curs = null;
// MAIN PROGRAM START ============================
public static void main(String[] args) {
try {
// CONNECT TO DATABASE CODE
} catch(Exception log) {
// YOU FAILED CODE
}
f_prepare(); // PREPARE THE STATEMENTS
ArrayList<Integer> list_id = new ArrayList<Integer>();
ArrayList<String> list_name = new ArrayList<String>();
ArrayList<Integer> list_info = new ArrayList<String>();
ArrayList<String> list_extra = new ArrayList<String>();
try {
users_sel.setInt(1, 1);
users_curs = users_sel.executeQuery();
// RETRIEVE ROWS FROM USERS
while (users_curs.next()) {
int lv_u_id = users_curs.getInt("u_id");
String lv_u_name = users_curs.getString("u_name");
uinfo_sel.setInt(1, lv_u_id);
uinfo_curs = uinfo_sel.executeQuery();
// RETRIEVE DATA FROM UINFO RELATIVE TO USER
String lv_ui_info = uinfo_curs.getString("ui_info");
String lv_ui_extra = uinfo_curs.getString("ui_extra");
// STORE DATA I WANT IN THESE ARRAYS
list_id.add(lv_u_id);
list_name.add(lv_u_name);
list_info.add(lv_ui_info);
list_extra.add(lv_ui_extra);
}
} catch(SQLException log) {
// EVERYTHING BROKE
}
// MAKING SURE IT WORKED
System.out.println(
list_id.get(0) +
list_name.get(0) +
list_info.get(0) +
list_extra.get(0)
);
// TESTING WITH ARBITRARY ROWS
System.out.println(
list_id.get(2) +
list_name.get(5) +
list_info.get(9) +
list_extra.get(14)
);
}
// PREPARE STATEMENTS SEPARATELY =================
public static void f_prepare() {
String lv_sql = null;
try {
lv_sql = "select * from users where u_id >= ?"
users_sel = gv_conn.prepareStatement(lv_sql);
lv_sql = "select * from uinfo where ui_u_id = ?"
uinfo_sel = gv_conn.prepareStatement(lv_sql)
} catch(SQLException log) {
// IT WON'T FAIL COZ I BELIEEEVE
}
}
}
class DBConn {
// connect to SQLite3 code
}
All in all this code works, I can hit the database once, get all the data I need, store it in variables and work with them as I please however this does not feel right and I think it's far from the most suited way to do this in Java considering I can do it with only 15 lines of code in Informix-4GL.
Can anyone give me advice on a better way to achieve a similar result?

In order to use Java effectively you need to use custom objects. What you have here is a lot of static methods inside a class. It seems that you are coming from a procedural background and if you try to use Java as a procedural language, you will not much value from using it. So first off create a type, you can plop it right inside your class or create it as a separate file:
class User
{
final int id;
final String name;
final String info;
final String extra;
User(int id, String name, String info, String extra)
{
this.id = id;
this.name = name;
this.info = info;
this.name = name;
}
void print()
{
System.out.println(id + name + info + extra);
}
}
Then the loop becomes:
List<User> list = new ArrayList<User>();
try {
users_sel.setInt(1, 1);
users_curs = users_sel.executeQuery();
// RETRIEVE ROWS FROM USERS
while (users_curs.next()) {
int lv_u_id = users_curs.getInt("u_id");
String lv_u_name = users_curs.getString("u_name");
uinfo_sel.setInt(1, lv_u_id);
uinfo_curs = uinfo_sel.executeQuery();
// RETRIEVE DATA FROM UINFO RELATIVE TO USER
String lv_ui_info = uinfo_curs.getString("ui_info");
String lv_ui_extra = uinfo_curs.getString("ui_extra");
User user = new User(lv_u_id, lv_u_name, lv_ui_info, lv_ui_extra);
// STORE DATA
list.add(user);
}
} catch(SQLException log) {
// EVERYTHING BROKE
}
// MAKING SURE IT WORKED
list.get(0).print();
This doesn't necessarily address the number of lines. Most people who use Java don't interact with databases with this low-level API but in general, if you are looking to get down to the fewest number of lines (a questionable goal) Java isn't going to be your best choice.

Your code is actually quite close to box stock JDBC.
The distinction is that in Java, rather than having a discrete collection of arrays per field, we'd have a simple Java Bean, and a collection of that.
Some examples:
public class ListItem {
Integer id;
String name;
Integer info;
String extra;
… constructors and setters/getters ellided …
}
List<ListItems> items = new ArrayList<>();
…
while(curs.next()) {
ListItem item = new ListItem();
item.setId(curs.getInt(1));
item.setName(curs.getString(2));
item.setInfo(curs.getInfo(3));
item.setExtra(curs.getString(4));
items.add(item);
}
This is more idiomatic, and of course does not touch on the several frameworks and libraries available to make DB access a bit easier.

Improve the search functionality

How can I improve the search functionality.? I have written some codes to search for something.The search was taking too much time. And the code snippets here,
I am pulling the data from the database using this method.,
OracleConnection connection = null;
OraclePreparedStatement ptmst = null;
OracleResultSet rs = null;
OracleCallableStatement cstmt = null;
StringBuffer strBfr = new StringBuffer();
ArrayList myList = new ArrayList();
try
{
connection = (OracleConnection) TransactionScope.getConnection();
strBfr.append("select distinct .......... ");
ptmst = (OraclePreparedStatement)connection.prepareStatement(strBfr.toString());
rs = (OracleResultSet)ptmst.executeQuery();
while (rs.next())
{
HashMap hashItems = new HashMap();
hashItems.put("first",rs.getString(1));
hashItems.put("second",rs.getString(2));
myList.add(hashItems);
}
}
catch (Exception e) {
}
finally {
try {
if (ptmst != null) {
ptmst.close();
}
} catch (Exception e) {
}
try {
if (connection != null) {
TransactionScope.releaseConnection(connection);
}
} catch (Exception e) {
}
}
return myList;
In my jsp:
ArrayList getValues = new ArrayList();
getValues = //calling Method here.
for(int i=0; i < getValues.size();i++)
{
HashMap quoteSrch=(HashMap)allPOV.get(i);
first = (String)quoteSrch.get("first");
second = (String)quoteSrch.get("second");
}
Query:
SELECT DISTINCT(mtl.segment1),
mtl.description ,
mtl.inventory_item_id ,
mtl.attribute16
FROM mtl_system_items_b mtl,
mtl_system_items_tl k
WHERE 1 =1
AND mtl.organization_id = ?
AND k.inventory_item_id = mtl.inventory_item_id
AND NVL(orderable_on_web_flag,'N')= 'Y'
AND NVL(web_status,'UNPUBLISHED') = 'PUBLISHED'
AND mtl.SEGMENT1 LIKE ? --Here is the search term

Make sure organization_id , inventory_item_id and especially SEGMENT1 is indexed in your table.
Your query is pretty standard , if that doesn't work then it seems like your DB server is responding slow which could be due to number of reasons like low space , low memory , slow disk/read etc.
You can then ask your DBA/Server admins to check that.

First you need to find out the real problem
Is it the DB query
Is it the Network (is the App and the DB located on the same machine?)
Once you have identified that it is the DB query, then it becomes more of a DB question.
How does the two tables look like?
Any index used?
How does the data look like (How many rows etc)
After you have analyzed this, you should be able to post the question differently and expect an answer. I am not a DB guy, but I am sure someone would be able to provide some pointers.

Tunning has to be done:
Check TransactionScope.getConnection(); is giving connection without any delay.
Instead of creating new HashMap hashItems = new HashMap(); you can use
while (rs.next()){
myList.add(rs.getString(1) + "delimiter" + rs.getString(2));
}
in jsp use
first = allPOV.get(i).split("delimter")[0];
second = allPOV.get(i).split("delimter")[1];
so that you can reduce memory.
If possible use limit in your query, and use index on SEGMENT1 link.