We Keep Coding

Share limited amount of resources between different threads

I write selenium tests and make it parallel via testng.
There are some tests which should use resource, and that resource cant be used in tests, while it using in another test.
So to make it clear let me describe in other words, I have 10 resources, and when some test start working with one of them, only 9 another resources should be available in another tests. If all 10 resources are busy, and another test attempts to get it, it should wait until one of test will finish execution and free it's resource.
Im trying to create provider which will control desired behaviour but it looks like I get deadlocks, because it hangs out some times at synchronized method call.
My plan is provider have 2 methods get() and remove()
get() called in test method to get resource
remove() called in method annotated with #AfterMethod annotation and this method is default method of specific interface which should be implemented in class, containing resource usage
Here is provider class:
public class ResourceProvider {
private static final Logger logger = LogManager.getLogger();
private static List<Resource> freeResources;
private static Map<String, List<Resource>> resourcesInUse;
static {
freeResources = new ArrayList<>();
//here is resource initialization to fill freeResources list
resourcesInUse = new HashMap<>();
}
public static synchronized Resource get() {
String testName = Thread.currentThread().getStackTrace()[2].getClassName()
+ "." + Thread.currentThread().getStackTrace()[2].getMethodName();
Resource resource = null;
logger.info(String.format("Attempt to get resource for %s test", testName));
for (int i = 0; i < 240; i++) {
if (freeResources.isEmpty()) {
try {
Thread.sleep(5_000);
} catch (InterruptedException e) {
e.printStackTrace();
}
} else {
resource = freeResources.get(0);
if (resourcesInUse.containsKey(testName)) {
resourcesInUse.get(testName).add(resource);
} else {
List<Resource> resources = new ArrayList<>();
resources.add(resource);
resourcesInUse.put(testName, resources);
}
freeResources.remove(resource);
break;
}
}
if (resource == null) {
throw new RuntimeException(String.format("There is no free resource for '%s' in 20 minutes", testName));
}
logger.info(String.format("Resource %s used in %s", resource, testName));
return resource;
}
public static synchronized boolean remove(ITestResult result) {
String testName = result.getMethod().getTestClass().getName() + "." + result.getMethod().getMethodName();
return remove(testName);
}
public static synchronized boolean remove(String testName) {
boolean isTestUseResource = resourcesInUse.containsKey(testName);
if (isTestUseResource) {
logger.info(String.format("Removing %s resources, used in %s", resourcesInUse.get(testName), testName));
freeResources.addAll(resourcesInUse.get(testName));
resourcesInUse.remove(testName);
}
return isTestUseResource;
}
Interface:
public interface RemoveResource {
#AfterMethod
default void removeResource(ITestResult result) {
ResourceProvider.remove(result);
}
But this code doesnt work good, it hangs out at remove() call sometimes.
May you help me to understand why I get hangs out and how to resolve it?

android - Check if the room database is populated on startup without Livedata

I'm fairly new to Android and I want to have a database in my app.
I'm introduced to Room the documents say it's the best way to implement databases in the android.
Now I have to pre-populate some data in the database, and make sure that it gets populated before the app startup.
I see that there are many things like LiveData, Repositories, ViewModels and MediatorLiveData.
But I just want to keep it plain and simple, without using the said things how can one find if the database has been populated before the application launch.
I'm getting loads of NullPointerExceptions.
I'm using onCreateCallback() to populate the database but when I try to get the item from database it produces NullPointerException and after some time it may or may not produce the same warning, and the question remains the same what is the best way to know when the database is populated completely.
Here is a Minimal Example
public class MainActivity extends AppCompatActivity {
private TextView nameView;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
nameView = findViewById(R.id.name);
new NamesAsyncTask().execute();
}
private class NamesAsyncTask extends AsyncTask<Void,Void,String> {
private NameDao mNameDao;
#Override
public String doInBackground(Void... params) {
NameDatabase db = NameDatabase.getDatabase(MainActivity.this);
mNameDao = db.nameDao();
String name = mNameDao.getNameByName("Body").name;
return name;
}
#Override
public void onPostExecute(String name) {
nameView.setText(name);
}
}
}
Entity
#Entity(tableName = "name")
public class Name {
#NonNull
#PrimaryKey(autoGenerate = true)
public Integer id;
#NonNull
#ColumnInfo(name = "name")
public String name ;
public Name(Integer id, String name) {
this.id = id;
this.name = name;
}
public Integer getId() {
return this.id;
}
public void setId(Integer id ) {
this.id = id;
}
public String getName() {
return this.name;
}
public void setName(String name) {
this.name = name;
}
}
Dao
#Dao
public interface NameDao {
#Insert
void insertAll(List<Name> names);
#Query("SELECT * from name")
List<Name> getAllNames();
#Query("DELETE FROM name")
void deleteAll();
#Query("SELECT * FROM name WHERE name = :name LIMIT 1")
Name getNameByName(String name);
#Query("SELECT * FROM name WHERE id = :id LIMIT 1")
Name getNameById(int id);
}
Database
#Database(entities = {Name.class}, version = 1)
public abstract class NameDatabase extends RoomDatabase {
public abstract NameDao nameDao();
private static NameDatabase INSTANCE;
public boolean setDatabaseCreated = false;
public static NameDatabase getDatabase(final Context context) {
if (INSTANCE == null) {
synchronized (NameDatabase.class) {
if (INSTANCE == null) {
INSTANCE = buildDatabase(context);
INSTANCE.updateDatabaseCreated(context);
}
}
}
return INSTANCE;
}
private static NameDatabase buildDatabase(final Context appContext) {
return Room.databaseBuilder(appContext, NameDatabase.class,
"name_database").addCallback(new Callback() {
#Override
public void onCreate(#NonNull SupportSQLiteDatabase db) {
super.onCreate(db);
Executors.newSingleThreadScheduledExecutor().execute(() -> {
// Add Delay to stimulate a long running opeartion
addDelay();
// Generate the data for pre-population
NameDatabase database = NameDatabase.getDatabase(appContext);
List<Name> names = createNames();
insertData(database, names);
// notify that the database was created and it's ready to be used
database.setDatabaseCreated();
});
}
}
).build();
}
private void updateDatabaseCreated(final Context context) {
if (context.getDatabasePath("name_database").exists()) {
setDatabaseCreated();
}
}
private boolean setDatabaseCreated() {
return this.setDatabaseCreated = true;
}
protected static List<Name> createNames() {
List<Name> cList = new ArrayList<>();
cList.add(new Name(1, "Body"));
cList.add(new Name(2, "Mind"));
cList.add(new Name(3, "Love"));
cList.add(new Name(4, "Community"));
cList.add(new Name(5, "Career"));
cList.add(new Name(6, "Money"));
cList.add(new Name(7, "Fun"));
cList.add(new Name(8, "Home"));
return cList;
}
private static void insertData(final NameDatabase database, final List<Name> names) {
database.runInTransaction(() -> {
database.nameDao().insertAll(names);
});
}
private static void addDelay() {
try {
Thread.sleep(4000);
} catch (InterruptedException ignored) {
}
}
}
Gives me the exception on String name = mNameDao.getNameByName("Body").name; this line, when I install the app for first time, however if I close the app and start again it does not give the exception anymore. I think because the database has not been populated yet.
I read a post Pre-Populate Database that says on the first call to db.getInstance(context); the database will be populated on in my case NameDatabase.getDatabase(MainActivity.this).
So what shall I do to know if the database has finished populating after the call?

I think because the database has not been populated yet.
Correct. You have forked one background thread (AsyncTask). That thread is forking a second background thread, via your getDatabase() call, as your database callback is forking its own thread via Executors.newSingleThreadScheduledExecutor().execute(). Your AsyncTask is not going to wait for that second thread.
Remove Executors.newSingleThreadScheduledExecutor().execute() from your callback. Initialize your database on the current thread (which, in this case, will be the AsyncTask thread). Make sure that you only access the database from a background thread, such as by having your database access be managed by a repository.

I hope I'm not late! Just a bit of a background before I answer.
I was also searching for a solution regarding this problem. I wanted a loading screen at startup of my application then it will go away when the database has finished pre-populating.
And I have come up with this (brilliant) solution: Have a thread that checks the sizes of the tables to wait. And if all entities are not size 0 then notify the main UI thread. (The 0 could also be the size of your entities when they finished inserting. And it's also better that way.)
One thing I want to note is that you don't have to make the variables in your entity class public. You already have getters/setters for them. I also removed your setDatabaseCreated boolean variable. (Believe me, I also tried to have a volatile variable for checking but it didn't work.)
Here's the solution: Create a Notifier class that notifies the main UI thread when the database has finished pre-populating. One problem that arises from this is memory leaks. Your database might take a long time to pre-populate and the user might do some configuration (like rotating the device for example) that will create multiple instances of the same activity. However, we can solve it with WeakReference.
And here's the code...
Notifier class
public abstract class DBPrePopulateNotifier {
private Activity activity;
public DBPrePopulateNotifier(Activity activity) {
this.activity = activity;
}
public void execute() {
new WaitDBToPrePopulateAsyncTask(this, activity).execute();
}
// This method will be called to set your UI controllers
// No memory leaks will be caused by this because we will use
// a weak reference of the activity
public abstract void onFinished(String name);
private static class WaitDBToPrePopulateAsyncTask extends AsyncTask<Void, Void, String> {
private static final int SLEEP_BY_MILLISECONDS = 500;
private WeakReference<Activity> weakReference;
private DBPrePopulateNotifier notifier;
private WaitDBToPrePopulateAsyncTask(DBPrePopulateNotifier notifier, Activity activity) {
// We use a weak reference of the activity to prevent memory leaks
weakReference = new WeakReference<>(activity);
this.notifier = notifier;
}
#Override
protected String doInBackground(Void... voids) {
int count;
Activity activity;
while (true) {
try {
// This is to prevent giving the pc too much unnecessary load
Thread.sleep(SLEEP_BY_MILLISECONDS);
}
catch (InterruptedException e) {
e.printStackTrace();
break;
}
// We check if the activity still exists, if not then stop looping
activity = weakReference.get();
if (activity == null || activity.isFinishing()) {
return null;
}
count = NameDatabase.getDatabase(activity).nameDao().getAllNames().size();
if (count == 0) {
continue;
}
// Add more if statements here if you have more tables.
// E.g.
// count = NameDatabase.getDatabase(activity).anotherDao().getAll().size();
// if (count == 0) continue;
break;
}
activity = weakReference.get();
// Just to make sure that the activity is still there
if (activity == null || activity.isFinishing()) {
return null;
}
// This is the piece of code you wanted to execute
NameDatabase db = NameDatabase.getDatabase(activity);
NameDao nameDao = db.nameDao();
return nameDao.getNameByName("Body").getName();
}
#Override
protected void onPostExecute(String name) {
super.onPostExecute(name);
// Check whether activity is still alive if not then return
Activity activity = weakReference.get();
if (activity == null|| activity.isFinishing()) {
return;
}
// No need worry about memory leaks because
// the code below won't be executed anyway
// if a configuration has been made to the
// activity because of the return statement
// above
notifier.onFinished(name);
}
}
}
MainActivity
public class MainActivity extends AppCompatActivity {
private TextView nameView;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
nameView = findViewById(R.id.name);
new DBPrePopulateNotifier(this) {
#Override
public void onFinished(String name) {
// You set your UI controllers here
// Don't worry and this won't cause any memory leaks
nameView.setText(name);
}
}.execute();
}
}
As you can see, our Notifier class has a thread in it that checks if the entities are not empty.
I didn't change anything in your other classes: Name, NameDao and NameDatabase except that I removed the boolean variable in NameDatabase and made private the variables in Name.
I hope that this answers your question perfectly. As you said, no LiveData, Repository, etc.
And I really hope I ain't late to answer!
Now I want to write down what I tried before I came up to the final solution.
Keep in mind that what I am trying to do here is for my app to show a progress bar (that infinite spinning circle) and put it away after the database has finished pre-populating.
Tried:
1. Thread inside thread
Practically, there's a thread that checks if the size of an entity is still 0. The query is done by another thread.
Outcome: Failed. Due to my lack of knowledge, you cannot start a thread within another thread. Threads can only be started from the main thread.
Tables' sizes loop checker
A thread that queries the tables to be checked if they have been initialized through an infinite loop. Only breaks if all sizes of the tables to be checked are greater than 0.
Outcome: Solution. This is by far the most elegant and working solution to this problem. It doesn't cause memory leaks because as soon as a configuration has been made, the thread that loops continually will break.
Static variable
A static volatile variable in the database class in which will turn to true when the thread has finished inserting the values.
Outcome: Failed. For unknown reason that I still search for, it won't run the thread for initializing the database. I have tried 3 versions of the code implementation but to no avail. Hence, a failure.
Initialize database then notify
A listener that is defined in the UI thread then passed by argument to the repository. All database initialization is done also in the repository. After populating the database, it will then notify/call the listener.
Outcome: Failed. Can cause memory leaks.
As always, happy coding!

Logging in onCreateCallback ofc!

Check if data exists in database and insert new if not found using multiple threads?

My requirement is something like this:
I have written a code which checks for data in database and if data is not found, it consumes third party rest API in order to get data. Some logic is performed and finally obtained data is stored in database.
public void exampleMethod(){
MyObject myObj = getObjFromDatabase(); //getting from db
if(myObj==null){ //not found in db
myObj = getObjFromThirdParty(); //consuming rest api
//some logic here in case myobj is not in db....
}else{
//some other logic here in case myobj is in db...
}
saveObjInDatabase(myObj); //saving it in database
}
I need it to be saved in database just once. Getting response from third party API takes some time and this method gets executed from multiple threads.
Now, the problem is I need it to be saved in database just once, but before one thread could save data in database another thread gets null from database, logic that should get executed only when "data is not in db" gets executed and saves same data more than once. (I am using mongoDB to store data)
How can I solve this problem? Thank you.

What you are asking about is caching. Here is an example that should work decently. It synchronizes on getObj in case the object needs to be loaded. If the object is fresh then getObj returns very quickly and hardly blocks the other threads, but if it needs to load the object then the other threads will wait until the object is loaded.
public class Test {
// used when you want to refresh the cached object
private static boolean refreshNext = false;
// a static reference to your object that all threads can use
private static MyObj cachedObj = null;
private static void message(String msg) {
System.out.println(
System.currentTimeMillis() + " : " + Thread.currentThread().getName() + " : " + msg);
}
private static void sleep(long milli) {
try { Thread.sleep(milli); } catch (Exception e) { }
}
// represents the object stored in the db
private static MyObj myObjInDb = null;
private static void saveObjInDb(MyObj obj) {
// TODO do real saving to db and handle errors
message("storing in db...");
myObjInDb = obj;
}
private static MyObj loadObjFromDb() {
// TODO do real loading from db and handle errors
message("getting from db...");
sleep(1000);
return myObjInDb;
}
private static MyObj loadObjFromVendor() {
// TODO do real fetching from vendor and handle errors
message("getting from vendor...");
sleep(2000);
return new MyObj();
}
private static MyObj loadObj() {
message("loading object...");
MyObj obj = loadObjFromDb();
if (obj == null) {
message("db didn't have it.");
obj = loadObjFromVendor();
saveObjInDb(obj);
}
return obj;
}
/** Returns the object, first loading and caching if needed. */
public static synchronized MyObj getObj() {
// only one thread can get the object at a time, in case it needs to be loaded.
if (cachedObj == null || refreshNext) {
// load and cache the object
cachedObj = loadObj();
refreshNext = false;
}
return cachedObj;
}
public static void exampleMethod() {
MyObj obj = getObj();
message(obj.toString());
// ... do stuff with obj
}
private static class MyObj {
public final String data = "I have data!";
#Override public String toString() { return data; }
}
public static void main(String[] args) {
for (int i = 0; i < 20; i++)
new Thread(Test::exampleMethod).start();
}
}

UI Thread blocking during API calls - is AsyncTask usage correct?

I'm rather new to creating apps for Android, but what I am trying to essentially do is sync an API (using Volley) completely to my App in the first instance, and then periodically thereafter (I gather Volley creates new threads itself).
Whilst the app is syncing with the API the main UI Thread is getting blocked (I'm seeing a lot of "Skipped frames" in the logs). I have tried running the code to get the data and create in the database within an AsyncTask thread, and while I can see that the AsyncTask threads were created the Main UI thread still is sluggish.
I was thinking that since the initial call is within an AsyncTask, any sub ones will be within the same AsyncTask thread?
I'm also not 100% sure if the anothe rpage exists, whether I can execute another AsyncTask query from within doInBackground(), or if I should just call doInBackground inself?
I hope the below will be enough to get some help/pointers, but if any more is required, I will gladly update this question.
MainActivity
public class MainActivity extends AppCompatActivity {
public SQLiteHandler sqlh;
public DatabaseManager dbm;
public API API;
private Handler handler=new Handler();
#Override
protected void onCreate(Bundle savedInstanceState) {
...
API = new API(getApplicationContext());
sqlh = new SQLiteHandler(getApplicationContext());
dbm = new DatabaseManager();
dbm.initializeInstance(sqlh);
...
// start 5 seconds after loading, then run every minute
handler = new Handler();
handler.postDelayed(new Runnable() {
#Override
public void run(){
getChanges("thisApiUrl");
API.processSyncOut();
handler.postDelayed(this, 60000);
}
},5000);
...
}
private class getChangesTask extends AsyncTask<String, Void, Void>(){
#Override
protected Void doInBackground(String... params) {
String url = params[0];
final SQLiteHandler sqlh = new SQLiteHandler(getApplicationContext());
final DatabaseManager dm = new DatabaseManager();
dm.initializeInstance(sqlh);
JSONObject response = API.doAction(url, new API.VolleyActionCallback() {
#Override
public void onActionSuccess(Object response) {
....
for (Map.Entry<String, JsonElement> entry : entrySet) {
....
HashMap<String, String> addRecord = new HashMap();
addRecord.put("id", dataObj.get("id").getAsString());
addRecord.put("data", dataObj.get("data").getAsString());
addRecord.put("updated_at", dataObj.get("updated_at").getAsString();
dm.createRecord(addRecord);
....
}
if (!json.getString("next").equals("")) {
// if another page exists, go to it
getAPIChangesTask task = new getAPIChangesTask();
task.execute(json.getString("next"));
}
....
}
});
}
}
private Void getChanges(String url){
getChangesTask task = new getChangesTask();
task.execute(url);
}
}
DatabaseManager
Just a side question, but can a writeable database also do reading at the same time? Thinking that I could probably combine both methods below?
public class DatabaseManager {
private int rOpenCounter;
private int wOpenCounter;
private static DatabaseManager instance;
private static SQLiteOpenHelper mDatabaseHelper;
private SQLiteDatabase rDatabase;
private SQLiteDatabase wDatabase;
public static synchronized void initializeInstance(SQLiteOpenHelper helper) {
if (instance == null) {
instance = new DatabaseManager();
mDatabaseHelper = helper;
}
}
public static synchronized DatabaseManager getInstance() {
if (instance == null) {
throw new IllegalStateException(DatabaseManager.class.getSimpleName() +
" is not initialized, call initializeInstance(..) method first.");
}
return instance;
}
public synchronized SQLiteDatabase getReadableDatabase() {
rOpenCounter++;
if(rOpenCounter == 1) {
// Opening new readable database
rDatabase = mDatabaseHelper.getReadableDatabase();
}
return rDatabase;
}
public synchronized SQLiteDatabase getWritableDatabase() {
wOpenCounter++;
if(wOpenCounter == 1) {
// Opening new writable database
wDatabase = mDatabaseHelper.getWritableDatabase();
}
return wDatabase;
}
public synchronized void closeReadableDatabase() {
rOpenCounter--;
if(rOpenCounter == 0) {
// Closing readable database
rDatabase.close();
}
}
public synchronized void closeWritableDatabase() {
wOpenCounter--;
if(wOpenCounter == 0) {
// Closing writable database
wDatabase.close();
}
}
public Long createRecord(HashMap<String, String> dataTable){
String id = dataTable.containsKey("id") ? dataTable.get("id") : "";
String data = dataTable.containsKey("data") ? dataTable.get("data") : "";
String updatedAt = dataTable.containsKey("updated_at") ? dataTable.get("updated_at") : "";
// we don't want ID to be set, as probably will be therefore updating something
if (!id.isEmpty()) {
SQLiteDatabase db = DatabaseManager.getInstance().getWritableDatabase();
ContentValues values = new ContentValues();
values.put("data", data);
values.put("updated_at", updatedAt);
// insert row
Long insertId = db.insert("this_table", null, values);
DatabaseManager.getInstance().closeWritableDatabase();
return insertId;
}
return null;
}
}
}

Developing Persisent Store in Blackberry Java

I currently have code to share a variable between two entry points in my application. The variable is the iconCount variable used to indicate how many notices the user has which is displayed on the home screen beside the icon. The way I've managed to do this is with a singleton and it (seems) to work fine at the moment. The issue is now that I do not want those notices to reset to zero when I completely turn off and turn on the phone. Should there be 7 notifications, I want there to be 7 notifications even after a device restart. For this I apparently need a persistent store integration which I've researched for a while.
So far my code for the bare singleton is:
public class MyAppIndicator{
public ApplicationIndicator _indicator;
public static MyAppIndicator _instance;
MyAppIndicator () {
setupIndicator();
}
public static MyAppIndicator getInstance() {
if (_instance == null) {
_instance = new MyAppIndicator ();
}
return(_instance);
}
public void setupIndicator() {
//Setup notification
if (_indicator == null) {
ApplicationIndicatorRegistry reg = ApplicationIndicatorRegistry.getInstance();
_indicator = reg.getApplicationIndicator();
if(_indicator == null) {
ApplicationIcon icon = new ApplicationIcon(EncodedImage.getEncodedImageResource ("notificationsdemo_jde.png"));
_indicator = reg.register(icon, false, true);
_indicator.setValue(0);
_indicator.setVisible(false);
}
}
}
public void setVisible1(boolean visible, int count) {
if (_indicator != null) {
if (visible) {
_indicator.setVisible(true);
_indicator.setValue(count); //UserInterface.incrementCount()
} else {
_indicator.setVisible(false);
}
}
}
}
I have been using the blackberry tutorial to figure out how to implement the persistable storage: http://supportforums.blackberry.com/t5/Java-Development/Storing-persistent-data/ta-p/442747
Now before I go any further I must stress I'm very new to java development so my coding might be completely wrong, but here is what I've tried to do:
public void setVisible1(boolean visible, int count) {
if (_indicator != null) {
if (visible) {
_indicator.setVisible(true);
_indicator.setValue(count); //UserInterface.incrementCount()
StoreInfo info = new StoreInfo();
info.incElement();
synchronized (persistentCount) {
//persistentCount.setContents(_data);
persistentCount.commit();
}
} else {
_indicator.setVisible(false);
}
}
}
static {
persistentCount = PersistentStore.getPersistentObject(0xdec6a67096f833cL);
synchronized (persistentCount) {
if (persistentCount.getContents() == null) {
persistentCount.setContents(new Vector()); //don't know what to do with this?
persistentCount.commit();
}
}
}
private static final class StoreInfo implements Persistable{
private int iconCount;
public StoreInfo(){}
public int getElement(){
return (int)iconCount;
}
public void incElement(){
iconCount++; //persistently increment icon variable
}
public void resetElement(){
iconCount=0; //when user checks application
}
}
The code above doesn't work which I'd expect somehow because I'm having trouble implementing the persistent portion. If anyone has any idea or input on how to accomplish this any assistance would be helpful. And of course thanks in advance.

In the example they have a variable called _data that holds the StoreInfo class, so first of all you should be keeping the StoreInfo in some variable. To do this have something like the following in your static initializer:
persistentCount = PersistentStore.getPersistentObject(0xdec6a67096f833cL);
synchronized (persistentCount) {
if (persistentCount.getContents() == null) {
persistentCount.setContents(new StoreInfo());
persistentCount.commit();
}
}
_data = (StoreInfo)persistentCount.getContents();
Now when you want to update it and save to the PersistentStore you can have something like:
_data.incElement();
synchronized(persistentCount) {
persistentCount.setContents(_data);
persistentCount.commit();
}
Assuming you're going to only ever have one instance of StoreInfo it could be better to put the commit code into the modifier methods so you don't forget to save the new values to the PersistentStore.