We Keep Coding

How to clear webview cache when server side content updated?

I have a HTML cache enabled WebView app that sends Javascript fetch request to server and gets images list from server, images on server updating every random hour like: today 11:00, 16:00, tomorrow 14:00, 19:00. Images count is always 20 and their names are always like: 0.png, 1.png, 2.png,... 20.png.
Problem is once when app caches images with names like that, after replacing/updating them with other same name images Web app is not showing them, shows just old first time cached images. I know that i can pass get parameter in javascript code while sending get request like: "site.com?time="+Date.getHours();
But i need cache update only on image update on server
Tried code below to clear cache every day but that is not enought:
//helper method for clearCache() , recursive
//returns number of deleted files
static int clearCacheFolder(final File dir, final int numDays) {
int deletedFiles = 0;
if (dir!= null && dir.isDirectory()) {
try {
for (File child:dir.listFiles()) {
//first delete subdirectories recursively
if (child.isDirectory()) {
deletedFiles += clearCacheFolder(child, numDays);
}
//then delete the files and subdirectories in this dir
//only empty directories can be deleted, so subdirs have been done first
if (child.lastModified() < new Date().getTime() - numDays * DateUtils.DAY_IN_MILLIS) {
if (child.delete()) {
deletedFiles++;
}
}
}
}
catch(Exception e) {
Log.e(TAG, String.format("Failed to clean the cache, error %s", e.getMessage()));
}
}
return deletedFiles;
}
/*
* Delete the files older than numDays days from the application cache
* 0 means all files.
*/
public static void clearCache(final Context context, final int numDays) {
Log.i(TAG, String.format("Starting cache prune, deleting files older than %d days", numDays));
int numDeletedFiles = clearCacheFolder(context.getCacheDir(), numDays);
Log.i(TAG, String.format("Cache pruning completed, %d files deleted", numDeletedFiles));
}

Java executable .jar file not working properly after compiling in NETBeans IDE

I am exporting a .jar file from a GUI built in Netbeans IDE. It works fine within Netbeans but once it is exported, some logic in the code does not execute properly.
I have tried debugging by running test cases displaying which items are passing through the if clauses. It seems to work well in IDE but not when it is run through .jar file.
for(Chamber chamber: chs)
{
if(!scheduledTools.containsKey(chamber))//unscheduled chambers
{
JOptionPane.showMessageDialog(this, chamber.getName()+": is unscheduled");
model.addElement(chamber.getName());
}
else//scheduled
{
if((scheduledTools.get(chamber).size()/range) > .25)
{
System.out.println("Range: " +range+" Scheduled: " +scheduledTools.get(chamber).size());
System.out.println("Cannot use this chambers:" +chamber.getName());
JOptionPane.showMessageDialog(this, chamber.getName()+": Cannot use this chamber");
}
else{
System.out.println("Scheduled but can use"+chamber.getName());
altModel.addElement(chamber.getName());
}
}
}
alternateChambers.setModel(altModel);alternateChambers.setSelectedIndex(0);
suggestedChambers.setModel(model);suggestedChambers.setSelectedIndex(0);
if(altModel.size()>1){JOptionPane.showMessageDialog(this, "Scheduled but can use these chambers:"+altModel);}
After making the necessary queries in seekDates method below:
private void seekDates(ResultSet rs) throws SQLException
{
ArrayList<String> entries;
//get how many dates we have so we can update progressbar
int results = 0;
setProgress(results);
if(rs.last()){results=rs.getRow();rs.beforeFirst();}
// processing returned data and printing into console
while(rs.next()) {
String scheduledChamber = rs.getString(2);
for(Chamber chamber : chambers){
//if the scheduled tool is a thermal chamber
//chambers get listed so far
if(!scheduledChamber.trim().toLowerCase().contains(chamber.getName().toLowerCase()))
{}//print chambers im not saving
else{
if(scheduledTools.containsKey(chamber))
{
//if key already used, just grab the list and add date
entries = scheduledTools.get(chamber);
entries.add(rs.getString(1).split(" ")[0]);
}
else
{
//if chamber hasnt been saved, add chamber and date
entries = new ArrayList<String>();
entries.add(rs.getString(1).split(" ")[0]);
scheduledTools.put(chamber, entries);
}
//System.out.println("Just saved this chamber:" +scheduledChamber+" with this date:" +scheduledTools.get(chamber));
}
;
}
publish(rs.getRow());
Thread.yield();
setProgress(100*(rs.getRow()/results));
}
}
The GUI should display the chambers found in the queries in one text field while displaying the rest of the chambers in another text field.
When I run this in the IDE. I get the correct outcome but once I compile using the package-for-store option in the build.xml file, the outcome then just lists all of the chambers in one text field.
In the .jar file, it seems that all of the chambers only satisfy the first if clause and none of the others.

How to scanning and deleting millions of rows in HBase

What Happened
All the data from last month was corrupted due to a bug in the system. So we have to delete and re-input these records manually. Basically, I want to delete all the rows inserted during a certain period of time. However, I found it difficult to scan and delete millions of rows in HBase.
Possible Solutions
I found two way to bulk delete:
The first one is to set a TTL, so that all the outdated record would be deleted automatically by the system. But I want to keep the records inserted before last month, so this solution does not work for me.
The second option is to write a client using the Java API:
public static void deleteTimeRange(String tableName, Long minTime, Long maxTime) {
Table table = null;
Connection connection = null;
try {
Scan scan = new Scan();
scan.setTimeRange(minTime, maxTime);
connection = HBaseOperator.getHbaseConnection();
table = connection.getTable(TableName.valueOf(tableName));
ResultScanner rs = table.getScanner(scan);
List<Delete> list = getDeleteList(rs);
if (list.size() > 0) {
table.delete(list);
}
} catch (Exception e) {
e.printStackTrace();
} finally {
if (null != table) {
try {
table.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if (connection != null) {
try {
connection.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
private static List<Delete> getDeleteList(ResultScanner rs) {
List<Delete> list = new ArrayList<>();
try {
for (Result r : rs) {
Delete d = new Delete(r.getRow());
list.add(d);
}
} finally {
rs.close();
}
return list;
}
But in this approach, all the records are stored in ResultScanner rs, so the heap size would be huge. And if the program crushes, it has to start from the beginning.
So, is there a better way to achieve the goal?

Don't know how many 'millions' you are dealing with in your table, but the simples thing is to not try to put them all into a List at once but to do it in more manageable steps by using the .next(n) function. Something like this:
for (Result row : rs.next(numRows))
{
Delete del = new Delete(row.getRow());
...
}
This way, you can control how many rows get returned from the server via a single RPC through the numRows parameter. Make sure it's large enough so as not to make too many round-trips to the server, but at the same time not too large to kill your heap. You can also use the BufferedMutator to operate on multiple Deletes at once.
Hope this helps.

I would suggest two improvements:
Use BufferedMutator to batch your deletes,  it does exactly what you need – keeps internal buffer of mutations and flushes it to HBase when buffer fills up, so you do not have to worry about keeping your own list, sizing and flushing it.
Improve your scan:
Use KeyOnlyFilter – since you do not need the values, no need to retrieve them
use scan.setCacheBlocks(false) - since you do a full-table scan, caching all blocks on the region server does not make much sense
tune scan.setCaching(N) and scan.setBatch(N) – the N will depend on the size of your keys, you should keep a balance between caching more and memory it will require; but since you only transfer keys, the N could be quite large, I suppose.
Here's an updated version of your code:
public static void deleteTimeRange(String tableName, Long minTime, Long maxTime) {
try (Connection connection = HBaseOperator.getHbaseConnection();
final Table table = connection.getTable(TableName.valueOf(tableName));
final BufferedMutator mutator = connection.getBufferedMutator(TableName.valueOf(tableName))) {
Scan scan = new Scan();
scan.setTimeRange(minTime, maxTime);
scan.setFilter(new KeyOnlyFilter());
scan.setCaching(1000);
scan.setBatch(1000);
scan.setCacheBlocks(false);
try (ResultScanner rs = table.getScanner(scan)) {
for (Result result : rs) {
mutator.mutate(new Delete(result.getRow()));
}
}
} catch (IOException e) {
e.printStackTrace();
}
}
Note the use of "try with resource" – if you omit that, make sure to .close() mutator, rs, table, and connection.

Spring inbound channel adapter - how to auto delete folders and files older than 10 days

Integration.xml - this will take all files in the directory
<int-file:inbound-channel-adapter id="delFiles" channel="delFiles"
directory="C:/abc/abc" use-watch-service="true" prevent-duplicates="false" auto-startup="true"
watch-events="CREATE,MODIFY">
<int:poller fixed-delay="1000"/>
<int-file:nio-locker/>
</int-file:inbound-channel-adapter>
I need to delete all files older than 10 days in that folder and sub folder. Can some one pls help?
Listener
#Transformer(inputChannel="delFiles")
public JobLaunchRequest deleteJob(Message<File> message) throws IOException {
Long timeStamp = message.getHeaders().getTimestamp();
return JobHandler.deleteJob(message.getPayload(), jobRepository, fileProcessor, timeStamp);
}
Handler
public static JobLaunchRequest deleteJob(File file, JobRepository jobRepository, Job fileProcessor, Long timeStamp) throws IOException {
//Is there a way in spring integration whether I can check this easily?
//How to check for folder and subfolders?
// This will check for files once it's dropped.
// How to run this job daily to check the file's age and delete?
}

This is not a <int-file:inbound-channel-adapter> responsibility. This one is really about polling files from the directory according filtering setting you provide.
If you are not interested in old files, you can implement a custom FileListFilter to skip files which are really so old.
If you still would like to delete those old files as some application functionality, you need to take a look into some other solution, something like #Scheduled method it iterate through files in that dir and remove them, e.g. once a day let's say at midnight.
You also can just remove processed files in the and of your logic. Since you use prevent-duplicates="false", you are going to poll the same file again and again.
To perform directory clean up you don't need Spring Integration:
public void recursiveDelete(File file) {
if (file != null && file.exists()) {
File[] files = file.listFiles();
if (files != null) {
for (File fyle : files) {
if (fyle.isDirectory()) {
recursiveDelete(fyle);
}
else {
if (fyle.lastModified() > 10 * 24 * 60 * 60 * 1000) {
fyle.delete();
}
}
}
}
}
}
(You might to improve this function a bit: haven't tested...)

What is an Efficient way of searching for files in storage in Android?

I am creating a music player app in android. In the MainActivity I'm listing all the (.mp3,.wav,.flac) files in a simple listview. I'm using this getSongs method to search for such files:
public void getSongs(File root) {
File listFile[] = root.listFiles();
if (listFile != null && listFile.length > 0) {
for (int i = 0; i < listFile.length; i++) {
if (listFile[i].isDirectory() && !listFile[i].isHidden()) {
getSongs(listFile[i]);
} else {
if (listFile[i].getName().endsWith(".mp3") ||
listFile[i].getName().endsWith(".MP3") ||
listFile[i].getName().endsWith(".flac")||
listFile[i].getName().endsWith(".wav")) {
songList.add(listFile[i]);
}
}
}
}
}
Then from the main activity I'm calling this:
File externalSD = new File("/storage/"); //gets the external but not internal!!
getSongs(externalSD);
File internalSD = new File(Environment.getExternalStorageDirectory().getAbsolutePath()); //gets the internal but not external!!
getSongs(internalSD);
Works well for me (I have only 100 songs) but in case you have a large number of such files it is really slow (my friend has around 2.5k songs..).
I would like to know how I can achieve this more efficienty.

You could try:
Using an AsyncTask to retrieve the songs in another thread.
Getting the song list from Android's media database instead - you will have to work with SQL and MediaStore.Audio. Remember to use an AsyncTask or a Loader.
The second option would be faster since you don't have to traverse the whole system.