I am developing an azure function using Java. I need to iterate all the files in the following folder
aDirectory/aSubdirectoryWithManyFiles/
There are many files in that path,:
aDirectory/aSubdirectoryWithManyFiles/file1
aDirectory/aSubdirectoryWithManyFiles/file2
aDirectory/aSubdirectoryWithManyFiles/file3
aDirectory/aSubdirectoryWithManyFiles/file4
aDirectory/aSubdirectoryWithManyFiles/file5
so I wrote the following code in order to get them all:
// myCloudBlobContainer is a CloudBlobContainer
// I expected to get all files thanks to the next row
Iterable<ListBlobItem> blobs = myCloudBlobContainer.listBlobs();
// The only blob found in the container is the directory itself
for (ListBlobItem blob : blobs) {
//log the current blob URI
if (blob instanceof CloudBlob) { // this never happens
CloudBlob cloudBlob = (CloudBlob) blob;
//make nice things with every found file
}
}
The only blob iterated in the for is the directory, noone of the expected files. so in logs i get only the following URI:
https://blablablabla.blob.core.windows.net/aDirectory/aSubdirectoryWithManyFiles/
What should I do in order to access every file?
And in case I would have more than one subdirectory, as in the following example?
aDirectory/aSubdirectoryWithManyFiles/files(1-5)
aDirectory/anotherSubdirectoryWithManyFiles/files(6-10)
Thanks in advance
Edit
In order to make methods testable, the project uses wrappers and interfaces instead of directly using directly a CloudBlobContainer; basically, the CloudBlobContainer is given by CloudBlobClient.getContainerReference("containername")
After the answer to this question, I changed teh code to the following
so I used listBlobs with parameters myCloudBlobContainer.listBlobs("aDirectory", true) and I wrote the following code in order to get them all:
// myCloudBlobClient is a CloudBlobClient
CloudBlobContainer myCloudBlobContainer = myCloudBlobClient.getContainerReference("containername")
// I expected to get all files thanks to the next row
Iterable<ListBlobItem> blobs = myCloudBlobContainer.listBlobs("aDirectory", true); // HERE THE CHANGE
// No blob found this time
for (ListBlobItem blob : blobs) { // NEVER IN THE FOR
//log the current blob URI
if (blob instanceof CloudBlob) {
CloudBlob cloudBlob = (CloudBlob) blob;
//make nice things with every found file
}
}
But this time, it doesn't go at all in the for...
I must say that the previous answer made me to waste time; the problem was in the fact that only one for is not enough to find files in folders. The first for finds the folders and subfolders, plus (maybe, i didn't check) files that are in the "root" (let's call it like that).
Having the folders, for each of them we have to cast as CloudBlobDirectory in order to see and iterate all contained files with another for.
Here the solution that works for me:
// myCloudBlobClient is a CloudBlobClient
CloudBlobContainer myCloudBlobContainer = myCloudBlobClient.getContainerReference("containername")
// I expected to get all files thanks to the next row
Iterable<ListBlobItem> blobs = myCloudBlobContainer.listBlobs();
// only directories here, another for needed to scan files
for (ListBlobItem blob : blobs) {
if (blob instanceof CloudBlobDirectory) {
CloudBlobDirectory directory = (CloudBlobDirectory)blob;
//next is in try/catch
Iterable<ListBlobItem> fileBlobs = directory.listBlobs();
for (ListBlobItem fileBlob : fileBlobs) {
if (fileBlob instanceof CloudBlob) {
CloudBlob cloudBlob = (CloudBlob) fileBlob;
//make nice things with every found file
}
}
} // else: may be we found a cloudBlob in root?
}
This helped me to find the right way:
https://social.msdn.microsoft.com/Forums/en-US/1cfdc91f-e588-4839-a878-9650339a0a06/list-all-blobs-in-c?forum=windowsazuredata
Try using the following override of listBlobs method:
listBlobs(String prefix, boolean useFlatBlobListing)
So your code would be:
Iterable<ListBlobItem> blobs = myCloudBlobContainer.listBlobs("aDirectory", true);
This will list all blobs inside "aDirectory" virtual folder in your blob container.
Related
I need to get single the GridFS file using Java driver 3.7+.
I have two collections with file in a database: photo.files and photo.chunks.
The photo.chunks collection contains the binary file like:
The photo.files collection contains the metadata of the document.
To find document using simple database I wrote:
Document doc = collection_messages.find(eq("flag", true)).first();
String messageText = (String) Objects.requireNonNull(doc).get("message");
I tried to find file and wrote in same way as with an example above, according to my collections on screens:
MongoDatabase database_photos = mongoClient.getDatabase("database_photos");
GridFSBucket photos_fs = GridFSBuckets.create(database_photos,
"photos");
...
...
GridFSFindIterable gridFSFile = photos_fs.find(eq("_id", new ObjectId()));
String file = Objects.requireNonNull(gridFSFile.first()).getMD5();
And like:
GridFSFindIterable gridFSFile = photos_fs.find(eq("_id", new ObjectId()));
String file = Objects.requireNonNull(gridFSFile.first()).getFilename();
But I get an error:
java.lang.NullPointerException
at java.util.Objects.requireNonNull(Objects.java:203)
at project.Bot.onUpdateReceived(Bot.java:832)
at java.util.ArrayList.forEach(ArrayList.java:1249)
Also I checked docs of 3.7 driver, but this example shows how to find several files, but I need single:
gridFSBucket.find().forEach(
new Block<GridFSFile>() {
public void apply(final GridFSFile gridFSFile) {
System.out.println(gridFSFile.getFilename());
}
});
Can someone show me an example how to realize it properly?
I mean getting data, e.g. in chunks collection by Object_id and md5 field also by Object_id in metadata collection.
Thanks in advance.
To find and use specific files:
photos_fs.find(eq("_id", objectId)).forEach(
(Block<GridFSFile>) gridFSFile -> {
// to do something
});
or as alternative, I can find specific field of the file.
It can be done firstly by creating objectId of the first file, then pass it to GridFSFindIterable object to get particular field and value from database and get finally file to convert into String.
MongoDatabase database_photos =
mongoClient.getDatabase("database_photos");
GridFSBucket photos_fs = GridFSBuckets.create(database_photos,
"photos");
...
...
ObjectId objectId = Objects.requireNonNull(photos_fs.find().first()).getObjectId();
GridFSFindIterable gridFSFindIterable = photos_fs.find(eq("_id", objectId));
GridFSFile gridFSFile = Objects.requireNonNull(gridFSFindIterable.first());
String file = Objects.requireNonNull(gridFSFile).getMD5();
But it checks files from photo.files not from photo.chunkscollection.
And I'm not sure that this way is code-safe, because of debug info, but it works despite the warning:
Inconvertible types; cannot cast 'com.mongodb.client.gridfs.model.GridFSFile' to 'com.mongodb.client.gridfs.GridFSFindIterableImpl'
As i am using v3 of google api,So instead of using parent and chidren list i have to use fileList, So now i want to search list of file inside a specific folder.
So someone can suggest me what to do?
Here is the code i am using to search the file :
private String searchFile(String mimeType,String fileName) throws IOException{
Drive driveService = getDriveService();
String fileId = null;
String pageToken = null;
do {
FileList result = driveService.files().list()
.setQ(mimeType)
.setSpaces("drive")
.setFields("nextPageToken, files(id, name)")
.setPageToken(pageToken)
.execute();
for(File f: result.getFiles()) {
System.out.printf("Found file: %s (%s)\n",
f.getName(), f.getId());
if(f.getName().equals(fileName)){
//fileFlag++;
fileId = f.getId();
}
}
pageToken = result.getNextPageToken();
} while (pageToken != null);
return fileId;
}
But in this method it giving me all the files that are generated which i don't want.I want to create a FileList which will give file inside a specific folder.
It is now possible to do it with the term parents in q parameter in drives:list. For example, if you want to find all spreadsheets in a folder with id folder_id you can do so using the following q parameter (I am using python in my example):
q="mimeType='application/vnd.google-apps.spreadsheet' and parents in '{}'".format(folder_id)
Remember that you should find out the id of the folder files inside of which you are looking for. You can do this using the same drives:list.
More information on drives:list method can be seen here, and you can read more about other terms you can put to q parameter here.
To search in a specific directory you have to specify the following:
q : name = '2021' and mimeType = 'application/vnd.google-apps.folder' and '1fJ9TFZOe8G9PUMfC2Ts06sRnEPJQo7zG' in parents
This examples search a folder called "2021" into folder with 1fJ9TFZOe8G9PUMfC2Ts06sRnEPJQo7zG
In my case, I'm writing a code in c++ and the request url would be:
string url = "https://www.googleapis.com/drive/v3/files?q=name+%3d+%272021%27+and+mimeType+%3d+%27application/vnd.google-apps.folder%27+and+trashed+%3d+false+and+%271fJ9TFZOe8G9PUMfC2Ts06sRnEPJQo7zG%27+in+parents";
Searching files by folder name is not yet supported. It's been requested in this google forum but so far, nothing yet. However, try to look for other alternative search filters available in Search for Files.
Be creative. For example make sure the files within a certain folder contains a unique keyword which you can then query using
fullText contains 'my_unique_keyword'
You can use this method to search the files from google drive:
Files.List request = this.driveService.files().list();
noOfRecords = 100;
request.setPageSize(noOfRecords);
request.setPageToken(nextPageToken);
String searchQuery = "(name contains 'Hello')";
if (StringUtils.isNotBlank(searchQuery)) {
request.setQ(searchQuery);
}
request.execute();
I'm trying to find my folder or view in my database. Which named Team Documents this folder has some filter option like By Date, By Category. But this returns me a null even the folder already exists.
String dbServer = "d23dbm95/23/A/IBM", dbFileName = "dbom\\farizan\\stsklb1.nsf";
public void runNotes()
{
Session session = null;
Database db = null;
View view = null;
Document doc = null;
try
{
NotesThread.sinitThread();
session = NotesFactory.createSession();
System.out.println("User = " + session.getUserName());
db = session.getDatabase(dbServer, dbFileName);
if(db.isOpen())
{
System.out.println("Title "+db.getTitle());
view = db.getView("Team Documents \\ By Date");
if(view == null)
{
System.out.println("still null");
}
}
}
catch(NotesException e)
{
e.printStackTrace();
}
}
I tried also to fill my getView() method like Team Documents. But still returns a null. Any approach to this problem?
While it would have been more helpful if you had included a link to a screenshot of your Domino Designer client's folder list, my best guess is that you have two folders, not one folder with "filter options". Also, my guess is that "Team Documents" is not actually a folder; it's just a prefix on the folder names that makes them appear to be nested in a parent folder.
If that's the case, you would need
iew = db.getView("Team Documents\\By Category");
Or
iew = db.getView("Team Documents\\By Date");
Note: No spaces before & after the backslashes.
If my assumptions above are not correct, then my suggestion would be to assign alias names to the folders in Domino Designer and use the aliases instead of the display names in your code. Frankly, that's always a good practice, because it allows your code to continue working even if you decide to change the display names.
Is it possible to rename a database already created in android?
On my apps update I would like to rename the old database and install a new then compare some values and finally delete the old.
I am doing the creation from an sqlite file in the assets folder. This is why I cannot rename all the tables and insert the new ones.
Clarification:
The old database will contain only one table that I need to compare values from against the new (from the update) database.
Both databases have been copied over from an sqlite file in the assets folder.
Once I have compared a values from the old database to new I will delete the old and use the new in its place with the values I compared.
What i was thinking of doing was rename the old create the new in its place and do everything above.
Just rename the File. Make sure the database is closed first!
Call this in your activity class:
private void renameDatabase()
{
File databaseFile = getDatabasePath("yourdb.whatever");
File oldDatabaseFile = new File(databaseFile.getParentFile(), "yourdb_old.whatever");
databaseFile.renameTo(oldDatabaseFile);
}
Response to clarification. Rename the old db (as above), copy the new one from the assets folder, open both databases and do your compare. Then delete the old file.
Lord Flash is right, you should delete the old db and copy the new oneā¦
Assuming you use a SQLiteOpenHelper, you could use a createDatabaseIfRequired(); method in getReadableDatabase() and getWritableDatabase()
private boolean checkOldDatabase() {
Log.d(Constants.LOGTAG, "OperationDbHelper.checkDatabase");
File f = new File(DB_PATH + OLD_DB_NAME);
return f.exists();
}
public void createDatabaseIfRequired() throws IOException, SQLiteException {
if (!checkOldDatabase()) {
// do db comparison / delete old db / copy new db
}
}
It's not possible to rename a sql table directly.
But you may copy it creating a new and deleting the old one.
Thanks to Kevin Galligan's answer, I was able to create a function in my Kotlin Android app that I can use whenever it might ever be necessary, to rename the database files.
If you're using Java, you'll need to change the syntax a bit but the code should hopefully be somewhat self-explanatory.
val x: String = "Hello"
//in Kotlin would be
String x = "Hello";
//in Java, for example.
Anyway, here's my code, feel free to ask questions if you have any:
private fun checkAndRenameDatabase(oldName: String, newName: String) {
val oldDatabaseFile: File = getDatabasePath(oldName)
val oldDatabaseJournal: File = getDatabasePath("${oldName}-journal")
// Can use this to check files beforehand, using breakpoints
//val files = oldDatabaseFile.parentFile.listFiles()
if(oldDatabaseFile.exists() || oldDatabaseJournal.exists()) {
db.close() // Ensure existing database is closed
val newDatabaseFile: File = getDatabasePath(newName)
val newDatabaseJournal: File = getDatabasePath("${newName}-journal")
if(oldDatabaseFile.exists()) {
if(newDatabaseFile.exists()) {
newDatabaseFile.delete()
}
oldDatabaseFile.renameTo(newDatabaseFile)
}
if(oldDatabaseJournal.exists()) {
if(newDatabaseJournal.exists()) {
newDatabaseJournal.delete()
}
oldDatabaseJournal.renameTo(newDatabaseJournal)
}
// Use with breakpoints to ensure files are now in order
//val newFiles = oldDatabaseFile.parentFile.listFiles()
// Re-open database with new name
db = SQLiteDBHelper(applicationContext, newName)
}
}
How do you set up a Test Blob Image using the yaml structure?
Also, what is the database structure for a BLOB file? (MySQL)
I have experienced the same kind of problem a while ago on a project. However as I could not find a way to solve this with the fixtures (as the database stores the blob object as a string as Pere explained above), I created a workaround to at least solve this problem in a test-case-scenario. I created the following file /app/job/Bootstrap.java:
import play.test.*;
import play.jobs.*;
import play.db.DB;
import models.*;
import java.util.List;
#OnApplicationStart
public class Bootstrap extends Job {
public void doJob() {
// Load default data if the database is empty
if(Item.count() == 0) {
Fixtures.loadModels("my_fixtures.yml");
List<Item> allItems = Item.findAll();
for (Item a: allItems){
DB.execute("UPDATE `Item` SET image='item_" + a.name.toLowerCase() + ".png|image/png' WHERE id=" + a.getId());
}
}
}
}
The first thing I do is filling the database with initial data if there are no 'Item' already stored in the database.
The second thing is iterating over all the 'Item' which play! just stored in the database, which are read from the "my_fixtures.yml" file. Here for each item the string field will get updated as shown in the example above.
I know this is not exactly the answer to question in the OP, but it gives some kind idea to work around this issue..
EDIT: In the example given above I assume that the pictures are uploaded manually to your attachment folder as given in your application.conf, and that each image name is like: "item_<item_name_in_lowercase>" with a ".png" extension
Well, play is quite weird on that point.
The blob is not saved into the database but in a upload folder defined in your application.conf. It is the path toward the file that is saved in the database.
I cannot check it right now, but I seem to recall they are saved as textuel representations (VARCHAR, TEXT)
The blob is saved in the file system, by default under "data/attachments" if I recall correctly, but you can change that in the configuration (application.conf)
In the database, it's stored as a String (varchar in most DB) with two components: the name and the mime type. It looks like:
12345asbcdefghi12345abcdfed|image/jpeg
The first part is the name of the file. When you upload a file Play generates a unique UUID as name to avoid collision. Yes, this means you are loosing the original name. (note: now I'm having doubts on the name part, I would swear it is lost, but I may be wrong!)
The second part (after the |) is the myme type. Play uses a magic-myme library to automatically detect it.
You can see the code here.
Here is a modified version of Unji's answer that loads the images from a folder in conf, please note that I have removed all the import statements:
/**
* A job executed when the application starts.
*/
#OnApplicationStart
public class Bootstrap extends Job {
/**
* Loads the initial data if there are no
* WebAdministrators at the database.
* <p>
* It loads images on the post with the following criteria:
* <ol>
* <li>file loaction: /conf/initialMedia/</li>
* <li>file name: {post.title.toCamelCase()}-{i}.jpg</li>
* </ol>
* Where i must start in 0.
* </p>
*/
#Override
public void doJob() {
// Check if the database is empty
if(WebAdministrator.count() == 0) {
Logger.info("Loading Initial Data.");
Fixtures.loadModels("initial-data.yml");
List<Post> posts = Post.findAll();
for (Post post: posts) {
Logger.info("Looking for files for post: [" + post.title + "]");
for (int i=0; true; i++) {
VirtualFile vf = VirtualFile.fromRelativePath("/conf/initialMedia/"
+ JavaExtensions.camelCase(post.title) + "-" + i + ".jpg");
File imageFile = vf.getRealFile();
if (imageFile.exists()) {
try {
Blob blobImage = new Blob();
blobImage.set(new FileInputStream(imageFile), MimeTypes.getContentType(imageFile.getName()));
MediaItem mediaItem = new Image(blobImage);
mediaItem.save();
post.mediaItems.add(mediaItem);
post.save();
Logger.info("File: [%s] Loaded", imageFile.getAbsolutePath());
} catch (FileNotFoundException e) {
// this should never happen.
}
} else {
Logger.info("Media Loaded for post [%s]: %d files.", post.title, i);
break;
}
}
}
}
}
}