We Keep Coding

Trying to add substrings from newLines in a large file to a list

I downloaded my extended listening history from Spotify and I am trying to make a program to turn the data into a list of artists without doubles I can easily make sense of. The file is rather huge because it has data on every stream I have done since 2016 (307790 lines of text in total). This is what 2 lines of the file looks like:
{"ts":"2016-10-30T18:12:51Z","username":"edgymemes69endmylifepls","platform":"Android OS 6.0.1 API 23 (HTC, 2PQ93)","ms_played":0,"conn_country":"US","ip_addr_decrypted":"68.199.250.233","user_agent_decrypted":"unknown","master_metadata_track_name":"Devil's Daughter (Holy War)","master_metadata_album_artist_name":"Ozzy Osbourne","master_metadata_album_album_name":"No Rest for the Wicked (Expanded Edition)","spotify_track_uri":"spotify:track:0pieqCWDpThDCd7gSkzx9w","episode_name":null,"episode_show_name":null,"spotify_episode_uri":null,"reason_start":"fwdbtn","reason_end":"fwdbtn","shuffle":true,"skipped":null,"offline":false,"offline_timestamp":0,"incognito_mode":false},
{"ts":"2021-03-26T18:15:15Z","username":"edgymemes69endmylifepls","platform":"Android OS 11 API 30 (samsung, SM-F700U1)","ms_played":254120,"conn_country":"US","ip_addr_decrypted":"67.82.66.3","user_agent_decrypted":"unknown","master_metadata_track_name":"Opportunist","master_metadata_album_artist_name":"Sworn In","master_metadata_album_album_name":"Start/End","spotify_track_uri":"spotify:track:3tA4jL0JFwFZRK9Q1WcfSZ","episode_name":null,"episode_show_name":null,"spotify_episode_uri":null,"reason_start":"fwdbtn","reason_end":"trackdone","shuffle":true,"skipped":null,"offline":false,"offline_timestamp":1616782259928,"incognito_mode":false},
It is formatted in the actual text file so that each stream is on its own line. NetBeans is telling me the exception is happening at line 19 and it only fails when I am looking for a substring bounded by the indexOf function. My code is below. I have no idea why this isn't working, any ideas?
import java.util.*;
public class MainClass {
public static void main(String args[]){
File dat = new File("SpotifyListeningData.txt");
List<String> list = new ArrayList<String>();
Scanner swag = null;
try {
swag = new Scanner(dat);
}
catch(Exception e) {
System.out.println("pranked");
}
while (swag.hasNextLine())
if (swag.nextLine().length() > 1)
if (list.contains(swag.nextLine().substring(swag.nextLine().indexOf("artist_name"), swag.nextLine().indexOf("master_metadata_album_album"))))
System.out.print("");
else
try {list.add(swag.nextLine().substring(swag.nextLine().indexOf("artist_name"), swag.nextLine().indexOf("master_metadata_album_album")));}
catch(Exception e) {}
System.out.println(list);
}
}

Find a JSON parser you like.
Create a class that with the fields you care about marked up to the parsers specs.
Read the file into a collection of objects. Most parsers will stream the contents so you're not string a massive string.
You can then load the data into objects and store that as you see fit. For your purposes, a TreeSet is probably what you want.

Your code will throw a lot of exceptions only because you don't use braces. Please do use braces in each blocks, whether it is if, else, loops, whatever. It's a good practice and prevent unnecessary bugs.
However, everytime scanner.nextLine() is called, it reads the next line from the file, so you need to avoid using that in this way.
The best way to deal with this is to write a class containing the fields same as the json in each line of the file. And map the json to the class and get desired field value from that.
Your way is too much risky and dependent on structure of the data, even on whitespaces. However, I fixed some lines in your code and this will work for your purpose, although I actually don't prefer operating string in this way.
while (swag.hasNextLine()) {
String swagNextLine = swag.nextLine();
if (swagNextLine.length() > 1) {
String toBeAdded = swagNextLine.substring(swagNextLine.indexOf("artist_name") + "artist_name".length() + 2
, swagNextLine.indexOf("master_metadata_album_album") - 2);
if (list.contains(toBeAdded)) {
System.out.print("Match");
} else {
try {
list.add(toBeAdded);
} catch (Exception e) {
System.out.println("Add to list failed");
}
}
System.out.println(list);
}
}

Adding files from folder to the list in android app

I'm trying to create a list of files in my sdcard (and then get a random one from this list).
I've read tutorials but none of those worked.
My code is as following:
try{
File file=new File("/sdcard");
File[] list = new File ("/sdcard").listFiles();
ArrayList<String> lista = new ArrayList<String>();
for (File f : list){
if (f.isFile()){
if (f.getName().startsWith("aa")){
lista.add(f.getName());
}
}
}
Random gen = new Random();
String s = lista.get(gen.nextInt(lista.size()-1)).toString();
wyswietl.setText(s);
}catch(NullPointerException e){
Log.e("nope", e.getMessage());
}
LogCat shows exceptions.
I've checked every single line - when I try to show lista.size() - it throws ResourcesNotFoundException.
What interesting is, changing String s into
String s = lista.get(1).toString()
works - it shows me one of the files in the folder.
So my question is: how can I fix this and get a list of files (which start with "aa") in /sdcard folder?

If you want to pick one random item in array list, I believe it should be
String s = lista.get(gen.nextInt(lista.size()));
Random.nextInt(int n) retrieves random value between 0-(n-1). See Random.nextInt() documentation.
ResourceNotFound exception I believe is related to failure to locate resource ID inside R.java not index out of bound exception.
TextView.setText() with integer value parameter, interprets integer value as a resource ID, see here. So if you call
atextView.setText(lista.size());
It will throw ResourceNotFoundException because it may not point to correct resource ID. If you want to display number of items in list then
atextView.setText(String.valueOf(lista.size()));

If lista.size () equals 0 then exception can occur...
( because gen.nextInt (-1) )
I hope you to prevent it.

Here is the different way to the filter on list of file based on file name !!!!
File f = new File("/sdcard");
String fileList[];
if(f.isDirectory()){
fileList = f.list(new FilenameFilter() {
#Override
public boolean accept(File dir, String name) {
// TODO Auto-generated method stub
if (name.startsWith("aa")){
return true;
}
return false;
}
});
'FileList' is an array where you get all the File object and then you can easily get the file name from that!!!!
Hope this will help!!!

I've found the solution why this code didn't work. The files weren't in the folder yet - i had to add names to the list manually, then in other function check if they are on sdcard (if not, there is need to copy; if yes, I can display the content).

Searching files in a directory and pairing them based on a common sub-string

I have been attempting to program a solution for ImageJ to process my images.
I understand how to get a directory, run commands on it, etc etc. However I've run into a situation where I now need to start using some type of search function in order to pair two images together in a directory full of image pairs.
I'm hoping that you guys can confirm I am on the right direction and that my idea is right. So far it is proving difficult for me to understand as I have less than even a month's worth of experience with Java. Being that this project is directly for my research I really do have plenty of drive to get it done I just need some direction in what functions are useful to me.
I initially thought of using regex but I saw that when you start processing a lot of images (especially with imagej which it seems does not dump data usage well, if that's the correct way to say it) that regex is very slow.
The general format of these images is:
someString_DAPI_0001.tif
someString_GFP_0001.tif
someString_DAPI_0002.tif
someString_GFP_0002.tif
someString_DAPI_0003.tif
someString_GFP_0003.tif
They are in alphabetical order so it should be able to go to the next image in the list. I'm just a bit lost on what functions I should use to accomplish this but I think my overall while structure is correct. Thanks to some help from Java forums. However I'm still stuck on where to go to next.
So far here is my code: Thanks to this SO answer for partial code
int count = 0;
getFile("C:\");
string DAPI;
string GFP;
private void getFile(String dirPath) {
File f = new File(dirPath);
File[] files = f.listFiles();
while (files.length > 0) {
if (/* File name contains "DAPI"*/){
DAPI = File f;
string substitute to get 'GFP' filename
store GFP file name into variable
do something(DAPI, GFP);
}
advance to next filename in list
}
}
As of right now I don't really know how to search for a string within a string. I've seen regex capture groups, and other solutions but I do not know the "best" one for processing hundreds of images.
I also have no clue what function would be used to substitute substrings.
I'd much appreciate it if you guys could point me towards the functions best for this case. I like to figure out how to make it on my own I just need help getting to the right information. Also want to make sure I am not making major logic mistakes here.

It doesn't seem like you need regex if your file names follow the simple pattern that you mentioned. You can simply iterate over the files and filter based on whether the filename contains DAPI e.g. see below. This code may be oversimplification of your requirements but I couldn't tell that based on the details you've provided.
import java.io.*;
public class Temp {
int count = 0;
private void getFile(String dirPath) {
File f = new File(dirPath);
File[] files = f.listFiles();
if (files != null) {
for (File file : files) {
if (file.getName().contains("DAPI")) {
String dapiFile = file.getName();
String gfpFile = dapiFile.replace("DAPI", "GFP");
doSomething(dapiFile, gfpFile);
}
}
}
}
//Do Something does nothing right now, expand on it.
private void doSomething(String dapiFile, String gfpFile) {
System.out.println(new File(dapiFile).getAbsolutePath());
System.out.println(new File(gfpFile).getAbsolutePath());
}
public static void main(String[] args) {
Temp app = new Temp();
app.getFile("C:\\tmp\\");
}
}
NOTE: As per Vogel612's answer, if you have Java 8 and like a functional solution you can have:
private void getFile(String dirPath) {
try {
Files.find(Paths.get(dirPath), 1, (path, basicFileAttributes) -> (path.toFile().getName().contains("DAPI"))).forEach(
dapiPath -> {
Path gfpPath = dapiPath.resolveSibling(dapiPath.getFileName().toString().replace("DAPI", "GFP"));
doSomething(dapiPath, gfpPath);
});
} catch (IOException e) {
e.printStackTrace();
}
}
//Dummy method does nothing yet.
private void doSomething(Path dapiPath, Path gfpPath) {
System.out.println(dapiPath.toAbsolutePath().toString());
System.out.println(gfpPath.toAbsolutePath().toString());
}

Using java.io.File is the wrong way to approach this problem. What you're looking for is a Stream-based solution using Files.find that would look something like this:
Files.find(dirPath, 1, (path, attributes) -> {
return path.getFileName().toString().contains("DAPI");
}).forEach(path -> {
Path gfpFile = path.resolveSibling(/*build GFP name*/);
doSomething(path, gfpFile);
});
What this does is:
Iterate over all Paths below dirPath 1 level deep (may be adjusted)
Check that the File's name contains "DAPI"
Use these files to find the relevant "GFP"-File
give them to doSomething
This is preferrable to the files solution because of multiple things:
It's significantly more informative when failing
It's cleaner and more terse than your File-Based solution and doesn't have to check for null
It's forward compatible, and thus preferrable over a File-Based solution
Files.find is available from Java 8 onwards

Adding File objects to an Array

I seem to be having an issue with not properly syntaxing my code, but as I've just started out with learning I seem to be missing the error. It's a homework assignment, where I need to use an Array of JxploreFile-objects. This is the part of the code I'm having trouble with:
private JxploreFile[] getSubFolders()
{
File subFiles[];
subFiles = file.listFiles();
File subFolders[];
int p = 0;
for(int i = 0; i < subFiles.length; i++)
{
if(subFiles[i].isDirectory() == true)
{
Array.set(subFolders, p, subFiles[i]);
}
}
JxploreFile foldersToReturn[] = new JxploreFile[subFolders.length];
for(int i=0; i < subFolders.length; i++)
{
foldersToReturn[i] = new JxploreFile(subFolders[i]);
}
return foldersToReturn;
}
Specifically, the for-loop where I'm trying to add the files marked as .isDirectory into a new Array. I've also tried other methods by placing each new file coming from the subFiles Array manually into the subFolders Array by declaring indexnumbers, but this also turned out faulty. At this point I'm out of ideas and I hope there is someone who can point me out the obvious, as I'm probably missing something reallly basic.
Edit:
I'm sorry for the incomplete post, It's the first time I actually post here as I usually try to filter my own problems out of the posts of others. The error I got was indeed that 'subFolders' had not been initialized yet, which I didn't understood because on the sixth line I wrote
File subFolders[];
which as far as I know should declare the variable subFolders to become an Array, or is this where I went wrong?
Also, my question might not have been specific enough, I'm looking for what causes the error (which I didn't mention at all): why 'subFiles' wasn't initialized.

The array subFolders has not been initialized properly. In order to use the array in the Array.set method it must be initialized and allocated with a size.
An alternative approach for this is to use a List instead. Lists are good when you are working with data that is more dynamic e.g. when you do not know the size of the array. Then you can simplify your code like this:
File[] subFiles = file.listFiles();
// Create the list
List<JxploreFile> subFolders = new ArrayList<>();
// Add all the sub folders (note that "file" is a bit magic since it
// is not specified anywhere in the original post
for (File subFile : file.listFiles()) {
if (subFile.isDirectory()) {
subFolders.add(new JxploreFile(subFile));
}
}
// Return an array
return subFolders.toArray(new JxploreFile[subFolders.size()]);
You can also simplify the whole thing even further by using a Java 8 stream like this:
return Arrays.stream(file.listFiles())
.filter(File::isDirectory)
.toArray(JxploreFile[]::new);
For more info:
Creating Objects in Java
Arrays

There is no question in your post, but anyway here the problems I found in your code :
File subFolders[];
int p = 0;
for(int i = 0; i < subFiles.length; i++)
{
if(subFiles[i].isDirectory() == true)
{
Array.set(subFolders, p, subFiles[i]);
}
}
when calling Array.set you never initialized subFolders which will throw a NullPointerException.
Also, you dont need to do
if(subFiles[i].isDirectory() == true)
you can simply do
if(subFiles[i].isDirectory())
as subFiles[i].isDirectory() is already a condition.

how do you search for something in an array in java

I wan to create a public, non-static method that does this:
getContent : This method should take as input a String filename and return a String. The method should search for a file with name filename in the array drive and return the data that is stored in that TxtFile. If no such file exists in the array drive, the method should return null.
I don't know how to search for something in an array. can someone show me how to do this?

Come on, try it your self:
Step1: Read all the file Names from a directory.
Step2: Store the list of Files to a List.
Step3: Iterate the list, Write a conditions with a use the File.getName() method to compare and the name and your input.
if(file.getName().equals(inputFileName)){
return "boooo! I have found You!"
}

You search in an array by iterating of it. Something like this (if you are inside a method):
for(String x : someStringArray){
if (somecondition(x)) return x;
}

this should be of help: File search
A definitive source for information.
EDIT: thought of something really simple:
String[] drives = new String[]{"C:\\", "D:\\"};
File file;
for(String drive : drives)
{
file = new File(drive + "abc.txt");
JOptionPane.showMessageDialog(null, file.exists());
}
you can replace the showMessageDialogBox with whatever you want to do if the file exists.