We Keep Coding

Trying to add substrings from newLines in a large file to a list

I downloaded my extended listening history from Spotify and I am trying to make a program to turn the data into a list of artists without doubles I can easily make sense of. The file is rather huge because it has data on every stream I have done since 2016 (307790 lines of text in total). This is what 2 lines of the file looks like:
{"ts":"2016-10-30T18:12:51Z","username":"edgymemes69endmylifepls","platform":"Android OS 6.0.1 API 23 (HTC, 2PQ93)","ms_played":0,"conn_country":"US","ip_addr_decrypted":"68.199.250.233","user_agent_decrypted":"unknown","master_metadata_track_name":"Devil's Daughter (Holy War)","master_metadata_album_artist_name":"Ozzy Osbourne","master_metadata_album_album_name":"No Rest for the Wicked (Expanded Edition)","spotify_track_uri":"spotify:track:0pieqCWDpThDCd7gSkzx9w","episode_name":null,"episode_show_name":null,"spotify_episode_uri":null,"reason_start":"fwdbtn","reason_end":"fwdbtn","shuffle":true,"skipped":null,"offline":false,"offline_timestamp":0,"incognito_mode":false},
{"ts":"2021-03-26T18:15:15Z","username":"edgymemes69endmylifepls","platform":"Android OS 11 API 30 (samsung, SM-F700U1)","ms_played":254120,"conn_country":"US","ip_addr_decrypted":"67.82.66.3","user_agent_decrypted":"unknown","master_metadata_track_name":"Opportunist","master_metadata_album_artist_name":"Sworn In","master_metadata_album_album_name":"Start/End","spotify_track_uri":"spotify:track:3tA4jL0JFwFZRK9Q1WcfSZ","episode_name":null,"episode_show_name":null,"spotify_episode_uri":null,"reason_start":"fwdbtn","reason_end":"trackdone","shuffle":true,"skipped":null,"offline":false,"offline_timestamp":1616782259928,"incognito_mode":false},
It is formatted in the actual text file so that each stream is on its own line. NetBeans is telling me the exception is happening at line 19 and it only fails when I am looking for a substring bounded by the indexOf function. My code is below. I have no idea why this isn't working, any ideas?
import java.util.*;
public class MainClass {
public static void main(String args[]){
File dat = new File("SpotifyListeningData.txt");
List<String> list = new ArrayList<String>();
Scanner swag = null;
try {
swag = new Scanner(dat);
}
catch(Exception e) {
System.out.println("pranked");
}
while (swag.hasNextLine())
if (swag.nextLine().length() > 1)
if (list.contains(swag.nextLine().substring(swag.nextLine().indexOf("artist_name"), swag.nextLine().indexOf("master_metadata_album_album"))))
System.out.print("");
else
try {list.add(swag.nextLine().substring(swag.nextLine().indexOf("artist_name"), swag.nextLine().indexOf("master_metadata_album_album")));}
catch(Exception e) {}
System.out.println(list);
}
}

Find a JSON parser you like.
Create a class that with the fields you care about marked up to the parsers specs.
Read the file into a collection of objects. Most parsers will stream the contents so you're not string a massive string.
You can then load the data into objects and store that as you see fit. For your purposes, a TreeSet is probably what you want.

Your code will throw a lot of exceptions only because you don't use braces. Please do use braces in each blocks, whether it is if, else, loops, whatever. It's a good practice and prevent unnecessary bugs.
However, everytime scanner.nextLine() is called, it reads the next line from the file, so you need to avoid using that in this way.
The best way to deal with this is to write a class containing the fields same as the json in each line of the file. And map the json to the class and get desired field value from that.
Your way is too much risky and dependent on structure of the data, even on whitespaces. However, I fixed some lines in your code and this will work for your purpose, although I actually don't prefer operating string in this way.
while (swag.hasNextLine()) {
String swagNextLine = swag.nextLine();
if (swagNextLine.length() > 1) {
String toBeAdded = swagNextLine.substring(swagNextLine.indexOf("artist_name") + "artist_name".length() + 2
, swagNextLine.indexOf("master_metadata_album_album") - 2);
if (list.contains(toBeAdded)) {
System.out.print("Match");
} else {
try {
list.add(toBeAdded);
} catch (Exception e) {
System.out.println("Add to list failed");
}
}
System.out.println(list);
}
}

Assigning New Object to a Generic Array Index

I'm POSITIVE that my title for this topic is not appropriate. Let me explain. The purpose of this is to duplicate a "Profile" application, where I have a profile and so would you. We both have our own followers and in this example, we both follow each other. What this method is needed to return is a cross reference based on whom you follow that I do not. I need this method to return to me a recommended Profile object that I do not already have in my array. Right now I'm having a difficult time with one line of code within a particular method.
One of my classes is a Set class that implements a SetInterface (provided by my professor) and also my Profile class that implements a ProfileInterface which was also provided. In my code for the Profile class, I have the following object: private Set<ProfileInterface> followBag = new Set<ProfileInterface>(); which utilizes the Array bag methods from my Set class with the ProfileInterface methods I've made.
Here is the method (not complete but can't move further without my problem being explained):
public ProfileInterface recommend(){
Set<ProfileInterface> recommended;
ProfileInterface thisProfile = new Profile();
for(int index = 0; index < followBag.getCurrentSize(); index++){
Set<ProfileInterface> follows = followBag[index].toArray();
for(int followedFollowers = 0; followedFollowers < follows.getCurrentSize(); followedFollowers++) {
if()
//if Profile's do not match, set recommended == the Profile
}
}
return recommended;
}
The purpose of this method is to parse through an array (Profile as this example) and then take each of those sub-Profiles and do a similar action. The reason for this much like "Twitter", "Facebook", or "LinkedIn"; where each Profile has followers. This method is meant to look through the highest Profiles follows and see if those subProfiles have any followers that aren't being followed by the highest one. This method is then meant to return that Profile as a recommended one to be followed. This is my first dealing with Array Bag data structures, as well as with generics. Through "IntelliJ", I'm receiving errors with the line Set<ProfileInterface> follows = followBag[index].toArray();. Let me explain the reason for this line. What I'm trying to do is take "my" profile (in this example), and see who I'm following. For each followed profile (or followBag[index]) I wish to see if followBag[index][index] == followBag[index] and continue to parse the array to see if it matches. But, due to my confusion with generics and array bag data structures, I'm having major difficulties figuring this out.
I'd like to do the following:
//for all of my followers
//look at a particular followed profile
//look at all of that profile's followers
//if they match one of my followers, do nothing
//else
//if they don't match, recommend that profile
//return that profile or null
My problem is that I do not know how to appropriately create an object of a Profile type that will allow me to return this object
(in my method above, the line Set<ProfileInterface> follows = followBag[index].toArray();)
I'm trying to make an index of my Profile set to an object that can later be compared where my difficulties are. I'd really appreciate any insight into how this should be done.
Much appreciated for all help and Cheers!

When you do:
Set<ProfileInterface> follows = followBag[index].toArray();
you're trying to use Set as Array. But you can't.
Java will not allow, because Set and Array are different classes, and Set does not support [] syntax.
That is why you get error. For usefollowBag as Array you have to convert it:
ProfileInterface[] profileArray = followBag.toArray(new ProfileInterface[followBag.size()]);
for(int i=0; i<profileArray.length; i++){
ProfileInterface profile = profileArray[i];
//do what you would like to do with array item
}
I believe, in your case, you don't need assign Set object to generic Array at all. Because you can enumerate Set as is.
public class Profile {
private Set<ProfileInterface> followBag = new HashSet<Profile>();
...
public Set<ProfileInterface> recommended(){
Set<ProfileInterface> recommendSet = new HashSet<ProfileInterface>();
for(Profile follower : followBag){
for(Profile subfollower : follower.followBag){
if(!this.followBag.contains(subfollower)){
recommendSet.add(subfollower);
}
}
}
return recommendSet;
}
}
I also added possibility of returning list of recommended profiles, because there is may be several.

Xtend: filter an ArrayList of Strings

I have an array list declared like this:
val aName = new ArrayList
I add names in this array via the add().
When I print them, I only want to print specific names (e.g. all with names "Charlie" and working at the department Finance).
In my for loop, I have this:
for (m: aName.filter[!CDirectoryFacade::instance.isNameUsed(toString)])
{
print(m)
}
The loop above did not print the name at all.
Because my function isNameUsed() did not receive the string as I expect, but rather it receives the address as a String
org.generator.myDsl.myDslGenerator#67bd0a26
However, I do not seem to have problem when I do not use filter().
for (m: aName)
{
if (!CDirectoryFacade::instance.isNameUsed(m))
{
print(m)
}
}
Can anyone suggest on how to use the filter() with Strings?

It's hard to tell because your code snippet doesn't provide any types and it is not clear where name comes from in the print expression.
But the code above should probably read like this:
for (m: aName.filter[!CDirectoryFacade::instance.isNameUsed(toString)])
{
print(m.name) // instead of print(name)
}
I could be even more concise:
aName.filter[!CDirectoryFacade::instance.isNameUsed(toString)].forEach[ print(name) ]

Making an add method for an ArrayList

So I am supposed to make an add method for an array list which adds a new movie object to the list if it doesnt exist, or if it finds a movie object with a similar title within the list, it just increases the quantity property of that object. Here is what I've got so far.
public void add(String title, double rating, int releaseYear){
if(this.myMovies.size() < 1)
{
Movie mymovie = new Movie(title, rating, releaseYear);
this.myMovies.add(mymovie);
}
else
{
for(int i = 0; i < this.myMovies.size(); i++)
{
Movie temp = this.myMovies.get(i);
if(temp.Title.equals(title)){
this.myMovies.get(i).quantity++;
break;
}
else
{
Movie mymovie = new Movie(title, rating, releaseYear);
this.myMovies.add(mymovie);
break;
}
}
}
}
My problem is that this ends up not taking account of similar names and doesn't increase the quantity but just adds another object to the list. I have a strong feeling that the problem lies within my For loop but I just can't identify it. Can anyone see anything that I may be doing wrong? Thank you!

You're testing only for equality, not similarity here:
if(temp.Title.equals(title)){
Instead, you should write a helper method to test for similarity based on whatever criteria are appropriate. For example:
if (isSimilar(temp.Title, title)){
and the isSimilar method might look something like this (assuming you don't need any input validation):
private void isSimilar(String title1, String title2) {
return title1.equalsIgnoreCase(title2)
|| title1.toLowerCase().contains(title2.toLowerCase())
|| title2.toLowerCase().contains(title1.toLowerCase());
}
or, perhaps more appropriately, like this (if you implement it in the Movie class):
private void isSimilar(otherMovie) {
return title.equalsIgnoreCase(otherMovie.title)
|| title.toLowerCase().contains(otherMovie.title.toLowerCase())
|| otherMovie.title.toLowerCase().contains(title.toLowerCase());
}
...in which case your if statement would also change slightly.
Keep in mind that I don't know what you consider 'similar'; only that the movies are considered similar if the names are similar.
A couple more comments:
Fields and method names generally start with a lowercase letter (so the field Movie.Title should instead be Movie.title).
It's usually preferable to loop over a Collection using an Iterator instead of using the raw index--partly because the Iterator should always know how to loop over the Collection efficiently.
Learn to use your IDE's debugger (it's probably very easy). Then you can step through each line of code to see exactly where your program is doing something unexpected.

I would do something like this:
public void add(String title, double rating, int releaseYear){
for(Movie m: myMovies.size())
{
if(m.Title.equals(title)){
m.quantity++;
return;
}
}
// movie with same title not found in the list -> insert
this.myMovies.add(new Movie(title, rating, releaseYear));
}
By the way: variable names should start with a lowercase character (Title -> title).

I'm addressing your "similarity" requirement. If you really want to do this properly it could be a lot of work. Essentially you have two strings and want to get a measure of the similarity. I am doing the same thing for figure captions and I plan to tackle it by:
splitting the title into words
lowercasing them
using them as features for classifier4J (http://classifier4j.sourceforge.net/)
That will go a long way based on simple word counts. But then you have the problem of stemming
(words that differ by endings - "Alien" and "Aliens"). If you go down this road you'll need to read up about Classification and Natural Language Processing

problems with RequestForFile() class that is using User class in Java

Okay I'll try to be direct.
I am working on a file sharing application that is based on a common Client/Serer architecture. I also have HandleClient class but that is not particularly important here.
What I wanna do is to allow users to search for a particular file that can be stored in shared folders of other users. For example, 3 users are connected with server and they all have their respective shared folders. One of them wants to do a search for a file named "Madonna" and the application should list all files containing that name and next to that file name there should be an information about user(s) that have/has a wanted file. That information can be either username or IPAddress. Here is the User class, the way it needs to be written (that's how my superiors wanted it):
import java.io.File;
import java.util.ArrayList;
import java.util.Scanner;
public class User {
public static String username;
public static String ipAddress;
public User(String username, String ipAddress) {
username = username.toLowerCase();
System.out.println(username + " " + ipAddress);
}
public static void fileList() {
Scanner userTyping = new Scanner(System.in);
String fileLocation = userTyping.nextLine();
File folder = new File(fileLocation);
File[] files = folder.listFiles();
ArrayList<String> list = new ArrayList<String>();
for (int i = 0; i < files.length; i++) {
list.add(i, files[i].toString().substring(fileLocation.length()));
System.out.println(list.get(i));
}
}
public static void main(String args[]) {
System.out.println("Insert the URL of your shared folder");
User.fileList();
}
}
This class stores attributes of a particular user (username, IPAddress) and also creates the list of files from the shared folder of that particular user. the list type is ArrayList, that's how it has to be, again, my superiors told me to.
On the other hand I need another class that is called RequestForFile(String fileName) whose purpose is to look for a certain file within ArrayLists of files from all users that are logged in at the moment of search.
This is how it should look, and this is where I especially need your help cause I get an error and I can't complete the class.
import java.util.ArrayList;
public class RequestForFile {
public RequestForFile(String fileName) {
User user = new User("Slavisha", "84.82.0.1");
ArrayList<User> listOfUsers = new ArrayList();
listOfUsers.add(user);
for (User someUser : listOfUsers) {
for (String request : User.fileList()) {
if (request.equals(fileName))
System.out.println(someUser + "has that file");
}
}
}
}
The idea is for user to look among the lists of other users and return the user(s) with a location of a wanted file.
GUI aside for now, I will get to it when I fix this problem.
Any help appreciated.
Thanks
I'm here to answer anything regarding this matter.

There are lots of problems here such as:
I don't think that this code can compile:
for (String request : User.fileList())
Because fileList() does not return anything. Then there's the question of why fileList() is static. That means that all User objects are sharing the same list. I guess that you have this becuase you are trying to test your user object from main().
I think instead you should have coded:
myUser = new User(...);
myUser.fileList()
and so fileList could not be static.
You have now explained your overall problem more clearly, but that reveals some deeper problems.
Let's start at the very top. Your request object: I think it represents one request for one user with one file definition. But it needs to go looking in the folders of many users. You add the the requesting user to a list, but what about the others. I think that this means that you need another class responsible for holding all the users.
Anyway lets have a class called UserManager.
UserMananger{
ArrayList<User> allTheUsers;
public UserManager() {
}
// methods here for adding and removing users from the list
// plus a method for doing the search
public ArrayList<FileDefinitions> findFile(request) [
// build the result
}
}

in the "line 14: for (String request : User.fileList()) {" I get this error: "void type not allowed here" and also "foreach not applicable to expression type"
You need to let User.fileList() return a List<String> and not void.
Thus, replace
public static void fileList() {
// ...
}
by
public static List<String> fileList() {
// ...
return list;
}
To learn more about basic Java programming, I can strongly recommend the Sun tutorials available in Trials Covering the Basics chapter here.

It looks like you're getting that error because the fileList() method needs to returns something that can be iterated through - which does not include void, which is what that method returns. As written, fileList is returning information to the console, which is great for your own debugging purposes, but it means that other methods can't get any of the information fileList sends to the console.
On a broader note, why is RequestForFile a separate class? If it just contains one method, you can just write it as a static method, or as a method in the class that's going to call it. Also, how will it get lists of other users? It looks like there's no way to do so as is, as you've hard-coded one user.
And looking at the answers, I'd strongly second djna's suggestion of having some class that acts as the controller/observer of all the Users.