Related
They want us to use a combination of ArrayList (or a regular array) and one of the Map Collections Hashmap or Tree Map. The conversion will involve reading the data file line by line reading it into the appropriate structure. Do this in a method called read_records. When the structure is filled, then send the structure to a method called print_records which will print it out in the following format:
"ABW" : {
"2020-03-13" : {
"iso_code" : "ABW",
"location" : "Aruba",
"date" : "2020-03-13",
"total_cases" : "2",
"new_cases" : "2",
"total_deaths" : "0",
"new_deaths" : "0",
"total_cases_per_million" : "18.733",
"new_cases_per_million" : "18.733",
"total_deaths_per_million" : "0.0",
"new_deaths_per_million" : "0.0",
"total_tests" : "",
"new_tests" : "",
"total_tests_per_thousand" : "",
"new_tests_per_thousand" : "",
"new_tests_smoothed" : "",
"new_tests_smoothed_per_thousand" : "",
"tests_units" : "",
"stringency_index" : "0.0",
"population" : "106766.0",
"population_density" : "584.8",
"median_age" : "41.2",
"aged_65_older" : "13.085",
"aged_70_older" : "7.452",
"gdp_per_capita" : "35973.781",
"extreme_poverty" : "",
"cvd_death_rate" : "",
"diabetes_prevalence" : "11.62",
"female_smokers" : "",
"male_smokers" : "",
"handwashing_facilities" : "",
},
.
.
.
},
.
.
.
Structure the program so that appropriate methods are used in a class called JSONParser. Ensure good program structure by dividing the processing into separate methods for reading, writing and or analyzing the input data.
Heres the code I have so far:
public class CSVParser {
private final int ISO_CODE = 0;
private final int DATE = 2;
public CSVParser(File f) throws FileNotFoundException {
try {
Scanner covid_data = new Scanner(f);
String header = covid_data.nextLine();
while(covid_data.hasNextLine()) {
String line = covid_data.nextLine();
String record[] = line.split(",");
String date = record[DATE];
System.out.println(record[ISO_CODE] + ":");
for(String s : record) System.out.println(" "+s);
System.out.println();
}
covid_data.close();
} catch(FileNotFoundException e) {
System.out.println("Problem with file");
}
}
}
Heres the Main Class:
import java.io.File;
import java.io.FileNotFoundException;
/*
*
Put the header comment here
*/
public class Main {
public static void main(String[] args) throws FileNotFoundException {
File f = new File("owid-covid-data.csv");
CSVParser csvfile = new CSVParser(f);
}
}
In a project, I'm trying to query the tweets of a particular user's handle and find the most common word in the user's tweets and also return the frequency of that most common word.
Below is my code:
public String mostPopularWord()
{
this.removeCommonEnglishWords();
this.sortAndRemoveEmpties();
Map<String, Integer> termsCount = new HashMap<>();
for(String term : terms)
{
Integer c = termsCount.get(term);
if(c==null)
c = new Integer(0);
c++;
termsCount.put(term, c);
}
Map.Entry<String,Integer> mostRepeated = null;
for(Map.Entry<String, Integer> curr: termsCount.entrySet())
{
if(mostRepeated == null || mostRepeated.getValue()<curr.getValue())
mostRepeated = curr;
}
//frequencyMax = termsCount.get(mostRepeated.getKey());
try
{
frequencyMax = termsCount.get(mostRepeated.getKey());
return mostRepeated.getKey();
}
catch (NullPointerException e)
{
System.out.println("Cannot find most popular word from the tweets.");
}
return "";
}
I also think it would help to show the codes for the first two methods I call in the method above, as shown below. They are all in the same class, with the following defined:
private Twitter twitter;
private PrintStream consolePrint;
private List<Status> statuses;
private List<String> terms;
private String popularWord;
private int frequencyMax;
#SuppressWarnings("unchecked")
public void sortAndRemoveEmpties()
{
Collections.sort(terms);
terms.removeAll(Arrays.asList("", null));
}
private void removeCommonEnglishWords()
{
Scanner sc = null;
try
{
sc = new Scanner(new File("commonWords.txt"));
}
catch(Exception e)
{
System.out.println("The file is not found");
}
List<String> commonWords = new ArrayList<String>();
int count = 0;
while(sc.hasNextLine())
{
count++;
commonWords.add(sc.nextLine());
}
Iterator<String> termIt = terms.iterator();
while(termIt.hasNext())
{
String term = termIt.next();
for(String word : commonWords)
if(term.equalsIgnoreCase(word))
termIt.remove();
}
}
I apologise for the rather long code snippets. But one frustrating thing is that even though my removeCommonEnglish() method is apparently right (discussed in another post), when I run the mostPopularWord(), it returns "the", which is clearly a part of the common English Words list that I have and meant to eliminate from the List terms. What might I be doing wrong?
UPDATE 1:
Here is the link ot the commonWords file:
https://drive.google.com/file/d/1VKNI-b883uQhfKLVg-L8QHgPTLNb22uS/view?usp=sharing
UPDATE 2: One thing I've noticed while debugging is that the
while(sc.hasNext())
in removeCommonEnglishWords() is entirely skipped. I don't understand why, though.
It can be more simple if you use stream like so :
String mostPopularWord() {
return terms.stream()
.collect(Collectors.groupingBy(s -> s, Collectors.counting()))
.entrySet().stream()
.sorted(Map.Entry.comparingByValue(Comparator.reverseOrder()))
.findFirst()
.map(Map.Entry::getKey)
.orElse("");
}
I tried your code. Here is what you will have to do. Replace the following part in removeCommonEnglishWords()
Iterator<String> termIt = terms.iterator();
while(termIt.hasNext())
{
String term = termIt.next();
for(String word : commonWords)
if(!term.equalsIgnoreCase(word))
reducedTerms.add( term );
}
with this:
List<String> reducedTerms = new ArrayList<>();
for( String term : this.terms ) {
if( !commonWords.contains( term ) ) reducedTerms.add( term );
}
this.terms = reducedTerms;
Since you hadn't provided the class, I created one with some assumptions, but I think this code will go through.
A slightly different approach using streams.
This uses the relatively common frequency count idiom using streams and stores them in a map.
It then does a simple scan to find the largest count obtained and either returns
that word or the string "No words found".
It also filters out the words in a Set<String> called ignore so you need to create that too.
import java.util.Arrays;
import java.util.Comparator;
import java.util.Map;
import java.util.Map.Entry;
import java.util.stream.Collectors;
Set<String> ignore = Set.of("the", "of", "and", "a",
"to", "in", "is", "that", "it", "he", "was",
"you", "for", "on", "are", "as", "with",
"his", "they", "at", "be", "this", "have",
"via", "from", "or", "one", "had", "by",
"but", "not", "what", "all", "were", "we",
"RT", "I", "&", "when", "your", "can",
"said", "there", "use", "an", "each",
"which", "she", "do", "how", "their", "if",
"will", "up", "about", "out", "many",
"then", "them", "these", "so", "some",
"her", "would", "make", "him", "into",
"has", "two", "go", "see", "no", "way",
"could", "my", "than", "been", "who", "its",
"did", "get", "may", "…", "#", "??", "I'm",
"me", "u", "just", "our", "like");
Map.Entry<String, Long> entry = terms.stream()
.filter(wd->!ignore.contains(wd)).map(String::trim)
.collect(Collectors.groupingBy(a -> a,
Collectors.counting()))
.entrySet().stream()
.collect(Collectors.maxBy(Comparator
.comparing(Entry::getValue)))
.orElse(Map.entry("No words found", 0L));
System.out.println(entry.getKey() + " " + entry.getValue());
I'm updating a 2014 Java project with the latest libraries and MongoDB Java Driver changed from 3.0 to 3.6. Most of the codes has been updated but there is a specific query pretty complicated which is giving me problems.
The Document is like this
{
"_id" : ObjectId("58750a67ae28bc28e0705b0f"),
"info": "description",
"parentId", "variable-id-here"
"issues": [
{"name": "a", "closed": true},
{"name": "b", "closed": false},
{"name": "c", "closed": true}
],
"bugs": [
{"name": "d", "closed": false},
{"name": "e", "closed": false},
{"name": "f", "closed": true}
],
"errors": [
{"name": "g", "closed": true},
{"name": "h", "closed": true},
{"name": "i", "closed": false}
]
}
(even if elements in arrays are similar, they can't be grouped in a single array in the document with an extra "type" key with values [issues, bugs, errors], but that's not the point)
The old script is this one
List<DBObject> aggregators = new ArrayList<>();
DBObject match = new BasicDBObject("$match", new BasicDBObject("parentId", myId));
DBObject project = new BasicDBObject();
List<String> domains = Arrays.asList("issues", "bugs", "errors");
for (Iterator<String> d = domains.iterator(); d.hasNext();) {
String domain = d.next();
//Reset values
aggregators = new ArrayList<>();
// Define MongoDB "$project" to find 'true' values on 'closed' flags
project = new BasicDBObject("$project", new BasicDBObject("closedProblems",
new BasicDBObject("$filter",
new BasicDBObject("input", "$"+domain)
.append("as", "myObject")
.append("cond", new BasicDBObject("$eq",
Arrays.<Object> asList("$$myObject.closed", true)
)
)
)
));
aggregators.add(match);
aggregators.add(project);
//db defined before. AggregationOutput is deprecated so must be changed
AggregationOutput output = db.getCollection("myColl").aggregate(aggregators);
// Now I can iterate results
for (DBObject result : output.results()) {
// ...
}
}
I tried to use projects, expressions, etc. but I can't find a way to duplicate the MongoDB project with new Aggregation methods.
The final result should use mongoTemplate.anyMethods to execute the Aggregation in order to join new project guidelines
Use below code in 3.6 driver.
DBCursor output = (DBCursor) db.getCollection("myColl").aggregate(aggregators, AggregationOptions.builder().build());
for (Iterator<DBObject> r = output.iterator(); output.hasNext();) {
DBObject result = r.next();
...
}
Update:
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
import static org.springframework.data.mongodb.core.aggregation.ArrayOperators.Filter.filter;
import static org.springframework.data.mongodb.core.aggregation.ComparisonOperators.Eq.valueOf;
import org.springframework.data.mongodb.core.query.Criteria;
import org.springframework.data.mongodb.core.aggregation.*;
List<AggregationOperation> aggregators;
MatchOperation match = Aggregation.match(Criteria.where("parentId").is(myId));
List<String> domains = Arrays.asList("issues", "bugs", "errors");
for (Iterator<String> d = domains.iterator(); d.hasNext();) {
String domain = d.next();
aggregators = new ArrayList<>();
ProjectionOperation project = project().and(filter(domain)
.as("myObject")
.by(valueOf(
"myObject.closed")
.equalToValue(
true)))
.as("closedProblems");
aggregators.add(match);
aggregators.add(project);
Aggregation aggregation = newAggregation(aggregators).withOptions(newAggregationOptions().cursor(new Document()).build());
AggregationResults<Document> results = mongoTemplate.aggregate(aggregation, "myColl", Document.class );
for (Document result : results) {
// ...
}
}
where I want to check which elements of the 1st file are missing in the second one.
Here is the form of the first one:
[
{
"pId": "pId1",
"Platform":["ios","and","web","winph","win"],
"Name": "ay",
"ShortDescription": "Mobi",
"Detail" : {
"IncentiveInformation": "ppp",
"DisplayName" : "vvv!",
"Description" : "mmm",
"TermsAndConditions": ".."
}
},
{
"pId": "pId2",
"Platform":["afasd","sdfsd","pppp","asdas","win"],
"Name": "ay",
"ShortDescription": "mob",
"PromotionDetail": {
"DebugMode": false,
"PromoDate": ["2015.01.01-00:01","2015.01.01-23:59"],
"IncentiveInformation": "PRO",
"Name": "iTunes",
"ShortDescription": "Punkte sammeln bei iTunes",
"DisplayName": null,
"Description": null,
"ImageURL": null,
"JumpToShopURL": "urlHere",
"JumpToShopName" : "Zu iTunes"
}
},
{
"pId": "pId3",
"Platform":["wqdsa"],
"Name": "poti",
"ShortDescription": "pun",
"ImageURL": "url.here",
"Promotion" : false,
"PromotionDetail": {
"DebugMode": false,
"PromoDate": ["2015.01.01-00:00","2015.01.01-23:59"],
"IncentiveInformation": "ppeur",
"Name": "namehere",
"ShortDescription": "tune",
"DisplayName": null,
"Description": null,
"ImageURL": null,
"JumpToShopURL": "noq",
"JumpToShopName" : "Zu"
}
}
]
and here is the form of the 2nd one:
{
"pList": [{
"shortName": "bb",
"longName": "bb",
"pId": "pId2",
"featured": true,
"pLog": "url.here",
"incentivation": "eu",
"details": {
"teaserImage": "image.url",
"description": "desc here",
"jumpToShopURL": "nurl",
"jumpToShopButton": "zubay",
"terms": [{
"headline": "Wichtig",
"body": "bodyline"
}]
}
}, {
"shortName": "one short name",
"longName": "bkp",
"pId": "pId1",
"featured": true,
"pLo": "some.pLo",
"incentivation": "1p",
"details": {
"teaserImage": "some.url",
"description": "desc",
"jumpToShopURL": "short url",
"jumpToShopButton": "Zuay",
"terms": [{
"headline": "Wichtig",
"body": "bodyhere"
}]
}
}]
}
Si I thought to save all the "pId" of the first one in a List(or array) and then iterate over that list and check for each one if the pId exists in the new one.
So I tried this, but it is not working..
Could anyone help me with that? I tried a bit and then I found that I have too many difficulties, to get the pIds saved in a list or an array.
So has someone an idea?
import java.io.*;
import org.json.*;
public class MainDriver {
public static void main(String[] args) throws JSONException {
String jsonData = readFile("C:\\Users\\kbelkhiria\\Desktop\\Karim_JSON\\alt.json");
JSONObject jobj = new JSONObject(jsonData);
JSONArray jarr = new JSONArray(jobj.getJSONArray("pList").toString());
for(int i = 0; i < jarr.length(); i++)
System.out.println("id: " + jarr.getString(i));
}
public static String readFile(String filename) {
String result = "";
try {
BufferedReader br = new BufferedReader(new FileReader(filename));
StringBuilder sb = new StringBuilder();
String line = br.readLine();
while (line != null) {
sb.append(line);
line = br.readLine();
}
result = sb.toString();
} catch(Exception e) {
e.printStackTrace();
}
return result;
}
}
For the 2nd form, you have an JSONObject, but it contains some errors. Please fix them, or use a the 1st form again.
SOLUTION
I found some errors in the second file so I suggest the following edits:
change "jumpToShopURL": nurl", to "jumpToShopURL": null,
add a comma at the end of "description": "desc"
add a comma at the end of "jumpToShopURL": "short url"
For the code, you can use the following lines:
/*first file*/
String jsonData = readFile("C:\\Users\\kbelkhiria\\Desktop\\Karim_JSON\\alt.json");
JSONArray jarr = new JSONArray(jsonData);
/*array of first file's ids*/
ArrayList<String> srcArray = new ArrayList<>();
for(int i = 0; i < jarr.length(); i++) {
srcArray.add(jarr.getJSONObject(i).getString("pId"));
}
/*second file*/
// second form in a seperate file
JSONObject obj = new JSONObject(readFile("C:\\Users\\kbelkhiria\\Desktop\\Karim_JSON\\alt2.json"));
JSONArray arr = obj.getJSONArray("pList");
/*array of second file's ids*/
ArrayList<String> dstArray = new ArrayList<>();
for(int i = 0; i < arr.length(); i++) {
dstArray.add(jarr.getJSONObject(i).getString("pId"));
}
for (String string : srcArray) {
if (dstArray.indexOf(string)==-1)
System.out.println(string + " is missing in the second file");
}
Luckily for you there are already developed libraries to parse any JSON string, like the one's you provided. One of the most popular is
org.json
Using this you can write code similar to this:
import org.json.*;
String myString = ".." // The String representation you provided
JSONObject obj = new JSONObject(myString);
JSONArray arr = obj.getJSONArray("pList");
Another popular library for the same task is GSON
One possible solution using Jackson is the following:
private static final String JSON1 = // first json;
private static final String JSON2 = //second json;
public static void main(String[] args) throws IOException {
ObjectMapper mapper = new ObjectMapper();
List<LinkedHashMap> list1 = Arrays.asList(mapper.readValue(JSON1, LinkedHashMap[].class));
List<LinkedHashMap> list2 = Arrays.asList(mapper.readValue(mapper.readTree(JSON2).get("pList").toString(), LinkedHashMap[].class));
List<LinkedHashMap> missingItens = new ArrayList<>();
for (LinkedHashMap o1 : list1) {
if (!objectExistsInList(o1.get("pId").toString(), list2)) {
missingItens.add(o1);
}
}
}
private static boolean objectExistsInList(String pIdValue, List<LinkedHashMap> objs) {
for (LinkedHashMap map : objs) {
if (map.containsValue(pIdValue)) {
return true;
}
}
return false;
}
Please keep in mind this is a very specific implementation to the given JSONs.
I've got a list of all the file names in a folder and a list of files which have been manually "checked" by a developer. How would I go about comparing the two arrays such that we print out only those which are not contained in the master list.
public static void main(String[] args) throws java.lang.Exception {
String[] list = {"my_purchases", "my_reservation_history", "my_reservations", "my_sales", "my_wallet", "notifications", "order_confirmation", "payment", "payment_methods", "pricing", "privacy", "privacy_policy", "profile_menu", "ratings", "register", "reviews", "search_listings", "search_listings_forms", "submit_listing", "submit_listing_forms", "terms_of_service", "transaction_history", "trust_verification", "unsubscribe", "user", "verify_email", "verify_shipping", "404", "account_menu", "auth", "base", "dashboard_base", "dashboard_menu", "fiveohthree", "footer", "header", "header_menu", "listings_menu", "main_searchbar", "primary_navbar"};
String[] checked = {"404", "account_menu", "auth", "base", "dashboard_base", "dashboard_menu", "fiveohthree", "footer", "header", "header_menu", "listings_menu"};
ArrayList<String> ar = new ArrayList<String>();
for(int i = 0; i < checked.length; i++)
{
if(!Arrays.asList(list).contains(checked[i]))
ar.add(checked[i]);
}
}
Change your loop to :
ArrayList<String> ar = new ArrayList<String>();
for(int i = 0; i < checked.length; i++) {
if(!Arrays.asList(list).contains(checked[i]))
ar.add(checked[i]);
}
ArrayList ar should be outside of the for loop. Otherwise ar will be created every time when element of checked array exists in list.
Edit:
if(!Arrays.asList(list).contains(checked))
With this statement you are checking whether the checked reference is not the element of list. It should be checked[i] to check whether the element of checked exists in list or not.
If you want to print elements in list that are not in checked. Then use :
for(int i = 0; i < list.length; i++) {
if(!Arrays.asList(checked).contains(list[i]))
ar.add(list[i]);
}
System.out.println(ar);
Your updated solution seems kind of odd to me, not sure why you would add list[i] to the result list. Generally this sounds like something hashsets are made for:
String[] list = { "my_purchases", "my_reservation_history","my_reservations","my_sales", "my_wallet", "notifications", "order_confirmation", "payment", "payment_methods", "pricing", "privacy", "privacy_policy", "profile_menu", "ratings", "register", "reviews", "search_listings", "search_listings_forms", "submit_listing", "submit_listing_forms", "terms_of_service", "transaction_history", "trust_verification", "unsubscribe", "user", "verify_email", "verify_shipping", "404", "account_menu", "auth", "base", "dashboard_base", "dashboard_menu", "fiveohthree", "footer", "header", "header_menu", "listings_menu", "main_searchbar", "primary_navbar"};
String[] checked = { "404", "account_menu", "auth", "base", "dashboard_base", "dashboard_menu", "fiveohthree", "footer", "header", "header_menu", "listings_menu"};
HashSet<String> s1 = new HashSet<String>(Arrays.asList(checked));
s1.removeAll(Arrays.asList(list));
System.out.println(s1);
for (String s: checked) { // go through all in second list
if (! list.contains(s)) { // if string not in master list
System.out.println(s); // print that string
}
}
First of all, I think your code has some errors:
s1 is not defined
ar is not defined
you mean to use Arrays.toString instead of Array.toString
So I fixed your code (using Java 8) and it should work like that:
public static void main(String[] args) throws java.lang.Exception {
String[] list = {"my_purchases", "my_reservation_history", "my_reservations", "my_sales", "my_wallet", "notifications", "order_confirmation", "payment", "payment_methods", "pricing", "privacy", "privacy_policy", "profile_menu", "ratings", "register", "reviews", "search_listings", "search_listings_forms", "submit_listing", "submit_listing_forms", "terms_of_service", "transaction_history", "trust_verification", "unsubscribe", "user", "verify_email", "verify_shipping", "404", "account_menu", "auth", "base", "dashboard_base", "dashboard_menu", "fiveohthree", "footer", "header", "header_menu", "listings_menu", "main_searchbar", "primary_navbar"};
String[] checked = {"404", "account_menu", "auth", "base", "dashboard_base", "dashboard_menu", "fiveohthree", "footer", "header", "header_menu", "listings_menu"};
final List<String> result = Stream.of(list)
.filter(listEntry -> Stream.of(checked)
.filter(checkedEntry -> checkedEntry.equals(listEntry)).findFirst().orElse(null) == null)
.collect(Collectors.toList());
System.out.println(result);
}
If you don't want to use Java 8, you have to replace the usage of Streams and filters and collect with the appropriate functions in Java 7 (see e.g., Satya's post).
Anyways, I should mention that there are better (regarding performance) implementations to solve your problem, e.g.,
you could sort your lists prior to searching for duplicates,
you could use, e.g., hash-based implementations to increase the speed when searching for duplicates,
you could move the code outside of the inner loop,
and many more