This question already has answers here:
Most efficient way to increment a Map value in Java
(28 answers)
Closed 6 months ago.
Sorry, I'm totally new to Java. I'm trying to count the same URLs present in objects and I'm not able to figure out how to count them.
I was reading this article https://www.geeksforgeeks.org/passing-and-returning-objects-in-java/
and wondered if this is a way to count URLs object or another way. I'm stuck here for two days.
Can anyone please guide me?
class Main() {
private HashMap<String, String> keyMap;
private static int count = 0;
public String newUrl(String url) {
//Completed: if the url is present in HashMap then return shortUrl else add it and return shortUrl
return shortUrl
}
public Integer countURL(String url){
//int count = Collections.frequency(keyMap.values(), url);
return count;
}
public static void main(String args[]) {
Main a = new Main();
String url = a.newUrl("http://example.com");
String url1 = a.newUrl("http://example2.com");
String url2 = a.newUrl("http://example2.com");
System.out.println(url);
System.out.println(url1);
System.out.println(url2);;
System.out.println(a.countURL("http://example2.com")); //2 if not then 0
}
}
You can add this to your newUrl method:
public String newUrl(String url){
if (!keyMap.containsKey(url)) {
keyMap.put(url, 1);
return url;
}else {
keyMap.replace(url, keyMap.get(url));
return url;
}
}
and in your countUrl method you can just return keyMap.get(url)
public Integer countURL(String url){
return keyMap.get(url);
}
Related
I'm working on web scraper and I can't solve problem I'm having for the second day in row.
The problem with this method is when the bot is supposed to visit the website, harvest all URL's, and add the ones of them it didn't visit already to List< String> "toVisit"
Problematic code:
Elements temp = userAgent.visit(currentUrl).findEvery("<a href>");
for (Element e : temp) {
String x = e.getAt("href");
if(!visited.contains(x)) {
toVisit.add(x);
}
}
However, the if statement doesn't filter (or filter it in way I didn't find out) url's and I have no idea why.
I tried delete the "!" in the statement and create an else part and paste toVisit.add(x) there, but it didn't help.
When I print every url, the bot visits the same ones two or even five times.
EDIT (visited defined)
static List<String> visited = new ArrayList<String>();
EDIT2 (whole code)
import java.util.ArrayList;
import java.util.List;
import com.jaunt.*;
public class b03 {
static String currentUrl = "https://stackoverflow.com";
static String stayAt = currentUrl;
static String searchingTerm = "";
static int toSearch = 50;
static List<String> toVisit = new ArrayList<String>();
static List<String> visited = new ArrayList<String>();
static UserAgent userAgent = new UserAgent();
public static void main(String[] args) {
System.out.println("*started searching...*");
while(visited.size() < toSearch)
visitUrl(currentUrl);
System.out.println("\n\n*done*\n\n");
}
public static void visitUrl(String url) {
visited.add(url);
evaluateUrls();
searchTerm();
toVisit.remove(0);
currentUrl = toVisit.get(0);
}
public static void searchTerm() {
//if(userAgent.doc.getTextContent().contains(searchingTerm))
System.out.println(visited.size() +") "+ currentUrl);
}
public static void evaluateUrls() {
try {
Elements temp = userAgent.visit(currentUrl).findEvery("<a href>");
for (Element e : temp) {
String x = e.getAt("href");
if(!visited.contains(x) && x.contains(stayAt)) {
toVisit.add(x);
}
}
}catch (Exception e) {
System.out.println(e);
}
}
}
Your bot visits the some urls several times because you add them several times to the toVisit list.
To illustrate this: let's assume that the first few links that your bot find on the stackoverflow site are the links to "home" (stackoverflow.com), tags (stackoverflow.com/tags), users (stackoverflow.com/users) and jobs (stackoverflow.jobs) and your bot adds three of those to the toVisit list.
Next it visits the tags page (stackoverflow.com/tags). This page contains again links to the same four urls as before. Since you didn't yet visit the users and the jobs subpage it will add those a second time to the toVisit list.
To fix this, you should only add urls to the toVisit list that are not in the visited list and not in the toVisit list:
if (!visited.contains(x) && !toVisit.contains(x) && x.contains(stayAt)) {
toVisit.add(x);
}
I can not try this code because of the jaunt lib
Split your code, make it readable.
Dont use "static" as much as possible.
Hope it helps
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;
import com.jaunt.*;
public class B03 {
static String currentUrl = "https://stackoverflow.com";
static String stayAt = currentUrl;
static String searchingTerm = "";
static int toSearch = 50;
static List<String> toVisit = new ArrayList<String>();
static List<String> visited = new ArrayList<String>();
static UserAgent userAgent = new UserAgent();
public static void main(String[] args) {
System.out.println("*started searching...*");
toVisit.add(currentUrl);
while(toVisit.size() > 0 && visited.size() < toSearch){
visitUrl(toVisit.get(0));
}
System.out.println("\n\n*done*\n\n");
}
public static void visitUrl(String url) {
List<String> ee = evaluateUrls(url);
searchTerm(url);
visited.add(url);
toVisit.remove(url);
toVisit.addAll(ee.stream().filter( e -> !visited.contains(e)).collect(Collectors.toList()));
toVisit.remove(0);
}
public static void searchTerm(String currentUrl) {
//if(userAgent.doc.getTextContent().contains(searchingTerm))
System.out.println(visited.size() +") "+ currentUrl);
}
public List<String> evaluateUrls(String currentUrl) {
List<String> subUrls = new ArrayList<>();
try {
Elements temp = userAgent.visit(currentUrl).findEvery("<a href>");
for (Element e : temp) {
String x = e.getAt("href");
subUrls.add(x);
}
}catch (Exception e) {
System.out.println(e);
}
return subUrls;
}
}
I Have a multithreaded environment in android app. I use a singleton class to store data. This singleton class contains a arraylist that is accessed using a synchronized method.
The app uses this arraylist to render images in app.
Initial problem : Concurrent modification error use to come so I made the get arraylist function syncronized.
Current Problem:Concurrent modification error not coming but in between empty arraylist returned (maybe when there is concurrent access).
Objective : I want to detect when Concurrent modification so that Instead of empty arraylist being return I can return last state of the arraylist.
public synchronized List<FrameData> getCurrentDataToShow() {
List<FrameData> lisCurrDataToShow = new ArrayList<FrameData>();
//for (FrameData fd : listFrameData) {//concurrent modification exception
//todo iterator test
Iterator<FrameData> iterator = listFrameData.iterator();
while (iterator.hasNext()) {
FrameData fd = iterator.next();
long currentTimeInMillis = java.lang.System.currentTimeMillis();
if ((currentTimeInMillis > fd.getStartDate().getTime() && currentTimeInMillis < fd.getEndDate().getTime()) || (fd.isAllDay() && DateUtils.isToday(fd.getStartDate().getTime()))) {
if (new File(ImageFrameActivity.ROOT_FOLDER_FILES + fd.getFileName()).exists()) {
lisCurrDataToShow.add(fd);
}
}
}
if (lisCurrDataToShow.size() == 0) {
lisCurrDataToShow.add(new FrameData(defaultFileName, null, null, null, String.valueOf(120), false));
}
return lisCurrDataToShow;
}
Referred to Detecting concurrent modifications?
Please help!
EDIT1:
This problem occurs rarely not everytime.
If a threads is accessing getCurrentDataToShow() and another thread tries to access this function what will the function return?? I'm new to multithreading , please guide
Edit 2
in oncreate following methods of singleton are called periodically
DataModelManager.getInstance().getCurrentDataToShow();
DataModelManager.getInstance().parseData(responseString);
Complete singleton class
public class DataModelManager {
private static DataModelManager dataModelManager;
private ImageFrameActivity imageFrameAct;
private String defaultFileName;
public List<FrameData> listFrameData = new ArrayList<FrameData>();
// public CopyOnWriteArrayList<FrameData> listFrameData= new CopyOnWriteArrayList<FrameData>();
private String screensaverName;
private boolean isToDownloadDeafultFiles;
private String tickerMsg = null;
private boolean showTicker = false;
private boolean showHotspot = false;
private String hotspotFileName=null;
public String getDefaultFileName() {
return defaultFileName;
}
public boolean isToDownloadDeafultFiles() {
return isToDownloadDeafultFiles;
}
public void setToDownloadDeafultFiles(boolean isToDownloadDeafultFiles) {
this.isToDownloadDeafultFiles = isToDownloadDeafultFiles;
}
private String fileNames;
private DataModelManager() {
}
public static DataModelManager getInstance() {
if (dataModelManager == null) {
synchronized (DataModelManager.class) {
if (dataModelManager == null) {
dataModelManager = new DataModelManager();
}
}
}
return dataModelManager;
}
private synchronized void addImageData(FrameData frameData) {
//Log.d("Frame Data","Start date "+frameData.getStartDate()+ " " +"end date "+frameData.getEndDate());
listFrameData.add(frameData);
}
public synchronized void parseData(String jsonStr) throws JSONException {
listFrameData.clear();
if (jsonStr == null) {
return;
}
List<String> listFileNames = new ArrayList<String>();
JSONArray jsonArr = new JSONArray(jsonStr);
int length = jsonArr.length();
for (int i = 0; i < length; i++) {
JSONObject jsonObj = jsonArr.getJSONObject(i);
dataModelManager.addImageData(new FrameData(jsonObj.optString("filename", ""), jsonObj.optString("start", ""), jsonObj.optString("end", ""), jsonObj.optString("filetype", ""), jsonObj.optString("playTime", ""), jsonObj.optBoolean("allDay", false)));
listFileNames.add(jsonObj.optString("filename", ""));
}
fileNames = listFileNames.toString();
}
public void setDefaultFileData(String jsonStr) throws JSONException {
JSONObject jsonObj = new JSONObject(jsonStr);
defaultFileName = jsonObj.optString("default_image", "");
screensaverName = jsonObj.optString("default_screensaver ", "");
}
#Override
public String toString() {
return fileNames.replace("[", "").replace("]", "") + "," + defaultFileName + "," + screensaverName;
}
public FrameData getFrameData(int index) {
return listFrameData.get(index);
}
public synchronized List<FrameData> getCurrentDataToShow() {
List<FrameData> lisCurrDataToShow = new ArrayList<FrameData>();
// for (FrameData fd : listFrameData) {//concurrent modification exception
//todo iterator test
Iterator<FrameData> iterator = listFrameData.iterator();
while (iterator.hasNext()) {
FrameData fd = iterator.next();
long currentTimeInMillis = java.lang.System.currentTimeMillis();
if ((currentTimeInMillis > fd.getStartDate().getTime() && currentTimeInMillis < fd.getEndDate().getTime()) || (fd.isAllDay() && DateUtils.isToday(fd.getStartDate().getTime()))) {
if (new File(ImageFrameActivity.ROOT_FOLDER_FILES + fd.getFileName()).exists()) {
lisCurrDataToShow.add(fd);
}
}
}
if (lisCurrDataToShow.size() == 0) {
lisCurrDataToShow.add(new FrameData(defaultFileName, null, null, null, String.valueOf(120), false));
}
return lisCurrDataToShow;
}
public String getCurrentFileNames() {
String currFileNames = "";
List<FrameData> currFrameData = getCurrentDataToShow();
for (FrameData data : currFrameData) {
currFileNames += "," + data.getFileName();
}
return currFileNames;
}
public ImageFrameActivity getImageFrameAct() {
return imageFrameAct;
}
public void setImageFrameAct(ImageFrameActivity imageFrameAct) {
this.imageFrameAct = imageFrameAct;
}
}
This is the only part of your question that is currently answerable:
If a threads is accessing getCurrentDataToShow() and another thread tries to access this function what will the function return?
It depends on whether you are calling getCurrentDataToShow() on the same target object; i.e. what this is.
If this is the same for both calls, then the first call will complete before the second call starts.
If this is different, you will be locking on different objects, and the two calls could overlap. Two threads need to lock the same object to achieve mutual exclusion.
In either case, this method is not changing the listFrameData collection. Hence it doesn't matter whether the calls overlap! However, apparently something else is changing the contents of the collection. If that code is not synchronizing at all, or if it is synchronizing on a different lock, then that could be a source of problems.
Now you say that you are not seeing ConcurrentModificationException's at the moment. That suggests (but does not prove) that there isn't a synchronization problem at all. And that suggests (but does not prove) that your current problem is a logic error.
But (as I commented above) there are reasons to doubt that the code you have shown us is an true reflection of your real code. You need to supply an MVCE if you want a more definite diagnosis.
I am writing a basic program to get the school website of many high schools, but for some reason the program randomly started giving me NullPointerExceptions. By randomly I mean, I tested a set of code that I know worked 100% before this started happening and it no longer works. It is also random in the sense that sometimes the program will return 5 schools and other times it will get a NullPointerException at the first school. What I have currently is shown below and the NullPointer exception happens at System.out.println() and pw.write(). I feel like this means the query isn't returning anything, but it was just a few moments ago.
public static void main(String[] args) throws IOException {
File infile = new File("inputschools.txt");
Scanner in = new Scanner(infile);
PrintWriter pw = new PrintWriter("outputschools.txt");
while(in.hasNextLine()){
String address = "http://ajax.googleapis.com/ajax/services/search/web?v=1.0&start=1&q=";
String query = in.nextLine() + " high school ohio";
String charset = "UTF-8";
URL url = new URL(address + URLEncoder.encode(query, charset));
Reader reader = new InputStreamReader(url.openStream(), charset);
GoogleResults results = new Gson().fromJson(reader, GoogleResults.class);
System.out.println("URL: " + results.getResponseData().getResults().get(0).getUrl() + "\n");
pw.write(results.getResponseData().getResults().get(0).getUrl() + ",");
}
}
}
EDIT: This is the stack trace as requested, but I don't think it will prove very useful.
Exception in thread "main" java.lang.NullPointerException
at GoogleSearch.main(GoogleSearch.java:32)
Line 32 is the println() segment.
EDIT2:
This is the code for the actual GoogleResults object. I stepped through the program and for some reason the GoogleResult is null in the second iteration. Can anyone see/think of why this is happening?
class GoogleResults{
private ResponseData responseData;
public ResponseData getResponseData() { return responseData; }
public void setResponseData(ResponseData responseData) { this.responseData = responseData; }
public String toString() { return "ResponseData[" + responseData + "]"; }
static class ResponseData {
private List<Result> results;
public List<Result> getResults() { return results; }
public void setResults(List<Result> results) { this.results = results; }
public String toString() { return "Results[" + results + "]"; }
}
static class Result {
private String url;
private String title;
public String getUrl() { return url; }
public String getTitle() { return title; }
public void setUrl(String url) { this.url = url; }
public void setTitle(String title) { this.title = title; }
public String toString() { return "Result[url:" + url +",title:" + title + "]"; }
}
}
This question already has answers here:
How to parse a dynamic JSON key in a Nested JSON result?
(5 answers)
Closed 7 years ago.
I have been looking for parsing JSON data in java/android. unfortunately, there is no JSON that same as mine. i have JSON data that include weird number, looks like :
{
"formules": [{"1":
{
"formule": "Linear Motion",
"url": "qp1"
},"2":
{
"formule": "Constant Acceleration Motion",
"url": "qp2"
},"3":
{
"formule": "Projectile Motion",
"url": "qp3"
}
}
]
}
Please help me how to parse this in Java/android. Thanks
try this
JSONObject jsonObject = new JSONObject(string);
JSONArray jsonArray = jsonObject.getJSONArray("formules");
JSONObject jsonObject1 = jsonArray.getJSONObject(0);
Now you can access object "1" as
JSONObject json = jsonObject1.getJSONObject("1");
or use iterator to iterate as below
Iterator keys = jsonObject1.keys();
while(keys.hasNext()) {
// loop to get the dynamic key
String currentDynamicKey = (String)keys.next();
JSONObject json = jsonObject1.getJSONObject(currentDynamicKey);
}
let me know if it works
For parsing Json in Android, I have found the Gson Library to be helpful
http://mvnrepository.com/artifact/com.google.code.gson/gson/2.3
What it would require is creating a pojo class that represents your object. Might look something like
public class ClassPojo
{
private Formules[] formules;
public Formules[] getFormules ()
{
return formules;
}
public void setFormules (Formules[] formules)
{
this.formules = formules;
}
#Override
public String toString()
{
return "ClassPojo [formules = "+formules+"]";
}
}
public class Formules
{
private Formule 3;
private Forumle 2;
private Formule 1;
}
public class Formule
{
private String formule;
private String url;
public String getFormule ()
{
return formule;
}
public void setFormule (String formule)
{
this.formule = formule;
}
public String getUrl ()
{
return url;
}
public void setUrl (String url)
{
this.url = url;
}
#Override
public String toString()
{
return "ClassPojo [formule = "+formule+", url = "+url+"]";
}
}
then to convert it to and from JSon,you could use
//Convert to JSON
ClassPojo pojo = new ClassPojo();
Gson gson = new Gson();
String json = gson.toJson(pojo);
//COnvert back to Java object
ClassPojo pojo = gson.fromJson(json,ClassPojo.class);
I need to retrieve the host name from the address bar using java code. For example, if "www.stackoverflow.com/questions", is there in the address bar, I have to get "www.stackoverflow.com". Please help me.
If you know a bit about format, it can be done very easily with this code :
String http = "www.stackoverflow.com/questions";
String url = http.substring(0, http.indexOf("/"));
System.out.println(url);
http = "http://www.stackoverflow.com/questions";
String nohttp = http.substring(7, http.length());
url = nohttp.substring(0, nohttp.indexOf("/"));
System.out.println(url);
Or you can use some "nicer" approach with methods :
public static void main(String[] args) {
System.out.println(getHostname("www.stackoverflow.com/questions"));
System.out.println(getHostname("http://www.stackoverflow.com/questions"));
}
public static boolean isHttp(String s){
if (s.indexOf("http://") == 0){
return true;
} else {
return false;
}
}
public static String getHostname(String url){
String nativeUrl = url;
if (isHttp(nativeUrl)){
url = url.substring(7);
}
url = url.substring(0, url.indexOf("/"));
if (isHttp(nativeUrl)){
url = "http://" + url;
}
return url;
}
Output
www.stackoverflow.com
http://www.stackoverflow.com