website looks somewhat like this:
dynamic webpage
i need to copy all text from a dynamic webpage[keeps changeing for each second like there is a countdown/loading] for each second to a string
previously i tried this code,
private class BackTask extends AsyncTask<String, Integer, String> {
#Override
protected void onPreExecute() {}
protected String doInBackground(String... address) {
String output = "";
try {
java.net.URL url = new java.net.URL(address[0]);
java.io.BufferedReader in = new java.io.BufferedReader(new java.io.InputStreamReader(url.openStream()));
String line;
while ((line = in.readLine()) != null) {
output += line;
}
in.close(); } catch (java.net.MalformedURLException e) {
output = e.getMessage();
} catch (java.io.IOException e) {
output = e.getMessage();
} catch (Exception e) {
output = e.toString();
}
return output;
}
protected void onProgressUpdate(Integer... values) {}
protected void onPostExecute(String s){
textview1.setText(s); }
it only gives source code of webpage
and it only works on static webpage
what that code gives me is :
<html>
<body>
<p id="demo"></p>
<script>
var myVar = setInterval(myTimer, 1000);
function myTimer() {
var d = new Date();
document.getElementById("demo").innerHTML = d.toLocaleTimeString();
}
</script>
</body>
</html>
but, what i need i need is just
17:07:24
i know Timer on Java.Utils
but what i need is just to copy all text from a webview/webpage and assign it to a string,
like, go to a random webpage > SELECT ALL > COPY > and PASTE it on notepad [manually]
Two ways you can render values on a webpage
Assign backend values to a JavaScript object or use expression language.
For reference go throw this link https://www.baeldung.com/spring-expression-language
Related
My plan is do analyze an image via tensorflow and send the result to an app. Der result is written in a json file. The json file looks like that:
{"file": "image.jpg", "objects": [{"bbox": [177, 14, 508, 773], "label": "spanishguitar", "prob": 0.7284}]
my class in android studio is:
`
private class Downloadjson extends AsyncTask<Void, Void, Void> {
String name;
String label;
public Downloadjson(String name) {
this.name = name;
}
#Override
protected Void doInBackground(Void... params) {
String url = SERVER_ADRESS + name;
sleep(5000);
//animation start
for (int i = 0; i < 600; i++) {
HttpClient client = new DefaultHttpClient(getHttpRequestParams());
HttpGet getJson = new HttpGet(url);
try {
HttpResponse jsonResponse = client.execute(getJson);
if (200 == jsonResponse.getStatusLine().getStatusCode()) {
try {
InputStream inputStream = jsonResponse.getEntity().getContent();
String json = IOUtils.toString(inputStream);
Downloadjson downloadjson = new Gson().fromJson(json, Downloadjson.class);
String label = downloadjson.label;
TextView Result = (TextView) findViewById(R.id.textView);
Result.setText("Your instrument could be a " + downloadjson.label);
} catch (Exception e) {
e.printStackTrace();
}
}
} catch (Exception e) {
e.printStackTrace();
}
//exucte
sleep(5000);
}
//stop animation
return null;
}
`
When I change "label" to "file" it works and the output is "image.jpg". But i need the "label" array with the output "spanishguitar". So, I think the problem is, that the object is not "label" but "objects" (in my json file). And label is an array? Do you know what I have to change to get the "label" ?
As I understand, you just need to properly traverse the json and assign value for label. Please try:
String label = downloadjson.objects[0].label;
I am trying to call another async task inside an OnPostExecute. The 2nd task does not run at all it seems. I am unable to print anything from within to the logs.
#Override
protected void onPostExecute(String result) {
super.onPostExecute(result);
try {
JSONObject json = new JSONObject(result);
JSONArray lignes = json.getJSONArray("lignes");
populatelist(lignes);
}
catch (JSONException e) {
}
}
}
The populatelist function fills an array. Inside this function, I try to call the 2nd async task to get values based on this list.
protected void populatelist(JSONArray lignes){
try {
for(int i=0;i<lignes.length(); i++) {
JSONObject jsonas = lignes.getJSONObject(i);
String fdesignation = jsonas.getString("designation");
String fqtecde = jsonas.getString("qtecde");
String fcode_produit = jsonas.getString("code");
InfoStock(fcode_produit);
items.add(new PickingListProduitItem(fdesignation,"",fqtecde, ""));
}
}
catch(Exception e){
}
}
InfoStock() is the function that is used to return additional from a web service.
protected void InfoStock(String code_produit){
String stockURL = "http://" + mSharedPreferences.getString(Constants.SERVER_IP,"")+"//rest/v2/produit/info/code/"+ code_produit + "?stock=true";
try {
if (mDownloader != null && mDownloader.getStatus() == AsyncTask.Status.RUNNING) {
mDownloader.cancel(true);
mPDialog.dismiss();
}
mPDialog = new ProgressDialog(getApplicationContext());
mDownloader = new XMLDownloader(getApplicationContext(),mPDialog);
byte[][] downloadResults = mDownloader.execute(stockURL).get();
// Read stock info.
String s = new String(downloadResults[0], StandardCharsets.UTF_8);
JSONObject resp = new JSONObject(s);
PrixStockJSONParser psj = new PrixStockJSONParser(resp);
mRepInfoStock = psj.getRepInfoStock();
mRepInfoPrix = psj.getRepInfoPrix();
} catch (Exception ex) {
}
}
I am trying to set a value in the array <> created in the OnPostExecute Method. However there is no error message and the array is not populated at all. Even if I try to print a log from the InfoStock function, it does nothing.
Any suggestions are welcome.
Below is my code, lat_val and long_val is not getting updated with received value from JSON response in btnShowLoc(), it is referencing to the default value which is 0,0. I want the global variable to keep updating when ever referenced and updated with JSON response.
public class MainActivity extends Activity {
public static String lat_val = "0";
public static String long_val = "0";
public String readJSONFeed(String urlStr) {
StringBuilder stringBuilder = new StringBuilder();
try {
URL url = new URL(urlStr);
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setRequestProperty("SisApiKey", "4572c3c9-73cb-4958-9649-26c1e8df27e8");
urlConnection.setRequestProperty("SisSmartKey", "d1aebd25-774c-4e8a-b3a5-ee5a603cc603");
InputStream ins = urlConnection.getInputStream();
urlConnection.connect();
int statusCode = urlConnection.getResponseCode();
if (statusCode == 200) {
BufferedReader br = new BufferedReader(new InputStreamReader(ins));
String line;
while ((line = br.readLine()) != null) {
stringBuilder.append(line);
}
ins.close();
} else {
Log.d("JSON", "Failed to download file");
}
} catch (java.net.MalformedURLException e) {
e.printStackTrace();
} catch (java.io.IOException e) {
e.printStackTrace();
} catch (Exception e) {
Log.d("readJSONFeed", e.getLocalizedMessage());
}
return stringBuilder.toString();
}
public class ReadJSONFeedTask extends AsyncTask
<String, Void, String> {
protected String doInBackground(String... url) {
return readJSONFeed(url[0]);
}
protected void onPostExecute(String result) {
try {
JSONObject jsonObject = new JSONObject(result);
//JSONObject flags = new JSONObject(jsonObject.getString("flag"));
JSONObject locationItems = new JSONObject(jsonObject.getString("response"));
//Log.v("Location Details :", locationItems.toString());
String []dev_loc = locationItems.toString().split("[\\s*,\\s*]");
MainActivity.lat_val = dev_loc[0]; //"12.9934136";
MainActivity.long_val = dev_loc[1]; //"80.2464206";
} catch (Exception e) {
Log.d("ReadJSONFeedTask", e.getLocalizedMessage());
}
}
}
public void btnGetDevLoc(View view) {
String sp_val = String.valueOf(spinner1.getSelectedItem());
new ReadJSONFeedTask().execute(
"http://15.153.133.160:21743/sis/sie/api/v1/applications/bb9f05fb-a796-4b75-9db7-c999360ad185/virtualobjects/d77d3905-aa77-41b9-9034-b0052bfde405?secondString=HWE_ASSET_ANDROID"); // + sp_val);
}
public void btnShowLoc(View view) {
//lat_val = "12.9934136";
//long_val = "80.2464206";
Intent in = new Intent(MainActivity.this, MapActivity.class);
Bundle bundle = new Bundle();
bundle.putString("latitude", MainActivity.lat_val);
bundle.putString("longitude", MainActivity.long_val);
in.putExtras(bundle);
startActivity(in);
}
With the few information you have shared, and given that
btnGetDevLoc() and btnShowLoc()are the functions executed when clicked on buttons in the application defined in activity_main.xml
and that
First btnGetDevLoc() is called then btnShowLoc()
the first thing that pops out in my mind is that the AsyncTask has not yet finished updating the String values, when you call btnShowLoc().
So, if btnGetDevLoc() and btnShowLoc() are called sequentially, like
... onClick() {
btnGetDevLoc();
btnShowLoc();
}
then it's most likely due to what I said above. Remember that AsyncTask runs asynchronously (as the name says...).
You can test this really small program.
public static double var1 = 0.0;
public static void main(String[] args) {
new Thread(() -> {
var1 = 1.0;
}).start();
System.out.println(var1);
}
It will almost always print 0.0, because the value of var1 is not updated yet when the main thread prints it.
What you should do is place your btnShowLoc() call at the end of onPostExecute(String). This guarantees that your method is called only after you have updated the new values.
I can't Understand, when the btnGetDevLoc() and btnShowLoc() called? Can you post your whole MainActivity?
Edit :
It's seems like you call btnShowLoc() before your AsyncTask finish its proccess.
You can change your code this way to make sure your btnShowLoc() called after your AsyncTask :
public class ReadJSONFeedTask extends AsyncTask
<String, Void, String> {
protected String doInBackground(String... url) {
return readJSONFeed(url[0]);
}
protected void onPostExecute(String result) {
try {
JSONObject jsonObject = new JSONObject(result);
//JSONObject flags = new JSONObject(jsonObject.getString("flag"));
JSONObject locationItems = new JSONObject(jsonObject.getString("response"));
//Log.v("Location Details :", locationItems.toString());
String []dev_loc = locationItems.toString().split("[\\s*,\\s*]");
MainActivity.lat_val = dev_loc[0]; //"12.9934136";
MainActivity.long_val = dev_loc[1]; //"80.2464206";
btnShowLoc(dev_loc[0], dev_loc[1]);
} catch (Exception e) {
Log.d("ReadJSONFeedTask", e.getLocalizedMessage());
}
}
}
public void btnShowLoc(String latitude, String longitude) {
//lat_val = "12.9934136";
//long_val = "80.2464206";
Intent in = new Intent(MainActivity.this, MapActivity.class);
Bundle bundle = new Bundle();
bundle.putString("latitude", latitude);
bundle.putString("longitude", longitude);
in.putExtras(bundle);
startActivity(in);
}
I developing android app and now I have problem. Below is a part of my code, and it keeps skipping the "for" part. When I put a breakpoint inside for statement, it stops at the point, and executes the lines very well and makes an output that I want. When I just 'run' app, it skips that part so "String locations" value doesn't change. I googled and some say it's thread-related problem. So I put synchroinzed on the method, still not working. Any other suggestions?
UPDATE
I was trying to show code only related to the problem, but I think now showing the whole would be more useful for those who try to help so here's my entire code on showMapActivity. You can see I've tried some ways around and nothing worked. Saving path's information into String url is where I'm having problem. I tested, and other parts seem to work fine. I know my code is really massy, that was why I only posted parts of the code. TMap related classes are imported from .jar file.
public class showMapActivity extends Activity {
TMapData tmapdata=new TMapData();
TMapView tmapView;
TMapPoint origin, dest;
volatile ArrayList<TMapPoint> points=new ArrayList<>();
private TextView x;
private TextView y;
private HashMap<String,LatLng> coordinates;
private HashMap<LatLng,Double> finalpoint;
static private ConcurrentHashMap<Double,Double> path;
#Override
protected void onCreate(Bundle savedInstanceState){
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_show_map);
coordinates=new HashMap<>();
Intent intent=getIntent();
tmapView=new TMapView(this);
path=new ConcurrentHashMap<>();
coordinates=(HashMap<String,LatLng>)intent.getSerializableExtra("coordinate");
path=getPathPoints(coordinates);
int i=0;
String url=getUrl();
//String url = "https://maps.googleapis.com/maps/api/elevation/json?locations=";
//String locations="";
/*
Iterator<Double> keys= path.keySet().iterator();
while(keys.hasNext()){
Double key=keys.next();
//String lat=String.valueOf(key);
//String lng=String.valueOf(path.get(key));
locations=locations+String.valueOf(key)+","+String.valueOf(path.get(key));
if(keys.hasNext())
locations=locations+"|";
}path.entrySet()
*/
/*
for(ConcurrentHashMap.Entry<Double,Double> elem : path.entrySet())
{
String lat=String.valueOf(elem.getKey());
String lng=String.valueOf(elem.getValue());
locations=locations+lat+","+lng;
i++;
if(i!=path.size())
{
locations=locations+"|";
}
}
*/
//url=url+locations+"&key=AIzaSyDD88VFMPIfC5sr0XsFL0PDCE-QRN8gQto";
//String url=getUrl(path);
FetchUrl fetchUrl=new FetchUrl();
fetchUrl.execute(url);
}
private ConcurrentHashMap<Double,Double> getPathPoints(HashMap<String,LatLng> coordinates)
{
final ConcurrentHashMap<Double,Double> Path=new ConcurrentHashMap<>();
tmapView.setSKPMapApiKey("6bb5b7f3-1274-3c5e-ba93-790aee876673");
origin=new TMapPoint(coordinates.get("origin").latitude,coordinates.get("origin").longitude);
dest=new TMapPoint(coordinates.get("dest").latitude,coordinates.get("dest").longitude);
tmapdata.findPathData(origin, dest, new TMapData.FindPathDataListenerCallback() {
#Override
public void onFindPathData(TMapPolyLine polyLine) {
points=polyLine.getLinePoint();
for(TMapPoint point : points )
Path.put(point.getLatitude(),point.getLongitude());
}
});
return Path;
}
//ConcurrentHashMap<Double,Double> path
private synchronized String getUrl() {
int i=0;
String url = "https://maps.googleapis.com/maps/api/elevation/json?locations=";
String locations="";
for(HashMap.Entry<Double,Double> elem : path.entrySet())
{
String lat=String.valueOf(elem.getKey());
String lng=String.valueOf(elem.getValue());
locations=locations+lat+","+lng;
i++;
if(i!=path.size())
{
locations=locations+"|";
}
}
url=url+locations+"&key=AIzaSyDD88VFMPIfC5sr0XsFL0PDCE-QRN8gQto";
//https://maps.googleapis.com/maps/api/elevation/json?locations=
// 39.7391536,-104.9847034|36.455556,-116.866667&key=AIzaSyDD88VFMPIfC5sr0XsFL0PDCE-QRN8gQto
// Output format
return url;
}
private class FetchUrl extends AsyncTask<String, Void, String> {
#Override
protected String doInBackground(String... url) {
// For storing data from web service
String data = "";
try {
// Fetching the data from web service
//downloadURL
data = downloadUrl(url[0]);
Log.d("Background Task data", data.toString());
} catch (Exception e) {
Log.d("Background Task", e.toString());
}
return data;
}
#Override
protected void onPostExecute(String result) {
super.onPostExecute(result);
//ParserTask
ParserTask parserTask = new ParserTask();
// Invokes the thread for parsing the JSON data
parserTask.execute(result);
}
}
private String downloadUrl(String strUrl) throws IOException {
String data = "";
InputStream iStream = null;
HttpURLConnection urlConnection = null;
try {
URL url = new URL(strUrl);
// Creating an http connection to communicate with url
urlConnection = (HttpURLConnection) url.openConnection();
// Connecting to url
urlConnection.connect();
//읽은 데이터를 버퍼에 저장
// Reading data from url
iStream = urlConnection.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(iStream));
StringBuffer sb = new StringBuffer();
String line = "";
while ((line = br.readLine()) != null) {
sb.append(line);
}
data = sb.toString();
Log.d("downloadUrl", data.toString());
br.close();
} catch (Exception e) {
Log.d("Exception", e.toString());
} finally {
iStream.close();
urlConnection.disconnect();
}
return data;
}
private class ParserTask extends AsyncTask<String, Integer, ArrayList<Double>> {
// Parsing the data in non-ui thread
#Override
protected ArrayList<Double> doInBackground(String... jsonData) {
JSONObject jObject;
ArrayList<Double> altitude = null;
try {
jObject = new JSONObject(jsonData[0]);
Log.d("ParserTask",jsonData[0].toString());
//DataParser class 호출
DataParser parser = new DataParser();
Log.d("ParserTask", parser.toString());
// Starts parsing data
altitude = parser.parse(jObject);
Log.d("ParserTask","Getting Altitudes");
Log.d("ParserTask",altitude.toString());
} catch (Exception e) {
Log.d("ParserTask",e.toString());
e.printStackTrace();
}
return altitude;
}
// Executes in UI thread, after the parsing process
#Override
protected void onPostExecute(ArrayList<Double> result) {
finalpoint=new HashMap<>();
LatLng latLng;
int i=0;
for(HashMap.Entry<Double,Double> elem : path.entrySet() )
{
latLng=new LatLng(elem.getKey(),elem.getValue());
finalpoint.put(latLng,result.get(i++));
}
x = (TextView) findViewById(R.id.textView5);
y = (TextView) findViewById(R.id.textView6);
x.setText(String.valueOf(finalpoint.get(coordinates.get("origin"))));
y.setText(String.valueOf(finalpoint.get(coordinates.get("dest"))));
}
}
}
(Apologies for posting this as an answer - I don't yet have the required reputation to comment)
Simply adding synchronized to a method doesn't necessarily guarantee thread safety.
How and when is path being populated?
Update after additional information provided
The problem seems to be that the path points are being generated asynchronously, and you are trying to use them before the generation process has finished (or perhaps even started). This happens because the findPathData simply starts the generation process and returns immediately (i.e. before the generation process has finished). In your code, you then go on and build the URL which is supposed to contain the point data immediately. At this point the background point generation process may not have finished, and may not have even started. As a result the point map may be empty or incomplete, and your URL will not be generated as you expect.
You need to find a way to wait until all of the path point data has been returned by the asynchronous processing before creating the URL. This looks like it could be very difficult, if not impossible, with the version of the findPathData method you are using, because it returns points via the callback one at a time and you may not know how many will be generated.
I had a quick look at the API for TMapData and it has a findPathDataAll method which seems to generate all the points and return them in a single callback call rather than one by one. If this is indeed what it does (sorry, I can't read Korean), you could use this method and then generate the URL from the callback, because when it's called you know that the generation process has been completed. If you do this, be careful to make sure that you're on the main thread before interacting with the UI or Activity.
Hope that helps.
I need to convert HTML to plain text. My only requirement of formatting is to retain new lines in the plain text. New lines should be displayed not only in the case of <br> but other tags, e.g. <tr/>, </p> leads to a new line too.
Sample HTML pages for testing are:
http://www.particle.kth.se/~lindsey/JavaCourse/Book/Part1/Java/Chapter09/scannerConsole.html
http://www.javadb.com/write-to-file-using-bufferedwriter
Note that these are only random URLs.
I have tried out various libraries (JSoup, Javax.swing, Apache utils) mentioned in the answers to this StackOverflow question to convert HTML to plain text.
Example using JSoup:
public class JSoupTest {
#Test
public void SimpleParse() {
try {
Document doc = Jsoup.connect("http://www.particle.kth.se/~lindsey/JavaCourse/Book/Part1/Java/Chapter09/scannerConsole.html").get();
System.out.print(doc.text());
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
Example with HTMLEditorKit:
import javax.swing.text.html.*;
import javax.swing.text.html.parser.*;
public class Html2Text extends HTMLEditorKit.ParserCallback {
StringBuffer s;
public Html2Text() {}
public void parse(Reader in) throws IOException {
s = new StringBuffer();
ParserDelegator delegator = new ParserDelegator();
// the third parameter is TRUE to ignore charset directive
delegator.parse(in, this, Boolean.TRUE);
}
public void handleText(char[] text, int pos) {
s.append(text);
}
public String getText() {
return s.toString();
}
public static void main (String[] args) {
try {
// the HTML to convert
URL url = new URL("http://www.javadb.com/write-to-file-using-bufferedwriter");
URLConnection conn = url.openConnection();
BufferedReader reader = new BufferedReader(new InputStreamReader(url.openStream()));
String inputLine;
String finalContents = "";
while ((inputLine = reader.readLine()) != null) {
finalContents += "\n" + inputLine.replace("<br", "\n<br");
}
BufferedWriter writer = new BufferedWriter(new FileWriter("samples/testHtml.html"));
writer.write(finalContents);
writer.close();
FileReader in = new FileReader("samples/testHtml.html");
Html2Text parser = new Html2Text();
parser.parse(in);
in.close();
System.out.println(parser.getText());
}
catch (Exception e) {
e.printStackTrace();
}
}
}
Have your parser append text content and newlines to a StringBuilder.
final StringBuilder sb = new StringBuilder();
HTMLEditorKit.ParserCallback parserCallback = new HTMLEditorKit.ParserCallback() {
public boolean readyForNewline;
#Override
public void handleText(final char[] data, final int pos) {
String s = new String(data);
sb.append(s.trim());
readyForNewline = true;
}
#Override
public void handleStartTag(final HTML.Tag t, final MutableAttributeSet a, final int pos) {
if (readyForNewline && (t == HTML.Tag.DIV || t == HTML.Tag.BR || t == HTML.Tag.P)) {
sb.append("\n");
readyForNewline = false;
}
}
#Override
public void handleSimpleTag(final HTML.Tag t, final MutableAttributeSet a, final int pos) {
handleStartTag(t, a, pos);
}
};
new ParserDelegator().parse(new StringReader(html), parserCallback, false);
I would guess you could use the ParserCallback.
You would need to add code to support the tags that require special handling. There are:
handleStartTag
handleEndTag
handleSimpleTag
callbacks that should allow you to check for the tags you want to monitor and then append a newline character to your buffer.
Building on your example, with a hint from html to plain text? message:
import java.io.*;
import org.jsoup.*;
import org.jsoup.nodes.*;
public class TestJsoup
{
public void SimpleParse()
{
try
{
Document doc = Jsoup.connect("http://www.particle.kth.se/~lindsey/JavaCourse/Book/Part1/Java/Chapter09/scannerConsole.html").get();
// Trick for better formatting
doc.body().wrap("<pre></pre>");
String text = doc.text();
// Converting nbsp entities
text = text.replaceAll("\u00A0", " ");
System.out.print(text);
}
catch (IOException e)
{
e.printStackTrace();
}
}
public static void main(String args[])
{
TestJsoup tjs = new TestJsoup();
tjs.SimpleParse();
}
}
You can use XSLT for this purpose. Take a look at this link which addresses a similar problem.
Hope it is helpful.
I would use SAX. If your document is not well-formed XHTML, I would transform it with JTidy.
JSoup is not FreeMarker (or any other customer/non-HTML tag) compatible. Consider this as the most pure solution for converting Html to plain text.
http://stackoverflow.com/questions/1518675/open-source-java-library-for-html-to-text-conversion/1519726#1519726
My code:
return new net.htmlparser.jericho.Source(html).getRenderer().setMaxLineLength(Integer.MAX_VALUE).setNewLine(null).toString();