As a step in auto-filling some fields in a flow I need to scrape a URL that has its interesting content filled dynamically by JavaScript.
Based on a cars license plates I can look up information on a government website, set an option and a value and submit.
The resulting page will hold the desired content in different tabs so I'll also have to navigate those. (Sorry cant give direct link to this but selecting "Registreringsnummer", using "YN28579" as the value and pressing "Søg" will take you to the page.)
I've done this with a WebViewClient in another Activity so it is possible to navigate the website directly on the android phone/tablet. But in this other Activity I don't want to display the resulting page just scrape some of the data items.
Here is how I've done it with the WebViewClient:
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.webview);
((Button)findViewById(R.id.btnBack)).setVisibility(View.VISIBLE);
wv = (WebView) findViewById(R.id.wv);
reg = getIntent().getStringExtra("reg");
wv.getSettings().setJavaScriptEnabled(true);
wv.setWebViewClient(new WebViewClient() {
private ProgressDialog pd;
private int count = 0;
#Override
public void onPageFinished(WebView view, String url) {
if (count==1) {
view.loadUrl("javascript:document.getElementById('regnr').checked=true;"
+"document.getElementById('soegeord').value='"+reg+"';"
+"document.getElementById('searchForm').submit();"
+"DMR.WaitForLoad.on();");
} else if (count>=2) {
view.loadUrl("javascript:document.body.innerHTML " +
"= '<div class=\"tabNav\">'+document.getElementsByClassName('tabNav')[0].innerHTML+'</div>';" +
"document.getElementsByClassName('h-tab-content')[0].style.width='320px';" +
"document.getElementsByClassName('h-tab-btns')[0].style.width='320px';" +
"document.getElementsByClassName('h-tab-btns')[0].style.height='45px';" +
"document.getElementsByTagName('ul')[0].style.display='inline';" +
"document.head.appendChild='<meta name=\"viewport\" content=\"width=device-width\">';" +
"document.body.style.minWidth ='300px';");
if (pd!=null) {
pd.dismiss();
}
view.setVisibility(View.VISIBLE);
}
}
#Override
public void onPageStarted(WebView view, String url, Bitmap favicon) {
if (pd==null || !pd.isShowing()) {
pd = ProgressDialog.show(SkatActivity.this, "cars.dk", "Vent venligst...", true, false);
}
count++;
}
#Override
public boolean shouldOverrideUrlLoading(WebView view, String url) {
view.loadUrl(url);
return true;
}
});
wv.loadUrl("https://motorregister.skat.dk/dmr-front/appmanager/skat/dmr?_nfpb=true&_nfpb=true&_pageLabel=vis_koeretoej_side&_nfls=false");
}
I can't seem to figure out how to incorporate this in
Do you set the WebView as invisible? Found this question that seems to be working along those same lines, but I can't figure out how to get the working WebViewClient hidden but usable inside this other activity?
Related
I want to extract the html code from WebView and put it in TextView. I search it in internet and I found this code.
/* An instance of this class will be registered as a JavaScript interface */
class MyJavaScriptInterface
{
private TextView textview1;
public MyJavaScriptInterface(TextView textview1)
{
textview1 = textview1;
}
#SuppressWarnings("unused")
public void processContent(String aContent)
{
final String content = aContent;
textview1.post(new Runnable()
{
public void run()
{
textview1.setText(content);
}
});
}
}
webview1.getSettings()
.setJavaScriptEnabled(true);
webview1.addJavascriptInterface
(new MyJavaScriptInterface
(textview1), "INTERFACE");
webview1.loadUrl
("http://blog.depauptits.nl/?m=1");
webview1.setWebViewClient(new
WebViewClient()
{
#Override public boolean ...
shouldOverrideUrlLoading(WebView
view, String url)
{
return true; }
#Override public void onPageStarted
(WebView view, String url, Bitmap favicon)
{ } public void
onPageFinished(WebView view, String url) {
webview1.loadUrl("javascript:window
.LOADHTML.processHTML
('<html>'+document
.getElementsByTagName('html')
[0].innerHTML+'</html>');"); } });
The code is working. The app is running the WebView load its URL, but the TextView doesn't display the html code.
I'm asking your help please correct my code.
you can use jsoup dependency for fetch html from url
this is the implementation of build.gradle :-
implementation 'org.jsoup:jsoup:1.11.3'
String url = "http://www.google.com";
Document document = Jsoup.connect(url).get();
by this way you can get the document object that hold the full html along with ids and tags and all.
for more you can refer :
https://jsoup.org/
This is the tutorial you can refer :
https://www.tutorialspoint.com/jsoup/jsoup_load_url.htm
Basically, i want to fill a field on the paypal register form.
Here's my code:
final String paypal_url = "https://www.paypal.com/welcome/signup/#/email_password";
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
fillField();
}
private void fillField() {
WebView webView = (WebView) findViewById(R.id.webview);
webView.getSettings().setDomStorageEnabled(true);
webView.getSettings().setJavaScriptEnabled(true);
webView.loadUrl(paypal_url);
webView.setWebViewClient(new WebViewClient() {
#Override
public void onPageFinished(WebView view, String url) {
super.onPageFinished(view, url);
view.loadUrl("javascript:document.getElementById('paypalAccountData_firstName').value = 'test';");
}
});
}
And this is the result: it loads the page and then displays this:
I had already tried almost every solution on this site, without success (the field was not filled).
So, the problem was Paypal's redirection. The code Paul posted works, but only if it is loaded the US version of the site.
Changes:
final String paypal_url = "https://www.paypal.com/us/signup/account";
And
view.loadUrl( "javascript:window.onload= (function(){ document.getElementById('paypalAccountData_firstName').value = 'test';})();");
You need to wait the entire web page loads, including its content (id="paypalAccountData_firstName")
So you should change
view.loadUrl("javascript:document.getElementById('paypalAccountData_firstName').value = 'test';");
to
view.loadUrl( "javascript:window.onload= (function(){ document.getElementById('paypalAccountData_firstName').value = 'test';})();");
I am trying to make an android app from my website using webview such that I want a certain links in my website to open in landscape mode and others in portrait mode inside my webview. For this purpose, I have implemented below code inside the overridden function shouldOverrideUrlLoading which does the work, but not up to the mark. What this code does is, on clicking the landscape links the first time, it changes orientation of the present page to landscape mode but does not go to the website link and if I click it a second time when the current page is already in landscape mode, then it goes to the website link in landscape mode. This does the work, but it is very irritating because the link positions get changed in landscape mode and user will have to again search the link by scrolling and if the user clicks a portrait mode link during the second time, the present page turns to portrait mode but does not go to the website link. In short, the activity restarts due to orientation change, but the weird thing is this happens irrespective of the order of loadUrl and setRequestedOrientation functions and makes no difference on placing sleep ( for say 5 seconds ) before or after setRequestedOrientation, in either cases it sleeps ( for say 5 seconds ) first and then changes the orientation on first click and on second click, it sleeps first ( for say 5 seconds ) and then loads the url link and I am unable to understand the reason behind this strange behaviour ( Same feeling which quantum mechanics scientists experience ). I have tried everything like giving below code in AndroidManifest file :
android:configChanges="keyboard|keyboardHidden|orientation|screenSize"
And also tried overriding below functions in MainActivity file :
#Override
protected void onSaveInstanceState(Bundle outState )
{
super.onSaveInstanceState(outState);
mywebView.saveState(outState);
}
#Override
protected void onRestoreInstanceState(Bundle savedInstanceState)
{
super.onRestoreInstanceState(savedInstanceState);
mywebView.restoreState(savedInstanceState);
}
#Override
public void onConfigurationChanged(Configuration newConfig){
super.onConfigurationChanged(newConfig);
}
Experts please help in resolving this orientation change on first click only.
#Override
public boolean shouldOverrideUrlLoading(WebView view, String url) {
view.loadUrl(url);
/* try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}*/
if(url.contains("<<some keywords>>")){
setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE);
}
else{
setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_PORTRAIT);
}
return true;
}
Please try this code ! i have implemented shouldOverrideUrlLoading,onPageStarted ,onPageFinished .Screen Orientation is changed in onPageStarted if any condition satisfy .
public class WebViewActivity extends AppCompatActivity
{
private WebView mywebView;
private ProgressBar myprogressBar;
private static final String TAG = "WebViewActivity";
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_web_view);
myprogressBar = findViewById(R.id.myProgressBar);
myprogressBar.setVisibility(View.GONE);
mywebView = findViewById(R.id.myWebView);
mywebView.getSettings().setLoadsImagesAutomatically(true);
mywebView.getSettings().setJavaScriptEnabled(true);
mywebView.setScrollBarStyle(View.SCROLLBARS_INSIDE_OVERLAY);
mywebView.setWebViewClient(new MyBrowser());
mywebView.loadUrl("https://stackoverflow.com/");
}
#Override
protected void onSaveInstanceState(Bundle outState) {
super.onSaveInstanceState(outState);
mywebView.saveState(outState);
}
#Override
protected void onRestoreInstanceState(Bundle savedInstanceState) {
super.onRestoreInstanceState(savedInstanceState);
mywebView.restoreState(savedInstanceState);
}
#Override
public void onConfigurationChanged(Configuration newConfig) {
super.onConfigurationChanged(newConfig);
}
private class MyBrowser extends WebViewClient {
#Override
public boolean shouldOverrideUrlLoading(WebView view, String url) {
Log.d(TAG, "shouldOverrideUrlLoading: loading ");
myprogressBar.setVisibility(View.VISIBLE);
view.loadUrl(url);
return true;
}
#Override
public void onPageStarted(WebView view, String url, Bitmap favicon)
{
Log.d(TAG, "onPageStarted: started");
myprogressBar.setVisibility(View.VISIBLE);
/*
here you have to include the your keywords instead of tags [hardcoded string]
*/
if (url.contains("tags")) {
setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE);
} else {
setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_PORTRAIT);
}
super.onPageStarted(view, url, favicon);
}
#Override
public void onPageFinished(WebView view, String url) {
myprogressBar.setVisibility(View.GONE);
Log.d(TAG, "onPageFinished: finished");
super.onPageFinished(view, url);
}
}
}
I'm building a simple app, in which I use a webview in Android Studio, in the app, i search a movie, and then I download the torrent.
When I download in the web, it automatically send me to the utorrent app, but when I download it from my app, I can't open it.
Sorry for my English and for my poor knowledge.
My java code:
public class MainActivity extends AppCompatActivity {
WebView web;
EditText et_peli;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
web=(WebView)findViewById(R.id.web);
et_peli=(EditText)findViewById(R.id.et_peli);
}
public void buscar0nclick(View v){
WebSettings conf = web.getSettings();
conf.setJavaScriptEnabled(true);
web.loadUrl("http://www.miltorrents.com/?pTit=" + et_peli.getText().toString());
web.setWebViewClient(new WebViewClient() {
#Override
public void onReceivedError(WebView view, int errorCode,
String description, String failingUrl) {
Log.d("WEB_VIEW_TEST", "error code:" + errorCode + " - " + description);
}
#Override
public boolean shouldOverrideUrlLoading(WebView view, String url) {
if (url.endsWith(".torrent")) {
Uri source = Uri.parse(url);
DownloadManager.Request request = new DownloadManager.Request(source);
request.setDescription("Description for the DownloadManager Bar");
request.setTitle("YourApp");
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.HONEYCOMB) {
request.allowScanningByMediaScanner();
request.setNotificationVisibility(DownloadManager.Request.VISIBILITY_VISIBLE_NOTIFY_COMPLETED);
}
request.setDestinationInExternalPublicDir(Environment.DIRECTORY_DOWNLOADS, "SmartPigs.apk");
DownloadManager manager = (DownloadManager) getSystemService(Context.DOWNLOAD_SERVICE);
manager.enqueue(request);
}
else view.loadUrl(url);
return true;
}
});
}
}
Just to make sure I have this right, you want to send your user, once they download the torrent to the torrent's app to view the movie, correct?
I would check to see if that torrent app has a broadcast receiver that you can search for. If so, you could just pass along the information. If not, you have to look for a way to have that media file play within your app, or use one of the default movie players.
This is a part of my android app, i have created a webview...but when i click on any link inside the webview...nothing happens....here is my code...i want to launch the link either in any browser or any installed download manager...i m a newbie..please help me out with this
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
// Show the Up button in the action bar.
setupActionBar();
// no need to use title bar
requestWindowFeature(Window.FEATURE_NO_TITLE);
// set webview as main content only
mWeb = new WebView(this);
setContentView(mWeb);
// set Javascript
WebSettings settings = mWeb.getSettings();
settings.setJavaScriptEnabled(true);
// the init state of progress dialog
mProgress = ProgressDialog.show(this, "Loading", "Please wait for a moment...");
// add a WebViewClient for WebView, which actually handles loading data from web
mWeb.setWebViewClient(new WebViewClient() {
// load url
public boolean shouldOverrideUrlLoading(WebView view, String url) {
view.loadUrl(url);
return true;
}
// when finish loading page
public void onPageFinished(WebView view, String url) {
if(mProgress.isShowing()) {
mProgress.dismiss();
}
}
});
// set url for webview to load
mWeb.loadUrl("http://vesit-d7b.host22.com/Assignments/ECCF.html");
}
Please use this code and check that your url is being caught by the webview or not.
myweb.setWebViewClient(new WebViewClient() {
#Override
public void onPageStarted(WebView view, String url,
Bitmap favicon) {
Toast.makeText(this,url,Toast.LENGTH_SHORT).show();
}
#Override
public void onPageFinished(WebView view, String url) {
dialog.dismiss();
}
});
see what url is coming when you click the link.....
mWeb.setWebViewClient(new WebViewClient() {
// load url
public boolean shouldOverrideUrlLoading(WebView view, String url) {
Log.v("here","the link is ::" + url);
view.loadUrl(url);
return true;
}
and then copy it on browser to see whether it is a valid link or not..
#Ritesh remove the shouldOverrideUrlLoading method it will work