How do I perform Web Scraping in Android? [closed] - java

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I want to scrape my website and then use the data from the website to populate elements in my app, my website has login pages and certain pages only open after the login has been done.
I started working with HtmlUnit as it is a headless browser and completed the custom api in a java IDE, later i tried to use the jar i generated from the java IDE and found that there are incompatibility issues with HtmlUnit and Android.
Can anyone propose a solution to this problem?
Edit :
Since no one actually answered this question I am currently going with a work around using android's native WebView, settings its Visibility to invisible and then using javascript interfacing to a Java object, I can inject JS code to scrape any data.

Use Jsoup library for such purpose. Very handy and easy to use.
Start with this answer and follow documents and other examples.

Either you contribute to HtmlUnit to produce a version of HtmlUnit not using the missing dependencies from Android.
Or you can use an alternative method like this one, as this seems to be the path someone else go before you.
If a real headless browser able to manage any recent web features, would exist, it would mean a team would have developed it and then invest much effort in it (in supporting existing and coming features) consistently.
Apart from Opera, Chrome, IE, and Firefox browsers, there is no such team.
I would point out Chromium (CEF) as the most open and actively supported cross language wise. Try Cef for java

Related

How to choose best automation Tool combination with selenium [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
It might be opinion based question but I am really confused about choosing automation tools combination.
Till date I have been using Selenium webdriver,maven and Java to build automation frameworks. This seems totally working fine.
Now a days I feel that many automation engineers moving to use of WebdriverJS , WebdriverIO, NodeJS etc.
Someone told me that if we build automation framework using WebdriverJS/ WebdriverIO/NodeJS, Then execution will be faster than Java.
I know that I should choose tool based on my requirements but at the same time I feel that we can not use WebdriverJS/ WebdriverIO/NodeJS in some kind of website where all elements must be located by xpath as most of elements changing class / id dynamically.
So If anyone can just give me hint about which tools to choose for which kind of website then it would be really helpful. i.e For AngularJS website Node.Js is fine for automation with selenium.
As per the Selenium Official Webpage, Download Section, you can see in the Selenium Client & WebDriver Language Bindings section the following are listed :
Java
C#
Ruby
Python
Javascript (Node)
So from my personal take would be, these 5 variants which are the most widely used Selenium Clients while dealing with Automation Framework with Selenium
Documentation
Leaving aside Selinium's GitHub ducumentation, its pretty clear majority of the documentation available on internet is on Selenium (Java). It's because of Java's world wide popularity as a Language. So learning becomes easier.
Even the frequency of Selenium (Java) Client Releases would support my native view.
But, personally I have found Selinium's GitHub Documentation for all the client bindings are perfect and precise.
WebdriverIO
WebdriverIO is another variant of the Selenium Release. No doubt the WebdriverIO team have played it hard in building up the catch-up game with the Selenium Client Releases. But my take would be WebdriverIO is still evolving.
Functional Perspective:
From a functional perspective my take will be either on NodeJS or WebdriverJS. Reason is simple I still feel the Selenium-RC way is still in practice when we inject javascript into the Web Browser once the browser loads completely and then use its javascript to drive the Application Under Test within the browser.

Alternatives to Java applets [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
I'm planning to write a simple program that displays course prerequisites for students at my university in graph form (ie as a network of vertices and edges). I'd like to embed the program in a webpage to save people the hassle of downloading an executable.
Currently I'm looking at making my program a Java applet (Java also would give me access to the handy Swing library), but I don't like the fact that applets can't be viewed on most mobile devices.
What alternatives to applets exist for a project like this? I'd like to make it compatible with as many devices as possible, and also not have to build the graphics stuff from scratch.
One final consideration is I'm doing this mostly as a learning exercise. Ideally the tools I'd be working with would be helpful to know in the future.
Please don't use applets. They have been sufficiently deprecated.
The best way to do this is by using html/js/css. A lot of useful libraries exist that can help you with this task. jQuery seems obvious, but there's also d3.js or vis.js for displaying visual representations of data, and bootstrap for responsiveness (mobile friendliness).
You may use Angularjs with angular-chart for Showing graph in Web Browser.
If your graph data is dynamic you might use Nodejs and mongoDB for backend.
angular-chart is responsive and its easy to show dynamic graph. But as it uses HTML5 canvas some mobile browsers might not show its transitions smoothly depending on the device.
I personally do not prefer using applet in web browser when the same functionality can be achieved using great frameworks like Angularjs.
why dont you try to build your project through Servlet framework
by the way cgi were removed by servlet because of the handling of the request
applet uses the same concept

Enabling a completed Java app on the web [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I am not a Java pro. I need to enable my completed java applications to ONLINE-Versions. Kindly give me the simplest way.
I have tried to go the JSP and Servlet way, but seems exceedingly tiresome,there surely ought to be a direct way.
When you use a typical web application what do you think is happening? You're in a browser and it's displaying some pages. You click around and some server does some work, for example Amazon shows you a list of books you could buy, the server is passing data to the browser for it to display.
That's a big difference from your current Java Application where everything is happening in the same computer.
This split between the UI part of the application, in the Browser, and the server part that's doing the real business is one reason why Web programming is so different from simple Java programming.
Servlets and JSPs are the traditional Java way of doing things, but modern web apps make much greater use of JavaScript for all the UI and all the Java (or any other server technology) does is provide the data displayed by JavaScript.
Bottom line: Sorry, but there is a whole lot more to learn. My recommendation would be to grit your teeth and forget about what you've done so far and start studying JavaScript and HTML.

Can JSP and Python used together for same database? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I'm a student who work part-time at a start-up, which runs a website made with JSP.
We are planning an android app in this summer and I'm considering developing it with Python, which I'm interested in.
However, the service provided by the app is supposed to be added to the website made with JSP later. I'm afraid the difference of the language would cause any obstacle.
Since they will use a common database, I think using different languages to access it won't have any problem. I want to make sure that my guess is correct.
Pardon my poor English. I'd appreciate your answers.
I would recommend building the API services in the current language used by your start-up. If resources are limited, the company may not wish to maintain services in another language.
Though I would not recommend this, it is definitely possible for you to develop the portions of your Android application in Python and use a different language for the API services (assuming you are willing to use Python libraries like Kivy).
This will allow you to code a chunk of your application logic in Python.
To consider:
http://kivy.org/#home
https://code.google.com/p/android-scripting/
We are planning an android app in this summer and I'm considering
developing it with Python
Native Android apps are developed using Java.
However, the service provided by the app is supposed to be added to
the website made with JSP later. I'm afraid the difference of the
language would cause any obstacle.
You will need to create an API that communicates between Android and your database.

How to make java website [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I want to make HTML website of course using CSS etc. But instead of using php for dynamic functions I would like to use Java. I have heard that there are plenty of web frameworks as Spring etc.
The problem is always when I search for tutorial - How to make java web? It shows me results with - web application, applet, servlet, JSP. I can't find short simple answer explaining how to start with that and what is the right thing to start with.
Would you mind telling me what should I use to make what I want? Just some short answer explaining what is what for. I remind you: I want to do some dynamic stuff on web based on Java instead of PHP.
EDIT:
I know I have to use dedicated server with Tomcat probably I am just looking for way to start. Should I start with the framework or?
I personaly only used Thymeleaf tutorials to learn about java-web and I'm satisfied with them, it has good-explained tutorials but it mostly teaches you how to process html templates with java.
If you do not want to follow thymeleaf tutorials, you should search for JAVA EE or JSP tutorials on either google or youtube, both provide well explained video tutorials for beginners
For my webserver I used tomcat which is easy to use and good with debugging. I recommand it
EDIT: A web application may have one or more jsps (java server pages). Servlets are requests from the browser to the jsp.
You must have VPS or Dedicated Server to use, because shared web hostings doesn't support languages like Java, C, C++, Node.JS and so on...

Categories