how to web scrape autocompleting textfield - java

I am trying to fill a website form(compareraja.in) to search and compare mobile phones using java. I currently am using the jaunt library, but I just cant get to know, how to fill an autocompleting textfield, what i want is to select a particular item from the autocompleting list after i apply a certain initial letters to the textfield. Is it possible with jsoup or jaunt or htmlunit or any other library?
if yes, which is the most better and easiest choice ? Also how can it be done?

My clue is that first of all you have to retrieve the whole autocompletion list which appears after you've applied a certain letters. If you open the web page in e. g. Chrome, go to Developer Tools F12, Network Tab, then you will see that each time you type a letter in the textfield, a corresponding XML HTTP Request is logged in the list.
For example, I've typed "htc ":
On the Network Tab the last one XHR Header section contains all the necessary query parameters :
And Response section shows the received data, which is actually being shown in that autocompletion list:
So, you can just make GET XHR to URL http://www.compareraja.in/autocompletedata.ashx?q=htc+&c=mobiles&limit=150 (you can even click this link or paste it to the browser's address line to test), where your URL-encoded initial letters should be placed instead of htc+. It works fine without timestamp parameter for me.
After that it's easy to parse the response, splitting text by \n and ; chars, and fill the textfield with selected item.

Related

DocuSign REST API V2 make signature mandatory

Currently we use DocuSign to let users sign documents with 3 prepared anchors for SignHere, DateSigned and Location (which is a text tab). We do this by sending an envelope, with a signer and three anchored tabs. Now we have the new requirement to sign arbitrary documents with the same 3 mandantory tabs.
If I send the envelope with the same 3 anchors, the DocuSign GUI ignores these anchors and lets the user place any tab into the document. But if the user places any single tab, it DocuSign enables the "finish" button and when the user presses it, GET /v2.1/accounts/{accountId}/envelopes/{envelopeId}/recipients returns status COMPLETED.
On the other hand, if I set the SignHere tab to pageNumber 1, it works, but the tab is not where I want it and the user cannot move it. In general we do not know where we should place tabs in arbitrary documents.
Is there any way to force the user to add certain tabs AND to allow him/her to move them to a place were they should be?
Short answer - no, you cannot do this.
Long answer - when the envelope has no tabs (signing elements, fields) for a given recipient (signer) then the DocuSign UI enter what we call "free form signing" mode which enables the end-user to drag-and-drop tags/tabs (hence this is called a tagger) to the documents. There's no way to require a minimal number of tags, or any other requirement since it's freeform signing.
The best bet is to use fixed position for your tabs and place them somewhere on the document and or change the document to include the anchor strings you need.

I dont understand this output produced by selenium webdriver java

I'm trying to write an application for the game Path of Exile, that lists the items in my stash on trading websites automatically.
For this I have to retrieve the items in my stash from their website. For some reason the ".getText()" functions is behaving very weird on the website. I really can't see any mistakes I did with the x-path Expressions.
Example:
Here you can see a snippet from the HTML file I am working on
screenshot of browser debugg tool
In the screenshot you can see that the x-Path I am using is selecting a element with a text element, however when I iterate over the elements and get the text with the getText() function, it returns a empty String... I really have no clue what I am doing wrong, is it the website, that is denying me to access the field?
In case it helps I add here a screenshot of the source code for outputting the text fields
printing the text of the elements(SourceCode)
5 empty Strings as output
On your place I would try to get value instead of text.
try to replace
e.getText()
with
e.getAttribute("value")
or you can also try to play with .getCssValue()

Filling out a web form automatically with a text file

I request a web page in the browser and I should fill out the text boxes it has. They are too many text boxes and it takes too mush time too fill them and the result of this latency is an expired page. I have written those inputs in a text file (each line for each text box). I want to write a program that when it is running, it fills out the form and I just click the submit button of the website.
The form is something like the following image:
I have a form which somehow looks like the following image:
I searched a lot to find a way to fill out the text boxes automatically in C# but did not find anything. My question is that is there anyway to set the values of these text boxes in C# and send them back to server?
You can write a json object, then a javascript function that iterate that json and figure out by each value the selector of the correct input to use and set the correct value.
Then when the page open, use the dev console.
paste the json object, paste the function.
Run the function and the form should get fill out.
=)

SWT Browser focus on next and previous highlight text

I am developing a small application with SWT Browser widget. I am highlighting a search text word with
<a id="xyz" href=''><mark>test</mark></a>
in a HTML document. and replace all the search words in HTML Text in this way so we get all the search words highlighted.
htmltext.replaceAll("(?i)"+Pattern.quote(searchword), "\\<a id='xyz' href=''> <mark>$0\\</mark></a>
I want to implement functionality that if I click on next button, next highlighted word should get focus and if I click on previous button previous highlighted text should get focus. how can I accomplish Next and Previous Hit using Javascript in Eclipse RCP application.
This is best solved by combining JavaScript with Java code. It depends what kind of HTML content are you going to handle, if it's stateful (e.g. cannot reload), dynamic with lot of JS code, or plain static. In most cases, the best solution would involve most of logic to be written in JS and just minimal code in Java to bind JS actions to SWT GUI.
There's several things you need to implement:
keyword searching
toggling highlighting
toggling highlight from one word to another
1. Search: you realise that you won't be able to search for words that span through many HTML elements, like W<span>o</span>rd? If that's ok then you can just search and replace from Java as you do now. I'd go for individually tagging each word match with id: <span id="match1"> and remembering how many matches in total were found.
You could likely do such search on JS side as well by adding a function that iterates through DOM and searches for specific text and wraps it with another DOM object.
2. Toggling highlighting: It's best done in JavaScript. Append to your HTML a JS code fragment that toggles DOM element style. Something like:
`
function highlight(id) {
document.getElementById(id).className = 'highlighted'
}
You'll be able to call this JS from SWT by invoking swtBrowser.execute("highlight('match1')")
Further you should implement function that takes off highlighting.
3. Toggling highlighting between elements:
This can be done both on Java side and on JS side. I would probably go with JS and add two more functions: highlightNext() and highlightPrev() that would just call highlight() function with proper ids.
Then in Java you could make SWT buttons that call JS functions through SWTBrowser.execute().

HTML Dropdown v/s autocomplete textbox

In my project(working on Spring and hibernate) i need to keep around 22 HTML dropdown for a form, each dropdown have around 30,000+ entries every dropdown fetching data from database because of this page loading getting delayed (40+ sec) now i wanted to replace dropdown to similar function one, now i thought to keep autocomplete text box, now i wanted to know major performance issues in HTML Dropdown and autocomplete textbox or any suggestion or any alternative for this Thanks in advance.
I think use of autocomplete text box is more beneficial than dropdown.
In case of dropdown, data is loaded at once so it get delayed.
In this case only one request is made to database.
In case of autocomplete text box,data will load in exactly required text box.
So loading time for jsp reduces sharply.
(a) For autocomplete text box,there is no need of using any kind of plugin, as one can manage it easily.
(b) Use simple json format for providing input to text box using ajax.
(c) Define condition on text box for firing ajax request for showing text like..
want to get text after typing 3 characters etc...
You can also use jquery chosen plugin.
You can have normal select box with the jquery chosen plugin applied to that particular select box.
Refer http://davidwalsh.name/jquery-chosen

Categories