I would like to check the position of the elements on a web application on which I am working on. It's a stable application so the positions won't change so much. After searching on the internet I've found this method:
Point getLocation()
Where on the page is the top left-hand corner of the rendered element?
Returns: A point, containing the location of the top left-hand corner of the element
After playing a little bit with it I realized that the position depends on the window size and screen resolution (though I read it somewhere I still checked it just to make sure it's true).
Is there another way? I have a lot of elements so it will be tricky to just store some values depending on a set of resolutions and window dimensions.That would be a lot of work.
In the end nothing beats manual testing when it comes to visual stuff :D
It has to depend on the window size, because the layout of the page changes with the window size.
What you're asking for is essentially:
Can I get the position of elements on the page as if the window was
infinitely wide and infinitely high so that the result would be
static?
No, sorry, you can't do that, because you're probably the first person wanting to do such thing. The positions you're asking for are meaningless. Also, these numbers will most likely be different across the browsers.
I believe you have two options:
Set a fixed window size for this test. If you'll run the test at 1024*768 px every time, it usually won't break. In this case, your positions shouldn't change and you'll get your results.
Bonus idea: you can get a screenshot of the page and compare the images. If the page's content is more or less static, your screenshot will always be the same, so you can compare it to a reference screenshot.
Don't check the absolute numbers, try to check for example whether a WebElement contains the right text, images (and whether the images have been loaded) or even whether a WebElement is visually contained in another WebElement.
Related
I have some lists of images. They always only contain growing objects. For example:
image01:
image 01+n:
image 01+n+n:
image 01+n+n+n:
What do i like to get? (This is the third picture, but modified)
If two (or more) objects merge i like to get the center point of the merge (red point) and also two lines from this point to the background.
I already tried today to solve this problem. I thought it is not so hard, but unfortunately it does not seem to be that easy.
I tried this:
1) For every picture make all objects black
2) Get all objects on every picture. Get the center for each object.
3) Check on every image if on one object is more than one center of a polygon of the previous image. If this is the case two objects merged.
4) If a merge was detected take the centers of the two merged object of the previous frame. Connect them with the A* algorithm on the current frame. Search now the pixel which was white on the previous frame. This is the "merge point".
5) Try now to get the two nearest white points and use them to draw the "break lines" (blue lines on the last image).
Unfortunately this seems not to be that easy and programming this is also hard. Probably an already implemented algorithm (java) exists which could do this job much easier? I implemented everything which I described (quick and dirty), but it is slow and buggy. Something doesn't work and it is hard to debug it. But I think there should be an easier solution.
So your problem is software development and algorithm development.
Because you say buggy and slow: for slow I dont care for now, it has to work; but the buggy is an issue (software development).
Part 3 is essentially correct: you take an object in the current frame and if it contains two objects in the previous frame its a merge.
Your connecting line of centers idea is not correct because the centers can be anywhere and the line will not pass through the connecting region.
Check the change that has happened in the current frame from the previous frame: that will be a hallo around both previous objects. Check the growth (part of this hallo) that is closest to both previous objects. That is take the growth of one object and check which part of the growth is close to the second object; do vice versa. This will give you the merging area.
It's a full project.
My company is trying to perform a browser update from IE8 to IE11. They want my team to automate tests that determine if their applications' web-elements have shifted at all in IE11 compared to IE8.
I did some research and found that the .getLocation() method returns the (x,y) points of an element on the screen. My questions are: what do they use exactly to measure x and y coordinates (i.e. pixels / screen size)? Does it take into account the browser's tool bar at the top of the screen?
I have my doubts about the .getLocation()method so is there a better and more efficient way (than .getLocation()) of determining if a shift has occurred in a web-element's location from browser to browser?
From this link, it looks like the location is measured from the HTML content render pane and therefore ignores the browser window, etc.
You could use JavascriptExecutor and then .getBoundingClientRect but I'm not sure if that will return you any different results. My guess is .getLocation is a wrapper to the JS commands but I can't prove that.
You might consider using an existing tool that compares a page in two browsers. I remember using one years ago that displayed every difference in element position between two browsers but I don't remember now what it was.
I am trying to use the Java Robot class to create a bot to automate some tedious tasks for me, I have never used the Robot class. I have looked up the Class in the Java docs, usage seems straightforward but I have an issue of finding a certain image(I say image, I mean a certain part of the screen) effectively. Is there any other way other than loading 'x' ammount of pixels, checking them, checking the next ammount etc until I find the image I am looking for? Also is there any list of the Button and MouseButton identifiers needed for the Java Robot class as I cna not find any.
For the mouse button identifiers, you are supposed to use BUTTON1_MASK and other button mask constants from java.awt.event.MouseEvent. For example, to click the mouse you would do something like:
Robot r = new Robot();
r.mousePress(MouseEvent.BUTTON1_MASK);
r.mouseRelease(MouseEvent.BUTTON1_MASK);
I believe BUTTON1_MASK is the left mouse button, BUTTON2_MASK is the middle mouse button, and BUTTON3_MASK is the right mouse button, but it has been a month or so since I have used Robot.
As for the checking for an image, I have no idea how that is normally done. But the way you specified in your question where you just check every group of pixels shouldn't be too computationally expensive because you can get the screen image as an array of primitives, then just access the desired pixel with a bit of math. So when checking the "rectangle" of pixels that you are searching for your image in, only keep checking the pixels as long as the pixels keep matching. The moment you find a pixel that does not match, move onto the next "rectangle" of pixels. The probability that you will find a bunch of pixels that match the image that end up not being the image is extremely low, meaning that each rectangle will only need to check about 5 or fewer pixels on average. Any software that performs this task would have to check every pixel on the screen at least once (unless it makes a few shortcuts/assumptions based on probabilities of image variations occurring), and the algorithm I described would check each pixel about 5 times, so it is not that bad to implement, unless you have a huge image to check.
Hope this helps!
Problem Description
I am writing a Java application that lets programmers query for page elements on a web page by specifying visible attributes. One of the most important and difficult is Color.
To be specific, i need a way to get the user-visible color of web page elements using Selenium 2 and Webdriver. I want to be able to query for color values (#ff0000) or names (red).
One parameter should control the percentage of similar colors needed to be "dominating" enough. If set to 100% the element is not allowed to have any other color. If set to 50% the element needs to be halfway filled with the color.
There should be another parameter to control the "tolerance" of these colors. With a higher tolerance, red could also match the orange "Ask Question" button here on Stackoverflow.
Example
Given the well-known Stackoverflow web page, i highlighted the page element to check:
With a higher color tolerance and a not too high domination percentage, the following queries should return the specified result:
color('#FFEFC6') // exact match: true
color('yellow') // match in tolerance range: true
color('orange') // true
color('blue') // false
color('green') // false
My first approach
Best bet would be using CSS attributes like color and background-color. But these do not take images into account, which are needed for good color queries. Also, they could produce difficulties because of css selector inheritance and the handling of transparency. In addition, absolutely positioned elements with a higher z-index above the current element could produce unexpected results.
Given is the web page element to check. It is represented either as JavaScript DOM element (or JQuery object) or as RemoteWebElement in the Java bindings of Webdriver.
It is possible to take automated Screenshots of the current state of the web page (i am using Firefox), see here: Take a screenshot with Selenium WebDriver
The coordinates of the page element to check are known. Therefore, the screenshot image could cropped to that size and area and be analyzed somehow to check if the query returns true or false.
Implementation
I am not limited to Java in this case. JavaScript would be very nice because i am doing the other queries with the help of JQuery too. Performance matters. I am counting on you, i fear this is a very difficult task. Therefore i need your input.
UPDATE
I solved this issue by taking screenshots and analyzing the pixel data of the relevant part. That way i can deal with all kinds of background images and transparency. It's part of the Abmash framework, which is open source and free to anybody to use: Abmash on Github
Easiest way:
Get screenshot (save in memory)
Crop screenshot to the element top = el.offsetTop, left = el.offsetLeft, width = el.offsetWidth, height = el.offsetHeight
Get the pixel data for the cropped image
Loop through the pixels getting the total sums of the R, G, B elements then divide the total sums by the pixel count to get the average. Test the average color against your constraints.
If you really want to use JavaScript
You could send the pixel data to JavaScript for processing if you're intent on doing the final check in JavaScript.
Or you could send JavaScript the IMAGE URI for the cropped image. Then draw that IMG to a CANVAS then loop through the pixel with ctx2d.getImageData(...)
Only do the above if the element is an IMG or a has a background-image CSS. Just use color and background-color CSS checks otherwise.
i am facing an issue while using sikuli through java, if there are 2 elements of same kind(or similar image) it fails to click on the correct element. so i wanted to know if it is possible to make sikuli just work inside a particular region and can some one please explain how can it be done ??
Yes sikuli can work within a particular region. The challenge is defining a region that only contains one of your two elements. You define a region by x,y coordinates. You can also increase the size of a region based on the location of a unique pattern (image) on your display.
while exists("foo.png"):
hover("bar.png")
ClickMeRegion = find("bar.png").nearby(5).right()
ClickMeRegion.click("baz.png")
So in the above I look for image foo.png/bar.png/baz.png image pairs that are being displayed. First I hover on bar.png so that visually I can see which pair the script is looking at. Then I create a region extending 5 pixels around the center of bar.png and extend this to the right of the display. This highlights a single baz.png image. I can then click on the one baz.png that I am interested in.
For more info on regions see: http://doc.sikuli.org/region.html