Introduction
Have you ever worked for a combined
Web-Desktop project and your boss said “Make automation tests to test the Web
application but also to test integration between the Desktop and the Web
application and take into consideration to use open-source tools because out
budget is limited?”. Well, there is a solution for everything!
First, let’s better understand
what a Web-Desktop architecture means: some companies can offer a diverse
amount of software packages that must work together. It is important not only
to test individual packages but also the chain between them. The problem may be
that individual applications can have different bases. In the image below we
can observe that any change done in the Desktop application should be integrated
in the Web application.
Because we are dealing with both
testing a Web application and also with the integration Web-Desktop, a solution
would be to use:
- Selenium WebDriver for testing the Web
application;
- Selenium WebDriver and Sikuli for the integration.
Integrate Selenium
WebDriver and Sikuli in the Same IDE
It doesn't make sense to say too
much about Selenium WebDriver because this a free tool already well known and
commonly used.
Sikuli[1] is an open-source research
project that automates anything we see on the screen. It uses
image recognition to identify and control GUI components. It is useful when
there is no easy access to a GUI's internal or source code, this means that the
only requirement here is that the objects that are tested to be on the screen.
You’ve probably already guessed that in our testing
framework we are going to use Sikuli to simulate actions in the Desktop
application and then with Selenium, we will check the integration of the data
in the Web application. In order to achieve this we do not want to use an IDE
for Selenium and another one for Sikuli. How can we obtain this result?
- Install Java;
- Use Intellij IDEA + Maven for Selenium;
- The core of SikuliX is written in Java,
which means we can use the SikuliX API as a standard JAVA library in our
program. Download
sikulixapi.jar from Sikuli site;
- In the Maven pom.xml file add the following dependency;
- Import classes we need in our applications: import
org.sikuli.script.*;
- Because we will have to capture images on the screen to
identify object, installing the Sikuli software is anyway required. The reason
we want to use it is because Target Offset functionality will help us to
indicate where exactly we want to execute an action in the Sikuli object. Let’s
imagine a text with a label near it. If we want to click on the textbox and not
on the label, the functionality will help us to click on the specified
coordinates;
- The objects (image) should be saved on the disk and used in
Intellij IDEA.
Basic Framework for
Web-Desktop Integration
Having integrated sikulixapi.jar dependency in Maven,
we are now able to access Sikuli methods by declaring objects of type Screen,
Pattern and App:
The most important features about a screen object are: click,
double click, type. We can create functions to simulate these actions like described
below. It is better to keep all these helpers in a separate class:
Click:
DoubleClick:
Type:
What is the meaning of SikuliWaitElement method? To perform operations on an image, we
need to make sure that the image is visible on the screen. Therefore we can
expect a number of seconds until the picture appears on the screen. This is not
the safest solution because we cannot be sure that indeed the image will be
available after the number of seconds indicated (maybe the desktop application
is slow). An alternative would be to wait in a loop one second until reading a
desired result (while image does not exist, wait one second). But this is also not
a complete solution because it may happen that the object does not even exist,
which means that the execution of the test would not stop. The best way is to
wait a maximum number of seconds. If during this period we find the object, then
we can go to the next step, otherwise we will stop the test. Here is an
example:
IsImageAvailable function should verify if
the image is present or not on the screen.
For every image we should declare a constant. We should
also have a constant that gives us the path to the images folder. It is wise to
keep these constants in a separate class.
Then, we can use our image like below (the example
clicks on the desired image):
If we want to minimize the run time of our scripts it
is recommended to simulate keyboard functionalities by using TypeImage function previously
exemplified instead of using a lot of images. Below we can see an example of
how we can use this function. Suppose in the Remote Assistance Settings dialog
we want to change the settings in order to have
- Allow this computer to be
controlled remotely à unchecked
- Set the maximum amount of
time invitations can remain open à 6 Minutes
- Create invitations that can
only be used from computers running Windows Vista or later à checked
Initially the dialog looks like this:
We can write this code in order to obtain the desired
result after Allow this computer to be
controlled remotely image was captured:
Other Important Functionalities.
Screen Class [2]
Until now we discussed about the most used functions:
Click, DoubleClick, Type, Wait and Exists. But besides these, there are other
interesting functions in sikulixapi.jar library, like:
FindAll(image) searches the entire screen
for all the matching visual patterns and returns a list of locations of those
similar patterns. We can use this function for example for unchecking all
check-boxes on a page.
DragDrop(image1,
image2) takes two images as
arguments. The first image describes the source GUI object to drag and the
second image describes the appearance of the destination location where the GUI
object should be dragged to and dropped.
Drag(image) starts a drag-and-drop
operation by dragging at the given click point.
Hover(image) makes the mouse cursor hover
over an object.
RightClick(image) performs a mouse click on the
click point using the right button.
MouseDown(button) presses and holds left mouse. We can
simulate a combination of the button constants Button.LEFT, Button.MIDDLE, Button.RIGHT.
MouseMove(image) moves the mouse to a new
location indicated by the image.
MouseUp(button) releases mouse button.
Wheel(image, WHEEL_DOWN | WHEEL_UP, steps) moves the mouse pointer to a location
indicated by the image and turn the mouse wheel in the specified direction by
the specified number of steps.
Paste(image,
text) pastes the text at a click point.
Highlight(second) highlights
the region for some period of time.
Other Important Functionalities.
App Class [3]
Opening an application à App.open(application name).
Switching to a different application à App.focus(application name)
Closing an application à App.close(application name).
Other Important
Functionalities. Pattern Class [4]
GetFilename gets the filename of the
image contained in the Pattern object.
Conclusion
We succeeded, in this article, to find a solution to
automatically test the integration between two totally different systems in a
single IDE.
Even Sikuli is not an automation test tool that depends
on the code, it is a very usable and user-friendly tool. Testers that haven’t
worked with automation tools before will notice in a short time that they will
be able, by adding only a few lines of code, to perform very easily the main
actions like clicking, double-clicking and typing on an object.
Sikuli completely depends on what’s shown on the
screen, can be sensitive to resolution differences and also the tests are
content-sensitive and can be affected by the object layout changes. However,
Sikuli included in Java is an alternative for integrating two different systems
and, also, can be used to compare objects in the web page that are difficult to
catch with Selenium.
References
