Automate Your Testing Process Using Robot Framework

If you are a manual Quality Assurance and currently considering learning about automation testing out of curiosity or necessity, this article might be helpful in your exploration of automated testing tools.

To start, you should read about automation testing. You can implement this into your testing process by using Katalon.

Why choose Robot Framework?

Taken from the official website, “Robot Framework is a generic open source automation framework. It can be used for test automation and robotic process automation (RPA)”. By being open source, it means anyone can use it for free without any licensing or subscription cost.

Robot Framework requires little to no knowledge of programming languages, though it is more useful if you already knew Python, as Robot Framework is built in this language. Nevertheless, it is considerably easier for a manual QA to migrate to automation because the syntax of Robot Framework is much similar to plain English. Even the business or product team will easily understand the test suite that you write though you are not using Gherkin.

Robot Framework has a huge community and contributors that are actively developing more functionality. It already has an extensive collection of libraries for testing almost all kinds of tests and platforms. You can use Robot Framework for testing HTTP requests, databases, websites, mobile apps for Android and iOS, and even desktop applications.

Automate Tests using Robot Framework

Requirements

All the examples in this article are written based on the flow using Mac OS.

Python 3

As Robot Framework is built using Python, you need to have Python 3 installed on your machine to run it. For the details of the installation for Mac OS, please refer to this official website.

Follow the installation process, and once Python is installed, open your Terminal and type python3 -V and press Enter, you will get the version number of the installed Python.

Text Editor with Robot Framework plugin

You will also need an IDE or text editor installed for writing your scripts. An easy selection for using Robot Framework is using Pycharm, but you are free to choose whatever text editor you prefer. Just make sure that it has a Robot Framework plugin available to install, as it will help you a lot with syntax highlighting and code auto-completion. In this example, I use Visual Studio Code for writing my Robot Framework scripts, and I installed Robot Framework Language Server and Robocorp Code extensions by Robocorp from the VSCode Marketplace

In addition, because I use VSCode and the Robocorp extensions, there will be small configurations covered in the Installation section.

Installation

Creating Virtual Environment

Before installing Robot Framework, it is preferable to prepare a virtual environment for your automation project. A virtual environment in Python is used for isolating your project and for managing its dependencies, instead of installing them globally. To create a virtual environment, use this command:

python3 -m venv /path/to/new/virtual/environment

In this example, I created a directory named Projects. Inside it, I created the project directory named RFTrial, and inside it, I created a folder named venv to contain my virtual environment dependencies. Then I used the mentioned command to create the virtual environment using the venv folder as the target.

Installing Libraries

Now that you have a virtual environment, you can start installing Robot Framework. But first, you have to cd into your project directory and activate the virtual environment with this command

source ./venv/bin/activate

Then install the Robot Framework with this command

pip install robotframework

You can then check the installed version with this command robot –version

In order to create test cases, you will still need Selenium-based keywords library in your project, as the Robot Framework built-in library does not cater to all the interaction functions like in Selenium. Install the Selenium library with this command

pip install –upgrade robotframework-seleniumlibrary

Providing Browser Driver

You will also need a browser driver based on the browser that you want to run the test. In this example, I will run the test in Chrome, so I need to have chromedriver in my project. You can download it from this official site. Unzip the downloaded file, and put it into the /venv/bin directory.

Later if you run your test for the first time using Mac OS, you will get this error message “chromedriver” can’t be opened because Apple cannot check it for malicious software. To handle this, open your Terminal, then use this command

xattr -d com.apple.quarantine path/to/chromedriver

Extra: Configure Robocorp Extension

As I mentioned, because I use VSCode, there is a small configuration that I have to do before I can start writing my test. First, open the VSCode and open the project folder. Then use Command + Shift + P to bring up the command palette. Type robocorp and select Robocorp: Create Robot command.

Then select the Standard Robot Framework template option.

Select the Use workspace folder option.

Lastly, select the Create Robot anyways option.

This will create a Robot project structure in your project directory. Next, open the robot.yaml file from the project explorer on your left side. Modify the values of PATH and PYTHONPATH as ./venv/bin. Save the file, then it is done.

Design Pattern: Page Object Model

Page Object Model (POM) is a design pattern in which all necessary web elements in a web page are stored inside a class or a file. This can also include the keywords or the methods, which define the user actions to those web elements. This will help you in reducing code duplication and ease the maintenance, as you only define the web element locators and the actions once, inside the corresponding class file.

Without a Page Object Model, you may have duplicate web elements and keywords scattered all over your test suites. And when you need to update those codes, you will have to update them one by one. This will take too much time and effort, which is against the intention of implementing test automation in the first place.

In this example, we will have this test scenario:

• Open https://robotframework.org
• Scroll down to the Resources section, tap the Built-In tab
• Tap the Show More button
• Select the Screenshot library
• Select Take Screenshot keyword
• Verify that the Documentation contains this text “Takes a screenshot in JPEG format and embeds it into the log file.”

Based on those steps, we will interact with 2 pages, the Home page and the Screenshot Library page. Thus, we will separate the web elements and the keywords in those pages based on POM. We will put those files into the resources folder, while we will put the actual test cases file into the tests folder. We will also have a reports folder to save the automatically generated report. We will also create a CommonKeywords.resource file to contain keywords that will be used commonly across other user-defined keywords. The project structure would look like this:

Now we have to collect the web elements in the pages that we want to interact with, we will store them in the corresponding page file.

Defining Variables and Keywords

Different items in Robot Framework are grouped into sections, or sometimes called tables. Commonly we use Settings, Variables, Keywords, and Test Cases; but there are also Comments and Tasks sections. To define a section, we write it with three asterisks before and after each section name and separate them with one whitespace, like *** Setings ***, *** Variables ***, and so on.

The settings section is used for the documentation or definition of what the file does, and to import necessary libraries. Variables, Keywords, and Test Cases sections imply their respective functions. The comments section is used for additional comments if necessary. While the Tasks section is used for writing Robotic Process Automation when the Robot Framework is used to perform automated tasks instead of testing.

First, I define the keywords in the CommonKeywords.resource. The objective is to make sure that the item that we want to click is verified to be present within a certain timeout. Without the implicit wait, Robot Framework will execute the action immediately without waiting for the element to be loaded on the page, and it will cause a false fail. We define this high-level keyword just once, and we will call it into our page resource file.

Now based on our mentioned test cases, we need to gather all the web elements locators from the 2 pages and also define the keywords as the action to those web elements. We put the locators and the keywords into the corresponding page resource file.

Now we can call the keywords from our resources file after importing them into the test suite file.

Best practices

• Don’t repeat yourself (DRY) is a principle in software development aiming to reduce the repetition of patterns or code duplication, and replace it with abstraction. Notice that, when present, the Click item was defined and called in multiple keywords. If you need to use a block of keywords to be called into other user-defined keywords, you better wrap it into a higher-level keyword to avoid redundancy and save time.
• Parameterize the keywords. This will make your keywords flexible to receive inputs instead of hardcoding the input into it. Imagine a login test case where you must give correct and incorrect input. You will have to write 2 different keywords if you hardcode the input, while you can reuse one keyword if it accepts inputs. 
• When possible, always use implicit wait before performing an action to the web elements. Implicit wait means that you wait until the element is available to perform an action within a defined timeout, instead of waiting for an exact time like in the Sleep keyword. So if the element is ready even though the timeout has not passed, the keyword will be executed. Without the wait keyword, you will likely get a false fail because the keyword is executed while the element is not yet loaded.
• Use the Documentation value in the *** Settings *** section efficiently. It is better not to have it filled if you just repeat the file or test suite name. Keep the content brief and clear about the purpose of why the resource file or the test suite is written.
• Use Test Setup and Test Teardown in your test suite file. The keyword used as the test setup will always be executed before every test case, while the test teardown will be executed after every test case. You will not repeat yourself writing the same keywords before and after the test case.
• Use [Tags] in your test case definition. Later in the Running the tests section, you will know about running the test cases only with certain tags. Smoke testing will have a different list of test cases from full regression testing. You will not need to write separate test suites for them, you just need to call the tags.

Running the tests

You can run your test suite using the Terminal, with the base command:
robot path/to/test_suite.robot

You can also run multiple test suite files by separating the test suite path/name with a single space:
robot test_suite_one.robot test_suite_two.robot

If you want to run the whole folder containing your test suite, you can specify the folder path instead of the test suite file:
robot path/to/test_folder

In addition to the base command, you can also add options after the robot command. In example:

• Add –timestamp or -T for adding datetime into the report and log file name.
• Add –variable <var:value> or -v <var:value> for explicitly specifying the value inside your test case or keyword variable.
• Add –variablefile path/to/variable_file.py or -V path/to/variable_file.py if you have a python file containing variables.
• Add –outputdir path/to/reports_folder or -d path/to/reports_folder to specify where the generated report files should be stored.

Add –include tag or -i tag to run test cases only with certain tags.

Executing tests in multiple environments

As a QA, you are definitely required to run tests in multiple environments, thus the automated test that you develop should also be flexible to be executed in different environments.

In Robot Framework, you are allowed to use variables within variables for variable naming. Assume you have a login test suite to be executed in the test and production environments, wherein both environments have different URLs and user credentials. If you hardcode the url, email, and password into the test suite, you would have to make 2 test suites, each to run in one environment. Instead, you can do this in Robot Framework:

Notice that the variable names called in the Keyword definition also contain variables. Instead of hardcoding ${STAGE_URL} into the keyword, ${env} is used in the variable call. You can set the values of these ${env} and ${browser} variables when you run the test using the run command in Terminal, by adding (the short form) -v env:STAGE -v browser:chrome in the run command. This way, you decide the environment run command instead of specifying it inside the test suite.

You can execute the test run using the run command in Terminal. But before that, you must activate the virtual environment first. Like before we install the libraries, open your Terminal, cd to the project folder, then use command:

source ./venv/bin/activate

Finally, the run command to:

• execute all test cases with regression tag, from all test suites inside the tests folder,
• in the Production environment,
• using Chrome browser,
• and save the generated reports to the reports folder with DateTime in the report name,

would look like this:

robot -T -d reports -v env:PROD -v browser:chrome -i regression tests

Here is the log printed in the Terminal while the test is running.

Checking generated reports

As you can notice in the Terminal log shown above, there are 3 generated files after the test run, they are output, log, and report file. Each with date and time printed in the filename, and stored in the reports folder.

You can open the report file, and you can check the Statistics by Tag or by Suite to see the distribution of failed test cases.

If you click the LOG button at the top right corner of the report, you will switch to open the log file to see the details of the test execution.

You can check more details in this log file for any failed test cases, whether the cases failed because of a bug, changed element locators, or any precondition flow that has not been handled. In the details of failed test cases, you can see the attached screenshot of the failure state and which keyword is failed, also the cause of the failure.

Key Takeaways

1. Robot Framework is an open-source automation tool with an easy-to-read syntax much like plain English.
2. Robot Framework has extensive libraries to automate tests on many platforms, like website, mobile, and desktop applications.
3. Page Object Model design pattern is used to separate web element locators and keywords from test cases.
4. Best to use parameterized and reusable keywords to ease up maintenance.
5. It is advisable to use tags in the test case definition to easily call desired test cases from the run command.
6. Variable in variable naming is allowed to separate test data in different environments.
7. Robot Framework automatically generates html reports, which have detailed statistics, running duration, up to the causes and screenshots of the failed cases.

References

1. Robot Framework Standard Library
2. Robot Framework Selenium Library
3. Robocorp VSCode Extension
4. Robot Framework Best Practices