My Two-Month Journey with Robot Framework - Key Insights and Takeaways

For about two months now I have been working with Robot Framework for test automation, and I would like to summarize and share my experience after using this test automation tool in my day to day software quality assurance job.

As you may understand, two months is little time to get to know any tool, but I consider it enough time to form an initial opinion of the benefits and issues you may encounter when using this framework. I aim to help any person who is interested in using Robot Framework as a test automation tool in an organization, personal project, or who simply wants to learn more about it, by providing some insights and takeaways from my personal experience.

So what is Robot Framework?

I want to start by saying that Robot Framework is an open source framework written with Python that can be used mainly for web and mobile test automation. According to Wikipedia it was originally developed in 2005 at Nokia Network to be used as an automation tool for this company. Later in 2008 version 2.0 was released as an open source software.

Robot Framework gained its popularity because of its relatively short learning curve, mainly because it uses keywords that are easily read and written and has a pretty large series of libraries that extend its capabilities.

Robot Framework - The Advantages:

So let's begin with some of the benefits and pros I have come to gather after using Robot Framework.

1. Keyword-Driven Approach

As I mentioned it works using keywords instead of straight code, this can be seen as an advantage as it increases the readability of the tests for people with no programming knowledge (i.e. Product Managers).

Each test case is written as a simple text and it's made of keywords. This allows you to create complex functionalities using the keywords, for example inside a keyword you can use a conditional or a loop.

Robot Framework Docs

Robot Framework Docs

2. Easy to Learn and Use

As it's a framework built using this keyword approach and there is no need to actually know a programming language, the learning curve it's pretty short. Of course if you have programming knowledge it would be much easier to learn.

Robot Framework Docs

3. Rich Ecosystem

There is a wide range of libraries and tools that complement the framework. For starters it has a series of built-in libraries that provides main keywords to include in your tests, as an example we can mention:

• BuiltIn Library: this is the default library that comes with Robot Framework and provides essential keywords for flow control, variable handling, and logging.
• Selenium Library: provides keywords to interact with web browsers using Selenium. It allows you to automate web application testing.
• Collections Library: provides keywords for working with lists and dictionaries, allowing for efficient data manipulation.

It also has several external libraries that enhance the capabilities of Robot Framework, making it adaptable to a wide range of testing scenarios and automation tasks. Let's see some examples:

• RequestsLibrary: facilitates making HTTP requests, allowing for testing REST APIs. It is built on Python's 'requests' library.
• DatabaseLibrary: provides keywords to interact with databases, supporting both relational and non-relational databases.
• AppiumLibrary: extends Robot Framework's capabilities to mobile application testing, using the Appium automation framework.

As you can see this is a big advantage that allows you to enhance your testing coverage within the same ecosystem.

4. Test Execution Control

As other frameworks, Robot Framework allows you to use tags to easily plan the execution of your tests and execute them in any order you like depending on your test strategy, type of tests, etc.

Also in case you have a large test suite there is the possibility of parallel executions using Pabot, the parallel executor for Robot Framework.

5. Community and Documentation

Last but not least, this framework has a large and active community that provides tons of information and documentation to solve any possible issue you may bump into. According to stackshare.io more than 44 big companies actively use Robot Framework.

Robot Framework - The Disadvantages:

Now that I've mentioned the main benefits, I want to also share some disadvantages I found while using Robot Framework, that I consider other test automation tools can do better.

1. Python Dependency

As mentioned before, Robot Framework is built with Python, so if you want to extend its functionalities beyond the pre written keywords you must have Python knowledge. This may not be as common as JavaScript or Java for some testers or quality assurance engineers. So you may want to consider that there might be a learning curve there.

2. Test Reports

Although Robot Framework's executions give back pretty decent test reports that include step by step logs and images, I consider the lack of video reproduction of the execution a big disadvantage. In my experience, video reports help you debug in a big way in case your tests are failing. Take Cypress Test Runner as an example of a platform that includes detailed logs, images and a video of the execution.

3. IDE Support

Robot Framework can be used with almost any IDE (Integrated Development Environment), it even has its own RIDE (the R stands for Robot). Also has extensions that provide support and theoretically make writing easier.

But in my short experience using this framework in Visual Studio code, I can say that the suggested extension Robot Code does not provide as much support as you would expect. For example, no auto formatting is provided, autocompletion is limited, etc.

I also tried another extension, Robot Framework Language Server, that makes the experience a little bit better, but still no auto formatting, which is a big pain for me.

Conclusion

In my opinion 2 months is little time to have a strong opinion of any framework or language out there, as over time you can see more advantages or disadvantages that currently I cannot account for. But hopefully this preview can help you form a preliminary opinion and make a decision about Robot Framwork’s basic functionalities and limitations.

Lastly I want to conclude by saying that there is no best or worst automation tool per se, it depends on your specific testing needs and strategy. So questions like: “What is the technical expertise of your testing team?” “What is the size of your testing suite?’’ “What are my testing needs (mobile, web, both)? are going to be the key to make a decision about the best tool for you.