|
1 | 1 | ---
|
2 | 2 | date: 2018-11-16T00:00:00-00:00
|
3 |
| -description: "Automated testing made easy and efficient with Robot framework" |
| 3 | +description: "Simple, efficient, and faster way to test your network devices" |
| 4 | +draft: true |
4 | 5 | featured_image: "/images/nrelabs.png"
|
5 | 6 | tags: [verification, testing]
|
6 | 7 | title: "Using Robot Framework for Automated Testing"
|
7 | 8 | show_reading_time: true
|
8 | 9 | authors:
|
9 |
| - - name: Saim Khan |
10 |
| - link: https://github.com/saimkhan92 |
11 | 10 | - name: Lakshmi Rajan
|
12 | 11 | link: https://github.com/lara29
|
| 12 | + - name: Saim Khan |
| 13 | + link: https://github.com/saimkhan92 |
13 | 14 | ---
|
14 | 15 | `“Testing leads to failure, and failure leads to understanding”` - wisely said by the aviation-wizard Burt Rutan.
|
15 | 16 |
|
16 |
| -If you take any standard product lifecycle development workflow, you will see that `Testing` is an integral phase in the lifecycle of a product or a solution. However, not many engineers understand the importance of testing their product well enough to ensure that it conforms to quality standards. I cannot blame them, because testing sounds like a boring, dull, repetitive task. And this is true for manual network testing, where testers would verify the operation by running their test-cases one at a time on individual devices. This was painstakingly slow and also prone to lack of attention errors. |
| 17 | +If you take any standard product lifecycle development workflow, you will see that `Testing` is an integral phase in the lifecycle of product development. However, not many engineers understand the importance of testing their product well enough to ensure that it meets the quality standards. I cannot blame them, because testing sounds like a boring, dull, repetitive task. And this is true for manual network testing, where testers would verify the operation by running their test-cases one at a time on individual devices. This is painstakingly slow and also prone to lack of attention errors. |
17 | 18 |
|
18 |
| -However, the testing phase has now received a facelift, thanks to the latest developments in automation brought in the by DevOps industry. Traditionally, coding and scripting were only handled by programmers, who generally shied away from testing. With the introduction of the concept of reliability engineering, these lines have now been blurred. A programmer is now expected to know testing, and to use test automation frameworks to check their code for reliability. Similarly, network engineers are expected to use scripting or devops frameworks to test as well as deploy their networks. |
| 19 | +However, the testing phase has now received a facelift, thanks to the latest developments in automation brought in the by devops industry. Traditionally, coding and scripting were only handled by programmers, who generally shied away from testing. With the introduction of the concept of reliability engineering, these lines have now been blurred. A programmer is now expected to know testing, and to use test automation frameworks to check their code for reliability. Similarly, network engineers are expected to use scripting or devops frameworks to test as well as deploy their networks. |
19 | 20 |
|
20 |
| -As a network engineer, suppose you have been assigned a task to set up a data center. You have a hundred devices (can be a mix of switches, routers, and firewalls) that you need to have up and running as quickly as possible. The topology has been set up, and all your devices have been configured. Now, you need to verify the operation of your devices to makes sure that your data center is ready for operation. Doing this job manually is simply out of question, as it would take a lot of time, effort and resources which is not feasible for the current demand. What options do you have? |
| 21 | +As a network engineer, suppose you have been assigned the task to set up a data center. You have a hundred devices (can be a mix of switches, routers, and firewalls) that you need to have up and running as quickly as possible. The topology has been set up, and all your devices have been configured. Now, you need to verify the operation of your devices to make sure that your data center is ready for operation. Doing this job manually is simply out of question, as it would take a lot of time, effort and resources which is not feasible to meet the current demands. What options do you have? |
21 | 22 |
|
22 | 23 | ## Robot as a Test Automation Framework
|
23 | 24 | This is where I would like to introduce you all to the `Robot framework`.
|
24 | 25 |
|
25 | 26 | Robot is a `keyword-driven` test automation framework which provides a platform where you can write test-cases, and run them in a consistent, repeatable manner. It is widely used in many industries to perform acceptance testing. There are several libraries that can be plugged into Robot for enhancements, and it also has a thriving user community with online support.
|
26 | 27 |
|
27 |
| -<div style="text-align:center;"><a href="/images/robot-fwk-architecture.png"><img src="/images/robot-fwk-architecture.png" style="width: 350px;display: block;margin: 0 auto;" ></a></div> |
| 28 | +<div style="text-align:center;"><a href="/images/robot-fwk-architecture.png"><img src="/images/robot-fwk-architecture.png" style="width: 400px;display: block;margin: 0 auto;" ></a></div> |
28 | 29 |
|
29 |
| -Robot can be used to test just about any software library - we just need to provide the right hooks to the library that we plan to use. In case of networking devices, Robot framework can interact with your library that interfaces with your devices under test. In the case of Junos, we can use the Pyez library which is commonly used to interact with Junos devices. |
| 30 | +Robot can be used to test just about any software library - we just need to provide the right hooks to the library that we plan to use. In case of networking devices, Robot framework can interact with your library that interfaces with the devices under test. In the case of Junos, we can use the Pyez library which is commonly used to interact with Junos devices. |
30 | 31 |
|
31 |
| -This is where the significance of the feature “keyword-driven” comes into the picture. Robot test cases are invoked by means of “keywords” that are used to represent a function being tested. For example, you could define a keyword “Check System Alarms”, and internally this can execute either a Junos specific API or another vendor’s API and perform the necessary check on system alarms. The concept of keywords make Robot framework user-friendly since we can use words that convey a simple meaning to our test, and at the same time, it makes Robot vendor agnostic. |
| 32 | +This is where the significance of the feature “keyword-driven” comes into the picture. Robot test cases are invoked by means of “keywords” that are used to represent a function being tested. For example, you could define a keyword “Check System Alarms”, and internally this can execute either a Junos specific API or another vendor’s API and perform the necessary check on system alarms. The concept of keywords make Robot framework user-friendly since we can assign words that convey a simple meaning to our test, and at the same time, it makes Robot vendor-agnostic. |
32 | 33 |
|
33 | 34 | Robot also generates detailed HTML reports that give an in-depth view of each test-case that was run, and this makes it easy for you to understand why things went right/wrong. This would assist you in going back and making the right changes so that your topology works as intended.
|
34 | 35 |
|
35 |
| -<div style="text-align:center;"><a href="/images/robot-test-results.png"><img src="/images/robot-test-results.png" style="width: 350px;display: block;margin: 0 auto;" ></a></div> |
| 36 | +<div style="text-align:center;"><a href="/images/robot-test-results.png"><img src="/images/robot-test-results.png" style="width: 400px;display: block;margin: 0 auto;" ></a></div> |
36 | 37 |
|
37 |
| -Another nice feature of Robot is the ability to group test cases so that they represent a certain functionality or type of device under test. For example, if you have test cases that run only on firewalls, and another set that would run only on routers, you could categorize them into two “suites” and run them separately. |
| 38 | +Another nice feature of Robot is the ability to group test cases so that they represent a certain functionality or a type of device-under-test (DUT). For example, if you have test cases that run only on firewalls, and another set of test cases that would run only on routers, you could categorize them into two “suites” and run them separately. |
38 | 39 |
|
39 | 40 | To summarize, these are the key points about the Robot framework:
|
40 |
| -+ Robot framework is widely used in acceptance testing. |
41 |
| -+ Keyword driven, need not know programming, simple, only need to understand the results. |
42 |
| -+ Rich ecosystem of libraries that can be used. |
43 |
| -+ Does not care about the underlying system under test, i.e., vendor agnostic. |
44 |
| -+ Generates extensive reports in HTML format, gives a granular view into the tests performed, and the results. |
| 41 | +1. Robot framework is widely used in acceptance testing. |
| 42 | +2. Keyword driven, need not know programming, simple, only need to understand the results. |
| 43 | +3. A rich ecosystem of libraries that can be used. |
| 44 | +4. Does not care about the underlying system under test, i.e., vendor agnostic. |
| 45 | +5. Generates extensive reports in HTML format, gives a granular view into the tests performed, and the results. |
45 | 46 |
|
46 | 47 | ## Stages of Test Creation
|
47 | 48 |
|
48 |
| -1. The first and the most important stage in any testing phase is to come up a plan on what features to test, and how you would like to sequence them. This depends on how well you know the product. Even if you don’t know your product completely, I assure you that at the end of testing, you’ll get to know it well ;) |
49 |
| -2. Once your plan is set, go ahead and write your test-cases. |
| 49 | +1. The first and the most important stage in any testing phase is to come up with a plan on what features to test, and how you would like to sequence them. This depends on how well you know the product. Even if you don’t know your product completely, I assure you that at the end of testing, you’ll get to know it well ;) |
| 50 | +2. Once your plan is ready, go ahead and write your test-cases. |
50 | 51 | 3. Run your test-cases.
|
51 | 52 | 4. Review the result of each test-run. See if your test-case produced the intended result. Go back and change the device configuration so that the test passes. Or maybe, your test-case is not valid for a particular scenario - go back and improve your test-case.
|
52 | 53 |
|
53 |
| -Over a period of time, you will understand your product better and be able to finalize on your test-suites. Once this is done, you can run your test-suite on any number of devices multiple times as you wish. There is a multitude of options out there which you can leverage in order to create a test pipeline( for example, integrate it with Docker). |
| 54 | +Over a period of time, you will understand your product better and be able to finalize on your test-suites. Once this is done, you can run your test-suite on any number of devices multiple times as you wish. There is a multitude of options out there which you can leverage in order to create advanced test pipelines (for example, integrate it with Docker or a build server). |
54 | 55 |
|
55 | 56 | Whatever means you take, automation testing can significantly reduce the time to deployment of a solution, by providing the ability to identify errors or shortcomings in the design in a much shorter time frame. It also helps ensure the quality of your product or solution delivered. In the post-deployment stage, monitoring and testing production networks help in ensuring smooth operation of your network. Testing is a means to understand your product/solution better, and automated testing frameworks make this process even sweeter.
|
56 | 57 |
|
57 | 58 | If automated testing is something you are interested in and you would like to get a hands-on experience, we have created a simple tutorial on Robot framework. There are three chapters in the lesson which guide you through the basics of Robot, and we also provide a live environment where you can execute Robot test-cases. Check out our tutorial on the NRE-Labs - [Using Robot Framework for Automated Testing](https://labs.networkreliability.engineering/labs/?lessonId=29&lessonStage=1), and leave your comments and feedback.
|
58 | 59 |
|
59 |
| -And remember, all code is guilty until proven innocent ;) |
60 |
| - |
| 60 | +And remember, all code is guilty until proven innocent ;) |
0 commit comments