Selenium-Grid-Extras

Editor's Notes

• #2: By a show of hands, how many people have heard of selenium grid extras? Ok nice. Selenium Grid Extras was created by a guy named Dima Kovalenko when he worked at Groupon. It was created to make setting up and maintaining a selenium grid easy and affordable.
• #3: So who am I? I have been a contributor to the grid extras project for 2 years. I also help on the selenium irc channel and google group. And I am currently a Software Developer Engineer in Test at GoDaddy (in the Tempe Arizona office).
• #4: I wanted to first explain how the selenium grid works to give some background information. The selenium grid is just a proxy. You tell the hub what kind of environment you want to run your tests against, and the hub finds a node that meets those requirements and forwards the requests to that node. The node receives those commands by the selenium standalone server and tells the driver those commands. The driver then executes those commands in the browser, and the response is sent back to the hub, and then back to the where the tests kicked off from. You can see we have a hub and 3 nodes, each with a different browser. When the node registers to the hub, it tells it that it has IE11 (from its node config file).
• #5: So I bet your asking, why do we need anything besides the normal selenium grid? You set it up once and leave it. Well, as your nodes are staying up for extended periods of time, the memory usage will probably get higher, and you could be left with a lot of browser windows left opened, due to many reasons. When this happens, you’ll start to see more tests start to fail for unknown reasons.
• #6: Now onto Selenium Grid Extras. When you first run the selenium grid extras jar file, it will go through a bunch of questions about how you want to set it up. It will store all of these responses into it’s own config file, and start the grid extras service. Some of the things it will ask you is, which browsers you want to run on the node, if you want to auto restart the nodes, and if you want to record video. You can skip this manual step by creating this config file within a chef or puppet or bash script. This might come in handy if you automatically provision your grid nodes.
• #7: Selenium Grid Extras provides multiple different endpoints you can hit on each of the node and hub machines to maintain it. The default port is 3000, but this is now configurable in the latest version. Going to the /api endpoint, will show all of the valid endpoints with a description, the accepted parameters, default values, as well as the response.
• #8: I will now show a demo of setting up selenium grid extras. ***Local Selenium Grid Extras
• #9: The Selenium Grid Extras project solves a lot of the stability and maintainability issues with a normal selenium grid. It also provides new functionality which could be helpful in debugging your tests.
• #10: Auto update may seem like a bad idea sometimes, but it does have some benefits. If your browsers are NOT setup to disable automatic updates, updating the driver could save your tests from failing. The grid extras jar file can also be set up to auto update, which is a cool feature in my opinion. You’ll get the latest features without having to do anything, besides restart your grid extras service.
• #11: How many people were affected by Chrome updating, and your tests started to fail? There are compatibility issues with Chrome 54 and chromedriver versions less than 2.24. The project that we use Selenium Grid Extras for, was unaffected by this change since the chromedriver version was already at the latest. This is just one example of why auto updating the versions could be useful.
• #12: Show demo of updating from chromedriver 2.23 with Chrome 54, updating to 2.24 Local Selenium Grid Extras
• #13: Your nodes can accrue some leftover browsers. If you accidently forget to close the driver after the test run, or if the connection between the driver and the browser crashes, or if you kill a test in the middle, you could be left with browsers on the machines. Also, in general, the memory usage could go up if there are any memory leaks in any of the services running on the machine. Restarting the nodes after X amount of test runs, gives you a clean slate every so often. With auto restarting the node, Selenium Grid Extras also gives the option to setup auto login, when you first run the grid extras jar file.
• #14: If you ever had to change a node config file, for all your nodes, you know how painful it is to log into each machine and update the config file. Storing all the configs in a central location, on the hub with the case of Selenium Grid Extras, means you only have to log into one machine and can quickly update all of your node config files to save several hours. It is also easy to compare node config files to make sure your settings you want to be the same (like timeouts) are actually the same across nodes. One feature improvement that we are thinking about adding to this project, is having 1 node config file be used for multiple nodes.
• #15: The video of the test runs is perhaps the single best feature of Selenium Grid Extras. It is an OS level video, which means it will capture any OS level modal dialogs that appear. One caveat to this, is that if you have multiple browsers open at the same time, it will only capture the top most browser. This is another reason running a single browser per node is a good idea. This will greatly help with debugging and bug reporting. Imagine you want to create a bug for a failing test, you can easily upload the video and the developer can quickly see the issue and fix it. This is another huge time saver.
• #16: Run a test, show the video Local Selenium-Grid-Extras
• #17: Just like the OS level videos that are taken, you can take an OS level screenshot by hitting an endpoint on the node. These screenshots are currently being saved on the node machine. Inside of your code, you would call this endpoint to take an OS level screenshot on test failure, for example.
• #18: Sometimes, the mouse can interfere with the test. For this reason, the mouse is automatically moved out of the way, to the top left of the screen. Just make sure there isn’t a hot corner setup in the same location. This feature is extremely useful for scenarios where you need to mouse over an area to show something, like a dropdown.
• #19: The ability to kill processes on remote machines is also very helpful. You may not want to have auto restart of the nodes, and still clean up the machine.
• #20: Have VM with multiple chromes, call the kill all chrome, and refresh the url ***Non-Local Grid Extras
• #21: When a node starts up, it can check the browser versions of firefox, chrome, and IE to see if they changed. If they did, it will update the node config file with the new version. This can come in handy when Firefox or Chrome auto update (or even if you update it manually), and you want to get a quick idea of what versions of what browsers are currently on the machine by going to the grid extras or normal hub URL.
• #22: To run tests against IE, the IE Protected Mode must be set as the same value for all zones. It doesn’t matter if they are all set, or all unset, they just have to be the same value. Selenium Grid Extras can automatically do this for you each time the grid extras service starts. Another setting in Internet Explorer is mixed content. If this is set to disabled, and an iframe in your web page is http, but the webpage itself is https, the iframe won’t be displayed. Selenium Grid Extras can set the mixed content value as well.
• #23: Here is a summary what we went over.
• #24: ****See next two slides for screenshots.*****
• #25: Here, you can see the OS level screenshot of the machine. It also tells you what version of grid extras is currently running, and the IP of the machine.
• #26: This is what shows below the previous screenshot. It shows you some of the node config file values, as well as information about the machine (like CPU, RAM, JVM, and Hard Drive space). You can also download the videos that ran on that node.
• #27: SauceLabs is a great product. It has all of the features and advantages as Selenium Grid Extras (and then some). The normal selenium grid is good, but requires a lot of maintenance and your nodes won’t be as stable. Docker-selenium is an amazing project, which lets you very easily setup a chrome and/or firefox selenium grid. It is easy enough to setup your tests to start a new node for each test, and kill it afterwards. This would solve a lot of the stability issues, however one thing that docker-selenium doesn’t have is a video of the test run. It also does not have Internet Explorer, Edge, or Appium for iOS tests. There are no Windows containers from before Windows Server 2016. Selenium Grid Extras can be ran on any platform, and control any kind of selenium grid nodes. Whether it is Windows IE11, Chrome on OSX, or an OSX machine running Appium for iOS tests. There is still maintenance involved with Selenium Grid Extras. You still need to install java, the browsers, and download the selenium grid extras jar. You also need to update the browsers if they are set to not set auto update. It all comes down to a cost, feature, and maintenance tradeoff. SauceLabs may have a lot of great features, but you are paying for them with money. Selenium Grid Extras may come with a few less features, but you have to take the time to learn it and still setup your machines. Selenium-docker is great, but if you need to test against IE11 or Edge, you still need something else.
• #28: Add ability to auto download and upgrade Edge Driver. One issue here is that there is no easy url to see all the versions of the EdgeDriver. Plus, the version of Edge Driver depends on the version of Windows. Add a flag for updating past major and minor versions of drivers. Similar to the tilde and carat in the javascript package.json world. Better video names. Currently the video names are UUIDs with timestamps. What I am thinking, is an endpoint where the user would get to name the video. Allow screenshots to be downloadable via the grid extras web page. Per browser videos, instead of OS level screenshots. There has been some work done here, but it is not complete and I am unsure of the performance impact. We are also thinking of adding a docker container for grid extras. Documentation – It is sometimes hard for people to get started with using selenium grid extras.
• #29: We are looking for contributors for this project. If you use this project, or plan to use this project, and want to help out, please meet up with me after. You can help by adding new features, fixing bugs, adding tests, or writing documentation.
• #30: If you have questions, you can ask them here, or if you wanted , you can ask them on the Gitter, Github or IRC channel.