Selenium Python :
Today, companies and applications are widely structured on web-based systems. The information that these systems contain is dense and requires extensive processing. Various tasks are repeated, tedious and take a lot of time and money. These repeated tasks can be taken care by Web Automation. The common tasks in web automation includes form filling, screen scraping, data collection, transfer between applications, website testing and periodic reports generation.
Web Automation Tool
There are many tools available for automation. A variety of skill levels are required for the automation tool. A non-programmer might simply have to record some test scripts, whereas programmers and advanced testers need more advanced libraries and scripts.
Web browser automation tools work by logging the number of steps involved in the transaction and then play back it in the target web pages by injecting JavaScript and then tracking the results. The macro-like web automation tools are much more flexible and sophisticated.
One of the most popular web automation testing tools is Selenium. It was originally developed at the Thought Works in 2004 by Jason Huggins as an internal tool. Selenium supports automation in various popular browsers, languages and platforms.
It can be easily used on platforms such as Windows, Linux, Solaris and Macintosh. It also supports OS for mobile applications, such as iOS, mobile windows, and android.
Selenium supports different programming languages using drivers specific to each language. Selenium-supported language includes C #, Java, Perl, PHP, Python and Ruby. Test scripts in Selenium can be encoded using any supported language and run directly in almost all the modern web browsers.
Let us take a look at the main advantages of this automation tool before going to the deeper sessions of Selenium.
- Creating quicker report
- Allows frequent testing of regression
- Supports agile approach
- The countless iterations can be done without impasse
- Easier documentation
- Errors in manual testing can be easily detected
Initial Setup
We’ll have to do a couple of things to set up before we start. For functional tests to be written using Selenium Web Driver, a request must be sent to the Selenium server and test cases are then executed on different browsers. In our case we’ll be working with Google Chrome. Hence the very first step is to get chromedriver.exe to simulate the browser. Next step is to install selenium package using pip. If virtual environment is already there then simply type in the shell command line:
$ pip install selenium
Now we need to import the Selenium web driver to implement Python with Selenium. But before proceeding we would like to understand about Selenium Web Driver. It is a web-based automation testing framework that can test Web pages that have been initiated on different web browsers and operating systems.
Selenium WebDriver Client Library for Python allows us to use all the features of Selenium WebDriver and interact with Selenium Standalone Server in order to perform automated testing of browser-based apps both remote and distributed.
To import and configure dependencies to add libraries and functionalities, use the commands below to import Selenium Webdriver:
from selenium import webdriver
from selenium.webdriver.common.keys import keys
from selenium.import.*
Running your first Selenium WebDriver Automation Script using Python
Let’s create a Python script with WebDriver which uses Selenium classes and functions to automate browser interaction.
Here is a simple script to activate the browser:
driver = webdriver.Chrome()
driver.get("https://www.google.com")
assert "Google" in driver.title
element = driver.find_element_by_name("q")
element.send_keys("gogol")
element.send_keys(Keys.RETURN)
assert "No results found." not in driver.page_source
driver.close()
Running the above code will create an instance of Chrome WebDriver. The driver.get method navigates to the page address provided by the URL. The page is loaded fully before WebDriver returns control to the script. However, WebDriver may not know if the page is loaded completely if the page uses a lot of AJAX on load.
The next line asserts that the title contains the word “Google”:
assert “Google” in driver.title
Next statement tries to locate the input text by its name attribute using find_element_by_name method.
element = driver.find_element_by_name("q")
Now we send keys, it’s similar to using your keyboard to enter keys. Use Keys class imported from selenium.webdriver.common.keys to send special keys.
element.send_keys("gogol")
element.send_keys(Keys.RETURN)
You will get the results if there is any once the submission of the page is done. Assertion can be made to make sure that results are found:
assert “No results found.” not in driver.page_source
The final step closes the window. In this script close method is called which will close one tab only. However, if only one tab was open then the browser will exit by default:
driver.close()
Locating Elements
Once the page is loaded Selenium interacts with various elements on the page. There are a number of ways by which WebDriver find elements using one of the find_element_by_* methods. We can use the most suitable method for our case.
Locating element by id
You can use find_element_by_id method to locate element by its id:
element = driver.find_element_by_id('element_id')
Locating element by name
To locate element by name you can use find_element_by_name method:
element = driver.find_element_by_name('element_name')
Locating element by xpath
If there is no appropriate id or name attribute for the item, you want to locate then you can use XPath. It can be used either to locate the element in absolute terms or relative to an element which has an id or name attribute. For example, let’s consider a contact form:
<html> <body> <form id="contactForm"> <input name="name" type="text" /> <input name="email" type="email" /> <input name="phone" type="tel" /> <input name="continue" type="submit" /> </form> </body> </html>
To locate the form elements, you can use XPath like this:
contact_form = driver.find_element_by_xpath("/html/body/form[1]")
contact_form = driver.find_element_by_xpath("//form[1]")
contact_form = driver.find_element_by_xpath("//form[@id='contactForm']")
Locating hyperlinks by link text/partial link text
To locate a hyperlink, you can use find_element_by_link_text or find_element_by_partial_link_text. For example:
<a href="contact.html">Contact Us</a>
contact = driver.find_element_by_link_text('Contact Us')
contact = driver.find_element_by_partial_link_text('Cont')
Locating Elements by Tag Name
To locate an element by tag you can use find_element_by_tag. For example:
<p>Lorem ipsum dolor</p>
element = driver.find_element_by_tag_name('p')
Locating Elements by Class Name
To locate an element by class name you can use find_element_by_class_name. For example:
<h1 class="heading">This is a heading</h1>
element = driver.find_element_by_class_name('heading')
Locating Elements by CSS Selectors
This method can be used when you need to locate an element by CSS selector syntax. For example:
<p class="para"> Lorem ipsum dolor sit amet, consectetur adipiscing elit</p>
paragraph = driver.find_element_by_css_selector('p.para')
These are all the methods by which we can locate elements in a browser. Let’s write a test case using Selenium.
Writing Test Case with Selenium
Selenium is primarily used to write test cases. However, there is no testing tool/framework provided by the selenium package. We can use Python’s unittest module write test case. For our example we create an example unittest script and save it as google_search.py.
import unittest
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
class GoogleSearch(unittest.TestCase):
def setUp(self):
self.driver = webdriver.Chrome()
def test_search_in_google_com(self):
driver = self.driver
driver.get("http://www.google.com")
self.assertIn("Google", driver.title)
element = driver.find_element_by_name("g")
element.send_keys("gogol")
element.send_keys(Keys.RETURN)
assert "No results found." not in driver.page_source
def tearDown(self):
self.driver.close()
if __name__ == "__main__":
unittest.main()
Let’s understand what the code does. First, we import the unit test module which is a built-in Python module based on Java’s Junit. Then we create a class called GoogleSearch where the test case class is inherited from unittest.TestCase.
class GoogleSearch(unittest.TestCase):
The setUp is part of the initialization process, this method will be called before each test function you write in this test case class. An instance of Chrome WebDriver is created.
def setUp(self): self.driver = webdriver.Chrome()
In the test case method, the first line creates a local reference to the driver object created in setUp method.
def test_search_in_google_com(self): driver = self.driver
After every test method, the tearDown method will be called. This is a place where all the cleanup actions can be done. The browser window is closed in the current method.
def tearDown(self): self.driver.close()
The final lines can be used to run the test suite:
if __name__ == "__main__": unittest.main()
Selenium Python Conclusion :
Selenium is one of the widely used Web Browser Automation tools, offering plenty of functionality and control over all major web browsers. Although it is mainly used as a testing/automation tool for production or integration environment, but it can also be used as a web scraper.
