Article

Browser Automation: Automate Website Actions like Clicks or Data Extraction

Windows, Linux, Mac OSX
Posted:
January 30, 2019
How to automate web tasks with browser automation

With robotic process automation (RPA) revolutionizing the handling of routine tasks throughout organizations, many wonder if there are applications they may be missing. Automating browser tasks is one area that can simplify administrative processes and give valuable time back to key employees. Let’s look at how to apply automation to common website actions.

 Do these steps look familiar?

  1. Launch the web application.
  2. Enter username in the username field.
  3. Enter password in the password field.
  4. Click the sign in button.
  5. Navigate to the reports section.
  6. Enter the current date in the date field.
  7. Wait for results of all reports to display.
  8. Repeat the following sub-steps for each report:
    • Click on the link of the report.
    • Save the report to the network folder.
  9. Log off of the website.

Doing this for just one report might only take about 10 minutes, but what happens if you have to do this weekly, daily, or even hourly? Just working with a website or web application could take up hours of your day. With the internet being an essential part of any business, being able to automate website interaction is vital. Automation can handle important yet routine tasks like the ones above and free up your time for other, more important projects.  

  • Do you work for a real estate company that needs to download/update current listings? Automate it.
  • Do you work for a hospital that needs to share lab results in a timely manner? Automate it.
  • Is there a pricing report you need to pull from multiple websites and turn that into a daily report that needs to be emailed to various stakeholders? Automate it.

Examples of Website Actions That Can Be Automated

Clicking a button

One of the most common browser automation tasks is to automate the clicking of a button or link within a web page. The button click is used to navigate a website, confirm data entry operations, select a link to another page, or cancel navigation. You might say the button click is one of the most important operations for a manual or automated browser navigation sequence.

Website login

There are many sites that need to be navigated via automation but that are also password protected. Examples include a bank portal, vendor or trading partner site, and a customer portal. By automating the login and navigation process for a protected website, many hours of manual processing can be eliminated. Site credentials can also remain protected since they are never manually entered on a website. This is because the RPA solution can store the credentials, perform all site processing, and log all site interactions for compliance.

Website navigation

Navigating a website to upload files, download files, and enter or extract data is one of the main uses for an RPA tool. Being able to move to a selected control, scroll down a page, or determine which links to click is all part of the process of recording website navigation steps that need to be completed. Once identified, automation steps are entered into a cohesive and consistent automation process that is repeatable every day.

Wait for page to load

When automatically navigating a web application, an automation task is at the mercy of application performance and internet speed. A process needs to be able to wait until the web browser page has loaded completely before continuing forward. This is usually done by a combination of waiting for the page to load and then checking to make sure all the right information is displayed on the page by checking the current page HTML to ensure the desired information is displayed. Once loading is complete, an automation task may continue forward. Page load monitoring is also a good way to check website performance metrics by capturing load times and performance thresholds and reporting issues automatically to the appropriate application and network monitoring teams.

Fill in web forms

This is a great website action to automate for repetitive data entry tasks. Source data may come from another application screen or by automatically reading data from a database, Excel, or CSV file. It can then be entered automatically into an online form and accept the information via a button click. Auto-filling data can also be used to test response times of an online form. Website automation actions can be used as part of a web or software deployment QA test workflow or after making updates to a website.

Open/close a new tab or browser window

When a browser automation task runs, the handle of the current window is available for manipulating the window or tab that is currently open. The window can be minimized, maximized, or brought to the foreground as needed. Or maybe the window needs to be in a certain location such as the upper-left corner of the screen and must also be a specific size. It’s always a good idea to store the attributes of a web browser window for use later in a task when making decisions on content that gets displayed.

Extract data from a web page

When a data entry or data search task is performed, often there is a need to extract the results from the web page or download a file to be stored or imported into another automated data entry process, network folder, or document management system. The general idea is to inspect the page and get the desired value from any object on the selected page. Once a value is grabbed it can be stored for later use. Values might be an HTML tag, text or field value, a hyperlink to a file, or any other specific attribute that may need to be used during the process. Values may also be stored to a database file, Excel, CSV, or other document to be used in another process or further down in the currently running automation process. Inspecting the HTML page is also another great use case for extracting data from a page. Once a page has refreshed, the entire HTML document or selected elements are available to be inspected to ensure all expected data is available and a website navigation process workflow is proceeding as planned. 

But how exactly can you automate the clicking of a button or the typing of a username on a web page? Traditionally, coding is the solution (see below).

This is a VB.NET script to automate signing into Gmail. It doesn’t look too complex, but it sure isn't simple, either. Epecially if it means completely educating or re-educating ourselves on syntax and coding. Luckily, there's a simpler way to automate web activities. Don’t give up just yet, because Automate’s web browser automation features are here to help.

 Automate’s Web Browser Action

Before we go into the details of Automate’s web browser action, let’s look at Automate’s take on the automation of a Gmail login:

All we need are these four steps and less than five minutes to complete an Automate task to log into Gmail. The steps are displayed in clear language and everything is done via drag-and-drop, so no programming background is needed. For those who are concerned with password security, notice that you also have an option to either disguise the password or password-protect your entire Automate task.

Automate’s web browser action is designed to support common web browsers like Internet Explorer, Chrome, Safari, and Mozilla Firefox, as well as to handle different types of website interaction. Automate offers a comprehensive suite of web browser actions to automate any web activities, from simply clicking a button to extracting data from a table on a web page:

Web Browser Activity

Description

Click

Clicks a particular control in an established web browser session.

Close

Closes a web browser and ends a session created by the open activity.

Extract Source

Extracts an HTML source and places the result in a variable.

Extract Table

Extracts the contents of a given table in an established session.

Get Value

Gets the value of a given element in an established web browser session.

Open

Opens a web browser and establishes a session to associate other web browser activities.

Set Value

Sets the specified value onto an element in an established web browser session.

Navigate

Navigates to a specific page URL that's currently open.

Wait

Waits until the browser is ready for input before continuing to the next step.

Unlike the traditional method of identifying a button using the XY coordinates, Automate accesses the HTML elements on a web page. It evaluates or dissects a web page to distinguish each element by a unique identifier. An element can be anything on a web page: a button, a text box, a link, a table, etc. That means a user automating web interactions does not need to worry about the variation of window size or resolution—the Automate task will always find the specified button to click by that unique identifier.

The Automate Solution

Automate comes with more than 600 pre-built actions, so automation does not need to stop with our preset web Interaction actions. Like the scenario below, you can quickly extract data from an HTML table, populate an Excel spreadsheet, and upload the information to a FTP site.  All of this can be automated in just a few clicks. For related examples, check out the blog and webinar on data and web scraping

Automate is an automation platform that extends business process automation across all areas of your organization. An IT manager might have been commissioned to make the above task just for the finance department, but with a few tweaks, the same processes can gather data for marketing, sales, or even human resources. No code means no limits—virtually any business or IT process can be automated with Automate.

Get Started

See how you can flexibly automate browser actions and other processes. Start a 30-day trial of Automate today.