Automate Data Scraping for Web and More (Webinar Recap)

Businesses run on data. We’re all dealing with it, often painfully—slowly keying numbers into a database, manually copying and pasting data from a website into an Excel spreadsheet, or collecting and reformatting reports received as email attachments.

Richard Schoen and Pat Cameron of HelpSystems recently hosted a webinar to share how the Automate suite of products can streamline and integrate all your data-driven processes.


Complete Enterprise Automation

First, Richard introduced Automate Complete. As you start automating your workforce and key processes, keep in mind that where you’re at on the automation continuum will change as your business needs mature. You don’t want to get locked into an automation platform that only provides one type of automation.

The Automate product line covers everything from repeatable robotic process automation of basic tasks to DevOps enablement and integration of business applications and infrastructure—and it’s easy to scale as your requirements change.

Who Needs to Automate Data Extraction?

According to Richard, automation issues come from two main areas of an organization: IT departments and business departments. For IT teams, automation includes back-office processes and workflows like running scheduled jobs, reading and writing to databases, managing file transfers, and automating DevOps.

While the IT department is the traditional home of automation software, these days end users in a variety of business departments also benefit from automating processes and applications. This could include automated data extraction processes like copying and pasting information in reports. Intuitive automation solutions that don’t require programming skills are useful in these cases.

Automate Data Extraction Use Cases

You can jump to 11:21 in the video to see several examples of how Automate can be used with data-related processes. They include:

Web Data Extraction: Richard gives an example of using website interactivity to capture information from Google Finance and place the current stock price along with timings for service level tracking into a spreadsheet based on a template. After completing the data extraction, the Excel information can be emailed, uploaded to SharePoint or an FTP site, or entered into another business system or database line by line.

For almost any application or website that doesn’t have another way, such as an API, to interact with it, Automate website interactivity can be used.

Database Updates and Queries: This scenario is common for ETL and data warehousing users. With Automate, it’s easy to quickly build a workflow that reads a table from a database, summarizes the contents, and writes the data to a second database table, merging it with other data along the way if necessary. You can even use database updates—such as a change to an Oracle or SQL Server database—to trigger the start of a workflow. Learn more about automating database tasks with Automate.

Capture and Process Email Documents: This is often used by Accounts Payable departments that need to receive and route vendor invoices to the appropriate user, into a document management system, or into an advanced form processing workflow to extract header and detail information. Another use for automated email capture would be if an order processing department wants to start receiving orders electronically. IT could also use this type of automation to process service desk requests such as password resets or Active Directory onboarding that have the potential to be handled automatically.

Document Management Automation: Many companies have implemented document management systems for capturing or managing documents and delivering reports. Often these solutions need to pull in system-generated documents such as daily reports or invoices, proofs of delivery, and more. Incoming paper files might get scanned from desktop scanners and assigned file names based on order number or invoice number shipment number, or email is received and invoices or other attachments need to be automatically extracted and routed for processing. Automate has the ability to use information from the file name, document properties, or meta data to extract indexing information for a document management system.

Live Analytics Dashboards: Having access to current business data is critical to daily operations. Maybe you need to know if the orders you received today are going to get processed on time or if you have enough material in stock to meet demand, or maybe your management team needs a regularly-updated sales summary. You can use Automate and simple HTML templates to create self-updating, visual dashboards that automatically track critical business data. To learn more about automating business analytics dashboards, watch the on-demand webinar.

You can jump to 19:06 in the video for a demo of the product.

Ready to start automating your data scraping and extraction processes?