At one time, it was common wisdom that Internet application delivery would do away with tasks like batch processing and the scheduling of recurring tasks. That hasn’t happened yet; and it likely won’t in the foreseeable future.
The number of batch jobs has diminished greatly, but they have far from disappeared. There is still a need for recurring application processes to perform activities, such as database extracts, data warehousing, and business intel- ligence, which have increased the need for critical back- end computing. Those workload automation processes must run securely, and in the proper sequence. And those processes must be able to check prerequisites and recover from runtime anomalies.
Workload automation is, thus, still a critical IT function. This white paper describes the fundamentals of process scheduling, how to ensure that back-end processing is secure, reliable, and resilient, and how to prevent a small workload automation failure from snowballing into a disaster.
Workflow in a Service-Oriented Architecture
Whether or not they employ a complete service-oriented architecture (SOA), most organizations use at least some SOA concepts. Viewed at the highest level of abstraction, SOA is often described as employing “black box” application objects. However, under the covers, the architecture is more complex.
As depicted in Figure 1, SOA groups workflow components into three layers: The business process layer encompasses the steps a user performs to complete an activity. The application layer contains the code that performs application functions. Between the two, the services interface layer contains the points of data exchange between the business process and application layers.
The services interface layer is, in turn, subdivided into three sub-layers. The application services sub-layer contains low-level interfaces to application components, such as databases. The performance, resource utilization and throughput of these interfaces must be monitored to ensure application availability and proper functioning.
The business services sub-layer contains interfaces to business processes, such as an order entry screen. In this sub-layer, transaction integrity and service level metrics must be tracked to ensure process accuracy and adequate response times.
Components in the orchestration sub-layer tie one or more business services together. It is essential to monitor activity and performance in all of these layers and sub- layers.
Spotting Complex Problems
Sometimes multiple critical problems occur seemingly independently when they, in fact, have a common cause. Security Incident & Event Management (SIEM) allows you to track these related events. SIEM collects event records from a variety of sources, such as server logs, firewall event logs, and intrusion detection logs. It correlates the events over time, graphs those correlations, and draws conclusions to determine when an event that might be innocuous on its own could, in reality, be a critical situation.
The sorts of instrumentation described above are essential to ensure that your entire workload automation process remains secure and that it does not become a victim of background failures that will hobble your system should they accumulate.
Dashboards are also valuable for workflow automation, but as analytical tools rather than to facilitate workflows. They are like the speedometer on your car: They tell you how fast you’re going and how much is getting done, but they’re not really tracking workflow.
Another kind of dashboard, called a workflow dashboard, shows you the status of workflow processes and how well they’re operating. It operates at the business process level, rather than the business intelligence level, meaning that it’s not analytical. Instead, it provides a discreet measurement.
Critical Components of Workload Automation
Many applications include schedulers, but they generally lack the sophistication necessary to take into account interfaces with and dependencies on other applications. Thus, because most organizations employ multiple applications running on multiple operating systems, it’s difficult to get an integrated picture of operations across the entire IT environment. And, without that big picture, it’s difficult to manage the entire environment for peak performance.
Beyond their inability to operate across application interfaces and take dependencies into account, application-specific schedulers often don’t provide adequate security, auditing capabilities, or a means to notify an operator when there are errors or delays in processing. Furthermore, gathering the information from multiple schedulers to satisfy auditors, or to evaluate service-level agreement compliance, can be difficult, if not impossible.
Because of the lack of sophistication of application-specific schedulers, it is often necessary to schedule jobs manually, which consumes labor resources and introduces opportunities for errors. In contrast, workload automation via an enterprise job scheduler gives you the integrated big picture and allows you to manage systems by exceptions, instead of having to manage each process and application individually.
Automate Schedule is an easy-to-use, cross-platform enterprise scheduler. Because it employs a browser-based user interface, there is no need to install a client on your workstation to access the schedule across all your servers.
Automate Schedule's central console allows you to monitor and manage your enterprise-wide schedule from a single point of control, while also facilitating auditing across all of your servers.
Scheduling can be event-driven or time-based, including complex scheduling conditions. Events can include activities such as a file being added or changed, a process starting or ending, or a directory being updated. In addition to driving your schedule, Automate Schedule can monitor these events and trigger processes based on their completion.
Automate Schedule can send automated alerts using email or text messaging. In addition, it can send SNMP traps to an enterprise monitor or it can act as an enterprise monitor and capture traps sent from managed devices. Once those traps are received, Automate Schedule can trigger an event to run an error-recovery process, while simultaneously notifying the system administrator so he or she can verify that the process restarted.
Automate Schedule can automate any type of script, executable, or batch file on all platforms. In addition, an interface to SQL Server allows you to easily schedule SQL Server processes and procedures, grouping them with processes on other servers, if appropriate. You can also run a SQL Server job and then immediately FTP the resulting file to another server.
Automate Schedule allows you to customize return codes, so that you can be assured that, whatever exit code your application sends back upon completion, if it says it completed successfully, it actually did complete successfully.
Automate Schedule variables are another feature that you can use to create dynamic parameters that are passed to your script at runtime. For example, you can use those variables to calculate dates. You can also create static variables and pass the value to a script at runtime to streamline your scripting.
Security in Automate Schedule is role-based, making it easy to set up and maintain. You can be very specific about who has access to which options within the scheduler.
Automate Schedule can be implemented very quickly. In fact, you can install a server and an agent or two, set up a couple of jobs, and get them up and running in as little as 15 minutes.
Implementation begins by installing Automate Schedule's database and HTTP server. You then install agent software on each server that you want to include in your scheduling and monitoring network. This allows the servers to communicate with one another over TCP/IP using transport-layer security to encrypt and secure communications.
Monitoring and managing a multi-platform IT architecture that includes applications from a variety of vendors can be a complex, cumbersome, time-consuming endeavor. An enterprise scheduler can help you to simplify the job, automating many tasks and allowing you to manage by exception to increase data center productivity.
About Automate Software
Automate Schedule is the job scheduling solution of the Automate Software Suite of IT automation solutions. Automate is backed by HelpSystems, LLC, the industry leader in automated operations software for IBM Power Systems Servers. With over 30 years of scheduling experience, HelpSystems, LLC knows what it takes to deliver industrial strength, high-quality, and easy-to-use enterprise automation software. With products like Robot Schedule and Robot Schedule Enterprise, HelpSystems has helped more than 15,000 customers around the world automate their job schedules.
For more information, please call +1 952-746-4786 or request a free trial.