Automate is certainly the ideal tool to use when it comes to automating the procedure of interacting with controls within a window or navigating through a webpage, application or other types of interfaces. Automate includes numerous interactivity related actions that can impersonate any mouse or keyboard activity as well as virtually any form of desktop interactions performed manually.
This article will describe some of the more commonly used Interactivity actions and how they are used to navigate and interact with interfaces as well as supply possible solutions to some frequent interactivity and navigation related problems.
Interaction / Navigation
Automate includes numerous actions that can be used to interact with controls or objects within a window. The 'Send Keystrokes' action is one of the most commonly used of these actions. It is used primarily to simulate keystrokes being typed into an application or simulate the process of navigating through an interface. It can interact with standard or non standard Windows based interfaces because it does not need to detect any controls, objects or pixel coordinates. It will accurately move about a window just as long as that window is in the foreground.
The 'Move Mouse to Object' action moves the cursor to an object or control on a window such as a button, page tab, text box, column header or link. To select an object from an existing window, go to the 'General' tab of this action and drag the "A" icon to the desired object on the screen. If the object becomes highlighted, then Automate was able to recognize the object. Simply use a 'Click Mouse' action to simulate clicking the object.
Another possibility is to use the 'Press' action, which simulates moving the cursor to an object and clicking that object all within one step. This action only simulates a single click; therefore, it will work as long as the object it is pressing does not require a double click. You can select an object in the same manner as the 'Move Mouse to Object' action.
Both the 'Move Mouse to Object' and 'Press' actions communicate with the target control natively, therefore, the window where the control resides is not required to be in the foreground for the event to occur. Additionally, these actions will continue to detect the target control even if it has been moved to a different location within the window. A common usage for these actions is to click the icons in the system tray or desktop. Note that Automate uses Windows API (Application Program Interface) calls to find and dissect an object or control inside a window. As a result, the 'Press' and 'Move Mouse to Object' actions will work with only standard Windows controls and consequently may not work properly on any applications or interfaces such as Java Applets.
Automate can nonetheless interact with non Windows based interfaces by using the 'Move Mouse' action, which moves the mouse pointer to the specified pixel coordinates. This will work on external applications that do not support standard Windows conventions because the action relies on pixel coordinates instead of control locations.
Get Data / Set Data
Automate can simulate grabbing data from a window in a number of ways. The 'Get Text' action can retrieve the text from a control on a window and store it in a variable. The 'Set Text' action can set the text of a control on a window. Automate also includes the standard Windows Clipboard actions such as 'Copy,' 'Cut' and 'Paste' commands. Other 'Clipboard' actions include 'Get Clipboard' action, which retrieves the current contents of the clipboard as text and places it in an Automate variable, 'Set Clipboard' action which sets the clipboard to the text specified in the Text parameter. The 'Send Keystrokes' action can be appropriately used to Get or Set text in a window as well. To set data, you can simply use this action to tab to the correct field and enter text. To get data (for example, text within an edit box control), you will need to use 'Send Keystrokes' to highlight and copy it. Achieving this will require you to send tabs and arrows to navigate to what you need to copy. To get all data from a particular document, simply use the appropriate Hot Keys (shortcut keys specified by the underlined letters) to do so. For example, if you want to copy all of the text from a Word document, enter "{CTRL}a" to select all, and "{CTRL}c" to copy the selected text.
Selecting Items
There are several actions in Automate that can be used to select certain items in a window, depending on where the item(s) are located. The 'Select Item' action can select an item from a list, a tree or a combo box on a window. The 'Get Selected List Item' and 'Get Selected Tree Item' actions appropriately retrieve the currently selected item(s) of a list box or tree view. Note that only standard Windows list and tree controls will work with the 'Get Selected List Item' and 'Get Selected Tree Item' actions.
Miscellaneous Interactions
The 'Check' action is used to check, uncheck, or toggle a checkbox or a radio button control in a window. If the window is not a standard windows based interface, the 'Send Keystrokes' action to send the appropriate keys to accomplish this process instead. When selecting Windows menu items (such as the ones located under File, Edit or View), the most reliable way to accomplish this is to use the 'Send Keystrokes' action and simply enter the hot keys to navigate to the option you want. When selecting dropdown menus, you must first use an action to click the button that opens the dropdown menu. The 'Move Mouse to Object' along with the 'Click Mouse' action or the 'Press' action should accomplish this. Thereafter, use the 'Send Keystrokes' action to send the appropriate number of DOWN ARROW keys to highlight the specific dropdown item.
Common Problems / Solutions
When creating tasks that deal with interactive actions, it is best to initially figure out step by step exactly how to accomplish the job manually and thereafter, use the proper interactive actions in Automate to duplicate those steps. Certain interactivity related tasks may fail if a user is currently utilizing the keyboard or mouse or somehow interacting with the desktop while the task is running. For example, if a task includes a 'Send Keystrokes' step that is executed at the same time a user is typing with the keyboard, incorrect keystrokes can be mistakenly entered, causing the task to fail. Therefore, it is recommended that user activity does not take place while an interactive task runs. It is also recommended that two or more tasks which include interactivity actions are not run simultaneously. This is because the steps of one task may mistakenly interact with the execution of another task. For example, Task A can be sending keystrokes to a window that Task B is set to focus, therefore failing both tasks. To resolve this issue, set the priority level for each task that includes interactive actions to either 'Run Alone' or 'Limit number of simultaneous tasks" to "1." All other tasks can be set to wait in a queue of waiting tasks until the current task has completed.
Two of the most common errors that occur when a task interacts with an interface are "Window Not Found" and "Control Not Found." These errors generally occurs when Automate fails to detect a particular window or control in a window, usually because the window may not have opened or fully loaded as of yet. A very important thing to remember is that Automate generally executes each step of a task within milliseconds. Therefore, a rule of thumb when creating interactivity related tasks is to make certain that appropriate 'Wait' actions are always utilized.
For example, when opening a new window in which to interact with, use a 'Wait for Window' action to wait for the specified window to fully load before executing the next step. When the task needs to wait for a currently running program to finish, determine what event usually takes place indicating completion. For example, if a popup window appears informing the user that the program has completed, use the 'Wait for Window' action to wait for that window to appear then use the appropriate actions to close the window. If the application automatically closes or loses focus upon completion, you can adjust the 'Wait for Window' parameters to "Wait for window to close" or "Wait for Window to not be focused" instead. If the program writes to a file or creates a new file, you can use a 'Wait for File' action to monitor for the specified file to be modified or to be created. If the mouse pointer type is different indicating that a program is still running, use the 'Wait for Pointer' action, which will pause at the current line until the mouse pointer matches, or in this case, differs from the specified pointer type. If waiting for a certain process to start or end, use the 'Wait for Process' action.
If none of the above events may occur, you may be able to use the 'Wait for Pixel' action. This action can wait for a graphic or area of the screen to match (or stop matching) a "checksum" of pixel colors, which is very useful for detecting subtle changes in a window's contents that cannot be measured by any other means. If all else fails, you can always use the 'Pause Task' action to pause the task at the current line until the timeframe specified has elapsed.
If after following the above instructions, the "Control Not Found" error still occurs, this may have something to do with Automate failing to detect controls in a task that interacts with multiple windows that have identical names. A reliable solution would be to first make sure to have the "Status Bar" option checked in your browser. In IE, this would be 'View' -> 'Status Bar.' Checking this option will show a status bar in the bottom of your browser. Go to the 'Contents' tab of the 'Wait for Window' action and, along with the current control, add another control that Automate can detect by clicking the 'Add' button. This time, drag the "A" icon to the word "Done" shown in the Status Bar at the bottom left corner of your browser. Since the word "Done" will not appear until the webpage is fully loaded, the "Control Not Found" error should no longer occur.
Many users have pointed out that their interactivity related tasks fail to properly navigate through a window, although no errors are generated by Automate. This usually happens when the 'Send Keystrokes' action is used as the form of interaction. Many times, it appears that this action fails to enter the proper keys to the appropriate fields as well. In actuality, what is most likely happening is that the 'Send Keystrokes' action is trying to send keys to an inappropriate window because the correct window has not fully loaded or was not set to properly focus (brought to the foreground). The 'Send Keystrokes' action cannot detect the window or control in which to send keys to. It generally just sends keys to the window that is currently in the foreground. Therefore, many times, no errors are generated. To fix this problem, simply use the proper 'Wait' action or adjust the option "Add a pause before sending keys" (located in the 'Advanced' tab of the 'Send Keystrokes' action) to a larger amount of milliseconds. To properly bring a window to the foreground, use the 'Focus Window' action or "Maximize Window" action which usually brings the window to the front while maximizing. Other common reasons that the 'Send Keystrokes' action may fail is due to the fact that it sends the keys too fast. To resolve this, go to the 'Advanced' tab and adjust the amount under the "Add a delay before sending keystrokes" field.