Artificial intelligence (AI) layer-based process extraction for robotic process automation (RPA) is disclosed. Data collected by RPA robots and/or other sources may be analyzed to identify patterns that can be used to suggest or automatically generate RPA workflows. These AI layers may be used to recognize patterns of user or business system processes contained therein. Each AI layer may “sense” different characteristics in the data and be used individually or in concert with other AI layers to suggest RPA workflows.
Human-in-the-loop robot training using artificial intelligence (AI) for robotic process automation (RPA) is disclosed. This may be accomplished by a listener robot watching interactions of a user or another robot with a computing system. Based on the interactions by the user or robot with the computing system, the robot may be improved and/or personalized for the user or a group of users.
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
Dynamically updating, or retraining and updating, artificial intelligence (AI)/machine learning (ML) models in digital processes at runtime is disclosed. Production operation may not need to be stopped for AI/ML model update or retraining and update. The update steps and/or retraining steps for the AI/ML model may be included as part of the digital process. The AI/ML model update may be requested from internal logic (e.g., from the evaluation of a condition, by an expression that calls for the AI/ML model, etc.), external requests (e.g., from external triggers in a finite state machine (FSM), such as a file change, database data, a service call, etc.), or both. Automation of AI/ML model updates or retraining and updates may be provided, where the software reloads/reinitializes/re-instantiates with a retrained and/or updated AI/ML model after (and possibly immediately after) the AI/ML model becomes available.
A method for visualizing a process map is executed by a process map server. The method includes receiving a flowchart and a step-by-step recording related to a process. Generating a process map by combining the flowchart and the step-by-step recording and displaying the process map. The process map displays a task, step, and action related to the process. A detail window shows information associated with the process, and portions of the process, in response to user input. The action is based on information from the step-by-step recording.
A method is implemented by a controller executed on at least one processor. The method provides pre-authorized access to a robotic process automation for a resource associated with a job. The method includes causing, by the controller, the robotic process automation to assume a user identity during an authentication flow to enable access by the robotic process automation to a resource. The method includes issuing, by the controller, tokens to the robotic process automation during the authentication flow. The method includes enabling, by the controller via the tokens, the identity service that governs the resource to participate in operations of the controller to provide the pre-authorized access to the robotic process automation.
User interface (UI) object descriptors, UI object libraries, UI object repositories, and UI object browsers for robotic process automation (RPA) are disclosed. A UI object browser may be used for managing, reusing, and increasing the reliability of UI descriptors in a project. UI descriptors may be added to UI object libraries and be published or republished as UI object libraries for global reuse in a UI object repository. The UI object browser, UI object libraries, and UI object repository may facilitate reusability of UI element identification frameworks and derivatives thereof.
Probabilistic models may be used in a deterministic workflow for robotic process automation (RPA). Machine learning (ML) introduces a probabilistic framework where the outcome is not deterministic, and therefore, the steps are not deterministic. Deterministic workflows may be mixed with probabilistic workflows, or probabilistic activities may be inserted into deterministic workflows, in order to create more dynamic workflows. A supervisor system may be used to monitor an ML model and raise an alarm, disable an RPA robot, bypass an RPA robot, or roll back to a previous version of the ML model when an error is detected by a data drift detector, a concept drift detector, or both.
According to one or more embodiments, a method executed by a type cache service implemented as a computer program within a computing environment is provided. The method includes scanning feeds publishing unpacked packages and automatically detecting new activities on the feeds. The method includes indexing the new activities and all previously detected activities according to type and generating a type cache for the unpacked packages according to the indexing of the new activities and all previously detected activities.
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
G06F 12/0802 - Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
Intent-based automation that discovers automatable tasks and/or determines task variants in data is disclosed. Task capture data may be utilized to determine task variants in task mining data. Semantic understanding of user actions by artificial intelligence (AI)/machine learning (ML) model(s), for example, may be applied to determine the intent of the user rather than only focusing on what actions the user is performing on the computing system. Application logs and semantic understanding may be used to facilitate a more accurate determination of what the user actually intends to do. Task capture for individual user flows may be performed. Once these are captured, task capture algorithms and AI/ML models are used to determine which parts of the flows are similar and/or match and which parts are unique. The path through these flows can then be followed to build a process graph that includes decision points representing the unique flows.
G05B 19/418 - Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control (DNC), flexible manufacturing systems (FMS), integrated manufacturing systems (IMS), computer integrated manufacturing (CIM)
10.
DIGITAL ASSISTANT USING ROBOTIC PROCESS AUTOMATION
A method or system for executing one or more digital assistant tasks using robotic processing automation (RPA) includes populating, by a next generation digital assistant application, a drop down menu comprising of a series of options when a user performs an action in a software application. The method or system also includes populating, by the next generation digital assistant application, a side panel comprising a task to be performed by the next generation digital assistant application and a series of options for the user to select from. The method or system further includes updating, by the next generation digital assistant application, the side panel with additional options in response to a selected option from the series of options. The additional options are recommended actions for the user to select from in addition to the task performed by the next generation digital assistant application.
Disclosed herein is a computing system. The computing system includes a memory and a processor. The memory stores processor executable instructions for a workflow recommendation assistant engine. The processor is coupled to the memory. The processor executes the workflow recommendation assistant engine to cause the computing device to analyze images of a user interface corresponding to user activity, execute a pattern matching of the images with respect to existing automations, and provide a prompt indicating that an existing automation matches the user activity.
Systems and methods for presenting video of execution of a robotic process automation (RPA) workflow at a remote computing system are provided. Execution of the RPA workflow by a remote computing system is initiated. Video of the execution of the RPA workflow by the remote computing system is received at a local computing system. The video is presented at the local computing system.
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
G06Q 10/06 - Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
13.
TARGET-BASED SCHEMA IDENTIFICATION AND SEMANTIC MAPPING FOR ROBOTIC PROCESS AUTOMATION
Target-based schema identification and semantic mapping for robotic process automation (RPA) are disclosed. When looking at a source, such as a document, a web form, a user interface of a software application, a data file, etc., it is often difficult for software to determine which fields are labels and which are values associated with those labels. Since values have not yet been entered for various labels (e.g., first name, company, customer number, etc.), these labels are easier to detect than when the target also includes various values associated with the labels. A selection of an empty target may be received and target-based schema identification may be performed on the empty target, determining labels and a type of the target. Semantic matching may then be performed between a source and the target. These features may be performed at design time or runtime.
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
14.
TARGET-BASED SCHEMA IDENTIFICATION AND SEMANTIC MAPPING FOR ROBOTIC PROCESS AUTOMATION
Target-based schema identification and semantic mapping for robotic process automation (RPA) are disclosed. When looking at a source, such as a document, a web form, a user interface of a software application, a data file, etc., it is often difficult for software to determine which fields are labels and which are values associated with those labels. Since values have not yet been entered for various labels (e.g., first name, company, customer number, etc.), these labels are easier to detect than when the target also includes various values associated with the labels. A selection of an empty target may be received and target-based schema identification may be performed on the empty target, determining labels and a type of the target. Semantic matching may then be performed between a source and the target. These features may be performed at design time or runtime.
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
Systems and methods for annotating exclusive-or nodes or the exclusive outgoing paths of a process model of a process are provided. A process model representing execution of a process is received. Exclusive-or blocks in the process model are identified. Attribute data relating to an exclusive outgoing path from an exclusive-or node in each of the identified exclusive-or blocks are identified. At least one of the exclusive-or node or the exclusive outgoing paths are annotated based on the attribute data. The annotated at least one of the exclusive-or node or the exclusive outgoing paths are output.
G05B 19/418 - Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control (DNC), flexible manufacturing systems (FMS), integrated manufacturing systems (IMS), computer integrated manufacturing (CIM)
G06K 9/62 - Methods or arrangements for recognition using electronic means
16.
DETERMINING SEQUENCES OF INTERACTIONS, PROCESS EXTRACTION, AND ROBOT GENERATION USING GENERATIVE ARTIFICIAL INTELLIGENCE / MACHINE LEARNING MODELS
Use of generative artificial intelligence (AI)/machine learning (ML) models is disclosed to determine sequences of user interactions with computing systems, extract common processes, and generate robotic process automation (RPA) robots. The generative AI/ML model may be trained to recognize matching n-grams of user interactions and/or a beneficial end state. Recorded real user interactions may be analyzed, and matching sequences may be implemented as corresponding activities in an RPA workflow.
Use of generative artificial intelligence (AI)/machine learning (ML) models is disclosed to determine sequences of user interactions with computing systems, extract common processes, and generate robotic process automation (RPA) robots. The generative AI/ML model may be trained to recognize matching n-grams of user interactions and/or a beneficial end state. Recorded real user interactions may be analyzed, and matching sequences may be implemented as corresponding activities in an RPA workflow.
Disclosed herein is a computing device that includes a memory and a processor. The memory stores processor executable for a robotic process engine. The robotic process engine accesses a distributed packaged robotic process to procure code and generate a local robotic process. The code includes parameters, while local robotic process includes input fields in accordance with the parameters. The robotic process engine receives input arguments via the input fields of the local robotic process to generate a configuration and executes the local robotic process utilizing the configuration. The execution of the local robotic process mirrors an execution of the distributed packaged robotic process without changing the distributed packaged robotic process.
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
A robot design interface comprises tools for testing a robotic process automation (RPA) workflow. Some embodiments automatically generate a mock workflow comprising a duplicate of an original workflow, wherein a set of RPA activities are replaced with substitute activities for testing purposes. Some embodiments expose an intuitive interface co-displaying the substitute activities in parallel to their respective original activities and enabling a user to configure various mock parameters. Testing is then carried out on the mock workflow.
G06F 11/36 - Preventing errors by testing or debugging of software
20.
TRAINING A GENERATIVE ARTIFICIAL INTELLIGENCE / MACHINE LEARNING MODEL TO RECOGNIZE APPLICATIONS, SCREENS, AND USER INTERFACE ELEMENTS USING COMPUTER VISION
Techniques for training a generative artificial intelligence (AI) / machine learning (ML) model to recognize applications, screens, and UI elements using computer vision (CV) and to recognize user interactions with the applications, screens, and UI elements are disclosed. Optical character recognition (OCR) may also be used to assist in training the generative AI/ML model. Training of the generative AI/ML model may be performed without other system inputs such as system-level information (e.g., key presses, mouse clicks, locations, operating system operations, etc.) or application-level information (e.g., information from an application programming interface (API) from a software application executing on a computing system), or the training of the generative AI/ML model may be supplemented by other information, such as browser history, heat maps, file information, currently running applications and locations, system level and/or application-level information, etc.
Robot access control and governance for robotic process automation (RPA) is disclosed. A code analyzer of an RPA designer application, such as a workflow analyzer, may read access control and governance policy rules for an RPA designer application and analyze activities of an RPA workflow of the RPA designer application against the access control and governance policy rules. When one or more analyzed activities of the RPA workflow violate the access control and governance policy rules, the code analyzer prevents generation of an RPA robot or publication of the RPA workflow until the RPA workflow satisfies the access control and governance policy rules. When the analyzed activities of the RPA workflow comply with all required access control and governance policy rules, the RPA designer application may generate an RPA robot implementing the RPA workflow or publish the RPA workflow.
Development and deployment of multi-platform automations for robotic process automation (RPA) are disclosed. Hardware level commands, driver level commands, and/or application programming interface (API) calls are automatically and seamlessly substituted within an automation and/or within an RPA workflow at design time. Development of an RPA automation may occur on a first operating system, and the automated reconfiguration and deployment of the RPA automation may occur in a second, distinct operating system. An automation including a first set of hardware level commands, driver level commands, and/or API calls native to a first operating system may be received, ingested, or retrieve and the automation may be automatically reconfigured to include a second set of hardware level commands, driver level commands, and/or API calls native to an operating system. Accordingly, seamless and consistent development of automations that are functionally similar or functionally identical across a range of operating systems may be provided.
In some embodiments, a robotic process automation (RPA) agent executing within a browser window/tab interacts with an RPA driver executing outside of the browser. A bridge module establishes a communication channel between the RPA agent and the RPA driver. In one exemplary use case, the RPA agent exposes a robot design interface, while the RPA driver detects interactions of a user with a target user interface (e.g., an instance of a spreadsheet application, an email program, etc.) and transmits data characterizing the interactions to the RPA agent for constructing a robot specification.
H04L 67/12 - Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06F 3/0484 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
Systems and methods for automatically benchmarking a process of an organization are provided. A process of an organization is extracted from a database of process data. A semantic understanding of the process of the organization is determined. The process of the organization is benchmarked to a standardized process based on the semantic understanding. Results of the benchmarking are output.
Systems and methods for executing a robotic process automation (RPA) workflow are provided. The RPA workflow is executed by a first robot. The execution of the RPA workflow is suspended by the first robot. A current context of the RPA workflow is serialized at a time of the suspension and the current context of the RPA workflow is stored. The execution of the RPA workflow is resumed by a second robot based on a triggering condition by retrieving the current context of the RPA workflow. The first robot and the second robot may be the same robot or different robots.
Artificial intelligence (AI)-based process identification, extraction, and automation for robotic process automation (RPA) is disclosed. Listeners may be deployed to user computing systems to collect data pertaining to user actions. The data collected by the listeners may then be sent to one or more servers and be stored in a database. This data may be analyzed by AI layers to recognize patterns of user behavioral processes therein. These recognized processes may then be distilled into respective RPA workflows and deployed to automate the processes.
A computing device including a memory and a processor is provided. The memory stores processor executable instructions for an entity engine. The processor is coupled to the memory. The processor executes the entity engine to cause the computing device to model entities, which hold or classify data. The processor executes the entity engine to cause the computing device to store in the memory a list identifying each of the entities and the entities themselves in correspondence with the list. The processor executes the entity engine to cause the computing device to provide, in response to a selection input from an external system, access to the entities based on the list. The access includes providing the list to the external system, receiving the selection input identifying a first entity of the entities, and exporting the first entity from the memory to the external system.
Systems and methods for automatically generating a knowledge graph are provided. Entity data, process data, user data, and system data of an organization are extracted from one or more business data sources. A knowledge graph defining relationships between the entities data, the process data, the user data, and the system data is generated. The knowledge graph is output.
Systems and methods for automatically provisioning recommendations for optimizing a process are provided. A knowledge graph for an organization is generated. The knowledge graph for the organization is compared with an optimized knowledge graph. Recommendations for optimizing a process of the organization are generated based on the comparing. The recommendations for optimizing the process are output.
Systems and methods for automatically creating a data model are provided. A semantic understanding of entities stored in one or more business data sources is determined. The entities are extracted from the one or more business data sources based on the semantic understanding. A data model for the entities is created. The data model is output.
In some embodiments, a robotic process automation (RPA) design application provides a user-friendly graphical user interface that unifies the design of automation activities performed on desktop computers with the design of automation activities performed on mobile computing devices such as smartphones and wearable computers. Some embodiments connect to a model device acting as a substitute for an actual automation target device (e.g., smartphone of specific make and model) and display a model GUI mirroring the output of the respective model device. Some embodiments further enable the user to design an automation workflow by directly interacting with the model GUI.
In some embodiments, a robotic process automation (RPA) design application provides a user-friendly graphical user interface that unifies the design of automation activities performed on desktop computers with the design of automation activities performed on mobile computing devices such as smartphones and wearable computers. Some embodiments connect to a model device acting as a substitute for an actual automation target device (e.g., smartphone of specific make and model) and display a model GUI mirroring the output of the respective model device. Some embodiments further enable the user to design an automation workflow by directly interacting with the model GUI.
G06F 30/13 - Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
G06F 30/27 - Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
G06N 3/008 - Artificial life, i.e. computing arrangements simulating life based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
A system and method provide an integrated automation solution that links multiple systems and applications of a contact center operation and provides a unified support interface and unified knowledge base that delivers relevant data in real-time to assist contact center personnel during a customer interaction. Robotic Process Automation (RPA) is used for automating workflows and processes with robots (e.g., attended and/or unattended) that perform various tasks and activities for capturing information (data, documents, etc.) from multiple front-end and/or back-end systems and applications to provide the necessary data and information in real-time during a contact center session.
In some embodiments, a robotic process automation (RPA) agent executing within a browser window/tab interacts with an RPA driver executing outside of the browser. A bridge module establishes a communication channel between the RPA agent and the RPA driver. In one exemplary use case, the RPA agent receives a robot specification from a remote server, the specification indicating at least one RPA activity, and communicates details of the respective activity to the RPA driver via the communication channel. The RPA driver identifies a runtime target for the RPA activity within the target web page and executes the respective activity.
A software robot is configured to automatically identify a target element (e.g., a button, a form field, etc.) within a user interface (UI) according to a set of attributes of the target element specified in the source-code of the respective UI. The robot's code specification includes a multiplicity flag which, when set, causes the robot to search for the target element within multiple instances of a UI object matching a subset of the set attributes (for instance, within all windows having a specific name, within all browser tabs, etc.)
G06F 9/44 - Arrangements for executing specific programs
G06F 9/451 - Execution arrangements for user interfaces
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
In some embodiments, a robotic process automation (RPA) agent executing within a first browser window/tab interacts with an RPA driver injected into a target web page displayed within a second browser window/tab. A bridge module establishes a communication channel between the RPA agent and the RPA driver. In one exemplary use case, the RPA agent exposes a robot design interface, while the RPA driver detects interactions of a user with the target web page and transmits data characterizing the interactions to the RPA agent for constructing a robot specification.
G06F 3/0484 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
In some embodiments, a robotic process automation (RPA) agent executing within a first browser window/tab interacts with an RPA driver injected into a target web page displayed within a second browser window/tab. A bridge module establishes a communication channel between the RPA agent and the RPA driver. In one exemplary use case, the RPA agent receives a robot specification from a remote server, the specification indicating at least one RPA activity, and communicates details of the respective activity to the RPA driver via the communication channel. The RPA driver identifies a runtime target for the RPA activity within the target web page and executes the respective activity.
H04L 67/12 - Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
Disclosed herein is a computing system that includes a memory and a processor coupled to the memory. The memory storing processor executable instructions for an interface engine that integrates robotic processes into a graphic user interface of the computing system. The processor executes the interface engine to cause the computing device to receive inputs via a menu of the graphic user interface and to automatically determine the robotic processes for display in response to the inputs. The interface engine further generates a list including selectable links corresponding to the robotic processes and displays the list in association with the menu.
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
G06F 3/04817 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
39.
AUTOMATION WINDOWS FOR ROBOTIC PROCESS AUTOMATION USING MULTIPLE DESKTOPS
Automation windows for robotic process automation (RPA) using multiple desktops are disclosed. One or more robot desktops are launched, and one or more RPA robots operate in the robot desktop(s). The robot desktops may not initially be the active desktop. When the robot(s) execute their automations, if an activity in the RPA robot workflow needs the robot desktop to be the active desktop, the active desktop is switched to the appropriate robot desktop when the appropriate robot desktop is not already active, and activit(ies) of the RPA workflow that require the robot desktop to be the active desktop are executed. In some embodiments, after RPA workflow execution finishes, the robot desktop, the RPA robot, or both, are automatically closed.
Multi-session automation windows for robotic process automation (RPA) for attended or unattended robots are disclosed. The sessions use the same credentials. A robot session is created and hosted as a window including the user interfaces (UIs) of applications of a window associated with a user session. Running multiple sessions allows a robot to operate in this robot session while the user interacts with the user session. The user may thus be able to interact with applications that the robot is not using or the user and the robot may be able to interact with the same application if that application is capable of this functionality. The user and the robot may both be interacting with the same application instances and file system.
According to one or more embodiments, a method is provided. The method is implemented by a trigger engine stored on a memory as processor executable instructions. The processor executable instructions being executed by a processor. The trigger engine operates as an intermediary for robotic process automations of a software platform. The method includes tracking operations within external systems and registering available events with respect to the operations into a database accessible by the robotic process automations. The method also includes enabling an active event of the external systems to be visible via a trigger of the trigger engine to the robotic process automations.
Multi-target libraries, projects, and activities for robotic process automation (RPA) are disclosed. Some embodiments multiple target platforms can be handled in the same project. The target platform(s) can be specified at the automation and/or activity level in order to provide the supported functionality for each. This may also allow previously built automations to be applied to new target frameworks without starting from scratch.
Process evolution for robotic process automation (RPA) and RPA workflow micro-optimization are disclosed. Initially, an RPA implementation may be scientifically planned, potentially using artificial intelligence (AI). Embedded analytics may be used to measure, report, and align RPA operations with strategic business outcomes. RPA may then be implemented by deploying AI skills (e.g., in the form of machine learning (ML) models) through an AI fabric that seamlessly applies, scales, manages AI for RPA workflows of robots. This cycle of planning, measuring, and reporting may be repeated, potentially guided by more and more AI, to iteratively improve the effectiveness of RPA for a business. RPA implementations may also be identified and implemented based on their estimated return on investment (ROI).
G06Q 10/0637 - Strategic management or analysis, e.g. setting a goal or target of an organisation; Planning actions based on goals; Analysis or evaluation of effectiveness of goals
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
GRAPHICAL ELEMENT SEARCH TECHNIQUE SELECTION, FUZZY LOGIC SELECTION OF ANCHORS AND TARGETS, AND/OR HIERARCHICAL GRAPHICAL ELEMENT IDENTIFICATION FOR ROBOTIC PROCESS AUTOMATION
Graphical element search technique selection, fuzzy logic selection for anchors and targets, and hierarchical graphical element identification for robotic process automation (RPA) are disclosed. The fuzzy logic selection of anchors and targets may be part of a larger, tiered, or hierarchical process for identifying graphical elements in the UI. When a selector for a UI element is not found with at least a confidence threshold, similar elements potentially corresponding to the selector for a UI element target may be searched based on fuzzy matching of the target and corresponding anchor(s). Geometric matching may also be employed between the target UI element and its respective anchor(s). The combination of fuzzy matching and geometric matching may allow for more flexible and accurate identification of the exact selector with which an RPA robot is attempting to interact.
G05B 19/042 - Programme control other than numerical control, i.e. in sequence controllers or logic controllers using digital processors
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
G06F 16/535 - Filtering based on additional data, e.g. user or group profiles
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
Systems and methods for integration of applications are provided. A request for data associated with a second application is received from a first application. The data associated with the second application is generated using one or more process extension APIs. The one or more process extension APIs generate the data using one or more native APIs of the second application. The data is transmitted to the first application.
Systems and methods for automatically assigning labels to one or more types of non-conforming behavior of execution of a process are provided. An aligned process defining non-conforming behavior of execution of a process is received. One or more types of the non-conforming behavior of the execution of the process is identified from the aligned process. Labels identifying the one or more types are assigned to the non-conforming behavior. The labels assigned to the non-conforming behavior are output.
G05B 19/406 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by monitoring or safety
G06F 40/40 - Processing or translation of natural language
47.
Systems and Methods for Dynamically Binding Robotic Process Automation (RPA) Robots to Resources
In some embodiments, a workflow specification includes a set of default characteristics of a resource required by a robotic process automation (RPA) robot tasked with executing the respective workflow. Some embodiments enable a user to change the respective default characteristics (e.g., relocate and/or rename a file) without changing the workflow specification per se. Such changes may be performed via a user interface of an RPA orchestrator managing the execution of multiple RPA robots.
Using long-running workflows with artificial intelligence flows to manage the training/retraining lifecycle of artificial intelligence (AI)/machine learning (ML) models is disclosed. Validation may be desired when an AI/ML model is called by a robotic process automation (RPA) robot executing the long-running workflow. This validation includes dynamic input from users. The RPA robot receives the dynamic input from the users and uses this data for training a replacement AI/ML model or retraining the called AI/ML model. The state of the long-running workflow may be preserved, both in training and serving. Long-running workflows may be used to keep track of where the current execution is in the ML model lifecycle.
Semantic matching between a source screen or source data and a target screen using semantic artificial intelligence (AI) for robotic process automation (RPA) workflows is disclosed. The source data or source screen and the target screen are selected on a matching interface, semantic matching is performed between the source data/screen and the target screen using an artificial intelligence/machine learning (AI/ML) model, and matching graphical elements and unmatched graphical elements are highlighted, allowing the developer to see which graphical elements match and which do not. The matching interface may also provide a confidence score of the individual matches, provide an overall mapping score, and allow the developer to hide/unhide the matched/unmatched graphical elements. Activities of an RPA workflow may be automatically created based on the semantic mapping that can be executed to perform the automation.
G06F 3/0481 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
50.
MAPPING INTERACTIVE UI ELEMENTS TO RPA OBJECT REPOSITORIES FOR RPA DEVELOPMENT
Systems and methods for mapping interactive UI (user interface) elements to an RPA (robotic process automation) object repository are provided. User input selecting a window of an application displayed on a display device is received. In response to receiving the user input selecting the window of the application, interactive UI elements in the window of the application are automatically identified. User input selecting one or more of the identified interactive UI elements in the window of the application is received. The one or more selected interactive UI elements are stored in an RPA object repository of an RPA system.
G06F 3/0484 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06F 9/451 - Execution arrangements for user interfaces
Disclosed herein is a method. The method is implemented by an authentication engine stored on a memory as processor executable instructions. The processor executable instructions are executed by at least one processor. The method (i.e., as implemented by the authentication engine) includes determining an authentication mechanism for an external system to a software platform, generating an interface, executing the authentication mechanism within the interface, authenticating an entity within the interface, and provisioning element instance details with respect to the authentication of the entity.
Semantic matching between a source screen or source data and a target screen using semantic artificial intelligence (AI) for robotic process automation (RPA) workflows is disclosed. The source data or source screen and the target screen are selected on a matching interface, semantic matching is performed between the source data/screen and the target screen using an artificial intelligence/machine learning (AI/ML) model, and matching graphical elements and unmatched graphical elements are highlighted, allowing the developer to see which graphical elements match and which do not. The matching interface may also provide a confidence score of the individual matches, provide an overall mapping score, and allow the developer to hide/unhide the matched/unmatched graphical elements. Activities of an RPA workflow may be automatically created based on the semantic mapping that can be executed to perform the automation.
G06F 3/0481 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
53.
AUTOMATIC DATA TRANSFER BETWEEN A SOURCE AND A TARGET USING SEMANTIC ARTIFICIAL INTELLIGENCE FOR ROBOTIC PROCESS AUTOMATION
Automatic data transfer between a source and a target using semantic artificial intelligence (AI) for robotic process automation (RPA) is disclosed. A user may be provided with the option of selecting a source and a target and indicating through an intuitive user interface that he or she would like to copy data from the source to the destination, regardless of format. This may be done at design time or at run time. For instance, the source and/or target may be a web page, a graphical user interface (GUI) of an application, an image, a file explorer, a spreadsheet, a relational database, a flat file source, any other suitable format, or any combination thereof. The source and the target may have different formats. The source, target, or both may not necessarily be visible to the user.
G05B 13/04 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators
Disclosed herein is a system. The system includes a memory and a processor. The memory stores processor executable instructions for a migration engine. The processor is coupled to the memory. The processor executes the migration engine to cause the system to implement an export operation for an on premises system to mine for data corresponding to automations or user specific arrangements. The processor, also, executes the migration engine to cause the system to implement an import operation of the data to a cloud environment to replicate the automations or user specific arrangements.
Semantic matching between a source screen or source data and a target screen using semantic artificial intelligence (AI) for robotic process automation (RPA) workflows is disclosed. The source data or source screen and the target screen are selected on a matching interface, semantic matching is performed between the source data/screen and the target screen using an artificial intelligence/machine learning (AI/ML) model, and matching graphical elements and unmatched graphical elements are highlighted, allowing the developer to see which graphical elements match and which do not. The matching interface may also provide a confidence score of the individual matches, provide an overall mapping score, and allow the developer to hide/unhide the matched/unmatched graphical elements. Activities of an RPA workflow may be automatically created based on the semantic mapping that can be executed to perform the automation.
G06F 3/0481 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
G06F 40/289 - Phrasal analysis, e.g. finite state techniques or chunking
G06K 9/34 - Segmentation of touching or overlapping patterns in the image field
56.
AUTOMATIC DATA TRANSFER BETWEEN A SOURCE AND A TARGET USING SEMANTIC ARTIFICIAL INTELLIGENCE FOR ROBOTIC PROCESS AUTOMATION
Automatic data transfer between a source and a target using semantic artificial intelligence (AI) for robotic process automation (RPA) is disclosed. A user may be provided with the option of selecting a source and a target and indicating through an intuitive user interface that he or she would like to copy data from the source to the destination, regardless of format. This may be done at design time or at run time. For instance, the source and/or target may be a web page, a graphical user interface (GUI) of an application, an image, a file explorer, a spreadsheet, a relational database, a flat file source, any other suitable format, or any combination thereof. The source and the target may have different formats. The source, target, or both may not necessarily be visible to the user.
G05B 13/04 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators
Systems and methods for configuring an RPA (robotic process automation) platform to perform a candidate process automation are provided. Discovery data relating to a candidate process automation is generated. RPA platform design components for configuring an RPA platform to perform the candidate process automation are generated based on the discovery data. The RPA platform design components are presented to a user via a user interface.
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
58.
PRECONFIGURED ROBOTS FOR ROBOTIC PROCESS AUTOMATION
Systems and methods for configuring an RPA (robotic process automation) platform to perform a candidate process automation are provided. Discovery data relating to a candidate process automation is generated. RPA platform design components for configuring an RPA platform to perform the candidate process automation are generated based on the discovery data. The RPA platform is configured based on the RPA platform design components.
Controlling and provisioning a robot of a virtual machine (VM) includes transmitting a connection request between a first service installed in a virtual machine and a second service. The robot is associated with at least one process running on the virtual machine. The virtual machine is authenticated based on a token associated with the second service and the virtual machine. A connection is established between the first service and the second service. A command is transmitted associated with the controlling of the robot from the second service to the first service based on the authentication of the virtual machine. The command is associated with a corresponding command identifier for identifying a type of the command. The command is then executed for controlling the robot.
A digital assistant may execute one or more tasks using robotic processing automation (RPA). The digital assistant (or robot) assigns a workflow to a robot to monitor for one or more triggers. The one or more triggers comprise one or more events causing a robot to perform an automated tasks with or without user involvement. The robot also identifies the one or more triggers during the monitoring of the one or more triggers, and loads a workflow associated with the one or more identified triggers. The robot further includes executing the loaded workflow to perform one or more tasks associated with the one or more triggers.
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
G06Q 40/00 - Finance; Insurance; Tax strategies; Processing of corporate or income taxes
61.
Artificial intelligence layer-based process extraction for robotic process automation
Artificial intelligence (AI) layer-based process extraction for robotic process automation (RPA) is disclosed. Data collected by RPA robots and/or other sources may be analyzed to identify patterns that can be used to suggest or automatically generate RPA workflows. These AI layers may be used to recognize patterns of user or business system processes contained therein. Each AI layer may “sense” different characteristics in the data and be used individually or in concert with other AI layers to suggest RPA workflows.
A system and a method for verification of execution of an activity are provided. The method comprises receiving a user input indicative of enablement of the verification, and displaying, in response to the reception of the user input, a target element comprising a menu for selecting an edit action. The method further comprises receiving, in response to the selection of the edit action, a verification element, and determining a status of the activity, wherein the status of the activity comprises either of successful execution of the activity or non-successful execution of the activity. Further, the method comprises generating a verification response based on the status of the activity and the verification element.
G06F 9/451 - Execution arrangements for user interfaces
G06F 11/07 - Responding to the occurrence of a fault, e.g. fault tolerance
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06F 3/0484 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
G06V 10/75 - Image or video pattern matching; Proximity measures in feature spaces using context analysis; Selection of dictionaries
63.
System and computer-implemented method for testing an application using an automation bot
A system and a method for performing a test of an application using an automation bot are provided. The method comprises accessing the application to be tested. The method comprises executing the test of the application using the automation bot. The automation bot is configured to interact with one or more other applications. The one or more other applications are different from the application. The method comprises determining one or more test results of the application based on the execution of the test. Further, the method comprises generating a notification indicative of the determined one or more test results.
A software robot is configured to automatically identify a target element (e.g., a button, a form field, etc.) within a user interface (UI) according to a set of attributes of the target element specified in the source-code of the respective UI. The robot's code specification includes a multiplicity flag which, when set, causes the robot to search for the target element within multiple instances of a UI object matching a subset of the set attributes (for instance, within all windows having a specific name, within all browser tabs, etc.)
G06F 9/44 - Arrangements for executing specific programs
G06F 9/451 - Execution arrangements for user interfaces
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
Systems and methods for generating a process tree of a process are provided. An event log of execution of a process is received. User constraints on one or more activities of the process are received from a user. A process tree is generated from the event log based on the user constraints. The process tree is output.
A Computer Vision (CV) model generated by a Machine Learning (ML) system may be retrained for more accurate computer image analysis in Robotic Process Automation (RPA). A designer application may receive a selection of a misidentified or non-identified graphical component in an image form a user, determine representative data of an area of the image that includes the selection, and transmit the representative data and the image to an image database. A reviewer may execute the CV model, or cause the CV model to be executed, to confirm that the error exists, and if so, send the image and a correct label to an ML system for retraining. While the CV model is being retrained, an alternative image recognition model may be used to identify the misidentified or non-identified graphical component.
Systems and methods for generating a process tree of a process are provided. An event log of execution of a process is received. User constraints on one or more activities of the process are received from a user. A process tree is generated from the event log based on the user constraints. The process tree is output. The user constraints comprise at least one of inclusion constraints defining one or more activities that must be represented in the process tree or exclusion constraints defining one or more activities that must not be represented in the process tree.
Multiple anchors may be utilized for robotic process automation (RPA) of a user interface (UI). The multiple anchors may be utilized to determine relationships between elements in the captured image of the Ul for RPA. The results of the anchoring may be utilized for training or retraining of a machine learning (ML) component.
Systems and methods for visually representing a process graph are provided. A process graph representing execution of a process is received. One or more gateway nodes in the process graph are folded into their from-nodes based on a number of incoming edges and a number of outgoing edges of the one or more gateway nodes. The process graph according to the folded one or more gateway nodes is output.
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
Screen response validation of robot execution for robotic process automation (RPA) is disclosed. Whether text, screen changes, images, and/or other expected visual actions occur in an application executing on a computing system that an RPA robot is interacting with may be recognized. Where the robot has been typing may be determined and the physical position on the screen based on the current resolution of where one or more characters, images, windows, etc. appeared may be provided. The physical position of these elements, or the lack thereof, may allow determination of which field(s) the robot is typing in and what the associated application is for the purpose of validation that the application and computing system are responding as intended. When the expected screen changes do not occur, the robot can stop and throw an exception, go back and attempt the intended interaction again, restart the workflow, or take another suitable action.
G06F 11/07 - Responding to the occurrence of a fault, e.g. fault tolerance
G06F 5/06 - Methods or arrangements for data conversion without changing the order or content of the data handled for changing the speed of data flow, i.e. speed regularising
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Disclosed herein is a system. The system includes a memory and a processor. The memory stores processor executable instructions for a recognition engine. The processor is coupled to the memory. The processor executes the processor executable to cause the system to define a plurality of baseline entities to be identified from documents in a workflow and digitize the one or documents to generate corresponding document object models. The recognition engine further causes the system to train a model by using as inputs the corresponding document object models and tagged files and determine, using the model, plurality of target entities from target documents.
Some embodiments address unique challenges of provisioning RPA software to airgapped hosts, and in particular, provisioning RPA machine learning components and training corpora of substantial size, and provisioning to multiple airgapped hosts having distinct hardware and/or software specifications. To reduce costs associated with data traffic and manipulation, some embodiments bundle together multiple RPA components and/or training corpora into an aggregate package comprising a deduplicated collection of software libraries. Individual RPA components are then automatically reconstructed from the aggregate package and distributed to airgapped hosts.
Systems and methods for operating an RPA (robotic process automation) services delivery platform for implementing a plurality of RPA services on premises of a customer are provided. An installer for installing a plurality of RPA services on one or more computing systems located on premises of a customer is generating using the RPA services delivery platform. One or more of the plurality of RPA services installed on the one or more computing systems using the installer are maintained using the RPA services delivery platform.
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
A computing device including a memory and a processor is provided. The memory stores processor executable instructions for an entity engine. The processor is coupled to the memory. The processor executes the entity engine to cause the computing device to model entities, which hold or classify data. The processor executes the entity engine to cause the computing device to store in the memory a list identifying each of the entities and the entities themselves in correspondence with the list. The processor executes the entity engine to cause the computing device to provide, in response to a selection input from an external system, access to the entities based on the list. The access includes providing the list to the external system, receiving the selection input identifying a first entity of the entities, and exporting the first entity from the memory to the external system.
Disclosed herein is a computing system. The computing system includes a memory and a processor. The memory stores processor executable instructions for a workflow recommendation assistant engine. The processor is coupled to the memory. The processor executes the workflow recommendation assistant engine to cause the computing device to analyze images of a user interface corresponding to user activity, execute a pattern matching of the images with respect to existing automations, and provide a prompt indicating that an existing automation matches the user activity.
Artificial intelligence (AI)-based process identification, extraction, and automation for robotic process automation (RPA) is disclosed. Listeners may be deployed to user computing systems to collect data pertaining to user actions. The data collected by the listeners may then be sent to one or more servers and be stored in a database. This data may be analyzed by AI layers to recognize patterns of user behavioral processes therein. These recognized processes may then be distilled into respective RPA workflows and deployed to automate the processes.
Web-based robotic process automation (RPA) designer systems that allow RPA developers to design and implement web serverless automations, user interface (UI) automations, and other automations are disclosed. Such web-based RPA designer systems may allow a developer to sign in through the cloud and obtain a list of template projects, developer-designed projects, services, activities, etc. Thus, RPA development may be centralized and cloud-based, reducing the local processing and memory requirements on a user's computing system and centralizing RPA designer functionality, enabling better compliance. Automations generated by the web-based RPA designer systems may be deployed and executed in virtual machines (VMs), containers, or operating system sessions.
Web-based robotic process automation (RPA) designer systems that allow RPA developers to design and implement web serverless automations, user interface (UI) automations, and other automations are disclosed. Such web-based RPA designer systems may allow a developer to sign in through the cloud and obtain a list of template projects, developer-designed projects, services, activities, etc. Thus, RPA development may be centralized and cloud-based, reducing the local processing and memory requirements on a user's computing system and centralizing RPA designer functionality, enabling better compliance. Automations generated by the web-based RPA designer systems may be deployed and executed in virtual machines (VMs), containers, or operating system sessions.
Web-based robotic process automation (RPA) designer systems that allow RPA developers to design and implement web serverless automations, user interface (UI) automations, and other automations are disclosed. Such web-based RPA designer systems may allow a developer to sign in through the cloud and obtain a list of template projects, developer-designed projects, services, activities, etc. Thus, RPA development may be centralized and cloud-based, reducing the local processing and memory requirements on a user's computing system and centralizing RPA designer functionality, enabling better compliance. Automations generated by the web-based RPA designer systems may be deployed and executed in virtual machines (VMs), containers, or operating system sessions.
Web-based robotic process automation (RPA) designer systems that allow RPA developers to design and implement web serverless automations, user interface (UI) automations, and other automations are disclosed. Such web-based RPA designer systems may allow a developer to sign in through the cloud and obtain a list of template projects, developer-designed projects, services, activities, etc. Thus, RPA development may be centralized and cloud-based, reducing the local processing and memory requirements on a user's computing system and centralizing RPA designer functionality, enabling better compliance. Automations generated by the web-based RPA designer systems may be deployed and executed in virtual machines (VMs), containers, or operating system sessions.
Systems and methods for monitoring a robotic process automation (RPA) system are provided. Job execution data for one or more jobs in the RPA system is determined based on logs of the RPA system. The job execution data for the one or more jobs in the RPA system is caused to be displayed in substantially real time.
A system and a computer-implemented method for validating label data includes receiving the label data and segmenting it into one or more parts using a first machine learning model. Further, from the segmented label data a first plurality of attributes, including text and images, are extracted. The method further includes receiving ground truth data associated with the label data and extracting a second plurality of attributes from the ground truth data. The first and second plurality of attributes are then compared using a second machine learning model and the result of comparison are displayed on a three pane user interface. Further, the label data is validated based on the displayed results.
In some embodiments, a robotic process automation (RPA) robot is configured to search for a target element within a first part of a document currently exposed within a user interface. When the search fails, the robot may automatically actuate a scroll control of the respective UI to cause it to bring another part of the respective document into view. The robot may then continue searching for the RPA target within the newly revealed part of the document. In some embodiments, the robot automatically determines whether the respective document is scrollable, and identifies the scroll control according to a type of target application (e.g., spreadsheet vs. web browser).
A computer system that generates a universal resource locator (URL). The URL is associated with resuming at least one automated process of a running workflow process that includes a plurality of automated processes. The URL is provided to an external system/application. An input of the URL is received from the external system/application, in response to a trigger event at the external system/application. Based on the input of the URL, the at least one automated process is resumed by at least one headless robot.
Automation of a process running in a first session via robotic process automation (RPA) robot(s) running in a second session is disclosed. In some aspects, a form is displayed in a user session, but one or more attended RPA robots that retrieve and/or interact with data for an application in the first session run in one or more other sessions. In this manner, the operation of the RPA robot(s) may not prevent the user from using other applications or instances when the RPA robot(s) are running, but the data modifications made or facilitated by the RPA robot(s) may be visible to the user in the first session window.
Automation of a process running in a first session via robotic process automation (RPA) robot(s) running in a second session is disclosed. In some aspects, a form is displayed in a user session, but one or more attended RPA robots that retrieve and/or interact with data for an application in the first session run in one or more other sessions. In this manner, the operation of the RPA robot(s) may not prevent the user from using other applications or instances when the RPA robot(s) are running, but the data modifications made or facilitated by the RPA robot(s) may be visible to the user in the first session window.
G06F 15/16 - Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
G06F 11/34 - Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation
A method and/or apparatus for creating and/or editing a machine pool with bring your own machine (BYOM) includes creating and/or editing a machine pool with a static list of machines. A user input machine list and an existing machine list are retrieved, and the user input machine list and existing machine list are compared to identify one or more changes between the user input machine list and existing machine list. Next, a new machine specification is created when the one or more changes between the user input machine list and existing machine list are identified. The one or more machines are then moved to the new machine specification.
Systems and methods for allocating computing environments for completing an RPA (robotic process automation) workload are provided. A request for completing an RPA workload is received. A number of computing environments to allocate for completing the RPA workload is calculated based on a selected one of a plurality of RPA autoscaling strategies. The calculated number of computing environments is allocated for allocating one or more RPA robots to complete the RPA workload. The computing environments may be virtual machines.
G05B 19/418 - Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control (DNC), flexible manufacturing systems (FMS), integrated manufacturing systems (IMS), computer integrated manufacturing (CIM)
Disclosed herein is a method implemented by a task mining engine. The task mining engine is stored as processor executable code on a memory. The processor executable code is executed by a processor that is communicatively coupled to the memory. The method includes receiving recorded tasks identifying user activity with respect to a computing environment and clustering the recorded user tasks into steps by processing and scoring each recorded user task. The method also includes extracting step sequences that identify similar combinations or repeated combinations of the steps to mimic the user activity.
G06F 9/48 - Program initiating; Program switching, e.g. by interrupt
G06F 11/34 - Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation
G06F 16/28 - Databases characterised by their database models, e.g. relational or object models
G06F 18/2113 - Selection of the most significant subset of features by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
G06F 18/2321 - Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
G06N 3/008 - Artificial life, i.e. computing arrangements simulating life based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
Systems and methods for determining conformance of a process based on a process model of the process and an event log of an execution of the process are provided. The process model is divided into one or more control regions and reachable nodes are determined for each node in the process model. Conformance of the process is determined by comparing transitions from source activities to destination activities in the event log with the reachable nodes based on the one or more control regions.
Disclosed herein is a computing system that includes a memory and a processor coupled to the memory. The memory storing processor executable instructions for an interface engine that integrates robotic processes into a graphic user interface of the computing system. The processor executes the interface engine to cause the computing system to receive inputs via a menu of the graphic user interface and to automatically determine the robotic processes for display in response to the inputs. The interface engine further generates a list including selectable links corresponding to the robotic processes and displays the list in association with the menu.
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G05B 19/4155 - Numerical control (NC), i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
G06F 3/04817 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
93.
System and computer-implemented method for managing robotic process automation (RPA) robots
A system for managing one or more robots is provided. The system is configured to resolve the one or more issues or faults that lead to failure of execution of one or more automation processes executed by the one or more robots. The system is configured to receive information of an issue associated with at least one robot of the one or more robots and further configured to obtain job log data, associated with the at least one robot, for the issue. The system is further configured to determine, using a trained machine learning model, a corrective action, and its associated confidence score for resolving the received issue, based on the job log data and an analysis performed by the trained machine learning model. Further, system performs the corrective action based on the confidence score and the analysis, for managing the one or more robots.
Probabilistic models may be used in a deterministic workflow for robotic process automation (RPA). Machine learning (ML) introduces a probabilistic framework where the outcome is not deterministic, and therefore, the steps are not deterministic. Deterministic workflows may be mixed with probabilistic workflows, or probabilistic activities may be inserted into deterministic workflows, in order to create more dynamic workflows. A supervisor system may be used to monitor an ML model and raise an alarm, disable an RPA robot, bypass an RPA robot, or roll back to a previous version of the ML model when an error is detected by a data drift detector, a concept drift detector, or both.
Use of artificial intelligence (AI)/machine learning (ML) models is disclosed to determine sequences of user interactions with computing systems, extract common processes, and generate robotic process automation (RPA) robots. The AI/ML model may be trained to recognize matching n-grams of user interactions and/or a beneficial end state. Recorded real user interactions may be analyzed, and matching sequences may be implemented as corresponding activities in an RPA workflow.
G06F 17/18 - Complex mathematical operations for evaluating statistical data
96.
GRAPHICAL ELEMENT DETECTION USING A COMBINED SERIES AND DELAYED PARALLEL EXECUTION UNIFIED TARGET TECHNIQUE, A DEFAULT GRAPHICAL ELEMENT DETECTION TECHNIQUE, OR BOTH
Graphical element detection using a combined series and delayed parallel execution unified target technique that potentially uses a plurality of graphical element detection techniques, performs default user interface (UI) element detection technique configuration at the application and/or UI type level, or both, is disclosed. The unified target merges multiple techniques of identifying and automating UI elements into a single cohesive approach. A unified target descriptor chains together multiple types of UI descriptors in series, uses them in parallel, or uses at least one technique first for a period of time and then runs at least one other technique in parallel or alternatively if the first technique does not find a match within the time period.
A robot design interface comprises tools for testing a robotic process automation (RPA) workflow. Some embodiments automatically generate a mock workflow comprising a duplicate of the original workflow wherein a set of RPA activities are replaced with substitute activities for testing purposes. Some embodiments expose an intuitive interface co-displaying the substitute activities in parallel to their respective original activities and enabling a user to configure various mock parameters. Testing is then carried out on the mock workflow.
Automation windows for RPA for attended or unattended robots are disclosed. A child session is created and hosted as a window including the UIs of applications of a window associated with a parent session. Running multiple sessions allows a robot to operate in this child session while the user interacts with the parent session. The user may thus be able to interact with applications that the robot is not using or the user and the robot may be able to interact with the same application if that application is capable of this functionality. The user and the robot are both interacting with the same application instances and file system. Changes made via the robot and the user in an application will be made as if a single user made them, rather than having the user and the robot each work with separate versions of the applications and file systems.
Detection of typed and/or pasted text, caret tracking, and active element detection for a computing system are disclosed. The location on the screen associated with a computing system where the user has been typing or pasting text, potentially including hot keys or other keys that do not cause visible characters to appear, can be identified and the physical position on the screen where typing or pasting occurred can be provided based on the current resolution of where one or more characters appeared, where the cursor was blinking, or both. This can be done by identifying locations on the screen where changes occurred and performing text recognition and/or caret detection on these locations. The physical position of the typing or pasting activity allows determination of an active or focused element in an application displayed on the screen.
G06F 3/0481 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
G06V 20/40 - Scenes; Scene-specific elements in video content
A system and a computer-implemented method for validating label data includes receiving the label data and segmenting it into one or more parts using a first machine learning model. Further, from the segmented label data a first plurality of attributes, including text and images, are extracted. The method further includes receiving ground truth data associated with the label data and extracting a second plurality of attributes from the ground truth data. The first and second plurality of attributes are then compared using a second machine learning model and the result of comparison are displayed on a three pane user interface. Further, the label data is validated based on the displayed results.
patents
