DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicant
The following is a Final Office action. In response to Examiner’s Non-Final Rejection of 11/17/25, Applicant, on 1/27/26, amended claims. Claims 1-6 are pending in this application and have been rejected below.
Response to Amendment
The amendments are acknowledged.
The double patenting is withdrawn in light of the Terminal Disclaimer filed 1/27/26 and approved on 2/21/26.
Reasons for Subject Matter Eligibility under 35 USC 101
The claim 1 overcomes the 101 rejections because the claim is now : a) a computer-implemented method receiving a stream of desktop-actions that relate to User Interface (UI) data-handling operations of application, b) operating an unsupervised task segmentation module on the stream of data of desktop-actions to identify sequences of desktop-actions, creating word embeddings of the UI data-handling operations for each desktop-action and c) implementing unsupervised topic-segmentation Natural Language Processing on the created vector of embeddings to determine cutting points in an integer sequence of desktop operations, such that semantic similarity level of embeddings is maximized; d) identifying repetitive segments for automation, e) executing code instructions and logic, of the identified repetitive segments at runtime, as a dynamic linked library interacting with one or more applications. When viewing the claim as a whole, this when combined with the earlier limitations is viewed as a practical application under step 2a, prong 2, as the claim is improving another technology when viewing all the limitations listed above (See MPEP 2106.05a) and/or is viewed as a using a judicial exception in a meaningful way under MPEP 2106.05(e). Claims 2-6 depend from claim 1 and are eligible for the same reasons.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-6 are rejected under 35 U.S.C. 103 as being unpatentable over Bialy (US 2022/0318713), Smutko (US 2020/0219033), and Van Grondelle (US 2014/0156336).
Concerning independent claim 1, Bialy discloses:
A computer-implemented method for unsupervised task segmentation (Bialy see par 18 - Embodiments may identify in an unsupervised manner the existence of case ID in the collected data and use it for connecting actions which are spread around the data which in turn may allow discovery of novel connections of continuous business processes which would be hidden otherwise. See par 22 - See par 36 - Automation finder 24 may preprocess collected low level actions or events; and form, segment or split (typically in an unsupervised manner) the stream of user actions into sentences each forming a segment, possibly bounded by time, of actions that form a sequence of a subset of the user actions. Identified sentences may describe an instance of a task.) comprising:
receiving a stream of data of desktop-actions (Bialy – see par 36 - Automation finder 24 may preprocess collected low level actions or events; and form, segment or split (typically in an unsupervised manner) the stream of user actions into sentences each forming a segment, possibly bounded by time, of actions that form a sequence of a subset of the user actions),
wherein each desktop-action relates to User Interface (UI) data-handling operations of applications (Bialy – See par 26 - Events may be generated by users or administrators (e.g., agents of an organization) of client systems or devices, e.g. user terminals, based on input and processing requests to the client devices, such as input and data while performing operations (e.g. user input to applications) on the client devices. see par 33 – For example RT local interface 8 may receive or collect a user's desktop activity or actions, e.g. low-level user action information or descriptions, and send or transmit them to a remote analytics server 20 (e.g. as JSON or other files); See par 108 - Typically automation actions such as business process actions are performed on screen elements (e.g. buttons, windows, dropdown menus, text entry fields) in various applications), and
wherein each desktop-action is labeled with an action related integer identification (id) (Bialy – see par 42 - During a sentence segmentation process, actions may be processed and sentences may be identified by iterating over the database of event logs or actions. Sentences may be labelled using, e.g. numbers, and those sentence labels may be used or assigned to segment actions into sentences, by assigning each action to a sentence label.);
operating an unsupervised task segmentation module on the stream of data of desktop-actions to identify one or more sequences of desktop-actions (Bialy – See par 16 - Embodiments may collect actions (e.g. user entry or input to computer programs) and determine or extract, from the collected corpus or DB (database) of actions, identifiers or case IDs appearing in the corpus, and form the actions into sentences (e.g. sequences of subsets of actions which are part of the same business process), in part using the case IDs. see par 36 - Automation finder 24 may identify automation opportunities by discovering repetitive sequences of actions, e.g. based on sentences, for example using desktop analytics and machine-learning. Automation finder 24 may include an artificial intelligence (AI) server or capability and other functionality discussed herein such as a clustering module and a case ID parser or text search process. Automation finder 24 may preprocess collected low level actions or events; and form, segment or split (typically in an unsupervised manner) the stream of user actions into sentences each forming a segment, possibly bounded by time, of actions that form a sequence of a subset of the user actions. Identified sentences may describe an instance of a task),
wherein each sequence of desktop-actions is identified as a sequence to achieve a task (Bialy – see par 36 – Identified sentences may describe an instance of a task.), and
wherein each task is a business process that is operated by a user and the business process is appropriate for automation (Bialy – see par 36- Automation finder 24 may identify automation opportunities by discovering repetitive sequences of actions, e.g. based on sentences, for example using desktop analytics and machine-learning. Automation finder 24 may perform sequence mining, sequential pattern mining, finding repetitive sequences in data that contains a set of sentences; and a find process function, grouping the previously found sequences into processes, each process potentially describing a business process. see par 107 - An automation sequence may be created which may include a series of actions executed by a computer system to substitute for actions taken by a user operating a computer system. Automation sequences may be identified by sequential pattern mining as repetitive sequences having corresponding user actions that are consecutive and/or within the same time-frame and are repeated within a stream of user actions, typically across different users),
said unsupervised task segmentation module comprising:
(i) creating an integer sequence from the action related integer id, such that desktop-actions in the stream of data are consecutively concatenated (Bialy – see par 42 - During a sentence segmentation process, actions may be processed and sentences may be identified by iterating over the database of event logs or actions. Sentences may be labelled using, e.g. numbers, and those sentence labels may be used or assigned to segment actions into sentences, by assigning each action to a sentence label. While iterating over actions in the preliminary processing phase, a state machine may examine the action's properties and deduce the action's sequence, serial or sentence number, which in one embodiment is an incremental property maintained within the state machine; It may be changed, e.g. incremented, when a condition associated with determining that the next action does not belong to the current sentence (the current sentence containing the previous set of actions), is met. See par 45 - A process such as one shown in Table 2 and Table 3 may, for each action, automatically enter a case ID if no case ID exists, and use the case ID to identify connections between actions or sequences of actions that otherwise seem unconnected. see par 69 - Each action may have a set of gaps and consecutive appearances, each set of gaps and consecutive appearances being a list of integers.);
(ii) creating word embeddings of the UI data-handling operations of applications for each desktop-action based on the integer id thereof to yield a vector of embeddings (Bialy – see par 90 - For a clustering algorithm each word may be represented using, for example, a scaled and weighted f-dimensional vector (f being the number of features) of the above features. see par 91-92 - After creating groups or clusters, the cluster including the case IDs or identifiers may be selected. An improved clustering process according to some embodiments may, without user intervention or direction, and without a-priori knowledge regarding the format of case IDs, find case IDs in actions. ; and
(iii) implementing unsupervised topic-segmentation Natural Language Processing (NLP) module on the created vector of embeddings to determine one or more cutting-points in the integer sequence to yield one or more segments (Bialy – See par 28 - Multiple specific instances of similar actions may share the same generalized name or description. see par 56 - An action duration may be, for example, the time period (e.g. in seconds) that passed from the beginning of the current action until the beginning of the next action if performed by the same user. Table 220 includes, for each entry corresponding to an action in table 200, a case ID 222 if able to be parsed or extracted from the text of the action, and a sequence or sentence number 224 assigned by a process as described herein. In the specific example shown, a process may merge actions having sentence numbers 1 and 3 into a newly numbered sentence, as these actions share a common case ID; see par 36 – split (in unsupervised manner) stream of user actions into sentences each forming a segment; See par 42, 52 – sentence segmentation processes actions; sentence number may remain static while iterating and while assigned to actions. It may be changed, e.g. incremented, when a condition associated with determining that the next action does not belong to the current sentence, is met. See par 66 - For every word in the dictionary, a list of consecutive appearances in the list of actions sorted by user or agent first and then by time (e.g. a word or term's appearance in a string representing an action) and a list of gaps may be compiled for the sake of temporal features, for example, within the sorted list of collected actions used to build sentences; see par 90-91 - words may be grouped or clustered, to partition the words into groups having similar traits; for a clustering algorithm each word may be represented using, for example, a scaled and weighted f-dimensional vector (f being the number of features) of the above features.) such that semantic similarity level of embeddings in each yielded segment is maximized… (Bialy – see par 58 - specific instances of case ID or the case ID format can then be used by text searching, or parsing or text analysis in order to identify case ID appearances in the sentence building and merging phase and to extract actual case IDs. This may be performed by analyzing the raw action data before generalization, for example using ML to identify probable words in the actions' text corpus, represent statistical features for each word, cluster them (e.g. according to similar features), and identify the cluster containing the case ID words, e.g. according to its statical traits, thus inferring what words represent the case ID; see par 90 - words may be grouped or clustered, to partition the words into groups having similar traits; features may be weighted, for example, using the same weights P.sub.i: as used to calculate distance D; see par 92 - The most compact cluster, having the lowest compactness score, may indicate the cluster or group which has the most similar members in feature terms.)
Bialy discloses “Sentences may be associated with a case ID via each action, where the action includes as an attribute or is assigned in a database a case ID and sentence number, or sequence or sentence counter, where the sentences are ordered sequentially. Thus a series of actions A, B, C, D, each having sentence number 3, may form the four actions in sentence 3. A state machine, an example of which described in Tables 2 and 3, may use the case ID and other properties to determine every action's “sn_counter” or sentence counter when iterating over all the actions. Different sequences of actions or sentences may be merged or combined to larger sentences or sequences by for example taking actions with the same case ID, sorted by time to create a larger sentence” (See par 44). Bialy discloses that clusters are for words with similar features (See par 92). Bialy discloses that sequences can be merged or combined, even into one sentence (See par 106).
Smutko also discloses “cutting points” as claimed above (Smutko – see par 36 - further, each user action, such as user action 500a, is further associated with a “description” field that summarizes the action's information. This field acts as a unique identifier of the user action's functionality. This allows different actions to be associated with different descriptions while the same actions can be linked by the same or similar identifiers (e.g., the same or similar descriptions and identifiers within a vector space or other representation). see par 47 - In flowchart 400a, sequence mining from a sentence database begins at step 402. The sentence database includes the sentences of user actions broken up from a stream of data from an event log of desktop events. At step 404, a length for sequences is set having a threshold length that is used for identification of initial sequences based on their support within the user action sentences. The length may be utilized so that sequences of only a meaningful length are determined and identified in the user action sentences. See par 53 - Clustering of sequences may further be performed to identify the same or similar sequences and select a sequence of those for automation. In this regard, sequence descriptions are converted to vector representation of the user actions, at step 430. Actions may be represented as vectors using word2vec or other technique to generate word vectors from user actions. For example, a vectorization process may generate a vector from word and/or text input of the user action. Each description and/or parts of the description can be a vector. A vector representation of each sequence's description may then be generated through averaging the vectors of the descriptions included in the sequence. See par 55 - At step 434, using the distance between the sequences, clustering is performed to identify sequences with other similar sequences. A cluster represent sequences having similar business functionalities, such as sequences that implement the same or similar business processes for automation.), as well as:
“…and a number of non-complete business processes is reduced” (Smutko – see par 19 - The desktop event may also be associated with particular user actions, which is the execution of the particular desktop event (e.g., the input) within a certain application, at a certain time, and/or according to another parameter (e.g., interface location, elements, etc.). These user actions may be joined into discreet sequences of actions, which define a particular execution of a process by a user. For example, a concatenation of several actions, which may or may not be consecutive but may be joined by a temporal similarity, may correspond to a complete business flow executed by a user, such as a process. see par 31 - According to some embodiments, automation finder engine 130 may be trained with one or more encodings or features so that it is able to predict and/or cluster similar desktop events into business processes for automation; see par 48 - A minimum sequence threshold may be utilized to omit smaller recurring individual user actions of sequences and less likely sequences, which may be further tuned based on the needs of the system and/or administrator),
“wherein each cutting-point of the one or more cutting-points in the integer sequence indicates an end of a first segment and a beginning of a second consecutive segment” (Smutko – see par 55-56 - At step 434, using the distance between the sequences, clustering is performed to identify sequences with other similar sequences. A cluster represent sequences having similar business functionalities, such as sequences that implement the same or similar business processes for automation. When performing clustering, the following algorithm may be used to determine a modularity Q of a graph of those clusters within vector space. Using this method, at step 436, an iterative finding of smaller clusters may be performed, wherein in each iteration, each sequence is associated with its cluster. Then for the sequence, the change in modularity is determined from its cluster. After determining the modularity for a sequence with its cluster, the sequence is then removed from the cluster and is associated with the cluster for its neighbor sequences. Iterating this over a database, the i-th sequence is associated with a cluster that maximizes the modularity.), and
Bialy and Smutko disclose:
(iv) identifying repetitive segments for automation thereof (Bialy – See par 32 - A real-time (RT) local interface 8 (e.g. a NICE Attended Robot provided by NICE, Ltd.) executing on terminals 2 may collect user action data, execute an automation sequence in place of user input or provide or display a recommended next action to a user, according to automations created. see par 107- Automation sequences may be identified by sequential pattern mining as repetitive sequences having corresponding user actions that are consecutive and/or within the same time-frame and are repeated within a stream of user actions, typically across different users; see par 110 - FIG. 5 shows a high-level block diagram of an exemplary computing device which may be used with embodiments of the present invention) by providing the one or more segments which represent complete tasks to a routine mining module (Bialy – see par 36 - automation finder 24 may perform sequence mining, sequential pattern mining, finding repetitive sequences in data that contains a set of sentences; and a find process function, grouping the previously found sequences into processes, each process potentially describing a business process. The sequences may be filtered to identify the best ones of those sequences that have the highest ROI
See also Smutko – see par 19 - For example, a concatenation of several actions, which may or may not be consecutive but may be joined by a temporal similarity, may correspond to a complete business flow executed by a user, such as a process. These processes may therefore be identifiable and utilized for automation based on significant sequences of actions, or processes, that have a highest or best ROI for robotic automation. See par 58 - For example, a template may correspond to a business flow or process where a user performs some initial desktop event, and then may perform multiple sub-events within that event to adjust the data initially provided in the event. An exemplary template is shown in FIG. 5C having static data and dynamic data entered to a static template. For example, an initial copy-paste of a template may be filled with multiple copy-pastes of dynamic data into portions of the template).
(v) transforming the identified repetitive segments for automation of the complete tasks into code (Bialy – See par 32 - A real-time (RT) local interface 8 (e.g. a NICE Attended Robot provided by NICE, Ltd.) executing on terminals 2 may collect user action data, execute an automation sequence in place of user input. see par 107- Automation sequences may be identified by sequential pattern mining as repetitive sequences having corresponding user actions that are consecutive and/or within the same time-frame and are repeated within a stream of user actions, typically across different users; see par 108 - An automation sequence may include actions input by a bot to software applications; an automation sequence may have a process on a computer system perform this automation sequence for the user, to automatically and quickly complete the login process for the user. Typically automation actions such as business process actions are performed on screen elements (e.g. buttons, windows, dropdown menus, text entry fields) in various applications. see par 110 - FIG. 5 shows a high-level block diagram of an exemplary computing device which may be used with embodiments of the present invention).
Bialy discloses having bots (par 108) and instructions/code/executable code to perform tasks (see par 32, 107-108, 111-112). Smutko discloses having computer program instructions and code in any programming language (See par 26).
Van Grondelle discloses:
“(vi) executing the code at runtime as a dynamic linked library which is interacting with one or more applications, wherein said code is a set of instructions and logic.” (Van Grondelle – see par 58 - Some or all aspects of the invention may be implemented in a computer program product, i.e. a collection of computer program instructions stored on a computer readable storage device for execution by a computer. The instructions of the present invention may be in any interpretable or executable code mechanism, including but not limited to scripts, interpretable programs, dynamic link libraries (DLLs) or Java classes. The instructions can be provided as complete executable programs, as modifications to existing programs or extensions ("plugins") for existing programs.)
Bialy, Smutko, and Van Grondelle are analogous art as they are directed to identifying processes for automation (see Bialy Abstract, par 36; Smutko Abstract, par 13; Van Grondelle par 36 – executing business process). 1) Bialy discloses “Different sequences of actions or sentences may be merged or combined to larger sentences or sequences by for example taking actions with the same case ID, sorted by time to create a larger sentence” (See par 44); looking at conditions to determine next action does not belong to the current sentence (See par 42, 52), that clusters are for words with similar features (See par 92); that sequences can be merged or combined, even into one sentence (See par 106). Smutko improves upon Bialy by disclosing that it can look at “similar” sequences for automation, using only sequences of a meaningful length (See par 36, 47, 53, 55), where actions correspond to a “complete business flow” (par 19) that has clusters representing “similar” business functionalities and clusters with neighbor sequences and maximizes modularity (See par 55-56). One of ordinary skill in the art would be motivated to further include use sequences of a meaningful length as well as looking at similarity relative and associating with neighbor sequences to efficiently improve upon the merging or combining sentences for clustering in Bialy (See par 44, 92, 106). 2) Bialy discloses preprocessing events in an unsupervised manner and using desktop monitoring systems with interfaces (see par 36, 101). Smutko discloses using Word2vec to generate vectors of user actions (See par 53). Van Grondelle improves upon Bialy and Smutko by disclosing the use of DLL (dynamic link Library) to execute code by the computer. One of ordinary skill in the art would be motivated to further include a known “dynamic link library” for executing code/programs by a computer to efficiently improve upon the execution/code by a computer for automation in Bialy and Ma.
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the creating of sequences for automation sequences in Bialy to further explicitly use the filtering of partial tasks, and identifying sequences of tasks based on similarity in Smutko, and to further use a “dynamic link library” as disclosed in Van Grondelle, since the claimed invention is merely a combination of old elements, and in combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable and there is a reasonable expectation of success.
Concerning claim 2, Bialy and Smutko disclose:
The computer-implemented method of claim 1, wherein the similarity level of desktop-actions in each yielded segment is maximized by: (i) defining a target function, wherein the target function is
PNG
media_image1.png
82
232
media_image1.png
Greyscale
whereby:
PNG
media_image2.png
56
514
media_image2.png
Greyscale
For purposes of claim interpretation, Applicant’s specification [0046-0048] as filed state:
PNG
media_image3.png
528
698
media_image3.png
Greyscale
Bialy discloses the limitations based on broadest reasonable interpretation in light of the specification – See par 87-90 -
PNG
media_image4.png
452
500
media_image4.png
Greyscale
See also par 90-92 - A process may select the cluster with a smallest compactness score, e.g. the smallest mean distance between members, as the cluster including all case IDs (typically no other cluster will include a case ID). One embodiment may use a Levinshtein distance as the measure to evaluate clusters, e.g. determining the Levinshtein distance among all pairs within each cluster, or using the distance matrices, then averaging all the distances within each cluster, then picking cluster with minimum average distance as the cluster including case IDs. Other methods of evaluating clusters may be used. The most compact cluster, having the lowest compactness score, may indicate the cluster or group which has the most similar members in feature terms.
See also Smutko – see par 54 - At step 432, distance calculations are performed on sequences such that a distance between two sequences can be determined by weighing the Euclidean distance between vector representations of the two sequences with the Jaccard similarity between the two sequences. See par 55 - ] At step 434, using the distance between the sequences, clustering is performed to identify sequences with other similar sequences. A cluster represent sequences having similar business functionalities, such as sequences that implement the same or similar business processes for automation).
Bialy discloses that sequences can be merged or combined, even into one sentence (See par 106). It does not disclose the “penalty” aspect as best understood in the similarity calculation.
Smutko discloses:
PNG
media_image5.png
54
496
media_image5.png
Greyscale
For purposes of claim interpretation, Applicant’s specification [0056-0058] as filed state:
PNG
media_image6.png
500
684
media_image6.png
Greyscale
Smutko discloses the limitations based on broadest reasonable interpretation in light of the specification – See par 19 - Clients 110 may be utilized in order to collect user desktop events and provide event logs and processing history associated with the desktop events. A desktop event may correspond to any desktop activity of a user, for example, a text copy through mouse clicks and/or keyboard commands, a text paste, a left or right mouse click, a selection of a tab command, input of text data, or other input and activity of the user. The desktop event may also be associated with particular user actions, which is the execution of the particular desktop event (e.g., the input) within a certain application, at a certain time, and/or according to another parameter (e.g., interface location, elements, etc.). see par 47 - In flowchart 400a, sequence mining from a sentence database begins at step 402. The sentence database includes the sentences of user actions broken up from a stream of data from an event log of desktop events. At step 404, a length for sequences is set having a threshold length that is used for identification of initial sequences based on their support within the user action sentences. The length may be utilized so that sequences of only a meaningful length are determined and identified in the user action sentences. Thus, when determining a sequence data set from processing a sentence data set based on the initial minimum support for those sequences within the sentence data set, dynamically tuning the support may allow for manipulation of a size of the resulting sequence data set so that only meaningful sequences may be identified and used for automation; see par 55-56 - After determining the modularity for a sequence with its cluster, the sequence is then removed from the cluster and is associated with the cluster for its neighbor sequences. Iterating this over a database, the i-th sequence is associated with a cluster that maximizes the modularity.).
Obvious to combine Bialy and Smutko for the same reasons as claim 1.
Concerning claim 3, Bialy discloses:
The computer-implemented method of claim 1, wherein the UI data-handling operations of applications are collected from computer-devices of users by a Real-Time (RT) client that is running on each user computer-device (Bialy – see par 32 - A real-time (RT) local interface 8 (e.g. a NICE Attended Robot provided by NICE, Ltd.) executing on terminals 2 may collect user action data, execute an automation sequence in place of user input or provide or display a recommended next action to a user, according to automations created. see par 33 - RT local interface 8 may act as desk-top monitoring or client data collection software such as an activity recorder or action recorder and may monitor input to programs 6. For example RT local interface 8 may receive or collect a user's desktop activity or actions, e.g. low-level user action information or descriptions, and send or transmit them to a remote analytics server 20) and sends user desktop-actions to an RT server to be combined and exported to a database (Bialy – see par 33 - Remote analytics server 20 may collect or receive data such as user action information or descriptions and transmit or export them to for example a database 34.).
Concerning claim 4, Bialy discloses discovering more “significant” business processes for automation (See par 18), and mentions attributes of actions being “relevant” (See par 55).
The computer-implemented method of claim 1, wherein the preprocessing of the stream of data of desktop-actions comprises: removing UI data-handling operations that have been predetermined as insignificant (Bialy – see par 36 - Automation finder 24 may perform sequence mining, sequential pattern mining, finding repetitive sequences in data that contains a set of sentences; and a find process function, grouping the previously found sequences into processes, each process potentially describing a business process. The sequences may be filtered to identify the best ones of those sequences that have the highest ROI. Once significant ones are identified and named, those may be used to build one or more templates that permit entry of dynamic text when form filling or otherwise executing a business process.
See also Smutko – see par 14 - sentences of user actions act as a combination of several actions that express a particular business functionality. Using these sentences, repetitive sequences may be identified, which may be those sequences that have corresponding user actions that are consecutive and/or within the same time-frame and are repeated within a stream of user actions; see par 48- For example, in FIG. 5B… By “most significant,” this means relative to other sequences that have a lower ROI. The term most significant can also refer to a pre-selected number or percentage of sequences that a user or enterprise wants to consider; see par 52 - sub-sequence removal may proceed by filtering the sequences that are contained within a different sequence so that the higher scored sequences will be preferred. … Those sub-sequences are identified using the aforementioned process, at step 428. If such sequences exist, those sub-sequences are removed based on their score and the score of any parent sequences. Thus, when considering whether a sequence is significant for automation, or more significant than another sequence, the processes described herein may consider the sequence's ROI based on time for input and execution of the sequence by a user with additional factors. This scoring of sequences allows identification of a sequence's importance and significance, and further allows a quantitative measurement to be determined that allows comparison of sequences within a log of desktop events.)
Obvious to combine Bialy and Smutko for the same reasons as claim 1.
Concerning claim 5, Bialy discloses:
The computer-implemented method of claim 1, wherein the unsupervised task segmentation module is pretrained to… (Bialy – see par 36 - Automation finder 24 may preprocess collected low level actions or events; and form, segment or split (typically in an unsupervised manner) the stream of user actions into sentences each forming a segment, possibly bounded by time, of actions that form a sequence of a subset of the user actions. Identified sentences may describe an instance of a task. see par 101 - The collected actions may form a corpus of actions, typically from different users, possibly across different days, but typically a corpus is generated from users at one organization. The actions may be preprocessed: e.g. each action may be processed to be represented as a string or another form, and generalized, for use for example in pattern mining.), and wherein each … is a desktop-action related to UI data-handling operations of applications (Bialy – see par 36 - Automation finder 24 may identify automation opportunities by discovering repetitive sequences of actions, e.g. based on sentences, for example using desktop analytics and machine-learning; See par 33 - For example RT local interface 8 may receive or collect a user's desktop activity or actions, e.g. low-level user action information or descriptions, and send or transmit them to a remote analytics server 20; See par 101 - computer-based actions may be received or collected, e.g. from desk-top monitoring systems (e.g. RT local interface 8 (see FIG. 1 – this is on the agent terminal) or other systems) executed by computers used by agents.)
Van Grondelle discloses considering “context” when inferring an activity to reach a goal (See par 10). To the extent that Bialy and Van Grondelle do not explicitly state the pretraining is for embeddings of words, Smutko discloses:
The computer-implemented method of claim 1, wherein the unsupervised task segmentation module is pretrained to “learn word embeddings”, and wherein each “word” is a desktop-action related to UI data-handling operations of applications (Smutko See par 53 - Clustering of sequences may further be performed to identify the same or similar sequences and select a sequence of those for automation. In this regard, sequence descriptions are converted to vector representation of the user actions, at step 430. Actions may be represented as vectors using word2vec or other technique to generate word vectors from user actions. For example, a vectorization process may generate a vector from word and/or text input of the user action. Each description and/or parts of the description can be a vector. In some embodiments, word vectors for actions may be attained by running methods like word2vec, FastText, or GloVe, each of which defines a way of learning word vectors with useful properties. In some embodiments, pre-trained vectors of a certain dimensionality may be used. In some embodiments, the embedding may include partial vectors related to portions of a word within the actions.)
Obvious to combine Bialy and Smutko and Van Grondelle for the same reasons as claim 1. Bialy discloses preprocessing events in an unsupervised manner and using desktop monitoring systems with interfaces (see par 36, 101). Van Grondelle discloses considering “context” when inferring an activity to reach a goal (See par 10). Smutko improves upon Bialy and Van Grondelle by disclosing that it pretraining vectors, where vectors are from sequence descriptions for user actions designated for automation (See par 53). One of ordinary skill in the art would be motivated to further include pretraining vectors to efficiently improve upon the unsupervised learning with general preprocessing in Bialy (See par 36, 101).
Concerning claim 6, Smutko discloses:
The computer-implemented method of claim 5, wherein the unsupervised task segmentation module is pretrained to learn word embeddings by a Word2Vec algorithm (see Smutko - In some embodiments, word vectors for actions may be attained by running methods like word2vec, FastText, or GloVe, each of which defines a way of learning word vectors with useful properties. In some embodiments, pre-trained vectors of a certain dimensionality may be used. In some embodiments, the embedding may include partial vectors related to portions of a word within the actions).
Obvious to combine Bialy and Smutko and Van Grondelle for the same reasons as claim 1.
Response to Arguments
Applicant's arguments filed 1/27/26 have been fully considered but they are not persuasive and/or are moot in view of the new rejections.
With regards to 103, Applicant argues 1) Bialy and Smutko do not disclose “(iii) implementing unsupervised topic-segmentation Natural Language Processing (NLP) module on the created vector of embeddings to determine one or more cutting-points in the integer sequence to yield one or more segments such that semantic similarity level of embeddings in each yielded segment is maximized.” Remarks, page 6-7. In response, Examiner respectfully disagrees. The arguments are not persuasive, as they do not substantively address any of the citations in the rejection. In addition, none of the citations from the references are addressed. Bialy discloses, as applied in previous limitation (ii), that the clustering algorithm looks at words represented by vectors/features in Bialy paragraph 0090. This discloses “embeddings” In limitation (iii), where 0090-0091 is also cited along with other citations that have not been addressed (par 28 – similar actions share a name/description; paragraph 56 merging actions with common case id, par 36 – unsupervised splitting of user actions into sentences to form segments). Then Bialy further discloses in paragraph 58 clustering according to similar features raw action data as well as paragraph 92 looking for the most compact cluster which has the most similar members in feature terms. In explaining the combination, Bialy is cited for disclosing that sequences can be merged or combined, even into one sentence (See par 106). Smutko discloses “and a number of non-complete business processes is reduced” as it discloses “concatenation of several actions, which may or may not be consecutive but may be joined by a temporal similarity, may correspond to a complete business flow executed by a user” (See par 19), clustering similar desktop events into business processes for automation (See par 31), and having a minimum sequence threshold to omit… less likely sequences (See par 48).
Applicant then further argues that Bialy and Smutko do not disclose “2. Embedding-based representation.” Remarks, page 7. In response, Examiner respectfully disagrees. Examiner is not sure which limitation this refers to. This section of Applicant’s arguments then argues that Bialy and Smutko do not disclose “creating embeddings for UI desktop-actions or leveraging semantic similarity metrics.” Examiner is not sure which limitation this refers to. Examiner’s best guess for first portion of argument is the limitation “each desktop-action relates to User Interface (UI) data-handling operations of applications”. Bialy discloses in paragraph 36 collecting actions from users and in paragraph 26 that user terminals are used and in paragraph 33 collecting a user’s desktop activity or actions. None of the citations have been addressed by Applicant. Examiner’s best guess for second portion of argument is the limitation “such that semantic similarity level of embeddings in each yielded segment is maximized” and this was addressed above.
Applicant then further argues that Bialy and Smutko do not disclose “3. Transformation into Executable Code.” Remarks, page 7-8. In response, Examiner respectfully disagrees. Examiner’s best guess is this refers to “(v) transforming the identified repetitive segments for automation of the complete tasks into code.” None of the citations from the references are addressed. Bialy paragraph 32 has collecting user action data then executing an automation sequence in place of user input; and paragraphs 107-108 which include among other things that automation sequences include actions input by a bot to software applications. Applicant then argues the “(vi) executing the code at runtime as a dynamic linked library which is interacting with one or more applications, wherein said code is a set of instructions and logic” and argues that Van Grondelle merely “suggests” and “mentions” a dynamic linked library in paragraph 58. First, in response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Second, Bialy already discloses automating into automation code. Van Grondelle discloses a known “dynamic link library” for executing code to improve upon the code execution by a computer for automation in Bialy and Ma.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IVAN R GOLDBERG whose telephone number is (571)270-7949. The examiner can normally be reached 830AM - 430PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anita Coupe can be reached at 571-270-3614. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IVAN R GOLDBERG/Primary Examiner, Art Unit 3619
Prosecution Insights