DETAILED ACTION
This action is responsive to the Applicant’s response filed 5/04/26.
As indicated in Applicant’s response, claims 1, 4-5,7-10, 12-14, 17, 19, 21-22 have been amended, claim 2 cancelled and claims 25-29 added. Claims 1, 3-29 are pending one next office action.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 3, 7-8, 14-15, 29 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Tomkins et al, USPubN: 2021/0295822 (herein Tomkins).
As per claim 1, Tomkins discloses a method for generating a software application test case for an application, the method comprising:
providing a machine learning model ( neural network - para 0104; neural network - para 0264);
training the machine learning model (e.g. learning model using different sets of training data in a sequence train an instance with a first set of training data train the pre-trained instance with a second set of training data, pre-trained machine learning model - para 0105) with a base user interaction (user interaction, modify vertex of the ontology graph, modifying a relationship based on the UI interaction between pair of n-grams, change an n-gram weight or value update to the value may cause further updates to the set of neural network weights or other learning model parameters - para 0322; train a machine learning model in a first stage based on set of queries and responses predetermined set of questions and answers - para 0105; plurality of documents … set of domain vectors … sent … via an application program interface provided via a user interface … metadata associated with documents .. indicate … information relevant to … documents such as … domains, a count of words … specified number of syllables – para 0037) training dataset including a textual representation (e.g. embedding vectors or … learned representations representing words … n-grams of the plurality of documents … where the ontology graph may be usable as an index for the … documents … first word … may be associated with a second word … via an edge that categorizes the relationship – para 0040; documents referenced by index of ontology graph from below; question-answer pairs based on the knowledge graph … question-answer pairs in a index – para 0095; graph-indexed documents – para 0093; associating the first vertex with the second document or the third document – para 0095; document, metadata, domain vector – para 0037; user may be permitted to update the second set of ontology graphs by adding additional words, phrases or other n-grams – para 0299) of graphical user interface (ontology graph, graph-indexed documents, index may … indicate documents related to each other via vertex adjacency – para 0093-0094; ontology graph – para 0037, 0040) user interactions (see above; questions and answers – para 0105; sent … via an application program interface provided via a user interface – para 0037) to produce a base model (see interactive elements, update a hierarchical set of graphs, edit a connection between a first vertex and a second vertex, UI element may re-arrange blocks representing workflow operations - para 0167-0169; ontology graphs- para 0170; visual indicators based on ontology graphs - para 0303; machine learning model in a first stage based on set of queries and responses predetermined set of questions and answers - para 0105);
training the base model (blocks representing workflow operations - para 0167-0169; visual indicators based on ontology graphs – para 0303; hierarchical associations between vertices of knowledge graphs and vertex adjacency … question-answer pairs based on the knowledge graph .. question-answer pairs in a index – para 0095; machine learning model in a first stage … based on set of queries and responses – para 0105) with a application-specific training dataset (then use the initial set of parameters of the pre-trained to generate output that is usable as an input for a set of task-specific layers during the training of model 1214 - para 0183) including a textual representation (see above) of user interaction with a graphical user interface (see above) of a specific application (see neural network and ontology, n-grams NL learning - para 0193-0194, 0200-0201) to provide a fine-tuned model (updating the learning model may perform a set of fine-tuning operations by the fine tune training function 1220, function 1220 may apply from dataset 1206 data specific to an account or organization - para 0186) for the specific application; and
generating by the fine-tuned model a software application test case (domain "medical tests" - para 030) including a sequence of multiple user actions (e.g. n-grams and associated set of visual indicators UI elements may be interacted with set of requests to a server based on an input or configuration of the UI, a second message may include a n-gram indicated by a user and an update value corresponding to the n-gram - para 0306-0307) on the graphical user interface of the specific application (e.g. para 0193-0194, 0200-0201).
As per claim 3, Tomkins discloses method of claim 1, wherein the machine learning model is a generative pre-trained transformer (generative pre-trained transformer - para 0160).
As per claim 7, Tomkins discloses method of claim 1, further comprising forming the application-specific training dataset (refer to claim 1), wherein forming the application-specific training dataset includes recording a sequence of multiple user actions (data ingestion, processing workflow, dynamically update a UI with a workflow blocks - para 0336; interactive elements, update a hierarchical set of graphs, edit a connection between a first vertex and a second vertex, UI element may re-arrange blocks representing workflow operations - para 0167-0169; the same user may be permitted to update the second set of ontology graphs by adding additional words, phrases or other n-grams – para 0299) on the graphical user interface of the specific application.
As per claim 8, Tomkins discloses method of claim 7, wherein the forming further includes expressing the sequence of multiple user actions in a standard format for data interchange (interaction with the user send to a structured data store as JSON document - para 0316 ; causes an UI to render text from a natural language text rendered text includes a set of visual indicators indicating words or n-grams send UI data that includes structured data used to display the UI interpret the JSON file and update a UI based on the JSON file - para 0304; dataset representing a data ingestion and processing workflow. For example … data encoded a YAML format include square brackets and curly brackets – para 0301).
As per claim 14, Tomkins discloses a system for generating a software application test case for an application, the system comprising: a processor; and a computer-readable memory communicatively coupled to the processor, wherein the computer-readable memory stores:
a machine learning model (refer to claim 1);
a base user interaction training dataset (refer to claim 1) including a textual representation (embedding vectors or … learned representations representing words … n-grams of the plurality of documents … where the ontology graph may be usable as an index for the … documents … first word … may be associated with a second word … via an edge that categorizes the relationship – para 0040; documents referenced by index of ontology graph from below; question-answer pairs based on the knowledge graph … question-answer pairs in a index – para 0095; graph-indexed documents – para 0093; associating the first vertex with the second document or the third document – para 0095; document, metadata, domain vector – para 0037; user may be permitted to update the second set of ontology graphs by adding additional words, phrases or other n-grams – para 0299) of
graphical user interface user interaction (refer to claim 1); and
an application-specific training dataset (then use the initial set of parameters of the pre-trained to generate output that is usable as an input for a set of task-specific layers during the training of model 1214 - para 0183) including a textual representation (see above) of user interaction with a graphical user interface (see above) of a specific application (see neural network and ontology, n-grams NL learning - para 0193-0194, 0200-0201) and
instructions that, when executed by the processor, cause the system to perform:
training the machine learning model with the base user interaction training dataset (refer to claim 1) to produce a base model (refer to claim 1) and training the base model with the application-specific training dataset to provide a fine-tuned model (refer to claim 1) for the specific application; and
generating by the fine-tuned model a software application test case (refer to claim 1) including a sequence of multiple user actions (refer to claim 1) on the graphical user interface of the specific application (see above).
As per claim 15, Tomkins discloses system of claim 14, wherein the machine learning model is a generative pretrained transformer. (refer to claim 3)
As per claim 29, Tomkins discloses a program product for generating a software application test case for an application, the program product comprising: a non-transitory computer-readable storage device; instructions stored in the non-transient computer-readable storage device, wherein the
instructions, when executed by a processor of a data processing system, cause the data processing system to perform:
training a machine learning model with a base user interaction training dataset
including a textual representation of graphical user interface user interaction, wherein the training with the base user interaction training dataset produces a base model;
training the base model with an application-specific training dataset including a
textual representation of user interaction with a graphical user interface of a specific
application to provide a fine-tuned model for the specific application; and
generating, by the fine-tuned model, a software application test case including a sequence of multiple user actions on the graphical user interface of the specific application.
( All of which having been addressed in claim 1)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4-6, 10, 13, 16-18, 20, 22 is/are rejected under § 35 U.S.C. 103 as being unpatentable over Tomkins et al, USPubN: 2021/0295822 (herein Tomkins) in view of Li et al, CN 114238070, (translation), 03-25-2022, 12 pgs (herein Li) and Fong, USPubN: 2018/0349256 (herein Fong)
As per claims 4-5, Tomkins does not explicitly disclose method of claim 1,
wherein the base user interaction training dataset comprises a large corpus of a plurality of manually scripted software test cases;
wherein each of the plurality of manually scripted software test cases of the corpus is provided in a standard format for data serialization and/or data interchange.
Tomkins discloses text-based dataset as corpus (corpuses implemented with ontological objectives - para 0122; UI elements, user clicks on the first button, interaction with a third button display text documents from a corpus of text data - para 0354) representing base for (interactive elements, update a hierarchical set of graphs, edit a connection between a first vertex and a second vertex, UI element may re-arrange blocks representing workflow operations - para 0167-0169; ontology graphs- para 0170) the user interaction training set (data ingestion, processing workflow, dynamically update a UI with a workflow blocks - para 0336) which can be provided in data interchange format or data serialization format (data serialization formats, XML, JSON - para 0336); where a script can be used to implement various processing modules (para 0116) or query instances directed at the ontology data/vertices (para 0047) where operations to verify precision of dataset (e.g. SQUAD test dataset) comprising question-answer retrieval achieve improvement to a role-based domain of the operations facilitating AI categorization from basis of relationships associated with the oncology vertices of the graph (para 0050).
Use of pre-training of semantic recognition is shown in Li as generating test case, provided as pre-obtained from test case store (pg. 2) in form of extracted test script (pg. 2) to judge on dis- ambiguity associated with a predicate representation of translated keywords (pg. 8) obtained based on a pyTorch pre-training model (pg. 3), the semantic analysis by the test as part of disambiguation of the n-grams grammar/aspect of the language/speech (claim 4-5, pg. 11; disambiguation to part-of-speech tagging process – pg. 3) and making each sentence component semantically clear (pg. 4); hence use of test script included in pre-training to re-arrange or perform disambiguation to language (keyword translation) or grammar destined for semantic recognition under a neural network deep learning/translation (pg. 8) is recognized.
Further, Fong discloses test cases generated based on received natural language strings that are destined for a trained neural network in conjunction with a reinforcement learning model (see abstract), including legacy test automation data for a pre-training to identify correlation of the received input/dataset to an intent and functional aspect of the language, using the test to perform the correlating (par 0007-0008), so that the pre-trained neural network can make use of weights or values associated with interconnected nodes of the network, where dataset into the pretraining include natural language description of steps or user interactions with visual elements (para 0009), the pre-training configured via test scripts to pre-classify NL description, subclassify a description and/or sentiment (para 0029) and assigning weights, the latter based on performance or accuracy score generated from the test script (para 0027-0028) in support of the reinforcement learning engine that is configured to identify set of actions and pre-defined values representative of computations that can be made available within a target application (para 0032-0033). Hence, use of test case or scripts underlying a pretraining of NL input or language representing user interactions into a reinforced learning engine is recognized.
Therefore, as HTML structure, script, schema (XML, JSON) can be viewed as manually scripted structure or rendition, it would have been obvious for one of ordinary skill in the art before the effective filing date of the invention to implement the pre-training in Tomkins so that disambiguation of description language or syntax elements as well as pre-classification thereof to facilitate a subsequent classification or semantic recognition engine, would be using test cases implemented as manually formed script or standard/structured test case format - as set forth in Fong and Li - in the sense that training set provided as base user interaction training set represent a corpus of manually scripted software test cases - as per test case store in Li - with each manually scripted software test case of the corpus provided in a standard format for data serialization and/or data interchange; because
script as a standard structured format utilized for manually implementing text-based test case can be executed without use of a dedicated compiler, enabling this standard format to be interpreted within HTML, a browser or web friendly environment, and provision of test cases retrievable from of a corpus prestored in available test cases storage as set forth above (as per Li) would accelerate test configuration and testing of data being pre-trained, as in Tomkins, in accordance with pre- classification of text stream or NL representation of user interactions for which input element of weights or indicative of intent can be correlated (via effect of the test script) from within the semantic or grammar context of interactive operations and accordingly be assigned/quantified with commensurate merits or weight values, based on which the quantified merits can be deployed (embedded) as a vector configuration input into the actual training (follow-up to the pre-training stage), enabling this (neural network) stage as in Tomkins, to assess and determine the most r appropriate set of actions or recommendations commensurate to the received language expressing the user described UI activities (as in Fong), on basis of the pre-classified items of significance and intent-driven correlation achieved from the test script portion of the pretrained model.
As per claim 6, Tomkins does not explicitly disclose method of claim 1, wherein the base user interaction training dataset comprises a plurality of software test cases for a wide range of applications, a wide range of use cases, and/or has a reasonable distribution of test case lengths.
But pre-store of test data in form of test cases that can be retrieved and activated as scripts is shown in Li data storage (test case store - pg. 2); hence available data storage of SW test cases and test data to provision for a wide range of test applications, range of use cases is recognized.
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the invention to implement provision of pre-training test and pre-classification of input data in Tomkins's ontology-based machine learning so that the base user interaction training dataset comprises a plurality of software test cases for a wide range of applications, a wide range of use cases provided as large corpus storage of test cases; because
pre-existing availability of test cases and potential capability thereby to provide for various
test scenarios, diverse types of use cases or wide variety of test applications would ease test
developers with prompt identification, retrieval and formation of a desired test case so that
immediate semantic analytics, statement recognition and identification of keyword significance by
one such test case executing according to pre-training stage as set forth above, would enhance the
lexical grouping or semantic pre-classification of natural language input when textual description representative of user data is to be trained by a subsequent machine learning stage as in Tomkins;
and that in-depth learning by the latter would be able to derive user intent and numerical
qualification on interactions/operations of significance associated with that intent, which in turn
would enable the training to generate set of actions or configuration suggestions, design or
deployment set provided as recommendations in response to the user request for deep learning as in
Tomkins.
As per claim 10, Tomkins does not explicitly disclose method of claim 1, further comprising:
inputting to the fine-tuned model a sequence of user actions on the graphical user interface of the specific application (para 0193-0194; 0200-0201) to generate a software application test case including a sequence of user actions on the graphical user interface of the application;
scoring the software application test case using a reward model;
training a reinforcement learning model; and
adjusting weights of the fine-tuned model based on the reinforcement learning model.
But input user interactions recorded in a natural language or standard format such as JSON is shown in Tomkins (para 0304; interaction with the user send to a structured data store as JSON document - para 0316)
Generating test case to support pre-classification of semantic of significance (intent-driven keywords or words) via a pre-training for effect of configuring vector input (para 0015, 0018) into a in-depth learning model is shown in Fong; where datasets into the pretraining include natural language description of steps or user interactions with visual elements (para 0009), the pre-training configured via test scripts to pre-classify NL description, subclassify a description/sentiment (para 0029) and assigning weights, the latter based on performance as part a reward model/function (para 0023-0025) that ingests and stores accuracy scores (para 0028) generated from the automation test script (para 0027-0028) in support of the reinforcement learning engine (para 0033-0035) that evaluates merits (para 0034, 0090) stored in the reward model with respect to vindicated significance of UI actions, the convolutional NN underlying this reinforcement learning following the paradigm of a specific application established over neural network inference or training per effect correlating and connecting features and their weights for their tuning over a period time of a task (UI actions) to be performed from neurons performance and tuning of weights included with a neural network (para 0030); hence training a reinforcement model with tuning and adjusting weights based performance at a stage and time of the trained features is recognized.
Hence, it would have been obvious for one of ordinary skill in the art before the effective filing date of the invention to implement the stages of training of user interface data in Tomkins in-depth model training so that finetuning or in-depth training of the model includes
1) inputting a sequence of user actions on a graphical user interface of an application to generate a software application test case - as in Fong - comprising a sequence of user actions on a graphical user interface of the specific application - as in Fong;
2) scoring and recording thereof by the software application test case using a reward model/function as in Fong;
3) training a reinforcement learning model as in Fong and adjusting weights of the fine-tuned model based on stage/performance of reinforcement learning model and re-evaluation of rewards; because
automated test cases to acquire performance metrics from data grouping by a pre-training classification of user disparate type interactions supplied as text-based input into a deep learning (reinforced learning engine or RLE) would help the pre-classified data to be assigned with score - as via a reward model - which in turn would enable re-evaluation of the quantified reward by a RLE (reinforcement learning engine) that correlates the performance reward in association with the pre- classification of original set of UI actions, so to grant proper merits to the functions or user actions recognized from the pre-training, and prioritize implementation to all functions/actions deemed most worthed of the reward, for the benefit of a relevant user.
As per claim 13, Tomkins does not explicitly disclose method of claim 1, further comprising generating executable code in a programming language to perform the software application test case.
But use of test cases implemented from natural language text provided in manually scripted file in Fong can alternatively be automated as compiled binaries of object codes (para 0013) Hence, it would have been obvious at the time of the invention for one skill in the art to implement testing of the preprocessed input stream into the AI finetuning sequences of machine learning so that base user interaction training dataset can be reorganized with test arrangement, where test case can be in script format as well as executable code in a programming language format, both automated or configured to perform the software application test case; because
immediate semantic analytics, statement recognition and identification of keyword significance by test cases notably from those provided in programming language and compiled in binary format for very fast execution as part of a pre-training stage as set forth above, would boost up performance associated with action geared for the lexical grouping or semantic pre-classification of natural language input when textual description representative of user data is to be trained by a subsequent machine learning stage as in Tomkins; and that in-depth learning by the latter would be able to derive user intent as well as re-organization, assessment of user interactions or UI operations of significance matching that intent, which in turn would enable the training to generate set of actions or configuration suggestion, design or deployment provided as recommendations in response to the user request for deep learning as in Tomkins.
As per claims 16-17, refer to rejection of claims 4-5 respectively.
As per claim 18, refer to rejection of claim 6.
As per claim 20, refer to claim 8
As per claim 22, Tomkins discloses system of claim 14, the memory further comprising a reward model and a reinforcement learning model.
(refer to rationale of claim 10)
Claim(s) 9 is/are rejected under 35 U.S.C. 102(a)(1) as being unpatentable over Tomkins et al, USPubN: 2021/0295822 (herein Tomkins) in view of Burgis et al, USPubN: 2022/0116415 (herein Burgis) and Chen, Wen-Ke, CN 114117240 (translation), 07-08-2022, 31 pgs (herein ChenWK)
As per claim 9, Tomkins does not explicitly disclose method of claim 1, wherein the application-specific training dataset comprises:
(i) data describing sequences of user actions performed by multiple users interacting with the specific application;
(ii) data describing sequences of user actions performed by one or more users interacting with the specific application on multiple devices; and/or
(iii) data describing sequences of user actions performed by one or more users interacting with the specific application multiple different times.
As for (ii) and (iii),
Burgis discloses extraction of data for vector configuration (para 0046-0047) or grouping of insights in relation to machine learning trained classifiers using actionable insights on determined intent and interactions (para 0160), where the training datasets comprise a) user interactions captured within one or more predetermined of time windows to be mapped to likelihood prediction for each ML based model (para 0162) and where extraction module for accessing interaction datasets includes b) interaction between communication devices, including customer device and service provider device, the classification of such interactions to identify actionable insight categories based in part on the intent of a corresponding interaction (para 0084). Hence training dataset describing sequences of actions performed by users interacting with the application at a number of different times - per a) and on a number of devices - per b) is recognized.
As for (i),
Training dataset obtained from social interaction log is shown in ChenWK as comprising plurality of reference social group interaction, where the training is trained over vector to seek whether convergence based on (interaction intention) vector representation of a first community interaction log will match convergence based on (interaction intention) vector representation of a second community interaction log (bottom pg. 28 to pg. 29), each interaction intention vector set as reference for a respective interaction log in a plurality of user community interaction activities (pg. 28). Hence, vector configuration for AI training comprising training dataset indicative of sequences of user actions performed by multiple users interacting with a same social application is recognized.
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the invention to implement content of training dataset in Tomkins so that for configuring a application-specific training, the corresponding dataset would comprise:
1) data describing sequences of user actions performed by multiple users interacting with a specific application - as set forth in ChenWK;
2) data describing sequences of user actions performed by one or more users interacting with a specific application on multiple devices - as set forth in Burgis;
3) data describing sequences of user actions performed by one or more users interacting with a specific application multiple/different times; as set forth in Burgis; because
in the endeavor of structuring/preparing training data associated with classifying interactive elements or interacting entities associated with business, application, network environment for effect of recommending the most optimal functions, software implementations and options by which interactions and communication of information between the entities improve in performance and become more secure and effective, proper recording of information associated with the above recommendation purpose, would necessitate inclusion of activity type, the context or scale thereof, nature and number of entities or machines involved, as well as time information and occurrence frequency with which the activities occur; and by documenting or logging sequence of interactions in accordance to number of interacting users, number of devices in which their interactivity occurs, the number of times by which they occur, this well-defined set of information can be employed (pre- trained) for prompt categorization by intent, pre-classification per significance score, such that, based on the weight allotted to respective pre-category from the pre-training, additional AI fine- tuning runs can be deployed to find the most optimal/actionable recommendation in form of deployable component or software that best respond to various type of interaction paradigm, including a type that befits a numeric scale of participants/users, a type that can support defined amount of machines or devices involved in the communication network and/or the type that can accommodate interactivity demand that repeated reoccurs, extends or cycles over a time period.
Claims 11, 23 is/are rejected under § 35 U.S.C. 103 as being unpatentable over Tomkins et al, USPubN: 2021/0295822 (herein Tomkins) in view of Li et al, CN 114238070, (translation), 03-25-2022, 12 pgs (herein Li) and Fong, USPubN: 2018/0349256 (herein Fong) further in view of Zhuang et al, USPubN: 2021/0174209 (herein Zhuang)
As per claim 11, Tomkins does not explicitly disclose method of claim 10, wherein the reinforcement learning model is a Proximal Policy Optimization (PPO) model.
However, use of machine learning to finetune over previous stage of pre-classification of relatively raw training set was a known practice, including the finetuning in form of reinforcement learning model such as in Fong (para 0165-0168, 0174; Fig. 2); and implementing this reinforcement learning into a Proximal Policy Optimization (PPO) model was also a known practice; and this is shown in Zhuang finetuning of neural networks, where consecutive update(s) to parameters of successive neural network runs includes improving a gradient based a loss function evaluated a later neural network relative to the loss function considered from a previous NN run, per a reinforcement learning algorithm that utilizes a proximal policy optimization algorithm (para 0012, 0024, 0036) for redefining a trust region of probability gradient.
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the invention to implement finetuning of convergence function by successive AI execution in Tomkins so that the AI fine-tuning effects can achieve a reinforcement learning that can adopt optimization algorithm in the type of Proximal Policy Optimization (PPO) technique as in Zhaung; because
PPO approach would better stabilize likelihood of swaying by the parameters being tuned under the reinforcement learning, thereby reducing unreliable zone to be processed for convergence in regard to a loss function revisited from respective neural network runs, making the reinforcement learning less vulnerable to updates volatility and unreliable convergence outcome.
As per claim 23, Tomkins discloses system of claim 22, wherein the reinforcement learning model is a Proximal Policy Optimization (PPO) model.
Claims 12, 24 is/are rejected under § 35 U.S.C. 103 as being unpatentable over Tomkins et al, USPubN: 2021/0295822 (herein Tomkins) in view of Paul, USPubN: 2022/0116873 (herein Paul)
As per claim 12, Tomkins does not explicitly disclose method of claim 1, further comprising:
providing a runtime agent;
requesting by the runtime agent a predicted next step from the fine-tuned model for an
the specific application at runtime of the specific application; and
executing the predicted next step on the specific application.
Paul discloses inferencing engine execution on decision state on corresponding video, media requests using scheduling assistance by a intelligent QoS runtime agent effect of fetching of a next request deemed in tune with a condition, in support of rendering decision update associated a trained model configured (para 0045) for predicting parameters configurable with streaming in accordance to QoS; hence, executing a predicted step based on loading/prefetching a conformant request invoked by a runtime agent to enable next decision rendering by an inference engine underlying QoS-conforming predictive training of streaming parameters entails requesting by a runtime agent input load into a next finetuning engine associated with a predictive model.
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the invention to implement AI finetuning and pre-training of raw data (incoming requests) as part of carrying out application-specific training in Tomkins so that pretrained information is under control of runtime agent to support loading of proper input into a finetuning phase of the AI sequenced execution in Tomkins, so that a request executing under the runtime agent feeds to proper input (compliant request) into a inference engine of the finetuning stage, based on which a next prediction step can render decision in accordance to flow of prediction by the AI finetuning paradigm; because
use of runtime agent as a pluggable and self-contained program entity configured to request proper loading in support for a desired predictive rendering by a training AI model would boost effectiveness of a pre-classifying stage as in Tomkins wherein employ of the agent software pre- positioned in conjunction with the pre-training to properly assist with proper input load for the AI training engine to render a next prediction step with an outcome deemed more conducive with the finetuning aim in the sense that effect of such rendering would progressively shorten time complexity in achieving input/output convergence targeted by the finetuning aspect of the training process.
As per claim 24, Tomkins discloses system of claim 14, further comprising a runtime agent.
(refer to rationale of claim 12)
Claims 19 is/are rejected under § 35 U.S.C. 103 as being unpatentable over Tomkins et al, USPubN: 2021/0295822 (herein Tomkins) in view of Yuile, USPubN: 2021/0157583 (herein Yuile)
As per claim 19, Tomkins does not explicitly disclose system of claim 14, wherein the instructions for cause the system to perform:
recording user actions on the graphical user interface of the specific application to provide a dataset of sequential user actions.
Yuile discloses code snippets articulating logging behavior of different applications for use in a architecture overview for program code analysis, runtime analysis and investigation analysis of code execution (para 0014-0015; Figs 5) the snippet provide as Log Message string (Fig. 9A, 9B) and illustrating logging of different applications inside an event-handling application (para 0061)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the invention to implement recording of user interactions and UI activities in the pretraining of Tomkins so that code snippet - as in Yuile - comprising instructions for recording user actions on a graphical user interface of the application to provide a dataset of sequential user actions; because
snippets of code can be portable, created on the fly, and easily integrated inside event- handling applications or program code analyses, or runtime analysis where behavior or events captured by the snippets when plugged inside these application can provide immediate insights or patterns by which the application can derive analytic data or behavioral intent of significance with which to render proposed options or problem solving component as response to clients requesting recommendations and analytics service.
Claims 21 is/are rejected under § 35 U.S.C. 103 as being unpatentable over Tomkins et al, USPubN: 2021/0295822 (herein Tomkins) in view of Sianez, USPubN: 2021/0191925 (herein Sianez)
As per claim 21, Tomkins does not explicitly disclose system of claim 14, further comprising a dictionary that: comprises a list of integers and a vocabulary of words or sub-words; and
defines a one-to-one correspondence between each integer of the list of integers and each
word or sub-word of the vocabulary.
Sianez discloses a processing framework (para 0046) with query of term identity as part of extracting feature of data batch from corpus of a dictionary (para 0035, 0049) that includes terms, references and numerical data in relation to words, pronouns, adverbs etc. metadata, and special characterizations, all as query features configured with identifiers, where each feature for such query term identifier can be associated with unique hash value, the numerical representation thereof susceptible of defining a feature vector provided as vector representation library supporting machine learning associated with text-to-speech training model (para 0031) Hence, use of dictionary corpus comprising words, sub-words, and numerical representation configured to provide one-to-one correspondence between a unique number (among a list of hash values) and each word of the vocabulary is recognized.
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the invention to implement the natural language processing and feature extraction of request in Tomkins pretraining stage so that understanding the natural language part of the user interaction in this stage would include corpus information -as in Sianez - from a dictionary, the latter comprising list of integers such as a hash identifier and a vocabulary of words (or sub-words) and defining a one-to-one correspondence between each integer of the list of integers (hash value) and each word or sub-word of the vocabulary as set forth in Sianez; because
use of a dictionary as corpus of words and sub-words serving as reference standard within the process of validating and categorizing natural language terms or concepts encountered with the incoming requests of Tomkins AI intelligence system would consolidate lexicographic weight of a given word and possibly allow intent therefrom to be identified, and use of numerical reference provided as number-word (one-to-one) correspondence as set forth above would enable this numerical representation to populate a vector configuration - i.e. the one or more significantly filtered features or lexicographic entity being represented as a numerical value extracted from a dictionary from the NL preprocessing stage -- SO that the so-expressed vectors can be input into a deeper learning stage whereby intent-driven formulation or application-specific text or UI patterns can be further subjected to a finetuning AI model, according to whose evaluation/assessment, the most optimal implementation recommendations can be determined and returned to the users whose interaction sequences are provided as incoming request into the initial processing stage.
Claims 25-26, 28 is/are rejected under § 35 U.S.C. 103 as being unpatentable over Tomkins et al, USPubN: 2021/0295822 (herein Tomkins) in view of Fong, USPubN: 2018/0349256 (herein Fong) and Eelen, USPubN: 2023/0259788 (herein Eelen)
As per claims 25-26, Tomkins does not explicitly disclose method of claim 1,
(i)wherein the textual representation of user interaction with the graphical user interface of the specific application is in a JSON format;
(ii) wherein the textual representation of user interaction with the graphical user interface of the specific application includes a data structure including key-value pairs.
As for (i)
However, user interaction with a ontology graph include words and text from document referred to by the vertices of the graph (e.g. user may be permitted to update the second set of ontology graphs by adding words, phrases, or other n-grams – para 0299 ), where reversed mapping of the UI effects previously captured/stored on a JSON file (where the rendered text includes a set of indicators indicating one or more words or … n-grams that map vertices of a ontology map …where the structured data may be used to update display of the UI. For example … send a JSON file to a client … the client computing device may … interpret the JSON file and update a UI based on the JSON file -para 0304) in Tomkins entails that a JSON file can be used to recapture or update user entries or definition made into the ontology map, notably when the workflow entries or manual ingestion have been recorded in a YAML format (dataset representing data ingestion and processing workflow … encoded a YAML format- para 0301) as one translatable into a exchangeable format of the JSON structure as source for reverse updating (set of structured data store as a JSON document to indicate one or more updates to the UI – para 0316; client computing device may … interpret the JSON file and update a UI based on the JSON file -para 0304) the UI or graphical representation.
Eelen discloses visual editor supporting visual programming and rendering of a graphical representation of a neural network model having edges connecting one or more layers, according to which, user editing on the graphical representation can be stored in a JSON data (para 0057), the JSON being a protocol used to capture image and data of the workflow of a training model, which can be used to update and retrain the dataset to find previously missed errors by the production process (para 0055) as part of the data preprocessing, engineering and model evaluation per an open-source software for machine learning (para 0056)
As for (ii)
Eelen discloses use of JSON as text-based lightweight language-independent interchange format to facilitate effect of read and write by humans and parsing by machines, where each JSON file can be used to store attribute-value pairs and arrays as well as complex data structures like lists, dictionaries or nested objects (para 0056) transportable via a JSON format (Fig. 8); e.g. for building training data, libraries and packages of a Tensorflow that enables training of a machine learning models
Similar to using Python framework in Eelen in association with Tensor protocol and python dictionary object, Fong discloses data provided as description in natural language being reconverted, reconfigured in key-value pairings as part of packaged data to support a reinforcement model (para 0162)
Therefore, based on reinterpreting of the tensor operations and optimizing tensorizing workflows (para 0343) to update the UI based on learning-adapted compiler and learning operations under a neural network in Tomkins, it would have been obvious for one of ordinary skill in the art before the effective filing date of the invention to implement use of YAML and JSON for capturing UI data and user interaction to the graphical ontology so that
textual representation of user interaction with the graphical user interface of the specific application can be recorded in a JSON format – as per Eelen;
wherein the textual representation of user interaction with the graphical user interface of the specific application such as a Python, Ruby or Tensor based framework machine learning application – as pee Eelen or Fong - includes a natural language data structured in form of key-value pairs as set forth in Fong; because
textual representation capturing user interactions and addition to a graphical modeling context or ontology representation thereof can be structured into a more human interpretable file format such as YAML file or a portable data interchange format of a JSON, which in turn can be transportable to different UI configuration contexts and used as a reference source by which to reproduce the user interactions on a UI or update the previously stored UI interaction due to the interchangeability afforded with this JSON methodology; and textual representation of the user interaction with a graphical model provided in natural language structured as value-key pairings a set forth above can facilitate a vector-based configuration of a dataset into a machine learning as set forth in Fong; the value-key pairs enabling establishing of correspondence between a value and a unique key as part of a deep management or security setting by which a object or an attribute has been uniquely recorded in a separate environment and uniquely identified and fetched into a different configuration environment for a trusted use thereon.
As per claim 28, Tomkins discloses system of claim 14, wherein the textual representation of user interaction with the graphical user interface of the specific application is in a JSON format.
(refer to rationale of claim 25)
Claims 27 is/are rejected under § 35 U.S.C. 103 as being unpatentable over Tomkins et al, USPubN: 2021/0295822 (herein Tomkins) in view of Fong, USPubN: 2018/0349256 (herein Fong) and Sianez, USPubN: 2021/0191925 (herein Sianez)
As per claim 27, Tomkins does not explicitly disclose method of claim 1, further comprising segmenting the textual representation of user interaction with the graphical user interface into sentences and tokenizing the sentences into words or sub-words to produce a sequence of integers corresponding to the words or sub-words using a dictionary that defines a one-to-one correspondence between each integer and each word or sub-word.
Fong discloses natural language stream decomposed into key-value pairings (para 0162) and this entails a structuring of initial NL stream in tokenized elements that define each, a one-to-one correspondence between a integer value and each word, as part of tokenized sections (para 0012, 0087) representation of the original natural language.
Further, tokenizing a stream via a dictionary that comprises a list of integers and a vocabulary of words or sub-words, and which defines a one-to-one correspondence between each integer of the list of integers and each word or sub-word of the vocabulary has been rendered with the rationale of claim 21 from above, with the teachings by Sianez using tokenized words or phrases (para 0094)
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the invention to implement structured decomposition of the text representation of user interaction with the graphical user interface into sentences and tokenizing thereof into words or sub-words to produce a sequence of integers corresponding to the words or sub-words using a dictionary that defines a one-to-one correspondence (e.g. key-value pairings in Fong or in word-integer one-to-one mapping in Sianez) between each integer and each word or sub-word in support for the configuration of ontology-based machine learning in Tomkins and so for the same reasons set forth with rationale of claim 21; the value-key pairs enabling establishing of correspondence between a value/word and a unique key as part of a deep management or security setting by which a object, a token or an attribute has been uniquely set in storage at a separate environment and uniquely fetched into a different configuration environment for a trusted use thereon.
Response to Arguments
Applicant's arguments filed 5/4/26 have been fully considered but they are not persuasive. Following are the Examiner’s observations in regard thereto.
(A) The Applicant has submitted that the rather conventional pre-training by Tomkins using ontology data and JSON as payload for rendering U is neither a user interaction based type dataset for training, nor an application-specific training dataset as amended in claim 1 (Applicant's Remarks pg. 11). The language of claim 1 as amended has been met with a adjusted grounds of rejection and any allegation on the merits of the newly introduced limitations cannot be viewed as compliant with the actual state of the prosecution.
(B) The Applicant has submitted that claims 2-3,7-8, 15 are patentable over Tomkins by virtue of their dependency to claim 1, and the rejections of claims 4-6, 9-13, 16-24 should be withdrawn as none of the cited Li, Fong, Burgis, Chen, Zhuang, Paul, Yuile, and Sianez reasonably teach or suggest training datasets including textual representation of user interaction with a graphical user interface (Applicant's Remarks pg. 12-13). The allowability of the independent claim 1 cannot be ascertained as the arguments provided mainly assert patentability of a language that is newly introduced and addressed a first time as per a prima facie case of prosecution responsive to the amendment; thus, allegation on patentability of the amended language is deemed largely premature.
In all, the claims as presented submitted stand rejected as presented in the latest prosecution set forth above.
Conclusion
THIS ACTION IS MADE FINAL. The Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Tuan A Vu whose telephone number is (571) 272-3735. The examiner can normally be reached on 8AM-4:30PM/Mon-Fri.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Chat Do can be reached on (571)272-3721.
The fax phone number for the organization where this application or proceeding is assigned is (571) 273-3735 ( for non-official correspondence - please consult Examiner before using) or 571-273-8300 ( for official correspondence) or redirected to customer service at 571-272-3609.
Any inquiry of a general nature or relating to the status of this application should be directed to the TC 2100 Group receptionist: 571-272-2100.
/Tuan A Vu/
Primary Examiner, Art Unit 2193
July 15, 2026
Prosecution Insights