DETAILED ACTION
This action is in response to the filing of 8-28-2025. Claims 1-20 are pending and have been considered below:
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 rejected under 35 U.S.C. 101 have been withdrawn.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 8, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Tommasi (US 2021/0374558 A1) in view of “Visualizing multi-dimension decision boundaries in 2d”, Migut et al., pages 1-19 © 2013.
Regarding claim 1, Tommasi teaches the invention substantially as claimed, including:
A method for enabling visual editing of machine learning models by one or more processors comprising: (in Tommasi [0065]: As previously mentioned, the present invention provides for visualization [visual] and exploration of probabilistic models [of machine learning models] in a computing environment.; in Tommasi [0020]: Which feedback may be used for enhancing a machine learning training operation and adjusting [editing], correcting, and/or updating one or more application/software library suggestions.; in Tommasi [0016]: According, various embodiments as described herein for providing intelligent library management in a computing environment by a processor [by one or more processors], are provided.)
providing visualization and exploration of an interactive representation of a plurality of datasets and decision boundaries of one or more machine learning models built upon multidimensional dataset; (in Tommasi [0065]: As previously mentioned, the present invention provides for visualization and exploration [providing visualization and exploration] of probabilistic models in a computing environment. A multidimensional dataset may be received. The multidimensional dataset may be processed according to booting operation parameters. A visualization and exploration of an interactive representation [of an interactive representation] of one or more probabilistic models [one or more machine learning models] using multidimensional dataset [built upon multidimensional dataset].; in Tommasi [0073]:The recommendation component 404, in association with the library component 408 and/or the training learning component 406, may rank the list of recommended libraries according to the degree of relevance or compatibility for implementation, integration, or replacements of one or more sections [decision boundaries] of the one or more application projects. The recommendation component 404, in association with the feedback component [interactive representation] 414, the library component 408 and/or the training/learning component 406, may initiate a machine learning component to collect feedback and learn the one or more application projects from the plurality of libraries, the data sources [a plurality of datasets], or a combination thereof.])
and editing behavior of the one or more machine learning models via the interactive representation using one or more logical rules or moving the decision boundaries of one or more machine learning models. (in Tommasi [0020]: the feedback from the application/software developers assists in learning that the suggestion is incorrect, non-applicable/non-compatible, useless, and/or a wrong recommendation, which feedback [via the interactive representation] may be used for enhancing a machine learning training operation and adjusting [editing the behavior of the one or more machine learning models], correcting, and/or updating one or more application/software library suggestions.; in Tommasi [0087]: A training operation may be one or a combination of any machine learning algorithm (e.g., Bayesian network, collaborative filtering, business rules [using one or more logical rules], etc.).)
Tommasi does not appear to explicitly teach
displaying a set of decision boundaries using an interactive representation for said one or
more machine learning models with respect to multiple dimensions of said multidimensional dataset;
obtaining a user feedback through a visual interactive interface;
Migut teaches the visualizing decision boundaries of multi-dimension models and further providing user input for feedback
Abstract: In many applications well informed decisions have to be made based on analysis of multi-dimensional data. The decision making process can be supported by various automated classification models. To obtain an intuitive understanding of the classification model interactive visualizations are essential. We argue that this is best done by a series of interactive 2D scatterplots. We define a set of characteristics of the multi-dimensional classification model that have to be visually represented. To present those characteristics for both linear and non-linear methods, we combine visualization of the Voronoi based representation of multi-dimensional decision boundaries in scatterplots with visualization of the distances to the multi-dimensional boundary of all the data elements. We use interactive decision point selection on the ROC curve to allow the decision maker to refine the threshold of the classification model and instantly observe the results. We show how the combination of those techniques allows exploration of multi-dimensional decision boundaries in 2D.
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Tommasi and Migut to include the decision boundary updates and ability to be displayed in Tommasi’s visualization and exploration system that edits behavior of machine learning models. One would have been motivated to make such a combination in order to enable the user to effectively and efficiently review content of interest through an improved feedback method.
Regarding claim 8, Tommasi and Migut teach similar features of claim 1 and further teach:
A system for enabling visual editing of machine learning models in a computing environment, comprising: one or more computers with executable instructions that when executed cause the system to: (in Tommasi [0050]: Computer system/server [in a computing environment] 12 may be described in the general context of computer system-executable instructions [with executable instructions], such as program modules, being executed by a computer system [one or more computers].)
Regarding claim 15, Tommasi and Migut teach similar features of claim 1 and further teach: A computer program product for enabling visual editing of machine learning models, the computer program product comprising: one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instruction comprising: (The functionality [the computer program product] 900 may be implemented as a method executed as instructions on a machine, where the instructions are included [program instructions collectively stored] on at least one computer readable medium [one or more computer readable storage medium] or on a non-transitory machine-readable storage medium. The functionality 900 may start in block 902.)
Claims 2, 9, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Tommasi and “Visualizing multi-dimension decision boundaries in 2d”, Migut et al., pages 1-19 © 2013 in further view of Lam et al. (US 2022/0138505 A1).
Regarding claim 2, Tommasi teaches the limitations from claim 1 as mentioned above.
However, Tommasi does not appear to explicitly teach wherein editing behavior of the one or more machine learning models further includes building one or more logic statements of the one or more logical rules.
However, Lam teaches wherein editing behavior of the one or more machine learning models further includes building one or more logic statements of the one or more logical rules. (in Lam [0014]: In some examples, a human-understandable representation of one or more of the rules may be generated, allowing human users to view or otherwise comprehend the nature of the rules [of the one or more logical rules] as applied to a given input sample.; in Lam [0019]: As used herein, a statement [building one or more logic statements] that an element is “for” a particular purpose may mean that the element performs a certain function or is configured to carry out one or more particular steps or operations, as described herein.; in lam [0021]: Unless otherwise indicated , “DNN Representation” refers to generating rules [of the one or more logical rules] or classifiers that approximate the behavior of the DNN , whereas “DNN Interpretation” refers to generation of human-understandable representations of the behavior of the DNN.; in Lam [0022]: As used herein, the terms "classification” and “categorization” are used interchangeably and synonymously (as are “classify” and “categorize”, “class and "category”, “classifier” and “categorizer”, etc.). In some examples, the behavior of a classification model may be described as classifying an input sample into a first category or a second category; it will be appreciated that each of the first and second category may include multiple categories, e.g., the first category may be " dog ” whereas the second category may include multiple categories (e.g., "cat”, “human”, and “truck”), none of which are "dog”, such that the second category may be regarded as "not dog" [building one or more logic statements].)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Tommasi and Lam before them, to include Lam’s use of logic statements in Tommasi’s visualization and exploration system that edits behavior of machine learning models. One would have been motivated to make such a combination in order to improve a user’s comprehension of the model by allowing the user to know which certain functions, steps or operations are being carried out as taught by Lam ([0019]).
Regarding claim 9, the claim recites similar limitation as corresponding claim 2 and is rejected for similar reasons as claim 2 using similar teachings and rationale.
Regarding claim 16, the claim recites similar limitation as corresponding claim 2 and is rejected for similar reasons as claim 2 using similar teachings and rationale.
Claims 3 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Tommasi and “Visualizing multi-dimension decision boundaries in 2d”, Migut et al., pages 1-19 © 2013 and Lam et al. (US 2022/0138505 A1) in further view of Thibodeaux (US 2023/0215206 A1).
Regarding claim 3, Tommasi teaches the limitations from claim 1 as mentioned above.
However, Tommasi does not appear to explicitly teach wherein editing behavior of the one or more machine learning models further includes: relabeling and generating a training dataset; and displaying one or more updated decision boundaries of one or more machine learning models.
However, Lam teaches wherein editing behavior of the one or more machine learning models further includes: relabeling and generating a training dataset; (in Lam [0109]: In some embodiments, the DNN 500 may be trained using the same input samples used by method 600. Each updated input sample [a training dataset] 512 may be relabeled [relabeling] based on its respective classification data 514 generated [generating] by the DNN 500: e.g., a first input sample 502(1) that is classified by the DNN 500 as most likely in the first category “dog” in the classification data 514 (which is shown as indicating 0.993 probability of being in category “dog” and 0.007 probability of being in category “cat”) may have its respective updated input sample 512 labelled as first category “dog”.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Tommasi and Lam before them, to include Lam’s ability to relabel and generate training data in Tommasi’s visualization and exploration system that edits behavior of machine learning models. One would have been motivated to make such a combination in order to increase the accuracy of the model by improving the probability of being in each of the respective updated classifications and categories as taught by Lam ([0109-0110]).
However, Tommasi and Migut disclose decision boundaries, Thibodeaux is also disclosed for displaying one or more updated decision boundaries of one or more machine learning models.
Thibodeaux teaches and displaying one or more updated decision boundaries of one or more machine learning models. (in Thibodeaux [0032]: Once all of the backend generated labels have been processed for regrouping (affirmative outcome to decision step 406), then the updated resume data is output and/or stored along with the initial bounding block data and regrouped bounding block data at step 408. As disclosed herein, the output and storage processing may be implemented by a combination of client-facing frontend code and backend code which are configured to store and/or display [displaying] the updated resume data that includes any regrouped bounding blocks [updated decision boundary] along with updated labels provided through user feedback.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Tommasi and Thibodeaux’s before them, to include Thibodeaux’s decision boundary updates and ability to be displayed in Tommasi’s visualization and exploration system that edits behavior of machine learning models. One would have been motivated to make such a combination in order to enable the user to visually see all detected blocks of text on their resume as bounding blocks, and to prompt the user to select all bounding blocks which relate to a backend generated label as taught by Thibodeaux ([0030]).
Regarding claim 10, the claim recites similar limitation as corresponding claim 3 and is rejected for similar reasons as claim 3 using similar teachings and rationale.
Claims 4 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Tommasi and “Visualizing multi-dimension decision boundaries in 2d”, Migut et al., pages 1-19 © 2013 in further view of Smith et al. (US 2020/0150937 A1) and Thibodeaux (US 2023/0215206 A1).
Regarding claim 4, Tommasi teaches the limitations from claim 1 as mentioned above.
However, Tommasi does not appear to explicitly teach wherein editing behavior of the one or more machine learning models further includes comparing previous decision boundaries with one or more relocated decision boundaries, wherein both the previous decision boundaries and the one or more relocated decision boundaries are displayed via the interactive representation.
However, Smith teaches wherein editing behavior of the one or more machine learning models further includes comparing previous decision boundaries with one or more relocated decision boundaries (in Smith [0035]: The machine learning model 200 has internal parameters that determine its decision boundary [previous decision boundaries] and that determine the output that the machine learning model 200 produces. After each training iteration, comprising inputting the input object 210 of a training example in to the machine learning model 200, the actual output 208 of the machine learning model 200 for the input object 210 is compared [comparing] to the desired output value 212. One or more internal parameters 202 of the machine learning model 200 may be adjusted such that, upon running the machine learning model 200 with the new parameters [one or more relocated decision boundaries], the produced output 208 will be closer to the desired output value 212. If the produced output 208 was already identical to the desired output value 212, then the internal parameters 202 of the machine learning model 200 may be adjusted to reinforce and strengthen those parameters that caused the correct output and reduce and weaken parameters that tended to move away from the correct output.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Tommasi and Smith before them, to include Smith’s comparison of decision boundaries in Tommasi’s visualization and exploration system that edits behavior of machine learning models. One would have been motivated to make such a combination in order to improve the model by adjusting to reinforce and strengthen those parameters that caused the correct output and reduce and weaken parameters that tended to move away from the correct output as taught by Smith ([0035]).
However, Tommasi in view of Smith does not appear to explicitly teach wherein both the previous decision boundaries and the one or more relocated decision boundaries are displayed via the interactive representation.
However, Thibodeaux further teaches wherein both the previous decision boundaries and the one or more relocated decision boundaries are displayed via the interactive representation. (in Thibodeaux [0032]: Once all of the backend generated labels have been processed for regrouping (affirmative outcome to decision step 406), then the updated resume data is output and/or stored along with the initial bounding block [previous decision boundaries] data and regrouped bounding block data at step 408. As disclosed herein, the output and storage processing may be implemented by a combination of client-facing frontend code and backend code [via the interactive representation] which are configured to store and/or display [displayed] the updated resume data that includes any regrouped bounding blocks [and the one or more relocated decision boundaries] along with updated labels provided through user feedback.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Tommasi, Smith and Thibodeaux before them, to include Thibodeaux’s ability to display decision boundaries in Tommasi and Smith’s visualization and exploration system that edits behavior of machine learning models by comparing decision boundaries. One would have been motivated to make such a combination in order to allow users to have a better understanding and interpretation of the model as taught by Thibodeaux ([0037-0038]).
Regarding claim 11, the claim recites similar limitation as corresponding claim 4 and is rejected for similar reasons as claim 4 using similar teachings and rationale.
Claims 5, 12, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Tommasi and “Visualizing multi-dimension decision boundaries in 2d”, Migut et al., pages 1-19 © 2013 in further view of Siebel et al. (US 2021/0263945 A1).
Regarding claim 5, Tommasi teaches the limitations from claim 1 as mentioned above.
However, Tommasi does not appear to explicitly teach wherein editing behavior of the one or more machine learning models further includes identifying and tracking each of one or more changes to the behavior of the one or more machine learning models.
However, Siebel further teaches wherein editing behavior of the one or more machine learning models further includes identifying and tracking each of the changes to the behavior of the one or more machine learning models. (in Siebel [0324]: Smart sensor and meter investment can be leveraged to derive accurate predictive models of behavior [the behavior of the one or more machine learning models], performance, or operations relating to the enterprise. ... The performance of one component or aspect of operations of an enterprise can be compared to identify outliers for potential responsive measures (e.g., improvements). The effectiveness of responsive measures can be tracked [tracking], measured, and quantified to identify [identifying] those that provide the highest impact [changes to the behavior] and greatest return on investment).
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Tommasi and Siebel before them, to include Siebel’s ability to identify and track in Tommasi’s visualization and exploration system that edits behavior of machine learning models. One would have been motivated to make such a combination in order to optimize enterprise operations and understand the return or investments as taught by Siebel ([0324]).
Also see Migut (abstract)
Regarding claim 12, the claim recites similar limitation as corresponding claim 5 and is rejected for similar reasons as claim 5 using similar teachings and rationale.
Regarding claim 18, the claim recites similar limitation as corresponding claim 5 and is rejected for similar reasons as claim 5 using similar teachings and rationale.
Claims 6, 13, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Tommasi and “Visualizing multi-dimension decision boundaries in 2d”, Migut et al., pages 1-19 © 2013 in further view of Ray et al. (US 2021/0073576 A1) and Lam et al. (US 2022/0138505 A1).
Regarding claim 6, Tommasi teaches the limitations of claim 1 as described above.
However, Tommasi does not appear to explicitly teach further including: receiving a dataset and one or more feedback decision rules for a plurality of predictions by the one or more machine learning models; generating an updated dataset based on the dataset and the one or more feedback decision rules; and moving a decision boundary of one or more machine learning models based on the updated dataset.
However, Ray teaches further including: receiving a dataset and one or more feedback decision rules for a plurality of predictions by the one or more machine learning models; (in Ray [0005]: The method comprises receiving, from a number of data sources, data [receiving a dataset].; in Ray [claim 8]: updating the trend clusters and correlation rules [decision rules] according to customer feedback [feedback] and updated data from the data sources regarding choices.; in Ray [0061]: This weighting enables more recent user choices to more strongly impact recommendations and decisions [for a plurality of predictions] by the predictive model [by the one or more machine learning models].)
generating an updated dataset based on the dataset and the one or more feedback decision rules; (in Ray [Claim 8]: updating the trend clusters and correlation rules according to customer feedback [and the one or more decision feedback rules] and updated data [generating an updated dataset] from the data sources [based on the dataset] regarding choices.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Tommasi and Ray before them, to include Ray’s feedback decision rules in Tommasi’s visualization and exploration system that edits behavior of machine learning models. One would have been motivated to make such a combination in order to improve the system by further refining future predictions for a greater accuracy as taught by Ray ([0070]).
However, Tommasi in view of Ray does not appear to explicitly teach and moving a decision boundary of one or more machine learning models based on the updated dataset.
Lam further teaches and moving a decision boundary of one or more machine learning models based on the updated dataset. (in Lam [0086]: In summary, the heatmap indicates pixel regions of an input sample (i.e. interpretation sample 302) that are highly relevant or salient to a given decision boundary of the rule extracted from the DNN. This may mean that, in some examples, changing the pixel values of the pixels of an input sample (e.g. first interpretation sample 302) in highly-relevant regions of the input sample (e.g. first interpretation sample 302) would be likely to cause the DNN to classify the changed input sample [based on the updated dataset] (e.g. the changed first interpretation sample 302) on the other side of the decision boundary [moving a decision boundary], relative to changes made to low relevance pixel regions of the input sample (e.g. first interpretation sample 302 ).
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Tommasi, Ray, and Lam before them, to include Lam’s movement of decision boundaries based on updated data in Tommasi and Ray’s visualization and exploration system that edits behavior of machine learning models by using feedback decision rules. One would have been motivated to make such a combination in order to improve the system by more accurately categorizing the input data as taught by Lam ([0086]).
Regarding claim 13, the claim recites similar limitation as corresponding claim 6 and is rejected for similar reasons as claim 6 using similar teachings and rationale.
Regarding claim 19, the claim recites similar limitation as corresponding claim 6 and is rejected for similar reasons as claim 6 using similar teachings and rationale.
Claims 7, 14, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Tommasi and “Visualizing multi-dimension decision boundaries in 2d”, Migut et al., pages 1-19 © 2013 in further view of Marzuoli et al. (US 10,666,792 B1) and Smith et al. (US 2020/0150937 A1).
Regarding claim 7, Tommasi teaches all the limitations of claim 1 as described above.
Tommasi teaches the multidimensional dataset (in Tommasi [0065]: A visualization and exploration of an interactive representation of one or more probabilistic models using multidimensional dataset [the multidimensional dataset].)
However, Tommasi does not appear to explicitly teach further including: assigning similarity scores to a plurality of sectors of [the multidimensional dataset]; applying a projection mapping operation on the plurality of sectors based on the similarity scores; and building a decision boundary based on the projection mapping operation.
However, Marzuoli further teaches further including: assigning similarity scores to a plurality of sectors of [the multidimensional dataset]; (in Marzuoli [Col. 3 Lines 2-6]: and computing a pair-wise similarity between all pairs of transcripts to create similarity matrix that contains a similarity score [assigning similarity scores] between each pair of transcripts [to a plurality of sectors of the ... dataset] as the cosine of the projections, which are normalized.)
applying a projection mapping operation on the plurality of sectors based on the similarity scores; (in Marzuoli [Col. 1 Lines 65-67, Col. 2 Lines 1-2]: According to further aspects of the invention, the computing step computes the pair-wise similarity between all pairs of transcripts to create a similarity matrix that contains a similarity score [based on the similarity scores] between each pair of transcripts [on the plurality of sectors] as the cosine of the projections [applying a projection mapping operation], which are normalized.)
the projection mapping operation (in Marzuoli [Col. 1 Lines 65-67, Col. 2 Lines 1-2]: the computing step computes the pair-wise similarity between all pairs of transcripts to create a similarity matrix that contains a similarity score between each pair of transcripts as the cosine of the projections [the projection mapping operation], which are normalized.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Tommasi and Marzuoli before them, to include Marzuoli’s projection mapping operation based on similarity score in Tommasi’s visualization and exploration system that edits behavior of machine learning models. One would have been motivated to make such a combination in order to allow for projections to be compared and to identify sets of identical transcripts for improved categorization as taught by Marzuoli ([Col. 6 Lines 22-65]).
However, Tommasi in view Marzuoli does not appear to explicitly teach building a decision boundary based on the projection mapping operation.
However, Smith further teaches building a decision boundary based on the projection mapping operation (in Smith [0035]: The machine learning model 200 has internal parameters that determine its decision boundary and that determine the output that the machine learning model 200 produces. After each training iteration, comprising inputting the input object 210 of a training example in to the machine learning model 200, the actual output 208 of the machine learning model 200 for the input object 210 is compared to the desired output value 212 [based on the projection mapping operation]. One or more internal parameters 202 of the machine learning model 200 may be adjusted such that, upon running the machine learning model 200 with the new parameters [building a decision boundary], the produced output 208 will be closer to the desired output value 212.)
According to a person of ordinary skill in the art using broadest reasonable interpretation, the process of comparing “the actual output” to “the desired output” can be taught upon as a projection mapping operation. This process also includes adjusting parameters, or building a decision boundary based on the projection mapping operation that has been created by the comparison of outputs.
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Tommasi, Marzuoli and Smith before them, to include Smith’s decision boundary being built upon a mapping of projections in Tommasi and Marzuoli’s visualization and exploration system that edits behavior of machine learning models that use projection mapping operations. One would have been motivated to make such a combination in order to reinforce and strengthen those parameters that caused the correct output and reduce and weaken parameters that tended to move away from the correct output as taught by Smith ([0035]).
Regarding claim 14, the claim recites similar limitation as corresponding claim 7 and is rejected for similar reasons as claim 7 using similar teachings and rationale.
Regarding claim 20, the claim recites similar limitation as corresponding claim 7 and is rejected for similar reasons as claim 7 using similar teachings and rationale.
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Tommasi and “Visualizing multi-dimension decision boundaries in 2d”, Migut et al., pages 1-19 © 2013 in further of Lam et al. (US 2022/0138505 A1), Thibodeaux (US 2023/0215206 A1) and Smith (US 2020/0150937 A1).
Regarding claim 17, Tommasi teaches the limitations from claim 15 as mentioned above.
However, Tommasi does not appear to explicitly teach wherein editing behavior of the one or more machine learning models further includes: relabeling and generating a training dataset; and displaying one or more updated decision boundaries of one or more machine learning models.; wherein editing behavior of the one or more machine learning models further includes comparing previous decision boundaries with one or more relocated decision boundaries.
However, Lam teaches wherein editing behavior of the one or more machine learning models further includes: relabeling and generating a training dataset; (in Lam [0109]: In some embodiments, the DNN 500 may be trained using the same input samples used by method 600. Each updated input sample [a training dataset] 512 may be relabeled [relabeling] based on its respective classification data 514 generated [generating] by the DNN 500: e.g., a first input sample 502(1) that is classified by the DNN 500 as most likely in the first category “dog” in the classification data 514 (which is shown as indicating 0.993 probability of being in category “dog” and 0.007 probability of being in category “cat”) may have its respective updated input sample 512 labelled as first category “dog”. )
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Tommasi and Lam before them, to include Lam’s ability to relabel and generate training data in Tommasi’s visualization and exploration system that edits behavior of machine learning models. One would have been motivated to make such a combination in order to improve the probability of being in each of the respective updated classifications and categories as taught by Lam ([0109-0110]).
However, Tommasi in view of Lam does not explicitly teach and displaying one or more updated decision boundaries of one or more machine learning models; wherein editing behavior of the one or more machine learning models further includes comparing previous decision boundaries with one or more relocated decision boundaries.
Thibodeaux teaches and displaying one or more updated decision boundaries of one or more machine learning models. (in Thibodeaux [0032]: Once all of the backend generated labels have been processed for regrouping (affirmative outcome to decision step 406), then the updated resume data is output and/or stored along with the initial bounding block data and regrouped bounding block data at step 408. As disclosed herein, the output and storage processing may be implemented by a combination of client-facing frontend code and backend code which are configured to store and/or display [displaying] the updated resume data that includes any regrouped bounding blocks [updated decision boundary] along with updated labels provided through user feedback.)
wherein both the previous decision boundaries and the one or more relocated decision boundaries are displayed via the interactive representation. (in Thibodeaux [0032]: Once all of the backend generated labels have been processed for regrouping (affirmative outcome to decision step 406), then the updated resume data is output and/or stored along with the initial bounding block [previous decision boundaries] data and regrouped bounding block data at step 408. As disclosed herein, the output and storage processing may be implemented by a combination of client-facing frontend code and backend code [via the interactive representation] which are configured to store and/or display [displayed] the updated resume data that includes any regrouped bounding blocks [and the one or more relocated decision boundaries] along with updated labels provided through user feedback.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Tommasi, Lam and Thibodeaux’s before them, to include Thibodeaux’s decision boundary updates and ability to be displayed in Tommasi and Lam’s visualization and exploration system that edits behavior of machine learning models. One would have been motivated to make such a combination in order to enable the user to visually see all detected blocks of text on their resume as bounding blocks, and to prompt the user to select all bounding blocks which relate to a backend generated label as taught by Thibodeaux ([0030]).
However, Tommasi in view of Lam in view of Thibodeaux, does not appear to explicitly teach wherein editing behavior of the one or more machine learning models further includes comparing previous decision boundaries with one or more relocated decision boundaries.
However, Smith teaches wherein editing behavior of the one or more machine learning models further includes comparing previous decision boundaries with one or more relocated decision boundaries (in Smith [0035]: The machine learning model 200 has internal parameters that determine its decision boundary [previous decision boundaries] and that determine the output that the machine learning model 200 produces. After each training iteration, comprising inputting the input object 210 of a training example in to the machine learning model 200, the actual output 208 of the machine learning model 200 for the input object 210 is compared [comparing] to the desired output value 212. One or more internal parameters 202 of the machine learning model 200 may be adjusted such that, upon running the machine learning model 200 with the new parameters [one or more relocated decision boundaries], the produced output 208 will be closer to the desired output value 212. If the produced output 208 was already identical to the desired output value 212, then the internal parameters 202 of the machine learning model 200 may be adjusted to reinforce and strengthen those parameters that caused the correct output and reduce and weaken parameters that tended to move away from the correct output.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Tommasi, Lam, Thibodeaux and Smith before them, to include Smith’s comparison of decision boundaries in Tommasi, Lam and Thibodeaux’s visualization and exploration system that edits behavior of machine learning models. One would have been motivated to make such a combination in order to improve the model by adjusting to reinforce and strengthen those parameters that caused the correct output and reduce and weaken parameters that tended to move away from the correct output as taught by Smith ([0035]).
Response to Arguments
Applicants’ arguments have been considered but are moot because of the new grounds of rejection incorporating Migut.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Tommasi (2) (Patent No.: US 2021/0064635 A1) – “VISUALIZATION AND EXPLORATION OF PROBABILISTIC MODELS” relates to a visualization and exploration of an interactive representation of more or more probabilistic models using multidimensional datasets.
Liu et al. (Patent No.: US 2020/0183035 A1) – “DATA AUGMENTATION FOR SEISMIC INTERPRETATION SYSTEMS AND METHODS” relates to obtaining data and augmented the datasets based on the input data and geologic domain knowledge and/or geophysical domain knowledge.
Elprin et al. (Patent No.: US 11,315,039 B1) – “SYSTEMS AND METHODS FOR MODEL MANAGEMENT” relates to allowing users to visualize statistics about models and monitor models in real-time via a graphical user interface provided by the systems.
Applicants’ amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHERROD L KEATON whose telephone number is (571)270-1697. The examiner can normally be reached on MONDAY -FRIDAY 9:30-5.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle Bechtold can be reached on 571-272-4124. The fax phone number for the organization where this application or proceeding is assigned is 571-273-3800.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHERROD L KEATON/Primary Examiner, Art Unit 2148
7-28-2026
Prosecution Insights