DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant’s claim for the benefit of a prior-filed GR20220100590, filed on July 22, 2022, has been acknowledged.
Drawings
The drawings were received on 07/27/2022. These drawings are acceptable.
Information Disclosure Statement
The information disclosure statement (IDS) submitted 07/29/2022 has been considered by the examiner.
Response to Arguments
Applicant's arguments filed 12/16/2025 have been fully considered.
The remarks are directed to elements that have not been previously examined by the examiner. See rejection of amended claims below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e. an abstract idea) without significantly more.
Claim 1: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
… encoding,(Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed; see MPEP § 2106.04(a)(2), subsection III)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
receiving, from the first machine learning model configured to receive information associated with a data object, information associated with a predicted structure for the data table; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. receiving or transmitting data over a network)
the data object comprising a data table, wherein the predicted structure comprises at least a prediction of a row and column structure of the data table … detect failures of the first machine learning model (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to generally linking the use of a judicial exception to a particular technological environment or field of use. See 2106.05(h).)
[receiving], from the first machine learning model [configured to receive] information associated with a data object, information associated with a predicted structure for the data table; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to generally linking the use of a judicial exception to a particular technological environment or field of use. See 2106.05(h).)
… encoding, using a second machine learning model,… evaluating, using the second machine learning model … determining, using the second machine learning model… (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea; Thus claim limitations amount to mere instructions to apply the judicial exception using a computer/computing environment as a tool, as discussed in MPEP § 2106.05(f).)
evaluating, using the second machine learning model, the information associated with the data table using the encoded input channels; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea; Thus claim limitations amount to mere instructions to apply the judicial exception using a computer/computing environment as a tool, as discussed in MPEP § 2106.05(f).)
… information associated with the predicted structure for the data object to produce encoded input channels… the information associated with the data object with the encoded input channels … Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to generally linking the use of a judicial exception to a particular technological environment or field of use. See 2106.05(h).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and elements invoking computers or other machinery merely as a tool to perform the claimed process/judicial exception.
Secondly, the limitations directed to insufficient to transform the judicial exception to a patentable invention because the recitation are directed to insignificant solution activity for as noted above.
The courts have deemed these types of activity as well-known routine and convectional, see evidences noted below:
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink."
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 2: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Abstract idea recited in claim 1.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein the predicted structure further comprises a prediction of locations of spanning cells of the data table. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to generally linking the use of a judicial exception to a particular technological environment or field of use. See 2106.05(h).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use.. These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 3: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
… generate the predicted structure for the data object… monitor for failure of the structure prediction model. (Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed; see MPEP § 2106.04(a)(2), subsection III)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein: the first machine learning model comprises a structure prediction model configured to….; and the second machine learning model comprises a confidence model configured to monitor for failure of the structure prediction model. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and elements invoking computers or other machinery merely as a tool to perform the claimed process/judicial exception. These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 4: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Abstract idea recited in claim 1.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein the encoded input channels comprise a plurality of binary masks, a plurality of non-binary masks, or a combination thereof, each representative of locations of content of table elements of the data table relative to grid lines of the data table. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to generally linking the use of a judicial exception to a particular technological environment or field of use. See 2106.05(h).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use. These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 5: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
assessing a training prediction of a structure for the training data object with respect to the ground truth structure for the training data object. (Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed; see MPEP § 2106.04(a)(2), subsection III)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein the first machine learning model has been trained by: receiving a ground truth structure for a training data object; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).)
… receiving a ground truth structure for a training data object. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. receiving or transmitting data over a network)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and elements invoking computers or other machinery merely as a tool to perform the claimed process/judicial exception.
Secondly, the limitations directed to insufficient to transform the judicial exception to a patentable invention because the recitation are directed to insignificant solution activity for as noted above.
The courts have deemed these types of activity as well-known routine and convectional, see evidences noted below:
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink."
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 6: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Recites the abstract idea of claim 1.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein: the first machine learning model is configured to receive the information associated with the data object; the information associated with the predicted structure for the data object is generated by the first machine learning model based on the information associated with the data object; and the second machine learning model is configured to receive (i) the information associated with the data object received by the first machine learning model, and (ii) the information associated with the predicted structure for the data object generated by the first machine learning model. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).)
wherein: the first machine learning model is configured to receive the information associated with the data object; … and the second machine learning model is configured to receive (i) the information associated with the data object received by the first machine learning model,… (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. receiving or transmitting data over a network)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and elements invoking computers or other machinery merely as a tool to perform the claimed process/judicial exception.
Secondly, the limitations directed to insufficient to transform the judicial exception to a patentable invention because the recitation are directed to insignificant solution activity for as noted above.
The courts have deemed these types of activity as well-known routine and convectional, see evidences noted below:
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink."
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 7: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
wherein the determining of the probability of correctness of the predicted structure is based on a distribution function. (Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed; see MPEP § 2106.04(a)(2), subsection III; And Mathematical concept: Mathematical relationship (e.g. organizing information and manipulating information through mathematical correlation; see MPEP § 2106.04(a)(2))
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use. These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 8: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
… and generating an output comprising a predicted structure for the object data,…; …obtaining the input and the output; encoding the input and the output as a plurality of input channels; and determining a probability of correctness of the predicted boundaries of the predicted structure based on an evaluation of the plurality of input channels. (Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed; see MPEP § 2106.04(a)(2), subsection III)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
one or more memory components; one or more processing devices coupled to the one or more memory components; a first machine learning model coupled to the one or more processing devices, wherein the one or more processing devices are configured to cause the first machine learning model to perform operations comprising: … and a second machine learning model coupled to the one or more processing devices, wherein the one or more processing devices are configured to cause the second machine learning model to perform operations comprising: (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).)
receiving an input comprising object data of an object within a document; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. receiving or transmitting data over a network)
document, the object comprising a data table; …the predicted structure comprising predicted boundaries corresponding to the object corresponding to a row and column structure of the data table; … the encoded plurality of input channels indicating at least locations of the row and column structure in the predicted boundaries of the data table; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to generally linking the use of a judicial exception to a particular technological environment or field of use. See 2106.05(h).)
wherein the second machine learning model is used to detect failures of the first machine learning model. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and elements invoking computers or other machinery merely as a tool to perform the claimed process/judicial exception.
Secondly, the limitations directed to insufficient to transform the judicial exception to a patentable invention because the recitation are directed to insignificant solution activity for as noted above.
The courts have deemed these types of activity as well-known routine and convectional, see evidences noted below:
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink."
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 9 Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Incorporates abstract ideas recited in claim 8.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein each of the plurality of input channels is a binary or non-binary mask representative of a structural aspect of the predicted structure. (Claimed limitations are generally linking the use of a judicial exception to a particular technological environment or field of use, as discussed in MPEP § 2106.05(h))
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use. These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 10: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
wherein the determination of the probability of correctness of the predicted structure is based on a distribution function. (Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed; see MPEP § 2106.04(a)(2), subsection III; And Mathematical concept: Mathematical relationship (e.g. organizing information and manipulating information through mathematical correlation; see MPEP § 2106.04(a)(2))
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use. These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 11: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Incorporates abstract ideas recited in claim 8.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein: the predicted structure further comprises predicted locations of one or more elements of the data table within the row and column structure, wherein the one or more elements comprise one or more of textual content or an image contained in the data table; and if the predicted boundaries are correct, the data table contains each of the one or more of the textual content or the image within corresponding predicted boundaries. (Claimed limitations are generally linking the use of a judicial exception to a particular technological environment or field of use, as discussed in MPEP § 2106.05(h))
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use. These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 12: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
assessing a training prediction of a structure for the training object data with respect to the ground truth structure for the training object data.. (Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed; see MPEP § 2106.04(a)(2), subsection III)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein the first machine learning model has been trained by: receiving a ground truth structure for training object data; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).)
… receiving a ground truth structure for training object data. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. receiving or transmitting data over a network)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and elements invoking computers or other machinery merely as a tool to perform the claimed process/judicial exception.
Secondly, the limitations directed to insufficient to transform the judicial exception to a patentable invention because the recitation are directed to insignificant solution activity for as noted above.
The courts have deemed these types of activity as well-known routine and convectional, see evidences noted below:
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink."
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 13: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Abstract idea recited in claim 8.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein: the object data of the input comprises text; (Claimed limitations merely including instructions to implement an abstract idea on a computer and merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).)
… and the system comprises a third machine learning model is configured to apply a language model to the text to determine the probability of correctness. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and elements invoking computers or other machinery merely as a tool to perform the claimed process/judicial exception. These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 14: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
… encode the second input and the second output into a second plurality of input channels; and determine a probability of correctness of a predicted structure for the second object data (Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed; see MPEP § 2106.04(a)(2), subsection III)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein the second machine learning model is further configured to: receive a second input comprising second object data fed to a third machine learning model, and a second output from the third machine learning model, the third machine learning model being different from the first machine learning model; … the second object data predicted by the third machine learning model (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).)
… receive a second input comprising second object data fed to a third machine learning model… (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. receiving or transmitting data over a network)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and elements invoking computers or other machinery merely as a tool to perform the claimed process/judicial exception.
Secondly, the limitations directed to insufficient to transform the judicial exception to a patentable invention because the recitation are directed to insignificant solution activity for as noted above.
The courts have deemed these types of activity as well-known routine and convectional, see evidences noted below:
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink."
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 15: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
…, obtaining a label indicating a correctness of the predicted structure based on a comparison of the predicted structure with a ground-truth version of the predicted structure; and … determines the confidence of the output of the second machine learning model.. (Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed; see MPEP § 2106.04(a)(2), subsection III)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
receiving, from the second machine learning model, information associated with a predicted structure for a data object (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. receiving or transmitting data over a network)
(Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to generally linking the use of a judicial exception to a particular technological environment or field of use. See 2106.05(h).)
… and training the first machine learning model based on the information associated with the predicted structure for the data object … to generate a trained machine learning model that determines … of the output of the second machine learning model. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).)
a data object comprising a data table, wherein the predicted structure comprises at least a prediction of a row and column structure of the data table; (Claimed limitations are generally linking the use of a judicial exception to a particular technological environment or field of use, as discussed in MPEP § 2106.05(h))
trained first machine learning model is used in conjunction with the second machine learning model to detect failures of the second machine learning model. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and elements invoking computers or other machinery merely as a tool to perform the claimed process/judicial exception.
Secondly, the limitations directed to insufficient to transform the judicial exception to a patentable invention because the recitation are directed to insignificant solution activity for as noted above.
The courts have deemed these types of activity as well-known routine and convectional, see evidences noted below:
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink."
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 16: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Recites abstract in claim 15.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
the information associated with the predicted structure for the data object comprises coordinates for the structure for the data table, spanning information for one or more cells within the structure for the data table, or a combination thereof. (Claimed limitations are generally linking the use of a judicial exception to a particular technological environment or field of use, as discussed in MPEP § 2106.05(h))
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use. These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 17: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
…predicting, (Mental process for evaluating and making judgements for comparing observations; and Mathematical concepts for process the making mathematical relationships using functions for making comparisons)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein the training of the first machine learning model comprises: predicting, with the first machine learning model, … (Claimed limitations merely including instructions to implement an abstract idea on a computer and merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and elements invoking computers or other machinery merely as a tool to perform the claimed process/judicial exception. These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 18: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
wherein the minimizing of the error comprises determining a difference between the obtained label and the predicted likelihood of correctness of the predicted structure, defining a threshold,... (Mental process for evaluating and making judgements for comparing observations; and Mathematical concepts for process the making mathematical relationships using functions for making comparisons)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
… and performing an iterative optimization process to reduce the difference until the threshold is reached. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. Performing repetitive calculations)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use.
Secondly, the limitations directed to insufficient to transform the judicial exception to a patentable invention because the recitation are directed to insignificant solution activity for as noted above.
The courts have deemed these types of activity as well-known routine and convectional, see evidences noted below:
Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values); Bancorp Services v. Sun Life, 687 F.3d 1266, 1278, 103 USPQ2d 1425, 1433 (Fed. Cir. 2012) ("The computer required by some of Bancorp’s claims is employed only for its most basic function, the performance of repetitive calculations, and as such does not impose meaningful limits on the scope of those claims.");
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 19: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
… encode the information associated with the predicted structure for the data object to produce encoded input channels; evaluate the information associated with the data object with the encoded input channels; and based on the evaluation, determine a probability of correctness of the predicted structure for the data object, the confidence of the output … based on the probability of correctness. (Mental process for evaluating and making judgements for comparing observations; and Mathematical concepts for process the making mathematical relationships using functions for making comparisons)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein the generated trained machine learning model is configured to: receive, from the second machine learning model, information… (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. receiving or transmitting data over a network)
wherein the generated trained machine learning model is configured to: receive, from the second machine learning model, information …. the confidence of the output of the second machine… (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).)
… information associated with the predicted structure for the data object … the confidence of the output of the second machine being based on the probability of correctness. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to generally linking the use of a judicial exception to a particular technological environment or field of use. See 2106.05(h).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and elements invoking computers or other machinery merely as a tool to perform the claimed process/judicial exception.
Secondly, the limitations directed to insufficient to transform the judicial exception to a patentable invention because the recitation are directed to insignificant solution activity for as noted above.
The courts have deemed these types of activity as well-known routine and convectional, see evidences noted below:
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink."
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 20: Does claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
… minimizing an error, the minimizing of the error comprising determining a difference between coordinates of the predicted structure and the ground-truth version of the predicted structure,... (Mental process for evaluating and making judgements for comparing observations; and Mathematical concepts for process the making mathematical relationships using functions for making comparisons)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
… and performing an iterative optimization process to reduce the difference. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. Performing repetitive calculations)
wherein the second machine learning model has been trained by minimizing an error,… (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and elements invoking computers or other machinery merely as a tool to perform the claimed process/judicial exception.
Secondly, the limitations directed to insufficient to transform the judicial exception to a patentable invention because the recitation are directed to insignificant solution activity for as noted above.
The courts have deemed these types of activity as well-known routine and convectional, see evidences noted below:
Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values); Bancorp Services v. Sun Life, 687 F.3d 1266, 1278, 103 USPQ2d 1425, 1433 (Fed. Cir. 2012) ("The computer required by some of Bancorp’s claims is employed only for its most basic function, the performance of repetitive calculations, and as such does not impose meaningful limits on the scope of those claims.");
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Therefore, claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed a judicial exception and does not recite, when claim elements are examined or as an ordered combination, that are directed to what have the courts have identified as "significantly more”, than the identified abstract idea, see MPEP 2106.05.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-13, 14-16 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Salman et al. (US 20210406644, hereinafter ‘Sal’) in view of Smock et al. (US 20220335240, hereinafter ‘Moc’) and in further view of Doug et al. (US 20220027740, hereinafter ‘Do’).
Regarding independent claim 1, Sal limitation a method of determining a confidence of an output of a first machine learning model, the method comprising:
to receive information associated with a data object, information associated with a predicted structure for the data object comprising a data ; ([0148] FIG. 11 illustrates the methodology of the typicality learning component in computing typicality predictions from labels. First, a multi-class channel stack is extracted from the bounding boxes of a label representation [receiving, from the first machine learning model configured to receive information associated with a data object, information associated with a predicted structure for the data object]. The multi-class channel stack is an ensemble of channels where every channel corresponds to one of the classes of the label [information associated with a predicted structure for the data object]. Each channel is obtained by summing rectangles of ones in place of each bounding box pertaining to the corresponding class. Using stacks to represent bounding boxes is very convenient because, like images, stacks hold spatial information [receiving, from the first machine learning model configured to receive information associated with a data object, information associated with a predicted structure for the data object] and can easily be fed into a neural network. Moreover, it's an efficient way of representing labels: information is preserved, and the machine learning component can identify recurrent spatial arrangements among objects by training on labels from the pool of labeled data.)
encoding, using a second machine learning model, the information associated with the predicted structure for the data ([0008] In embodiments, the one or more additional components can include a typicality learning component that employs a computation model [encoding, using a second machine learning model, the information associated with the predicted structure for the data table to produce encoded input channels] that is trained to generate a typicality score related to typicality of a predicted label output by the inspection learning component and supplied as input to the computational model [using a second machine learning model]. The typicality score characterizes the level of divergence of the predicted label relative to the distribution of labels in a pool of labeled observations…)
evaluating, using the second machine learning model, the information associated with the data ([0008] In embodiments, the one or more additional components can include a typicality learning component that employs a computation model that is trained to generate a typicality score [evaluating, using the second machine learning model, the information associated with the data ] related to typicality of a predicted label output by the inspection learning component [evaluating, using the second machine learning model, the information associated with the data ] and supplied as input to the computational model. The typicality score characterizes the level of divergence of the predicted label relative to the distribution of labels in a pool of labeled observations…)
and based on the evaluating, determining, using the second machine learning model, a probability of correctness of the predicted structure for the data ([0008] In embodiments, the one or more additional components can include a typicality learning component that employs a computation model that is trained to generate a typicality score [based on the evaluating, determining, using the second machine learning model, a probability of correctness of the predicted structure for the data ] related to typicality of a predicted label output by the inspection learning component and supplied as input to the computational model. The typicality score [based on the evaluating, determining, using the second machine learning model, a probability of correctness of the predicted structure for the data ] characterizes the level of divergence of the predicted label [wherein the probability of correctness of the predicted structure is used to detect failures of the first machine learning model] relative to the distribution of labels in a pool of labeled observations…)
While Sal teaches labeling component for feeding claimed processed data as claimed a first machine learning model.
Sal does not expressly disclosed object detection of data object comprising a data table and the use of them as claimed the data object comprising a data table, wherein the predicted structure comprises at least a prediction of a row and column structure of the data table;
… the data table to produce encoded input channels, the encoded input channels indicating at least locations of the row and column structure in the predicted structure of the data table;
Moc does expressly disclosed object detection of data object comprising a data table and the use of them as claimed the data object comprising a data table, wherein the predicted structure comprises at least a prediction of a row and column structure of the data table; (in [0036] The values 212 of the attributes of the respective cells 172 depend on specific settings/configurations of the cells 172 in generation of the data table 170, and the main purpose of the settings/configurations is to facilitate the organization and presentation of the data. The values 212 of the attributes may be extracted from metadata of the electronic document containing the data table 170 [the data object comprising a data table, wherein the predicted structure comprises at least a prediction of a row and column structure of the data table] or from other data sources. One or more attributes of interest by the attribute obtaining module 210 may be preconfigured so as to obtain the specific values 212 of the respective attributes from the input data table 170. In some implementations, the attribute obtaining module 210 may represent the obtained values 212 of the attributes in a two-dimensional structure of rows and columns of the data table 170 [the data object comprising a data table, wherein the predicted structure comprises at least a prediction of a row and column structure of the data table], and the two-dimensional structure includes the values 212 of the attributes extracted for the respective cell 172...; And in [0048] To create this structured representation, several steps are taken, including: overlap is eliminated between any predicted bounding boxes of the same class, rows and columns [the data object comprising a data table, wherein the predicted structure comprises at least a prediction of a row and column structure of the data table] are expanded to fill any gaps between them within the table boundary, rows and columns are intersected to form grid cells, cells falling in the same supercell bounding box must be merged into a single cell, each word must be uniquely assigned to a cell, and cells must be labeled by their role using the column header and row header predictions )
… the data table to produce encoded input channels, the encoded input channels indicating at least locations of the row and column structure in the predicted structure of the data table; (As depicted in Fig 7 and in [0041] The input to the table detection model 115 is an image of a document page, which for example, can be rendered electronically, scanned, or photographed. The output of the model is a set of bounding box and class predictions, where a bounding box is denoted by four coordinates describing the locations [the data table to produce encoded input channels, the encoded input channels indicating at least locations of the row and column structure in the predicted structure of the data table] of the four edges of the bounding box rectangle, and the class prediction is both a class label and a confidence score between 0 and 1. And in [0070] Table detection involves determining the location [the data table to produce encoded input channels, the encoded input channels indicating at least locations of the row and column structure in the predicted structure of the data table] of the table within its surrounding document as shown in FIG. 7. )
Additionally Do teaches object detection of data object comprising a data table and the use of them as claimed the data object comprising a data table, wherein the predicted structure comprises at least a prediction of a row and column structure of the data table; (in [0098] At block 710, values of at least one attribute for a plurality of cells in a data table are obtained, the values of the at least one attribute indicating at least one of a semantic meaning of data filled in the plurality of cells or a structure of the data table. The plurality of cells are arranged in rows and columns in the data table [the data object comprising a data table, wherein the predicted structure comprises at least a prediction of a row and column structure of the data table;]. At block 720, a feature representation of the values of the at least one attribute is extracted using a first learning network… )
… the data table to produce encoded input channels, the encoded input channels indicating at least locations of the row and column structure in the predicted structure of the data table; (in [0067] In some implementations, the extraction of the row-wise feature representation and/or the column-wise feature representation may be implemented in the respective network sections 310 of the first learning network 222... Depending on the location of the network section 310 in the first learning network 222, the convolutional layer 410 may extract the feature representation of the values 212 of the attributes of the respective cells 172 in the data table 170 (when it is located at the network section 310-1) [the data table to produce encoded input channels, the encoded input channels indicating at least locations of the row and column structure in the predicted structure of the data table] or may extract the feature representation output by the preceding network section 310. And in [0031] …. The data table 170 may also be of an unfixed size, and then the size of the data table can be determined by regions where non-empty cells of the spreadsheet are located...)
Do, Moc and Sal are analogous art because both involve developing information retrieval and object recognition techniques using machine learning systems and algorithms.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the prior art for developing and implementing information retrieval and object recognition techniques of data tables/structures, as disclosed by Moc and Do with the method of developing machine learning assisted labeling process and system as disclosed by Sal.
One of ordinary skill in the arts would have been motivated to combine the disclosed methods disclosed by Ro, Moc and Sal as noted above. Doing so enables solutions for auto-formatting of a data table using a learning network, (Do, 0002); and enables a table's content to be used in automated downstream applications, such as data visualization, aggregation of data across multiple sources, and statistical analysis, (Moc, 0002 and Do, 0014).
Regarding claim 2, the rejection of claim 1 is incorporated and Moc further teaches the method of claim 1, wherein the predicted structure further comprises a prediction of locations of spanning cells of the data table (in [0033] The first three classes are columns, rows, and supercells (also called “merged cells” or “spanning cells”) [wherein the predicted structure further comprises a prediction of locations of spanning cells of the data table]. These three classes recognize the cell structure of the table. Each intersection of a column and a row forms a grid cell. The supercell object class detects [wherein the predicted structure further comprises a prediction of locations of spanning cells of the data table] when grid cells in the table are merged.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Sal and Moc for the same reasons disclosed above.
Regarding claim 3, the rejection of claim 1 is incorporated and Sal in combination with Moc and Do teaches the method of claim 1, wherein: the first machine learning model comprises a structure prediction model configured to generate the predicted structure for the data object; (predicted bounding boxes as claimed predicted structures, in [0047] Active learning can also be used for object detection. Object detection typically involves identifying one or more bounding boxes within an observation and assigning one or more classes to the bounding box(es) [wherein: the first machine learning model comprises a structure prediction model configured to generate the predicted structure for the data object]. The location of the bounding boxes in the observation and the assigned class(es) produced by object detection are commonly referred to as a prediction…. )
and the second machine learning model comprises a confidence model configured to monitor for failure of the structure prediction model. ([0190] In embodiments, the relevance score for a particular prediction output by the inspection learning component 209C′ can be computed from the probability of a particular prediction as output by the inspection learning component 209C′. For example, where the inspection learning component 209C′ outputs bounding boxes coordinates as well as probabilities of classes for each bounding box [and the second machine learning model comprises a confidence model configured to monitor for failure of the structure prediction model], the relevance score can be given by the following: …)
Examiner notes that claimed computer instructions for implement claimed processes operations as claimed model, in [0216] Various processes of present disclosure may be described herein in the general context of software or program modules, or the techniques and modules may be implemented in pure computing hardware. Software generally includes routines, programs, objects, components, data structures, and so forth that perform particular tasks or implement particular abstract data types. An implementation of these modules and techniques may be stored on or transmitted across some form of tangible computer-readable media. Computer-readable media can be any available data storage medium or media that is tangible and can be accessed by a computing device…
Regarding claim 4, the rejection of claim 1 is incorporated and Sal in combination with Moc and Do teaches the method of claim 1, wherein the encoded input channels comprise a plurality of binary masks, a plurality of non-binary masks, or a combination thereof, each representative of locations of content of table elements of the data table relative to grid lines of the data table. (As depicted in Figs. 11 and 12:
PNG
media_image1.png
556
825
media_image1.png
Greyscale
PNG
media_image2.png
763
1119
media_image2.png
Greyscale
Examiner notes that the annotated imaged are still noted for teaching the amended limitation wherein the encoded input channels comprise a plurality of binary masks, a plurality of non-binary masks, or a combination thereof, each representative of locations of content of table elements of the data table relative to grid lines of the dataAnd in [0053] In embodiments, the framework can employ an inspection learning component (or primary learning component) that generates a prediction corresponding to an unlabeled observation. The prediction includes a label associated with the unlabeled observation and one or more uncertainty scores associated with the label. In embodiments, the label can specify the location of one or more bounding boxes and one or more classes associated with the bounding box(es) [wherein the encoded input channels comprise a plurality of binary masks, a plurality of non-binary masks, or a combination thereof, each representative of locations of content of table elements of the data table relative to grid lines of the data]…)
Additionally Moc teaches each representative of locations of content of table elements of the data table relative to grid lines of the data table, in [0033] The first three classes are columns, rows, and supercells (also called “merged cells” or “spanning cells”). These three classes recognize the cell structure of the table. Each intersection of a column and a row forms a grid cell [each representative of locations of content of table elements of the data table relative to grid lines of the data table]. The supercell object class detects when grid cells in the table are merged… [0048] To create this structured representation, several steps are taken, including: overlap is eliminated between any predicted bounding boxes of the same class, rows and columns are expanded to fill any gaps between them within the table boundary, rows and columns are intersected to form grid cells [each representative of locations of content of table elements of the data table relative to grid lines of the data table], cells falling in the same supercell bounding box must be merged into a single cell, each word must be uniquely assigned to a cell, and cells must be labeled by their role using the column header and row header predictions.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Sal and Moc for the same reasons disclosed above.
Regarding claim 5, the rejection of claim 1 is incorporated and Sal in combination with Moc and Do teaches the method of claim 1, wherein the first machine learning model has been trained by: receiving a ground truth structure for a training data object; and assessing a training prediction of a structure for the training data object with respect to the ground truth structure for the training data object. (in [0053] In embodiments, the framework can employ an inspection learning component (or primary learning component) that generates a prediction corresponding to an unlabeled observation. The prediction includes a label associated with the unlabeled observation and one or more uncertainty scores associated with the label [wherein the first machine learning model has been trained by: receiving a ground truth structure for a training data object]. In embodiments, the label can specify the location of one or more bounding boxes and one or more classes associated with the bounding box(es). The uncertainty score(s) [and assessing a training prediction of a structure for the training data object with respect to the ground truth structure for the training data object] can characterize the confidence level for the associated label and/or the reproducibility of the associated unlabeled observation. In some instances (for example, in the case of a missed detection), the inspection learning component can be wrongly confident about its prediction. A typicality score can be generated by a typicality learning component directly from a label predicted by the inspection learning component. In embodiments, the typicality score can characterize the level of deviation of a label predicted [… assessing a training prediction of a structure for the training data object with respect to the ground truth structure for the training data object] by the inspection learning component relative to the distribution of labels in a pool of labeled observations [wherein the first machine learning model has been trained by: receiving a ground truth structure for a training data object]…)
Additionally Moc teaches wherein the first machine learning model has been trained by: receiving a ground truth structure for a training data object; and assessing a training prediction of a structure for the training data object with respect to the ground truth structure for the training data object. (in [0032] The joint modeling of table structure recognition and table interpretation as an object detection problem is achieved using six object classes in one example. The terms object, container, and bounding box are used somewhat interchangeably. Bounding boxes for objects from different classes can coincide and potentially overlap with one or more other bounding boxes. Modeling the structure and interpretation of a table jointly in this way is a much more robust and efficient mechanism than modeling them independently in parallel or in sequence. The model 125 is trained with labeled images of tables [wherein the first machine learning model has been trained by: receiving a ground truth structure for a training data object;], with the labels corresponding to the six different types of bounding boxes, which are referred to as objects, a nomenclature commonly used with object detection models... [0082] The table object recognition model may include a convolutional neural network followed by a transformer encoder and transformer decoder. Training data for the table object recognition model may include images of tables having labels for table objects within the images derived from corresponding structural information for each table…)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Sal and Moc for the same reasons disclosed above.
Regarding claim 6, the rejection of claim 1 is incorporated and Sal in combination with Moc and Do teaches the method of claim 1, wherein: the first machine learning model is configured to receive the information associated with the data object; the information associated with the predicted structure for the data object is generated by the first machine learning model based on the information associated with the data object; ([0148] FIG. 11 illustrates the methodology of the typicality learning component in computing typicality predictions from labels. First, a multi-class channel stack is extracted from the bounding boxes of a label representation. The multi-class channel stack is an ensemble of channels where every channel corresponds to one of the classes of the label. Each channel is obtained by summing rectangles of ones in place of each bounding box pertaining to the corresponding class. Using stacks to represent bounding boxes [wherein: the first machine learning model is configured to receive the information associated with the data object] is very convenient because, like images, stacks hold spatial information and can easily be fed into a neural network. Moreover, it's an efficient way of representing labels: information is preserved, and the machine learning component can identify recurrent spatial arrangements [wherein: the first machine learning model is configured to receive the information associated with the data object] among objects by training on labels from the pool of labeled data. And in [0008] In embodiments, the one or more additional components can include a typicality learning component that employs a computation model that is trained to generate a typicality score related to typicality of a predicted label output by the inspection learning component [the information associated with the predicted structure for the data object is generated by the first machine learning model based on the information associated with the data object] and supplied as input to the computational model. The typicality score characterizes the level of divergence of the predicted label relative to the distribution of labels in a pool of labeled observations…)
and the second machine learning model is configured to receive (i) the information associated with the data object received by the first machine learning model, and (ii) the information associated with the predicted structure for the data object generated by the first machine learning model. ([0008] In embodiments, the one or more additional components can include a typicality learning component that employs a computation model that is trained to generate a typicality score related to typicality of a predicted label output by the inspection learning component and supplied as input [and the second machine learning model is configured to receive (i) the information associated with the data object received by the first machine learning model, and (ii) the information associated with the predicted structure for the data object generated by the first machine learning model] to the computational model. The typicality score, characterizes the level of divergence of the predicted label relative to the distribution of labels in a pool of labeled observations [to receive (i) the information associated with the data object received by the first machine learning model] … )
Additionally, Do teaches wherein: the first machine learning model is configured to receive the information associated with the data object; the information associated with the predicted structure for the data object is generated by the first machine learning model based on the information associated with the data object;. (in [0045] According to the implementations of the subject matter described herein, the format determination module 220 implements, based on the learning network 222, mapping from the values 212 of the attributes (specifically, the vectorized representation of the values 212 of the attributes) to the formats 224 of the plurality of cells 172 [wherein: the first machine learning model is configured to receive the information associated with the data object]. The training process of the learning network 222 is required for determining the mapping from the values 212 [wherein: the first machine learning model is configured to receive the information associated with the data object;] of the attributes or the vectorized representation to the formats 224. For convenience of description, the learning network 222 is referred to as a first learning network … [0082] In some implementations, the row-wise and/or column-wise format consistency constraints may also be implemented based on a learning network. FIG. 6 illustrates such implementation. As shown in FIG. 6, in addition to the first learning network 222, the format determination module 220 includes a further learning network 620 (referred to as a second learning network). The first learning network 222 extracts the feature representation of the input values 212 of attributes and determines the coarse formats 612 [the information associated with the predicted structure for the data object is generated by the first machine learning model based on the information associated with the data object] for the respective cells 172 in the data table 170 based on the feature representation. The second learning network 620 determines the final formats 224 for the respective cells 172 based on the values 212 of the attributes and the coarse formats 612. The second learning network 620 performs coarse-to-fine refinement on the formats for the cells, in order to remove the local abnormal formatting results…)
and the second machine learning model is configured to receive (i) the information associated with the data object received by the first machine learning model, and (ii) the information associated with the predicted structure for the data object generated by the first machine learning model. ([0082] In some implementations, the row-wise and/or column-wise format consistency constraints may also be implemented based on a learning network. FIG. 6 illustrates such implementation. As shown in FIG. 6, in addition to the first learning network 222, the format determination module 220 includes a further learning network 620 (referred to as a second learning network). The first learning network 222 extracts the feature representation of the input values 212 of attributes and determines the coarse formats 612 for the respective cells 172 in the data table 170 based on the feature representation. The second learning network 620 determines the final formats 224 for the respective cells 172 [the second machine learning model is configured to receive (i) the information associated with the data object received by the first machine learning mode] based on the values 212 of the attributes and the coarse formats 612. The second learning network 620 performs coarse-to-fine refinement on the formats for the cells, in order to remove the local abnormal formatting results. [0083] Specifically, the second learning network 620 extracts a joint feature representation from the values 212 [the second machine learning model is configured to receive …, and (ii) the information associated with the predicted structure for the data object generated by the first machine learning model] of the attributes and the coarse formats 612, and maps the joint feature representation to the respective formats 224 for the plurality of cells. The second learning network 620 may include a plurality of network layers 622, 624, 626, and the like…)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Sal, Moc and Do for the same reasons disclosed above.
Regarding claim 7, the rejection of claim 1 is incorporated and Sal in combination with Moc and Do teaches the method of claim 1, wherein the determining of the probability of correctness of the predicted structure is based on a distribution function. (in [0008] In embodiments, the one or more additional components can include a typicality learning component that employs a computation model that is trained to generate a typicality score related to typicality of a predicted label output by the inspection learning component and supplied as input to the computational model. The typicality score characterizes the level of divergence of the predicted label relative to the distribution of labels [wherein the determining of the probability of correctness of the predicted structure is based on a distribution function] in a pool of labeled observations…)
Regarding independent claim 8, Sal limitation a system, comprising: one or more memory components; one or more processing devices coupled to the one or more memory components; ([0206] FIG. 13 illustrates an example device 2500, with a processor 2502 and memory 2504 that can be configured to implement various embodiments of the active learning framework as discussed in this disclosure. Memory 2504 can also host one or more databases and can include one or more forms of volatile data storage media such as random-access memory (RAM), and/or one or more forms of nonvolatile storage media (such as read-only memory (ROM), flash memory, and so forth).; And in 0216: …Some of the methods and processes described above, can be performed by a processor. The term “processor” should not be construed to limit the embodiments disclosed herein to any particular device type or system. The processor may include a computer system. The computer system may also include a computer processor (e.g., a microprocessor, microcontroller, digital signal processor, or general-purpose computer) for executing any of the methods and processes described above.)
a first machine learning model coupled to the one or more processing devices, wherein the one or more processing devices are configured to cause the first machine learning model to perform operations comprising: receiving an input comprising object data of an object within a document, the object comprising a data ([0148] FIG. 11 illustrates the methodology of the typicality learning component in computing typicality predictions from labels. First, a multi-class channel stack is extracted from the bounding boxes of a label representation [a first machine learning model coupled to the one or more processing devices, wherein the one or more processing devices are configured to cause the first machine learning model to perform operations comprising: receiving an input comprising object data of an object within a document, the object comprising a data]. The multi-class channel stack is an ensemble of channels where every channel corresponds to one of the classes of the label. Each channel is obtained by summing rectangles of ones in place of each bounding box pertaining to the corresponding class. Using stacks to represent bounding boxes is very convenient because, like images [receiving an input comprising object data of an object within a document, the object comprising a data…], stacks hold spatial information and can easily be fed into a neural network. Moreover, it's an efficient way of representing labels: information is preserved, and the machine learning component can identify recurrent spatial arrangements among objects by training on labels from the pool of labeled data.)
generating an output comprising a predicted structure for the object data, the predicted structure comprising predicted boundaries corresponding to ([0148] FIG. 11 illustrates the methodology of the typicality learning component in computing typicality predictions from labels. First, a multi-class channel stack is extracted from the bounding boxes [generating an output comprising a predicted structure for the object data, the predicted structure comprising predicted boundaries corresponding to ] of a label representation. The multi-class channel stack is an ensemble of channels where every channel corresponds to one of the classes of the label…)
and a second machine learning model coupled to the one or more processing devices, wherein the one or more processing devices are configured to cause the second machine learning model to perform operations comprising: obtaining the input and the output; encoding the input and the output as a plurality of input channels, … and determining a probability of correctness of the predicted boundaries of the predicted structure based on an evaluation of the plurality of input channels, wherein the second machine learning model is used to detect failures of the first machine learning model. ([0008] In embodiments, the one or more additional components can include a typicality learning component that employs a computation model [a second machine learning model coupled to the one or more processing devices, wherein the one or more processing devices are configured to cause the second machine learning model to perform operations comprising: obtaining the input and the output] that is trained to generate a typicality score [determining a probability of correctness of the predicted boundaries of the predicted structure based on an evaluation of the plurality of input channels, wherein the second machine learning model is used to detect failures of the first machine learning model] related to typicality of a predicted label output by the inspection learning component and supplied as input to the computational model [wherein the second machine learning model is used to detect failures of the first machine learning model]. The typicality score characterizes the level of divergence of the predicted label [determining a probability of correctness of the predicted boundaries of the predicted structure based on an evaluation of the plurality of input channels, wherein the second machine learning model is used to detect failures of the first machine learning model] relative to the distribution of labels in a pool of labeled observations…)
While Sal teaches labeling component for feeding claimed processed data as claimed a first machine learning model.
Sal does not expressly disclosed object detection of data object comprising a data table and the use of them as claimed the object comprising a data table, … the predicted structure comprising predicted boundaries corresponding to a row and column structure of the data table; …
… the encoded plurality of input channels indicating at least locations of the row and column structure in the predicted boundaries of the data table;
Moc does expressly disclosed object detection of data object comprising a data table and the use of them as claimed the object comprising a data table, … the predicted structure comprising predicted boundaries corresponding to a row and column structure of the data table; (in [0036] The values 212 of the attributes of the respective cells 172 depend on specific settings/configurations of the cells 172 in generation of the data table 170, and the main purpose of the settings/configurations is to facilitate the organization and presentation of the data. The values 212 of the attributes may be extracted from metadata of the electronic document containing the data table 170 [the object comprising a data table, … the predicted structure comprising predicted boundaries corresponding to a row and column structure of the data table;] or from other data sources. One or more attributes of interest by the attribute obtaining module 210 may be preconfigured so as to obtain the specific values 212 of the respective attributes from the input data table 170. In some implementations, the attribute obtaining module 210 may represent the obtained values 212 of the attributes in a two-dimensional structure of rows and columns of the data table 170 the object comprising a data table, … the predicted structure comprising predicted boundaries corresponding to a row and column structure of the data table], and the two-dimensional structure includes the values 212 of the attributes extracted for the respective cell 172...; And in [0048] To create this structured representation, several steps are taken, including: overlap is eliminated between any predicted bounding boxes of the same class, rows and columns [the object comprising a data table, … the predicted structure comprising predicted boundaries corresponding to a row and column structure of the data table] are expanded to fill any gaps between them within the table boundary, rows and columns are intersected to form grid cells, cells falling in the same supercell bounding box must be merged into a single cell, each word must be uniquely assigned to a cell, and cells must be labeled by their role using the column header and row header predictions )
… the encoded plurality of input channels indicating at least locations of the row and column structure in the predicted boundaries of the data table; (As depicted in Fig 7 and in [0041] The input to the table detection model 115 is an image of a document page, which for example, can be rendered electronically, scanned, or photographed. The output of the model is a set of bounding box and class predictions, where a bounding box is denoted by four coordinates describing the locations [… the encoded plurality of input channels indicating at least locations of the row and column structure in the predicted boundaries of the data table] of the four edges of the bounding box rectangle, and the class prediction is both a class label and a confidence score between 0 and 1. And in [0070] Table detection involves determining the location [… the encoded plurality of input channels indicating at least locations of the row and column structure in the predicted boundaries of the data table] of the table within its surrounding document as shown in FIG. 7. )
Additionally Do teaches object detection of data object comprising a data table and the use of them as claimed the object comprising a data table, … the predicted structure comprising predicted boundaries corresponding to a row and column structure of the data table; (in [0098] At block 710, values of at least one attribute for a plurality of cells in a data table are obtained, the values of the at least one attribute indicating at least one of a semantic meaning of data filled in the plurality of cells or a structure of the data table. The plurality of cells are arranged in rows and columns in the data table [the object comprising a data table, … the predicted structure comprising predicted boundaries corresponding to a row and column structure of the data table]. At block 720, a feature representation of the values of the at least one attribute is extracted using a first learning network… )
… the encoded plurality of input channels indicating at least locations of the row and column structure in the predicted boundaries of the data table; (in [0067] In some implementations, the extraction of the row-wise feature representation and/or the column-wise feature representation may be implemented in the respective network sections 310 of the first learning network 222... Depending on the location of the network section 310 in the first learning network 222, the convolutional layer 410 may extract the feature representation of the values 212 of the attributes of the respective cells 172 in the data table 170 (when it is located at the network section 310-1) [… the encoded plurality of input channels indicating at least locations of the row and column structure in the predicted boundaries of the data table] or may extract the feature representation output by the preceding network section 310. And in [0031] …. The data table 170 may also be of an unfixed size, and then the size of the data table can be determined by regions where non-empty cells of the spreadsheet are located...)
Do, Moc and Sal are analogous art because both involve developing information retrieval and object recognition techniques using machine learning systems and algorithms.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the prior art for developing and implementing information retrieval and object recognition techniques of data tables/structures, as disclosed by Moc and Do with the method of developing machine learning assisted labeling process and system as disclosed by Sal.
One of ordinary skill in the arts would have been motivated to combine the disclosed methods disclosed by Ro, Moc and Sal as noted above. Doing so enables solutions for auto-formatting of a data table using a learning network, (Do, 0002); and enables a table's content to be used in automated downstream applications, such as data visualization, aggregation of data across multiple sources, and statistical analysis, (Moc, 0002 and Do, 0014).
Regarding dependent claim 9, the rejection of claim 8 is incorporated; the claim limitations are similar to claim 4 and are thus rejected under the same rationale.
Regarding dependent claim 10, the rejection of claim 8 is incorporated; the claim limitations are similar to claim 7 and are thus rejected under the same rationale.
Regarding claim 11, the rejection of claim 8 is incorporated and Sal in combination with Moc and Do teaches the system of claim 8, wherein: the predicted structure further comprises predicted locations of one or more elements of the data table within the row and column structure, wherein the one or more elements comprise one or more of textual content or an image contained in the data table; and if the predicted boundaries are correct, the data table contains each of the one or more of the textual content or the image within corresponding predicted boundaries. (As depicted in Figs. 11 & 12:
PNG
media_image3.png
670
1123
media_image3.png
Greyscale
PNG
media_image4.png
773
1252
media_image4.png
Greyscale
Examiner notes the annotated section still read on amended claim limitation: wherein: the predicted structure further comprises predicted locations of one or more elements of the data table within the row and column structure, wherein the one or more elements comprise one or more of textual content or an image contained in the data table; and if the predicted boundaries are correct, the data table contains each of the one or more of the textual content or the image within corresponding predicted boundaries.)
Regarding dependent claim 12, the rejection of claim 8 is incorporated; the claim limitations are similar to claim 5 and are thus rejected under the same rationale.
Regarding claim 14, the rejection of claim 8 is incorporated and Sal in combination with Moc and Do teaches the system of claim 8, wherein the second machine learning model is further configured to: receive a second input comprising second object data fed to a third machine learning model, and a second output from the third machine learning model, the third machine learning model being different from the first machine learning model; ([0148] …. The multi-class channel stack is an ensemble of channels [ the ensemble wherein the second machine learning model is further configured to: receive a second input comprising second object data fed to a third machine learning model, and a second output from the third machine learning model, the third machine learning model being different from the first machine learning model; in the plurality of multi-stack channel model including third mol feed claimed data as depicted in Figs 11 and 12] where every channel corresponds to one of the classes of the label. Each channel is obtained by summing rectangles of ones in place of each bounding box pertaining to the corresponding class. Using stacks to represent bounding boxes is very convenient because, like images, stacks hold spatial information and can easily be fed into a neural network. Moreover, it's an efficient way of representing labels: information is preserved, and the machine learning component can identify recurrent spatial arrangements [alternatively wherein the second machine learning model is further configured to: receive a second input comprising second object data fed to a third machine learning model, and a second output from the third machine learning model, the third machine learning model being different from the first machine learning model; in the plurality of data objects feed to third machine learning model algorithm for identifying recurrent spatial arrangements]…
encode the second input and the second output into a second plurality of input channels; ([0148] FIG. 11 illustrates the methodology of the typicality learning component in computing typicality predictions from labels. First, a multi-class channel stack is extracted from the bounding boxes of a label representation. The multi-class channel stack is an ensemble of channels where every channel corresponds to one of the classes of the label [encode the second input and the second output into a second plurality of input channels of the corresponding class channel stack of encoded label objects as claimed encoded inputs]…)
and determine a probability of correctness of a predicted structure for the second object data predicted by the third machine learning model, based on an evaluation of the second plurality of input channels. ([0008] In embodiments, the one or more additional components can include a typicality learning component that employs a computation model that is trained to generate a typicality score [determine a probability of correctness of a predicted structure for the second object data predicted by the third machine learning model, based on an evaluation of the second plurality of input channels] related to typicality of a predicted label output by the inspection learning component and supplied as input to the computational model. The typicality score [and determine a probability of correctness of a predicted structure for the second object data predicted by the third machine learning model, based on an evaluation of the second plurality of input channels] characterizes the level of divergence of the predicted label relative to the distribution of labels in a pool of labeled observations…)
Regarding independent claim 15, Sal limitation a method of training a first machine learning model configured to determine a confidence of an output of a second machine learning model, the method comprising: ([0206] FIG. 13 illustrates an example device 2500, with a processor 2502 and memory 2504 that can be configured to implement various embodiments of the active learning framework as discussed in this disclosure. Memory 2504 can also host one or more databases and can include one or more forms of volatile data storage media such as random-access memory (RAM), and/or one or more forms of nonvolatile storage media (such as read-only memory (ROM), flash memory, and so forth).; And in 0216: …Some of the methods and processes described above, can be performed by a processor. The term “processor” should not be construed to limit the embodiments disclosed herein to any particular device type or system. The processor may include a computer system. The computer system may also include a computer processor (e.g., a microprocessor, microcontroller, digital signal processor, or general-purpose computer) for executing any of the methods and processes described above.)
receiving, from the second machine learning model, information associated with a predicted structure for a data object comprising a data ([0148] FIG. 11 illustrates the methodology of the typicality learning component in computing typicality predictions from labels. First, a multi-class channel stack is extracted from the bounding boxes of a label representation [obtaining a label indicating a correctness of the predicted structure based on a comparison of the predicted structure with a ground-truth version of the predicted structure]. The multi-class channel stack is an ensemble of channels where every channel corresponds to one of the classes of the label [information associated with a predicted structure for a data object comprising a data]. Each channel is obtained by summing rectangles of ones in place of each bounding box pertaining to the corresponding class. Using stacks to represent bounding boxes [receiving, from the second machine learning model, information associated with a predicted structure for a data object comprising a data] is very convenient because, like images, stacks hold spatial information and can easily be fed into a neural network. Moreover, it's an efficient way of representing labels: information is preserved, and the machine learning component can identify recurrent spatial arrangements among objects by training on labels from the pool of labeled data.)
and training the first machine learning model based on the information associated with the predicted structure for the data object and the label indicating the correctness of the predicted structure, to generate a trained machine learning model that determines the confidence of the output of the second machine learning model, wherein the trained first machine learning model is used in conjunction with the second machine learning model to detect failures of the second machine learning model. (in [0053] In embodiments, the framework can employ an inspection learning component (or primary learning component) that generates a prediction corresponding to an unlabeled observation… In embodiments, the label can specify the location of one or more bounding boxes and one or more classes associated with the bounding box(es).. A typicality score can be generated by a typicality learning component [training the first machine learning model based on the information associated with the predicted structure for the data object and the label indicating the correctness of the predicted structure,] directly from a label predicted by the inspection learning component. In embodiments, the typicality score can characterize the level of deviation of a label predicted [to generate a trained machine learning model that determines the confidence of the output of the second machine learning model, wherein the trained first machine learning model is used in conjunction with the second machine learning model to detect failures of the second machine learning model] by the inspection learning component relative to the distribution of labels in a pool of labeled observations…)
While Sal teaches labeling component for feeding claimed processed data as claimed a first machine learning model.
Sal does not expressly disclosed object detection of data object comprising a data table and the use of them as claimed a data object comprising a data table, wherein the predicted structure comprises at least a prediction of a row and column structure of the data table;
Moc does expressly disclosed object detection of data object comprising a data table and the use of them as claimed a data object comprising a data table, wherein the predicted structure comprises at least a prediction of a row and column structure of the data table; (in [0036] The values 212 of the attributes of the respective cells 172 depend on specific settings/configurations of the cells 172 in generation of the data table 170, and the main purpose of the settings/configurations is to facilitate the organization and presentation of the data. The values 212 of the attributes may be extracted from metadata of the electronic document containing the data table 170 [a data object comprising a data table, wherein the predicted structure comprises at least a prediction of a row and column structure of the data table] or from other data sources. One or more attributes of interest by the attribute obtaining module 210 may be preconfigured so as to obtain the specific values 212 of the respective attributes from the input data table 170. In some implementations, the attribute obtaining module 210 may represent the obtained values 212 of the attributes in a two-dimensional structure of rows and columns of the data table 170 [ a data object comprising a data table, wherein the predicted structure comprises at least a prediction of a row and column structure of the data table], and the two-dimensional structure includes the values 212 of the attributes extracted for the respective cell 172...; And in [0048] To create this structured representation, several steps are taken, including: overlap is eliminated between any predicted bounding boxes of the same class, rows and columns [a data object comprising a data table, wherein the predicted structure comprises at least a prediction of a row and column structure of the data table] are expanded to fill any gaps between them within the table boundary, rows and columns are intersected to form grid cells, cells falling in the same supercell bounding box must be merged into a single cell, each word must be uniquely assigned to a cell, and cells must be labeled by their role using the column header and row header predictions )
Additionally Do teaches object detection of data object comprising a data table and the use of them as claimed a data object comprising a data table, wherein the predicted structure comprises at least a prediction of a row and column structure of the data table; (in [0098] At block 710, values of at least one attribute for a plurality of cells in a data table are obtained, the values of the at least one attribute indicating at least one of a semantic meaning of data filled in the plurality of cells or a structure of the data table. The plurality of cells are arranged in rows and columns in the data table [a data object comprising a data table, wherein the predicted structure comprises at least a prediction of a row and column structure of the data table]. At block 720, a feature representation of the values of the at least one attribute is extracted using a first learning network… )
Do, Moc and Sal are analogous art because both involve developing information retrieval and object recognition techniques using machine learning systems and algorithms.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the prior art for developing and implementing information retrieval and object recognition techniques of data tables/structures, as disclosed by Moc and Do with the method of developing machine learning assisted labeling process and system as disclosed by Sal.
One of ordinary skill in the arts would have been motivated to combine the disclosed methods disclosed by Ro, Moc and Sal as noted above. Doing so enables solutions for auto-formatting of a data table using a learning network, (Do, 0002); and enables a table's content to be used in automated downstream applications, such as data visualization, aggregation of data across multiple sources, and statistical analysis, (Moc, 0002 and Do, 0014).
Regarding claim 16, the rejection of claim 15 is incorporated and Sal in combination with Moc and Do teaches the method of claim 15, the information associated with the predicted structure for the data object comprises coordinates for the structure for the data table, spanning information for one or more cells within the structure for the data table, or a combination thereof. (in [0047] Active learning can also be used for object detection. Object detection typically involves identifying one or more bounding boxes within an observation and assigning one or more classes to the bounding box(es). The location [coordinates for the structure for the data table] of the bounding boxes in the observation and the assigned class(es) [the information associated with the predicted structure for the data object comprises coordinates for the structure for the data table, spanning information for one or more cells within the structure for the data table, or a combination thereof] produced by object detection are commonly referred to as a prediction … )
Additionally Moc teaches (in [0033] The first three classes are columns, rows, and supercells (also called “merged cells” or “spanning cells”) [the information associated with the predicted structure for the data object comprises coordinates for the structure for the data table, spanning information for one or more cells within the structure for the data table, or a combination thereof]. These three classes recognize the cell structure of the table. Each intersection of a column and a row forms a grid cell. The supercell object class detects [the information associated with the predicted structure for the data object comprises coordinates for the structure for the data table, spanning information for one or more cells within the structure for the data table, or a combination thereof] when grid cells in the table are merged.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Sal and Moc for the same reasons disclosed above.
Regarding claim 19, the rejection of claim 15 is incorporated and Sal in combination with Moc and Do teaches the method of claim 15, wherein the generated trained machine learning model is configured to: receive, from the second machine learning model, information associated with the predicted structure for the data object; encode the information associated with the predicted structure for the data object to produce encoded input channels; evaluate the information associated with the data object with the encoded input channels; and based on the evaluation, determine a probability of correctness of the predicted structure for the data object, the confidence of the output of the second machine being based on the probability of correctness.
wherein the generated trained machine learning model is configured to: receive, from the second machine learning model, information associated with the predicted structure for the data object; ([0148] FIG. 11 illustrates the methodology of the typicality learning component in computing typicality predictions from labels. First, a multi-class channel stack is extracted from the bounding boxes of a label representation [wherein the generated trained machine learning model is configured to: receive, from the second machine learning model, information associated with the predicted structure for the data object]. The multi-class channel stack is an ensemble of channels where every channel corresponds to one of the classes of the label [information associated with the predicted structure for the data object]. Each channel is obtained by summing rectangles of ones in place of each bounding box pertaining to the corresponding class. Using stacks to represent bounding boxes is very convenient because, like images, stacks hold spatial information [receive, from the second machine learning model, information associated with the predicted structure for the data object in the plurality of models for each multi-class channel for modeling the stacks as depicted by Figs. 11 & 12] and can easily be fed into a neural network. Moreover, it's an efficient way of representing labels: information is preserved, and the machine learning component can identify recurrent spatial arrangements among objects by training on labels from the pool of labeled data.)
encoding, using a second machine learning model, the information associated with the predicted structure for the data object to produce encoded input channels; evaluating, using the second machine learning model, the information associated with the data object with the encoded input channels; and based on the evaluating, determining, using the second machine learning model, a probability of correctness of the predicted structure for the data object. ([0008] In embodiments, the one or more additional components can include a typicality learning component that employs a computation model [encoding, using a second machine learning model, the information associated with the predicted structure for the data object to produce encoded input channels; evaluating, using the second machine learning model, the information associated with the data object with the encoded input channels] that is trained to generate a typicality score [and based on the evaluating, determining, using the second machine learning model, a probability of correctness of the predicted structure for the data object] related to typicality of a predicted label output by the inspection learning component and supplied as input to the computational model [using the second machine learning model associated with the class in the plurality of classes]. The typicality score [determining, using the second machine learning model, a probability of correctness of the predicted structure for the data object] characterizes the level of divergence of the predicted label relative to the distribution of labels in a pool of labeled observations…)
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Sal in view of Moc and Do in further view of Buban et al. (US 12094231, hereinafter ‘Bub’).
Regarding claim 13, the rejection of claim 8 is incorporated and Sal in combination with Moc and Do teaches the system of claim 8, wherein: the object data of the input comprises text; ( As depicted in Figs. 11 & 12 data object including image and labelled text for the bounding boxes for clustering similar data objects by third algorithm)
and the system comprises a third machine learning model is configured to apply a language model to the text to determine the probability of correctness. (in [0195] In embodiments, the set of quality control components 211′ can also include a quality control component 211C′ that outputs a typicality score based upon the prediction output by the typicality learning component 209B. The typicality score indicates how much the prediction of the inspection learning component 209C′ deviates from the distribution of labels in the pool of labeled data 207C by analyzing its inner-structure [and the system comprises a third machine learning model is configured to apply a language model to the text ….]. [0196] FIG. 16 illustrates operations that can be part of the computation of the typicality score [and the system comprises a third machine learning model is configured to apply a language model to the text to determine the probability of correctness], where a predicted label 1601 output by the inspection learning component 209C′ (e.g., one or more bounding boxes with associated uncertainty score(s)) are converted to a multi-class channel stack 1603, for example as described above with respect to FIG. 13. The multi-class channel stack 1603 is supplied as input to the typicality learner component 209B′, which employs machine learning inference to extract features (1605) and detect feature outliers or localization errors in generating a typicality score (1607) for the predicted label 1601 [and the system comprises a third machine learning model is configured to apply a language model to the text to determine the probability of correctness].)
While Sal teaches the processing of image document for analyzing associated labels as claimed language model. Sal does not expressly disclosed a text process algorithm as a language model.
Rub discloses a text process algorithm as a language model, in 11:41-34: The system analyzes the extracted text content (512). The analysis can include post processing as described above with respect to FIG. 4. More broadly, the analysis can include using the extracted text to determine an operation or next step of the system. A data structure, as described above, can associate the encoded text of the document with particular labels of bounding boxes in which the encoded text occurs [the system comprises a third machine learning model is configured to apply a language model to the text to determine the probability of correctness]. Once particular text is parsed from the document, e.g., based on the type of label, the system can apply various appropriate natural language processing techniques [apply a language model to the text] or additional models to understand the meaning of the text and compare that to expected results [to determine the probability of correctness]. Values in the document can be compared to expected values for consistency…
Bub, Do, Moc and Sal are analogous art because both involve developing information retrieval and object recognition techniques using machine learning systems and algorithms.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the prior art for developing and implementing machine learning models to process input images and to predict locations and types of data found within the input image as disclosed by Bub with the method of developing machine learning assisted labeling process and system as collectively disclosed by Do, Moc and Sal.
One of ordinary skill in the arts would have been motivated to combine the disclosed methods disclosed by Bub, Do, Moc and Sal to develop and implement machine learning models that can be used for processing on the images, for example, to extract text content from within the bounding boxes and determine a semantic meaning of text portions and to match the meaning of the text portions with one or more corresponding system text descriptions, (Ran, Abstract); Doing so enables a computer vision model develop models trained to identify particular regions of a specific document and efficiently evaluate the content of the bounding boxes that is difficult with other techniques (Bub, 12:4-11).
Claims 17, 18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Sal in view of Moc and Do in further view of Fang et al. (US 20220301156, hereinafter ‘Fan’).
Regarding claim 17, the rejection of claim 15 is incorporated and Sal in combination with Moc and Do teaches the method of claim 15, wherein the training of the first machine learning model comprises: predicting, with the first machine learning model, a likelihood of correctness of the predicted structure; ([0190] In embodiments, the relevance score for a particular prediction output by the inspection learning component 209C′ can be computed from the probability of a particular prediction as output by the inspection learning component 209C′. For example, where the inspection learning component 209C′ outputs bounding boxes coordinates as well as probabilities of classes for each bounding box [wherein the training of the first machine learning model comprises: predicting, with the first machine learning model, a likelihood of correctness of the predicted structure], the relevance score can be given by the following:…)
and minimizing an error associated with the predicted likelihood of correctness and the obtained label. ([0073] In embodiments, the framework 200 includes a set of machine learning components (labeled 209), which include the relevance learning component 209A, the typicality learning component 209B and the inspection learning component 209C. Each one of these machine learning components can be trained during the training phase (e.g., blocks 255 to 261 of FIG. 2B) or during the final training (block 283 of FIG. 2B), or can be used for predictions in the active learning phase (e.g., blocks 265 to 279 of FIG. 2B) as illustrated in FIG. 5. Blocks 503 to 513 illustrate the training of a machine learning component based on a labeled observation. The observation is used as input (block 501) to the machine learning component (block 503) to generate a corresponding prediction (block 505). The prediction (block 505) and the ground truth label of the observation (block 509) [an error associated with the predicted likelihood of correctness and the obtained label] are used to compute an error (block 511), and such error is used to update the machine learning component (block 513) [and minimizing an error associated with the predicted likelihood of correctness and the obtained label]…)
While Sal teaches using error to update the machine learning component associated with claimed prediction elements.
Sal does not expressly disclose minimizing an error associated with the machine learning process.
Fan teaches minimizing an error associated with the machine learning process, in [0075] Error estimator 706, on the other hand, is trained using a “ground-truth error” determined using ground-truth classification label 710 and predicted classification label 708. In one example, the error may be a cross entropy loss between ground-truth classification label 710 and predicted classification label 708. Training of error estimator 706 aims to minimize [an error associated with the predicted likelihood of correctness and the obtained label] the difference between an estimated classification error 712 estimated by error estimator 706 and the “ground-truth error” determined using ground-truth classification label 710 and predicted classification label 708. In some embodiments, error estimator 706 may be implemented by a multi-layer perceptron or other networks.
Fan, Do, Moc and Sal are analogous art because both involve developing information retrieval and object recognition techniques using machine learning systems and algorithms.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the prior art for developing systems and methods for training learning models for analyzing medical images with an error estimator and applying the trained models for image analysis as disclosed by Fan with the method of developing machine learning assisted labeling process and system as collectively disclosed by Do, Moc and Sal.
One of ordinary skill in the arts would have been motivated to combine the disclosed methods disclosed by Bub, Do, Moc and Sal to develop and implement machine learning models configured to detect an object from medical images using an error estimation process based on the predicted bounding box and object labels, (Fan, 0013); Doing so allows an independent error estimator to be trained to learn the complex error patterns of arbitrary main model (Fan, 0036); and improve the training of learning models with low annotation cost using a novel error estimation model, (Fan, 0035).
Regarding claim 18, the rejection of claim 17 is incorporated and Sal in combination with Moc, Do and Fan teaches the method of claim 17, wherein the minimizing of the error comprises determining a difference between the obtained label … (in [0073] In embodiments, the framework 200 includes a set of machine learning components (labeled 209), which include the relevance learning component 209A, the typicality learning component 209B and the inspection learning component 209C. Each one of these machine learning components can be trained during the training phase (e.g., blocks 255 to 261 of FIG. 2B) or during the final training (block 283 of FIG. 2B), or can be used for predictions in the active learning phase (e.g., blocks 265 to 279 of FIG. 2B) as illustrated in FIG. 5. Blocks 503 to 513 illustrate the training of a machine learning component based on a labeled observation. The observation is used as input (block 501) to the machine learning component (block 503) to generate a corresponding prediction (block 505). The prediction (block 505) and the ground truth label of the observation (block 509) are used to compute an error (block 511), and such error is used to update the machine learning component (block 513) [wherein the minimizing of the error comprises determining a difference between the obtained label … ]…)
While Sal teaches using error to update the machine learning component associated with claimed prediction elements.
Sal does not expressly disclose minimizing an error associated with the machine learning process and defining a threshold as claimed in wherein the minimizing of the error comprises determining a difference between the obtained label and the predicted likelihood of correctness of the predicted structure, defining a threshold, and performing an iterative optimization process to reduce the difference until the threshold is reached.
Fan teaches minimizing an error associated with the machine learning process and defining a threshold as claimed in wherein the minimizing of the error comprises determining a difference between the obtained label and the predicted likelihood of correctness of the predicted structure, defining a threshold, in [0075] Error estimator 706, on the other hand, is trained using a “ground-truth error” determined using ground-truth classification label 710 and predicted classification label 708. In one example, the error may be a cross entropy loss between ground-truth classification label 710 and predicted classification label 708. Training of error estimator 706 aims to minimize [wherein the minimizing of the error comprises determining a difference between the obtained label and the predicted likelihood of correctness of the predicted structure, defining a threshold] the difference between an estimated classification error 712 estimated by error estimator 706 and the “ground-truth error” determined using ground-truth classification label 710 and predicted classification label 708 [determining a difference between the obtained label and the predicted likelihood of correctness of the predicted structure,]. In some embodiments, error estimator 706 may be implemented by a multi-layer perceptron or other networks.; And in [0101] In step S1310, it is determined whether the error is too high. In some embodiments, the determination can be made by the user as a result of the visual inspection. In some alternative embodiments, the determination can be made automatically by image analysis device 203 by, e.g., by comparing the error to a threshold [defining a threshold]. If the error is too high (S1310: Yes), image analysis device 203 may request user interaction to improve the learning model or request the learning model to be retrained by model training device 202 (step S1314). Image analysis device 203 repeat steps S1306-S1310 with the user-improved or retained new learning model. For example, the learning model may be updated using workflow 500 of FIG. 5, using the current learning model as the initial main model. Otherwise (S1310: No), image analysis device 203 may provide the image analysis results (step S1312), such as the classification label, the segmentation mask, or the bounding boxes.)
and performing an iterative optimization process to reduce the difference until the threshold is reached. in [0101] In step S1310, it is determined whether the error is too high. In some embodiments, the determination can be made by the user as a result of the visual inspection. In some alternative embodiments, the determination can be made automatically by image analysis device 203 by, e.g., by comparing the error to a threshold [defining a threshold,]. If the error is too high (S1310: Yes), image analysis device 203 may request user interaction to improve the learning model or request the learning model to be retrained by model training device 202 (step S1314) [performing an iterative optimization process to reduce the difference until the threshold is reached]. Image analysis device 203 repeat steps S1306-S1310 with the user-improved or retained new learning model. For example, the learning model may be updated using workflow 500 of FIG. 5, using the current learning model as the initial main model. Otherwise (S1310: No), image analysis device 203 may provide the image analysis results (step S1312), such as the classification label, the segmentation mask, or the bounding boxes.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Fan, Do, Moc and Sal for the same reasons disclosed above.
Regarding claim 20, the rejection of claim 15 is incorporated and Sal in combination with Moc and Do teaches the method of claim 15, wherein the second machine learning model has been trained by minimizing an error, (in [0073] In embodiments, the framework 200 includes a set of machine learning components (labeled 209), which include the relevance learning component 209A, the typicality learning component 209B and the inspection learning component 209C. Each one of these machine learning components can be trained during the training phase (e.g., blocks 255 to 261 of FIG. 2B) or during the final training (block 283 of FIG. 2B), or can be used for predictions in the active learning phase (e.g., blocks 265 to 279 of FIG. 2B) as illustrated in FIG. 5. Blocks 503 to 513 illustrate the training of a machine learning component based on a labeled observation. The observation is used as input (block 501) to the machine learning component (block 503) to generate a corresponding prediction (block 505). The prediction (block 505) and the ground truth label of the observation (block 509) are used to compute an error (block 511), and such error is used to update the machine learning component (block 513) [wherein the second machine learning model has been trained by minimizing an error ]…)
the minimizing of the error comprising determining a difference between coordinates of the predicted structure and the ground-truth version of the predicted structure, (in [0148] FIG. 11 illustrates the methodology of the typicality learning component in computing typicality predictions from labels. First, a multi-class channel stack is extracted from the bounding boxes [coordinates of the predicted structure] of a label representation [the ground-truth version of the predicted structure,]. The multi-class channel stack is an ensemble of channels where every channel corresponds to one of the classes of the label. Each channel is obtained by summing rectangles of ones in place of each bounding box pertaining to the corresponding class. Using stacks to represent bounding boxes is very convenient because, like images, stacks hold spatial information and can easily be fed into a neural network. Moreover, it's an efficient way of representing labels: information is preserved, and the machine learning component can identify recurrent spatial arrangements among objects by training on labels from the pool of labeled data.)
While Sal teaches using error to update the machine learning component associated with claimed prediction elements.
Sal does not expressly disclose minimizing an error associated with the machine learning process using an iterative process as claimed in performing an iterative optimization process to reduce the difference.
Fran does expressly teach minimizing an error associated with the machine learning process using an iterative process as claimed in performing an iterative optimization process to reduce the difference. in [0101] In step S1310, it is determined whether the error is too high. In some embodiments, the determination can be made by the user as a result of the visual inspection. In some alternative embodiments, the determination can be made automatically by image analysis device 203 by, e.g., by comparing the error to a threshold [performing an iterative optimization process to reduce the difference]. If the error is too high (S1310: Yes), image analysis device 203 may request user interaction to improve the learning model or request the learning model to be retrained by model training device 202 (step S1314) [performing an iterative optimization process to reduce the difference]. Image analysis device 203 repeat steps S1306-S1310 with the user-improved or retained new learning model. For example, the learning model may be updated using workflow 500 of FIG. 5, using the current learning model as the initial main model. Otherwise (S1310: No), image analysis device 203 may provide the image analysis results (step S1312), such as the classification label, the segmentation mask, or the bounding boxes.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Fan, Do, Moc and Sal for the same reasons disclosed above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Hu et al. (US 20220036548): teaches a deep learning neural network that can identify corpora lutea in the ovaries and a rules-based technique using a neural network model, a set of images with a bounding box around objects that are identified within the set of images based on coordinates predicted for the bounding box.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OLUWATOSIN ALABI whose telephone number is (571)272-0516. The examiner can normally be reached Monday-Friday, 8:00am-5:00pm EST..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OLUWATOSIN ALABI/ Primary Examiner, Art Unit 2129
Prosecution Insights