Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The instant application having Application No. 17663240 has a total of 9 claims pending in the application, of which claims 3, 6, and 9 have been cancelled.
Claim Rejections – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 1 is a process type claim. Claim 4 is a machine type claim. Claim 7 is a manufacture type claim. Therefore, claims 1-9 are directed to either a process, machine, manufacture or composition of matter.
As per claim 1,
2A Prong 1:
“wherein the training dataset is used for a) training a … model for generating predictions of interest, and b) identifying a plurality of test instances from the training dataset to be used by a proxy model to provide global explanations for the predictions of interest” The user mentally or with pencil and paper takes data to make predictions and uses a model to make the predictions and another model to explain the predictions.
“Sampling, … the training dataset for identifying the plurality of test instances” The user mentally or with pencil and paper looks through the training data for test data
“a) extracting a plurality of features from the training dataset with each of the plurality of features having continuous attribute values” The user mentally or with pencil and paper looks for relevant features within the data.
“b) binarizing each of the plurality of features to discretize the continuous attribute values to ‘1’ and ‘0’” The user mentally or with pencil and paper converts any continuous features to discretized binary).
“c) generating a concept lattice for the binarized plurality of features of the training dataset using Formal Concept Analysis (FCA) based approach, wherein the concept lattice comprises a plurality of concepts representing data of the training dataset and attribute relationships within the training dataset by arranging the training dataset into hierarchical groups based on commonality among the binarized plurality of features” The user mentally or with pencil and paper performs formal concept analysis to create a concept lattice showing relationships among the data).
“d) deriving and ranking a plurality of implication rules form the concept lattice to generate a ranked list, wherein the plurality of implication rules are derived based on the FCA approach, wherein the plurality of implication rules are shortlisted based on coverage and redundancy” The user mentally or with pencil and paper creates and ranks rules based on the concept lattice considering coverage and redundancy.
“e) selecting a predefined number of implication rules from the ranked list arrange din ascending order of rank, wherein instances that follow the implication rules are identified based on whether a particular sample “s” follows implication rule “r” or not” The user mentally or with pencil and paper chooses the best rules from the set and notes that the test instances match the rules as needed.
“f) identifying the plurality of test instances form the training dataset corresponding to each of the selected predefined number of implication rules, wherein the plurality of test instances are selected using redundancy criteria that selects non-redundant explanations by avoiding instances from training dataset with similar explanations” The user mentally or with pencil and paper uses the rules on the test instances and chooses based on redundancy and avoiding similar explanation).
“forwarding … the plurality of test instances to the proxy model to generate global explanations for the predictions of interest generated by the ML model using a Sub-Modular pick – local interpretable model-agnostic explanations (SP-Lime) approach”. The user mentally or with pencil and paper uses an SP-Lime model to provide explanations for the test instances.
“Evaluating the effectiveness of the data and proxy model based hybrid approach by computing false discovery rate metric, wherein the false discovery rate metric is defined as the total number of noisy features selected as features in the plurality of test instances” The user mentally or with pencil and paper calculates the false discovery rate by adding up the total number of noisy features of the selected features in the plurality of test instances.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
“A processor”, “one or more hardware processors” (mere instructions to apply the exception using a generic computer component);
“machine learning”, “ML model” (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: Machine learning model is a generic model with no additional limitations or details that would make it anything more than a generic off the shelf machine learning model.
“receiving … a training dataset” (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)).
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
“A processor”, “one or more hardware processors” (mere instructions to apply the exception using a generic computer component)
“machine learning”, “ML model” (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: Machine learning model is a generic model with no additional limitations or details that would make it anything more than a generic off the shelf machine learning model.
“receiving … a training dataset” (MPEP 2106.05(d)(II) indicate that merely “receiving and transmitting data” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer).
As per claim 2, this claim has additional mental steps similar to claim 1, and is rejected for similar reasons to claim 1.
As per claim 4,
2A Prong 1:
“wherein the training dataset is used for a) training a … model for generating predictions of interest, and b) identifying a plurality of test instances from the training dataset to be used by a proxy model to provide global explanations for the predictions of interest” The user mentally or with pencil and paper takes data to make predictions and uses a model to make the predictions and another model to explain the predictions.
“Sample the training dataset for identifying the plurality of test instances” The user mentally or with pencil and paper looks through the training data for test data
“a) extract a plurality of features from the training dataset with each of the plurality of features having continuous attribute values” The user mentally or with pencil and paper looks for relevant features within the data.
“b) binarize each of the plurality of features to discretize the continuous attribute values to ‘1’ and ‘0’” The user mentally or with pencil and paper converts any continuous features to discretized binary).
“c) generate a concept lattice for the binarized plurality of features of the training dataset using Formal Concept Analysis (FCA) based approach, wherein the concept lattice comprises a plurality of concepts representing data of the training dataset and attribute relationships within the training dataset by arranging the training dataset into hierarchical groups based on commonality among the binarized plurality of features” The user mentally or with pencil and paper performs formal concept analysis to create a concept lattice showing relationships among the data).
“d) derive and rank a plurality of implication rules form the concept lattice to generate a ranked list, wherein the plurality of implication rules are derived based on the FCA approach wherein the plurality of implication rules are shortlisted based on coverage and redundancy” The user mentally or with pencil and paper creates and ranks rules based on the concept lattice and coverage/redundancy.
“e) selecting a predefined number of implication rules from the ranked list arrange din ascending order of rank, wherein instances that follow the implication rules are identified based on whether a particular sample “s” follows implication rule “r” or not” The user mentally or with pencil and paper chooses the best rules from the set and determines which instances follow the rules).
“f) identify the plurality of test instances form the training dataset corresponding to each of the selected predefined number of implication rules wherein the plurality of test instances are selected using a redundancy criteria that selects non-redundant explanations by avoiding instances from training dataset with similar explanations” The user mentally or with pencil and paper uses the rules on the test instances to determine non-redundant and non-similar explanations. ).
“forward the plurality of test instances to the proxy model to generate global explanations for the predictions of interest generated by the ML model using a Sub-Modular pick – local interpretable model-agnostic explanations (SP-Lime) approach”. The user mentally or with pencil and paper uses an SP-Lime model to provide explanations for the test instances.
“Evaluating the effectiveness of the proxy model based hybrid approach by computing false discovery rate metric which is defined as the total number of noisy features selected as features in the plurality of test instances” The user mentally or with pencil and paper calculates the false discovery rate by adding up the total number of noisy features of the selected features in the plurality of test instances.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
“A processor”, “one or more Input/Output (I/O) interfaces”, “one or more hardware processors” (mere instructions to apply the exception using a generic computer component);
“machine learning”, “ML model” (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: Machine learning model is a generic model with no additional limitations or details that would make it anything more than a generic off the shelf machine learning model.
“receiving … a training dataset” (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)).
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
“A processor”, “one or more Input/Output (I/O) interfaces”, “one or more hardware processors” (mere instructions to apply the exception using a generic computer component)
“machine learning”, “ML model” (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: Machine learning model is a generic model with no additional limitations or details that would make it anything more than a generic off the shelf machine learning model.
“receive … a training dataset” (MPEP 2106.05(d)(II) indicate that merely “receiving and transmitting data” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer).
As per claim 5, this claim has additional mental steps similar to claim 4, and is rejected for similar reasons to claim 4.
As per claim 7,
2A Prong 1:
“wherein the training dataset is used for a) training a … model for generating predictions of interest, and b) identifying a plurality of test instances from the training dataset to be used by a proxy model to provide global explanations for the predictions of interest” The user mentally or with pencil and paper takes data to make predictions and uses a model to make the predictions and another model to explain the predictions.
“Sampling, … the training dataset for identifying the plurality of test instances” The user mentally or with pencil and paper looks through the training data for test data
“a) extracting a plurality of features from the training dataset with each of the plurality of features having continuous attribute values” The user mentally or with pencil and paper looks for relevant features within the data.
“b) binarizing each of the plurality of features to discretize the continuous attribute values to ‘1’ and ‘0’” The user mentally or with pencil and paper converts any continuous features to discretized binary).
“c) generating a concept lattice for the binarized plurality of features of the training dataset using Formal Concept Analysis (FCA) based approach, wherein the concept lattice comprises a plurality of concepts representing data of the training dataset and attribute relationships within the training dataset by arranging the training dataset into hierarchical groups based on commonality among the binarized plurality of features” The user mentally or with pencil and paper performs formal concept analysis to create a concept lattice showing relationships among the data).
“d) deriving and ranking a plurality of implication rules form the concept lattice to generate a ranked list, wherein the plurality of implication rules are derived based on the FCA approach wherein the plurality of implication rules are shortlisted based on coverage and redundancy” The user mentally or with pencil and paper creates and ranks rules based on the concept lattice and avoiding redundancy and maintaining coverage.
“e) selecting a predefined number of implication rules from the ranked list arrange din ascending order of rank, wherein instances that follow the implication rules are identified based on whether a particular sample “s” follows implication rule “r” or not” The user mentally or with pencil and paper chooses the best rules from the set and identifies test instances that follow the implication rules based on whether they meet the requirements of the rule).
“f) identifying the plurality of test instances form the training dataset corresponding to each of the selected predefined number of implication rules wherein the plurality of test instances are selected using redundancy criteria that selects non-redundant explanations by avoiding instances from training dataset with similar explanations” The user mentally or with pencil and paper uses the rules on the test instances and avoid redundant and similar explanations).
“forwarding … the plurality of test instances to the proxy model to generate global explanations for the predictions of interest generated by the ML model using a Sub-Modular pick – local interpretable model-agnostic explanations (SP-Lime) approach”. The user mentally or with pencil and paper uses an SP-Lime model to provide explanations for the test instances.
“evaluating the effectiveness of the proxy model based hybrid approach by computing false discovery rate metric which is defined as the total number of noisy feature selected as features in the plurality of test instances” The user mentally or with pencil and paper calculates the false discovery rate by adding up the total number of noisy features of the selected features in the plurality of test instances.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
“non-transitory computer readable mediums “one or more hardware processors” (mere instructions to apply the exception using a generic computer component);
“machine learning”, “ML model” (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: Machine learning model is a generic model with no additional limitations or details that would make it anything more than a generic off the shelf machine learning model.
“receiving … a training dataset” (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)).
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
“non-transitory computer readable mediums “one or more hardware processors” (mere instructions to apply the exception using a generic computer component)
“machine learning”, “ML model” (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: Machine learning model is a generic model with no additional limitations or details that would make it anything more than a generic off the shelf machine learning model.
“receiving … a training dataset” (MPEP 2106.05(d)(II) indicate that merely “receiving and transmitting data” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer).
As per claim 8, this claim has additional mental steps similar to claim 7, and is rejected for similar reasons to claim 7.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-2, 4-5, and 7-8 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
As per claims 1, 4, and 7, these claims call for “wherein instances that follow the implication rules are identified” in section e of each of the independent claims. The claim does not contain any “instances” and therefore it is confusing and unclear just what instances are being referred to here. This causes the claim to be unclear, and therefore rejected under U.S.C. 112(b) for failing to particularly point out and claim the intended invention. For the sake of examination, the Examiner will assume this claim refers to the “test instances” found in the receiving step of the claim.
As per claims 2, 5, and 8, these claims are rejected as being dependent on a claim rejected under U.S.C. 112(b).
As per claims 1, 4, and 7, The term “similar explanations” in step f is a relative term which renders the claim indefinite. The term “similar” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The term “similar” here has been found to be indefinite, and therefore rejected under U.S.C. 112(b) for failing to particularly point out and claim the intended invention.
As per claims 2, 5, and 8, these claims are rejected as being dependent on a claim rejected under U.S.C. 112(b).
As per claims 1, 4, and 7, these claims contain the term “noisy features” in the final line of the claim. However, the term “noisy” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The term “similar” here has been found to be indefinite, and therefore rejected under U.S.C. 112(b) for failing to particularly point out and claim the intended invention. This causes the term “noisy feature” to be unclear as there is no explanation as to how to determine whether a feature is “noisy.” This causes the claim to be unclear, and therefore rejected under U.S.C. 112(b) for failing to particularly point out and claim the intended invention.
As per claims 2, 5, and 8, these claims are rejected as being dependent on a claim rejected under U.S.C. 112(b).
Prior Art Rejection
As per claims 1-2, 4-5, and 7-8, no art rejections will be given for these claims. The claims contain known material as shown in the previous office action. The current amendments contain additions to having rules which are shortlisted based on coverage and redundancy, as well as match “instances” based upon whether a “particular sample” follows a rule. These aspects are met by the already cited Kashnitsky reference, which denotes going through the data to produce rules for each object (See Kashnitsky, Pg.5, particularly algorithm 1) as well as avoiding redundancy (Kashnitsky, Pg.25, numbers 2-5, which looks for the best features to add to improve discriminatory power, which avoids adding redundant features as this would not increase discrimination).
However, the cited reference do not disclose selecting tests based on non-redundancy and coverage. However, Vandewiele et al (WO 2007020602 A1) discloses selecting tests based on coverage (Vandewiele, Abstract) and avoiding redundancy (Vandewiele, Pg.2, second paragraph).
Further, the cited references also fail to disclose a false discovery rate metric, wherein the false discovery rate metric is defined as the total number of noisy features selected as features in the plurality of test instances. As shown above, the application fails to explicitly disclose what or how a “noisy feature” is defined, leading to a rejection under U.S.C. 112(b) for failing to particularly point out and claim the intended invention. Regardless, the Kim reference (“Controlling the False Discovery Rate for Feature Selection in High-resolution NMR Spectra”) discloses determining False Discovery Rates for determining false positives among all the hypothesis rejected (Pg.58, C2, last paragraph; Pg.59, C1, first paragraph) when combined with the previously cited reference, this would make it obvious to one of ordinary skill in the art that one could find “noisy features” that have been falsely chosen as good features (i.e. false positives) and use that to determine how effective the feature selection had been. (See Kim, Pg.60, Section 3).
So while each of the limitations individually appear to be met by the references, the combination themselves is non-obvious, as too many pieces must be combined together to meet the claimed whole, and therefore the claims are considered to be non-obvious over the prior art.
Response to Arguments
In pg.8-9, the Applicant argues in regards to the rejection under U.S.C. 101,
The hybrid approach disclosed in claimed subject matter combines the proxy model and data- based approach which provides a better explanation at a much-reduced cost. The reduction in cost is a result of using a lesser number of samples rather than taking all the samples in the proxy model, which is not a practical and economic choice for real world huge datasets. However, the FCA enables selection of the most appropriate samples to maintain accuracy of explanations.
Therefore, Applicant submits that the claimed subject matter cannot be considered as a mental process which could be performed using pen and paper since generating implication rules is challenging as the number of rules can be very large for a given domain and training data.
In response, the Examiner maintains the rejection as shown above. The Applicant’s argument seems to be that the claims reduce the number of samples used and thereby improves the costs of performing the actions of the model, and it would not be a mental step because of the size of the data involved. However, the question is not how long it would take a human being, or how many human beings it would take to process the required data. Merely using large volumes of data is not enough to make the claimed invention more than the abstract idea of taking the mental steps to reduce the size of the samples used. The purpose of a generic computer system is to take large volumes of data and process them. Merely placing an abstract idea on a generic computer is not enough to make the claim significantly more than the abstract idea, and therefore the rejection is maintained as shown above.
In pg.9, the Applicant further argues in regards to the rejection under U.S.C. 101,
In response, Applicants submits that the claimed subject matter comprises additional elements that amount to significantly more than the abstract idea in terms of using the structured sampling, wherein optimal test instances are chosen both in terms of quantity and quality to generate explanations and interpret the outcomes. Based on the calibration error metric, it is observed that the Guided-LIME is a closer approximate of the original blackbox ML model, thus improving accuracy in provided explanations.
The claimed subject matter evaluates the effectiveness of the proxy model based hybrid approach by computing false discovery rate metric which is defined as the total number of noisy features selected as important features in the plurality of test instances
In response, the Examiner maintains the rejection as shown above. The choosing of an optimal test set based on quantity/quality for a system is a mental process as described above. Merely stating that the model will find a “closer approximate of the original blackbox ML model” is not an argument that causes the claim to be significantly more than the abstract idea. Merely calculating a false discovery rate to determine the effectiveness of a model, a mental process, improves the abstract idea, not the system itself.
Applicant cites paragraphs and figures from their specification as support for showing an improvement that makes it significantly more than the abstract idea. These paragraphs and figures compares the instant approach to other LIME based approaches and sates that the instant application is superior. However, comparing the current claims to other algorithms does not show that the subject matter itself is statutory under U.S.C. 101. Merely comparing the current claims to other algorithms and stating that it is a “closer approximate of the original blackbox ML model” is not an improvement to a technology or demonstratably more than the abstract idea as described above. Therefore the rejection is maintained as shown above.
Applicant's remaining arguments with respect to claims 1-2, 4-5, and 7-8 have been considered but are moot in view of the new ground(s) of rejection or are repetitions of the argument made above and the rejections maintained for similar reasons.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BEN M RIFKIN whose telephone number is (571)272-9768. The examiner can normally be reached Monday-Friday 9 am - 5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached at (571) 270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BEN M RIFKIN/ Primary Examiner, Art Unit 2123
Prosecution Insights