Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claims 1-6 are presented for examination.
This is a Non-Final Action.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-3 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claim 1 recites "An binary classification device… a reliability addition distribution generator… a bias corrector to correct… a corrected reliability output...”. The body of the claim does not define any specific hardware (i.e. a processor coupled to a memory, wherein the memory stores the modules) to execute the recited device. The claim lacks the necessary physical articles or objects to constitute a machine or a manufacture within the meaning of 35 USC 101. Therefore, the claimed apparatus is not limited to embodiments which include the hardware necessary to enable any underlying functionality to be realized, instead being software per se, and is therefore non-statutory.
Claims 1-3 are dependent upon claim 10, respectively, do not add anything to correct the deficiency and therefore are likewise rejected.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6 rejected under 35 U.S.C. 103 as being unpatentable over Edgar et al. (US 2021/0035015) in view of Al-Kofahi et al. (US 2003/0101181) further in view of Pinhasi et al. (US 2017/0092262)
1. Edgar teaches, A binary classification device (Fig 1:110, 106 and 108; Paragraphs 40 and 41 – teaches Presence vs Absence which supplies the binary classification context. The model is trained using annotated/labeled samples, Edgar), comprising:
regarding a reliability added to a sample by the annotator (Fig 6:210 & 604, Paragraph 78 and 81 – teaches a human annotator supplies confidence with an annotation, and a machine-generated annotation can likewise have an associated confidence, thereby disclosing the inputs to the histogram calculations, Edgar)
Edgar does not explicitly teach,
a reliability addition distribution generator to calculate, for each of annotators, a reliability addition histogram;
a bias corrector to correct, by referring to a reference reliability addition distribution, the reliability addition histogram to a corrected reliability addition histogram having a same characteristic as a characteristic of the reference reliability addition distribution;
a corrected reliability output to correct the reliability added by the annotator by referring to the corrected reliability addition histogram.
However, Al-Kofahi teaches,
a reliability addition distribution generator to calculate, for each of annotators, a reliability addition histogram (Fig 6:131/134, 610, Paragraphs 114 & 118 – teaches a separate score histogram is calculated for each classifier. The proposed modification applies this per-source calculation to annotation confidence scores, thus teaching cumulative histograms, Al-Kofahi); and
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which said subject matter pertains to modify Edgar’s annotation-confidence processing by calculating a separate confidence-score histogram for each annotation source and normalizing individual scores using the corresponding histogram, as taught by Al-Kofahi (Fig 6; Paragraphs 114-1149). Edgar uses annotation confidence for aggregation, training-data selection, and weighting (Paragraphs 75, 78 and 85), providing a reason to place scores from different sources on a common percentile scale. The modification would predictably reduce differences in numerical scoring scales when comparing or aggregating confidence values.
However, Aggarwal teaches,
a bias corrector to correct, by referring to a reference reliability addition distribution, the reliability addition histogram to a corrected reliability addition histogram having a same characteristic as a characteristic of the reference reliability addition distribution (Fig 2:206; Fig 4:416; Paragraph 70 and 71 – teaches correction of an output score distribution to match a reference expert score distribution, wherein the distribution matching supports the claimed shared characteristic, Aggarwal);
a corrected reliability output to correct the reliability added by the annotator by referring to the corrected reliability addition histogram (Fig 2:206, 208; Fig 4:416, Paragraphs 71 and 73 – teaches correction of individual scores by mapping between corresponding percentiles of two distributions, followed by score generation, Aggarwal).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which said subject matter pertains to modify the combination of Edgar and Al-Kofahi by applying Aggarwal’s equipercentile transformation (Paragraphs 70-71) to match each annotator’s reliability histogram to a reference distribution and correct individual reliability values using corresponding percentiles in the resulting corrected histogram. This motivation would have been to address systematic scoring differences before Edgar’s confidence-based training selection and weighting (Paragraphs 75 and 85). A reasonable expectation of success follows because Al-Kofahi’s cumulative histograms already provide the percentile information used in Aggarwal’s mapping (Paragraphs 114-119), predictably enabling correction onto a common reference scale.
2. The combination of Edgar, Al-Kofahi and Aggarwal teaches, The binary classification device according to claim 1, wherein the reference reliability addition distribution is an addition distribution of the reliability added by a person selected by referring to a proficiency level among the annotators (Fig 2:206, Paragraph 63 & 71, Aggarwal in combination with Paragraph 78, Edgar – discloses Edgar evaluates human annotation confidence based on historical annotation accuracy (Paragraph 78), while Aggarwal uses experienced human graders and their expert-score distribution as a reference for correcting model-generated scores (Paragraph 63, 71). A POSITA would select human annotator from Edgar’s annotator’s based on historical annotation accuracy and use that person’s score distribution, calculated using Al-Kofahi’s histogram technique, as the reference distribution. This would provide a proficiency-based benchmark for correcting differences among annotators’ scoring distributions).
3. The combination of Edgar, Al-Kofahi and Aggarwal teaches, The binary classification device according to claim 1, wherein the bias corrector outputs a score adjusted by a standard deviation together with the corrected reliability addition histogram (Fig 5:530,540; Fig 6:610, Paragraphs 111, 119 & 121 – teaches calculation of scores adjusted using standard deviation; further teaches, using histogram internally and calculating standardized scores do not establish outputting the standardized score together with a corrected histogram; A Posita would incorporate Al-Kofahi’s standard-deviation score adjustment into the bias-correction processing of the Edgar-Al-Kofahi-Aggarwal combination and provide the resulting standardized score together with the corrected histogram. Al-Kofahi uses standardized scores for subsequent comparison and re-ranking (Paragraph 111) and permits combining its scoring components (Paragraph 121). Providing both outputs would make the standardized score available for downstream score comparison while retaining the corrected histogram for individual-reliability correction. Both operations apply known numerical transformations to the available scoring data, supporting a reasonable expectation of success).
Claims 4-6 are similar to claims 1-3 hence rejected similarly.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Pinhasi et al. (US 2017/0092262) – teaches using classifier confidence scores, subsequent score transformation and express matching to reference score distribution using histogram matching (Abstract, Fig 1-2, Paragraphs 50-59, 69,74-75, 120-123 and 176-177).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMRESH SINGH whose telephone number is (571)270-3560. The examiner can normally be reached Monday-Friday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ann J. Lo can be reached at (571) 272-9767. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMRESH SINGH/Primary Examiner, Art Unit 2159
Prosecution Insights