DETAILED ACTION
This action is written in response to the election dated 1/6/26. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 11-15 were not elected, and are withdrawn from consideration.
Subject Matter Eligibility
In determining whether the claims are subject matter eligible, the examiner has considered and applied the 2019 USPTO Patent Eligibility Guidelines, as well as guidance in the MPEP chapter 2106. The examiner finds that the independent claims are directed to the practical application of improving the labeling of a machine-learning perception system, ie object recognition.
Claim Rejections - 35 USC § 112(b) - Indefiniteness
The following is a quotation of the second paragraph of 35 U.S.C. 112:
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim 7 is rejected under 35 U.S.C. 112(b), as being indefinite for failing to particularly point out and distinctly claim the subject matter which applicant regards as the invention. The claim recites in part: “filtering, by a computing device, the results of the processing step to concentrate on specific perception weaknesses of engineering interest.” The phrase “of engineering interest” is a subjective term, and the Applicant provides no standard for measuring the scope of the claim. Therefore, the claim is indefinite. See MPEP 2173.05(b)(IV) (“Subjective terms”).
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-3, 5-10 and 16-17 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Mishra (US 2017/0103267 A1).
Regarding claims 1, 16 and 17, Mishra discloses a computer-implemented method (and a related computer system and non-transitory computer-readable medium) for improving the labeling performance of a machine-learned (ML) perception system on an input dataset, the method comprising processing, by a computing device, the input dataset through the perception system;
[0031] “It has been recognized that video imaging systems and platforms which analyze image and video content for detection and feature extraction can accumulate significant amounts of data suitable for training and learning analytics that can be leveraged to improve over time the classifiers used to perform the detection and feature extraction by employing a larger search space and generating additional and more complex classifiers through distributed processing.”
identifying, by a computing device, one or more perception weaknesses;
[0032] “The results of the feature analysis stage 22 are validated in a validation stage 24, which can be performed manually by analysts responsible for accepting or rejecting the results of the feature analyses (e.g., to identify misclassified objects). “
isolating, by a computing device, at least one scene from the input dataset that includes the one or more perception weaknesses;
[0034] “A human user can validate the output and provide negative feedback to the algorithm when the algorithm performs poorly. This feedback is stored in a dataset and the classifier is retrained using the learning platform 12. In supervised learning, the goal is typically to label scenic elements and perform object detection.”
relabeling the one or more perception weaknesses in the at least one scene to obtain at least one relabeled scene; and
[0034] “A human user can validate the output and provide negative feedback to the algorithm when the algorithm performs poorly.”
[0074] “ One method is to use an independent observer to classify objects independently from the trained classifier, or to use two trained classifiers with independent implementation and algorithms. The observer can generate validation data which can be compared against the output from the trained classifier.”
retraining, by a computing device, the perception system with the at least one relabeled scene;
[0034] “A human user can validate the output and provide negative feedback to the algorithm when the algorithm performs poorly. This feedback is stored in a dataset and the classifier is retrained using the learning platform 12. In supervised learning, the goal is typically to label scenic elements and perform object detection.” (Emphasis added.)
whereby the labeling performance of the ML perception system is improved.
[The Examiner notes that this limitation is an intended result.]
[0032] “FIG. 1(a) illustrates a video analysis environment 10 which includes a learning platform 12 to generate and/or improve a set of classifiers 14 used in analyzing video content, e.g., for detection and extraction of features within image frames of a video.“
Regarding independent claims 16 and 17, Mishra also discloses the generic computer components recited therein. (See eg [0006] ‘processors’ and [0069] ‘memory’.)
Regarding claim 2, Mishra discloses the further limitation wherein the step of identifying one or more perception weaknesses is performed by a defect detection engine.
[0083] “video analysis module 342”, further discussed at [0085]-[0086].
Regarding claim 3, Mishra discloses the further limitation wherein the one or more perception weaknesses comprise labeled objects in bounding boxes.
See fig. 9 (reproduced below) and 10.
PNG
media_image1.png
346
984
media_image1.png
Greyscale
[0067] “FIG. 9 illustrates how a classifier determines if an object of interest is located within a context. For this case, measurements are extracted from an image frame extracted from a traffic intersection video. The sliding window (box in leftmost image) is centered at each pixel within the frame and the pixel intensities are extracted (top 2nd column).”
Regarding claim 5, Mishra discloses the further limitation wherein the step of relabeling the at least one scene comprises sending the at least one scene to be labeled by a human.
[0032] “The results of the feature analysis stage 22 are validated in a validation stage 24, which can be performed manually by analysts responsible for accepting or rejecting the results of the feature analyses (e.g., to identify misclassified objects).”
Regarding claim 6, Mishra discloses the further limitation wherein the input dataset is comprised of one or more of baseline data, augmented data, ground truth data, and simulated data.
[The Examiner notes that this is a Markush group.]
‘baseline data’ :: “training data and labelled ground truth in a supervised learning mode”.
Regarding claim 7, Mishra discloses the further limitation comprising filtering, by a computing device, the results of the processing step to concentrate on specific perception weaknesses of engineering interest.
The Examiner interprets “engineering interest” according to its broadest reasonable interpretation as encompassing every perception weakness.
See also §112(b) rejection supra.
Regarding claim 8, Mishra discloses the further limitation comprising repeating the method and comparing the perception weaknesses with the perception weaknesses identified in the processing step that occurred prior to the relabeling to determine an amount of improvement in the performance of the perception system.
[0069] “From the distributed processing, a learning and training analysis stage 112 is performed using the results of each partially computed algorithm in aggregation to iteratively improve the classifier. For example, the aggregate results could be used to determine the best feature to distinguish two categories, then that feature could be suppressed to find the next best feature, and the process can continue iteratively until acceptable at least one termination criterion is achieved.”
Regarding claim 9, Mishra discloses the further limitation comprising repeating the steps of processing, identifying, isolating, relabeling and retraining until retraining the ML perception system with the newly labeled scenes does not meaningfully improve the labeling performance.
[0069] “From the distributed processing, a learning and training analysis stage 112 is performed using the results of each partially computed algorithm in aggregation to iteratively improve the classifier. For example, the aggregate results could be used to determine the best feature to distinguish two categories, then that feature could be suppressed to find the next best feature, and the process can continue iteratively until acceptable at least one termination criterion is achieved.” (Emphasis added.)
Regarding claim 10, Mishra discloses the further limitation comprising reengineering the ML perception system when the labeling performance does not meaningfully improve with the newly labeled scenes.
[0069] “From the distributed processing, a learning and training analysis stage 112 is performed using the results of each partially computed algorithm in aggregation to iteratively improve the classifier. For example, the aggregate results could be used to determine the best feature to distinguish two categories, then that feature could be suppressed to find the next best feature, and the process can continue iteratively until acceptable at least one termination criterion is achieved.”
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
The following are the references relied upon in the rejections below:
Mishra (US 2017/0103267 A1)
Farabet (US 2019/0303759 A1)
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Mishra and Farabet.
Regarding claim 4, Farabet discloses the further limitation which Mishra does not disclose wherein the step of processing the input dataset further comprises creating, by a computing device, an augmented dataset, processing, by a computing device, the augmented dataset through the perception system, and comparing, by a computing device, results of the input data set with results of the augmented dataset.
The Examiner interprets “augmented dataset” in view of the applicant’s specification at [0006] as encompassing a dataset that is augmented with additive noise. This technique is disclosed in Farabet
[0099] “GNNS sensors (e.g., GPS sensors) may be simulated within the simulation space to generate real-world coordinates. In order to this, noise functions may be used to approximate inaccuracy.”
[0108] “In some examples, the KPI evaluation component 802 may also provide for more complex comparisons. For example, the KPI evaluation component 802 may be as complex as running analytics on the two differing CAN streams to find deviations.”
[0175] “The processor(s) 1110 may further include a safety cluster engine that includes a dedicated processor subsystem to handle safety management for automotive applications. The safety cluster engine may include two or more processor cores, a tightly coupled RAM, support peripherals (e.g., timers, an interrupt controller, etc.), and/or routing logic. In a safety mode, the two or more cores may operate in a lockstep mode and function as a single core with comparison logic to detect any differences between their operations.”
At the time of filing, it would have been obvious to a person of ordinary skill to implement the data augmentation technique disclosed by Farabet with the Mishra system because this can provide for models which generalize better, thus improving accuracy on new (ie previously unseen) data. Both disclosures pertain to machine vision systems.
Additional Relevant Prior Art
The following references were identified by the Examiner as being relevant to the disclosed invention, but are not relied upon in any particular prior art rejection:
Wang discloses a computer vision system comprising techniques for image area classification. (US 2020/0242424 A1)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Vincent Gonzales whose telephone number is (571) 270-3837. The examiner can normally be reached on Monday-Friday 7 a.m. to 4 p.m. MT. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang, can be reached at (571) 270-7092.
Information regarding the status of an application may be obtained from the USPTO Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov.
/Vincent Gonzales/Primary Examiner, Art Unit 2124
Prosecution Insights