DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) filed on 6/11/2024, 10/3/2024, and 12/16/2024 were considered and placed on the file of record by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 5-7, 9, 13-15, 17, 20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Lepek et al. (US 2020/0281164) WO/2019/008591 published 2/10/2019.
Regarding claim 1, Lepek teaches a system, comprising:
an imaging device configured to capture images of insects (see para. 0036, 0100-0105, Lepek discusses capturing successive images of insects to perform classification using a recurrent neural network); and
a computing device in communication with the imaging device, and configured to at least:
instruct the imaging device to capture a sequence of images depicting at least a portion of an insect (see para. 0036, 0100-0105, Lepek discusses capturing successive images of insects to perform classification using a recurrent neural network);
use a first predictive model to determine a first output corresponding to a first classification of a first image of the sequence of images, the first output comprising a confidence measure of the first classification (see para. 0176, 0206, Lepek discusses generating probabilities of an object belonging to a class); and
generate classification information based at least in part on the first output (see para. 0172, 0206, 0240, 0276-0278, Lepek discusses RNN generating probabilities and their accuracy of each classification).
Regarding claim 5, Lepek teaches wherein the sequence of images comprises a set of chronological images of the insect (see para. 0036, 0100-0105, Lepek discusses capturing successive images of insects to perform classification using a recurrent neural network).
Regarding claim 6, Lepek teaches wherein the computing device is further configured to use the first predictive model to determine a second output corresponding to a second classification of a second image of the sequence of images, the second output comprising a second confidence measure of the second classification (see para. 0036, 0100-0105, Lepek discusses capturing successive images of insects to perform classification using a recurrent neural network; see para. 0172, 0206, 0240, 0276-0278, Lepek discusses RNN generating probabilities and their accuracy of each classification).
Regarding claim 7, Lepek teaches wherein the computing device is further configured to use a second predictive model to determine a set of third outputs corresponding to a third second classification of the sequence of images based at least in part on the first output and the second output from the first predictive model; and generating the classification information is further based at least in part on the set of third outputs (see para. 0036, 0100-0105, Lepek discusses capturing successive images of insects to perform classification using a recurrent neural network; see para. 0172, 0206, 0240, 0276-0278, Lepek discusses RNN generating probabilities and their accuracy of each classification).
Claim 9 is rejected as applied to claim 1 as pertaining to a corresponding method.
Claim 13 is rejected as applied to claim 5 as pertaining to a corresponding method.
Claim 14 is rejected as applied to claim 6 as pertaining to a corresponding method.
Claim 15 is rejected as applied to claim 7 as pertaining to a corresponding method.
Claim 17 is rejected as applied to claim 1 as pertaining to a corresponding non-transitory computer-readable media.
Claim 20 is rejected as applied to claim 7 as pertaining to a corresponding non-transitory computer-readable media.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2-4, 10-12, 18, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Lepek et al. (US 2020/0281164) WO/2019/008591 published 2/10/2019 in view of Jiang et al. “Automatic Soccer Video Event Detection Based On A Deep Neural Network Combined CNN and RNN.”
Regarding claim 2, Lepek does not expressly disclose wherein: the computing device is further configured to use a second predictive model to determine a second output corresponding to a second classification of the sequence of images based at least in part on the first output from the first predictive model; and generating the classification information is further based at least in part on the second output.
However, Jiang teaches wherein: the computing device is further configured to use a second predictive model to determine a second output corresponding to a second classification of the sequence of images based at least in part on the first output from the first predictive model (see figure 1, Jiang discusses a first CNN model extracting features from successive images and second RNN model identifying features from the output of the first CNN model); and
generating the classification information is further based at least in part on the second output (see figure 1, Jiang discusses generating event classification based on the first CNN model and second RNN model).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Lepek with Jiang to derive at the invention of claim 2. The result would have been expected, routine, and predictable in order to perform insect object classification.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Lepek in this manner in order to improve insect object classification by applying multiple predictive models that classify different elements in image data therefore applying a more comprehensive search of features. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Lepek, while the teaching of Jiang continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of applying different predictive models that improve feature extraction of images to properly classify objects. The Lepek and Jiang systems perform object classification, therefore a person having ordinary skill in the art would have reasonable expectation of success in the combination yielding predictable results. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Regarding claim 3, Lepek teaches wherein the first predictive model comprises a core deep neural network model and the second predictive model comprises a recurrent neural network model (see figure 1, Jiang discusses a first CNN model extracting features from successive images and second RNN model identifying features from the output of the first CNN model).
The same motivation of claim 2 is applied to claim 3. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Lepek with Jiang to derive at the invention of claim 3. The result would have been expected, routine, and predictable in order to perform insect object classification.
Regarding claim 4, Lepek teaches wherein the second predictive model is trained using multiple subsets of multiple sequences of labeled images, each sequence of the multiple sequences depicting a different insect (see para. 0102, 0177, 0185-0186, Lepek discusses training the neural network with a large label database of insect images).
The same motivation of claim 2 is applied to claim 4. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Lepek with Jiang to derive at the invention of claim 4. The result would have been expected, routine, and predictable in order to perform insect object classification.
Claim 10 is rejected as applied to claim 2 as pertaining to a corresponding method.
Claim 11 is rejected as applied to claim 3 as pertaining to a corresponding method.
Claim 12 is rejected as applied to claim 4 as pertaining to a corresponding method.
Claim 18 is rejected as applied to claim 2 as pertaining to a corresponding non-transitory computer-readable media.
Claim 19 is rejected as applied to the combination of claim 3 and claim 4 as pertaining to a corresponding non-transitory computer-readable media.
Claims 8, 16 are rejected under 35 U.S.C. 103 as being unpatentable over Lepek et al. (US 2020/0281164) WO/2019/008591 published 2/10/2019 in view of Angel et al. (US 11,138,901).
Regarding claim 8, Lepek teaches wherein: the classification information identifies a category to which the first classification corresponds; and the computing device is further configured to: use the first predictive model to determine a second output corresponding to a second classification of one or more images of the additional set of images, the second output comprising an additional confidence measure of the second classification (see para. 0036, 0100-0105, Lepek discusses capturing successive images of insects to perform classification using a recurrent neural network; see para. 0172, 0206, 0240, 0276-0278, Lepek discusses RNN generating probabilities and their accuracy of each classification); and
generate updated classification information based at least in part on the second output (see para. 0172, 0206, 0240, 0276-0278, Lepek discusses RNN generating probability outputs and their accuracy of each classification).
Lepek does not expressly disclose to instruct the imaging device to capture an additional set of images when the confidence measure fails to meet a confidence threshold for the category. However, Angel teaches to instruct the imaging device to capture an additional set of images when the confidence measure fails to meet a confidence threshold for the category (see col. 7 lines 32-38, Angel discusses a system that if the object is not identified or is identified at below a threshold confidence level, one or more of the remaining cameras may initiate capturing more images of the object).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Lepek with Angel to derive at the invention of claim 2. The result would have been expected, routine, and predictable in order to perform insect object classification.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Lepek in this manner in order to improve insect object classification by applying a models to multiple image data therefore applying a more comprehensive search of features. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Lepek, while the teaching of Angel continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of applying a predictive model to multiple images to obtain a satisfactory classification. The Lepek and Angel systems perform object classification, therefore a person having ordinary skill in the art would have reasonable expectation of success in the combination yielding predictable results. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Claim 16 is rejected as applied to claim 8 as pertaining to a corresponding method.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ranson et al. (US 2019/0340743) discusses train the neural network comprises a labeled set of images to perform arthropod recognition using a machine learning algorithm that generates an output score.
Landwehr et al. (US 7,496,228) discusses detecting and classifying insects and other arthropods in images.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KENNY A CESE whose telephone number is (571) 270-1896. The examiner can normally be reached on Monday – Friday, 9am – 4pm.
If attempts to reach the primary examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached on (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Kenny A Cese/
Primary Examiner, Art Unit 2663
Prosecution Insights