Detailed Action
1. Claims 1-33 are pending in this Application.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless -
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
3. Claims 1-3, 5-6, 8-11, 13-15,18-20,22-23, 25-27 and 29-31 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Xiaolei Zhang et al., ( hereafter Xiaolei ),” Automatic pest identification system in the greenhouse based on deep learning and machine vision” PUBLISHED 28 September 2023.
As to claim 18, Xiaolei teaches A system for identifying pests within a closed environment (Title, Abstract, Automatic pest identification system in the greenhouse based on deep learning and machine vision), comprising:
(A) at least one substrate (Fig.1 that show greenhouse bench where the pests live and grow. The substrate corresponds to surface, or material of the greenhouse where the pests live and grow).
(B) a scanner configured to scan the at least one substrate to generate an image of the at least one substrate(Page 3 right col., section 2.2 2nd par., The pest-image collecting device consisted of an industrial camera and a cantilever bracket (Figure 2C). A Sony IMX226 camera with 3280*2464 resolution was installed in a cantilever and aimed at a piece of sticky paper at an appropriate distance to capture clear image);
(C) a non-transitory computer-readable storage medium having computer-executable instructions stored thereon, which when executed by one or more processors, cause the one or more processors to perform operations (Figs. 8 and 11,, page 5 left col. section 2.3.1, YOLOv5 model for identifying pests. Figs. 8 and 11 , show a computer simulation results generated using the algorithm of YOLOv5 model. It is known that the YOLO algorithm that runs on a computer is called a computer vision or deep learning algorithm designed specifically for object detection. Further The model could simultaneously process four images in the batch normalization layer, which decreased the GPU memory usage by using a relatively small mini-batch ) comprising:
(1) detecting one or more pests within the image using a neural network that has been trained using training data and a selected training algorithm, the training data comprising data associated with features of pest-types and tags associated with the pest-types and the selected training algorithm includes a deep learning framework (Abstract, page 12 section 4 1st par. Automatic pest identification system in the greenhouse based on deep learning and machine vision. This study cover a trapping system with yellow sticky paper and LED light for automatic pest image collection, and proposed an YOLOv5 model and improved YOLOv5 model with copy-pasting data augmentation for pest recognition, and apply both the YOLOv5 model and improved YOLOv5 model for pest recognition. It is known that YYOLOv5 is a deep learning model. It is a popular computer vision system used for real-time object detection, built using deep convolutional neural networks and written in the PyTorch framework ); and
(2) generating a report indicating presence of the detected one or more pests within the closed environment (Fig.7 and tale 2, Fig. 7 and2 display the classification results of the detected pests. The classification result includes tobacco whiteflies, thrips, winged aphids, leaf miners, fruit flies, and houseflies).
Claim 1 is rejected the same as claim 18 except claim 1 is directed to a method claim. All the limitations of claim 1 are addressed in claim 18. Thus, argument analogous to that presented above for claim 18 is applicable to claim 1.
As to claim 19, Xiaolei teaches the deep learning framework is one selected from the group consisting of: Tensorflow, Keras, PyTorch, MxNet, Chainer Caffe, Theano, Deeplearning4j, CNTK and Torch ( as discussed in claim 18 above Xiaolei specifically teaches YYOLOv5 and modified YYOLOv5. It is well-known that YYOLOv5 is a deep learning model. It is a popular computer vision system used for real-time object detection, built using deep convolutional neural networks and written in the PyTorch framework).
Claim 2 is rejected the same as claim 19 except claim 2 is directed to a method claim. All the limitations of claim 2 are addressed in claim 19. Thus, argument analogous to that presented above for claim 19 is applicable to claim 2.
As to claim 20, Xiaolei teaches wherein the trained neural network is a machine learning recognition model(Abstract, Automatic pest identification system in the greenhouse based on deep learning and machine vision. As discuss above in claim 18 It is well-known that YYOLOv5 is a deep learning model. It is a popular computer vision system used for real-time object detection, built using deep convolutional neural networks and written in the PyTorch framework).
Claim 3 is rejected the same as claim 20 except claim 3 is directed to a method claim. All the limitations of claim 3 are addressed in claim 20. Thus, argument analogous to that presented above for claim 20 is applicable to claim 3.
As to claim 22, Xiaolei teaches the trained neural network is a deep learning image recognition model (Title, Abstract Automatic pest identification system in the greenhouse based on deep learning and machine vision)
Claim 5 is rejected the same as claim 22 except claim 5 is directed to a method claim. All the limitations of claim 5 are addressed in claim 22. Thus, argument analogous to that presented above for claim 22 is applicable to claim 5.
As to claim 23, Xiaolei teaches the deep learning image recognition model is one selected from the group consisting of: Faster Region-based Convolutional Neural Network (RCNN), Single Shot Detector (SSD), and You Only Look Once (YOLO) ( see Table 1 ).
Claim 6 is rejected the same as claim 23 except claim 6 is directed to a method claim. All the limitations of claim 6 are addressed in claim 23. Thus, argument analogous to that presented above for claim 23 is applicable to claim 6.
As to claim 8, Xiaolei teaches the step of scanning the at least one substrate on a scanner to generate the image(Page 3 right col., section 2.2 2nd par., The pest-image collecting device consisted of an industrial camera and a cantilever bracket (Figure 2C). A Sony IMX226 camera with 3280*2464 resolution was installed in a cantilever and aimed at a piece of sticky paper at an appropriate distance to capture clear image)
As to claim 25, Xiaolei teaches the image is a composite image of a plurality of substrates each previously positioned at a different location within the closed environment (Fig. 4, page 5 Section 2.3.1 1st par., Fig.4 illustrates Mosaic data augmentation adopted in the YOLOv5 model. (A) Workflow of mosaic data augmentation, and (B) Pest image after mosaic data augmentation, where Mosaic data augmentation combines four different training images into a single new image in a specific four-way grid layout. Specifically, the workflow of mosaic data augmentation was as follows:1) random selection of four images from the original training dataset; 2) random rotation, scaling, flipping, and adjustment of the brightness and chromaticity of the four selected images; and 3) combining the images and box layout stitching into new images).
Claim 9 is rejected the same as claim 25 except claim 9 is directed to a method claim. All the limitations of claim 9 are addressed in claim 25. Thus, argument analogous to that presented above for claim 25 is applicable to claim 9.
As to claim 26, Xiaolei teaches the substrate comprises an adhesive ( page 12 section 4 1st par. To obtain high-quality images, we designed a sticky board trap that combines yellow sticky paper and LED pest-trap lamps to achieve all-weather pest-trapping effects..
Claim 10 is rejected the same as claim 26 except claim 10 is directed to a method claim. All the limitations of claim 10 are addressed in claim 26. Thus, argument analogous to that presented above for claim 26 is applicable to claim 10.
As to claim 27, Xiaolei teaches the report indicates at least one of class of the detected one or more pests ( Fig.4, Table 3, page 8 section 3.1, Fig.4 illustrates number of the detected one or more pests, and percentage of each type of the one or more pests. TABLE 3 illustrates the pest detection accuracies and average detection time of the models. The improved YOLOv5 model raises overall pest-detection precision from 64% to 96% using copy-pasting data augmentation. It reaches a top precision of 99% for leaf miners and fruit flies, and 98% for aphids and houseflies)
Claim 11 is rejected the same as claim 27 except claim 11 is directed to a method claim. All the limitations of claim 11 are addressed in claim 27. Thus, argument analogous to that presented above for claim 27 is applicable to claim 11
As to claim 29, Xiaolei teaches the step of detecting one or more pests that are not associated with the training data ( page 7, section 2.5 1st par., the tp (true positive) represents the number of correct positive predictions, fp (false positive) represents the number of incorrect positive predictions, and fn (false negative) represents the number of incorrect negative predictions.)
Claim 13 is rejected the same as claim 29 except claim 13 is directed to a method claim. All the limitations of claim 13 are addressed in claim 29. Thus, argument analogous to that presented above for claim 29 is applicable to claim 13.
As to claim 30, Xiaolei teaches the step of detecting one or more pests that are not associated with the training data is performed using unsupervised or semi-supervised learning algorithms( Abstract, Inherent A modified YOLOv5 model typically uses semi-supervised learning algorithms rather than unsupervised learning when it is adapted to handle unlabelled data)
Claim 14 is rejected the same as claim 30 except claim 14 is directed to a method claim. All the limitations of claim 14 are addressed in claim 30. Thus, argument analogous to that presented above for claim 30 is applicable to claim 14.
As to claim 31, Xiaolei teaches the one or more pests comprise crop pests (Fig.7 and tale 2, Fig. 7 and2 display the classification results of the detected pests. The classification result includes tobacco whiteflies, thrips, winged aphids, leaf miners, fruit flies)
Claim 15 is rejected the same as claim 31 except claim 15 is directed to a method claim. All the limitations of claim 15 are addressed in claim 31. Thus, argument analogous to that presented above for claim 31 is applicable to claim 15.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4. Claim 16-17 and 32-33 are rejected under 35 U.S.C. 103 as being unpatentable over Xiaolei, “Automatic pest identification system in the greenhouse based on deep learning and machine vision”, in view of Humpston et al., (hereafter Humpston ), US 20220155054 A, pub. 05/19/2022
As claim 32, Xiaolei teaches the crop pests comprise aphids, thrips(Abstract Fig.7, miners, aphids, fruit flies, thrips, and houseflies);
however it is noted that Xiaolei does not teach the crop pests comprise beetles and mites.
On the other hand Humpston teaches the crop pests comprise beetles and mites ([0033],[0035). Specific examples of beneficial organisms may include certain types of parasitic wasps, predatory mites, beetle).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Xiaolei’s pet detection algorithm to detect beetles and mites by incorporating Humpston’s pet detection method. The motivation to do so would have been to allow users of OSAWA to expand its pet detection capabilities.
Claim 16 is rejected the same as claim 32 except claim 16 is directed to a method claim. All the limitations of claim 16 are addressed in claim 32. Thus, argument analogous to that presented above for claim 16 is applicable to claim 32.
Regarding claim 33, while Xiaolei teaches the limitation of claim 18 but fails to teach the limitation of calm 33.
On the other hand Humpston teaches the closed environment comprises a vertical farm([033], Note that the functionality described in this patent document may be used with various types of growing areas, such as commercial greenhouses, vertical farms, and open fields.)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the vertical farm-based pet collection system taught by Humpston into the greenhouse-based pet collection system of Xiaolei, in order to expand the pet collection capabilities of Xiaolei's system
Claim 17 is rejected the same as claim 33 except claim 17 is directed to a method claim. All the limitations of claim 17 are addressed in claim 33. Thus, argument analogous to that presented above for claim 33 is applicable to claim 17.
5. Claims 4, 7, 12, 21, 24 ,28 are rejected under 35 U.S.C. 103 as being unpatentable over Xiaolei, “Automatic pest identification system in the greenhouse based on deep learning and machine vision”, in view of GEORGESCU et al., (hereafter GEORGESCU), WO2020243583 A1, pub. 12/03/2020.
As claim 21, Xiaolei teaches machine learning recognition model (Abstract, Fig.3), but not specifically teaches “wherein the machine learning recognition model is one selected from the group consisting of: Support Vector Machines (SVM) model, Bag of Feature model, and Viola-Jones model”
On the other hand GEORGESCU teaches wherein the machine learning recognition model is one selected from the group consisting of: Support Vector Machines (SVM) model ([196], A support vector machine or random forest algorithm).
Before the effective filing date of the claimed invention, a person of ordinary skill in the art would have found it obvious to incorporate the well-known support vector machine or random forest learning methods taught by Georgescu into the system of Xiaolei. This combination would have provided a motivation and suggestion to allow users of Xiaolei's system to compare pet detection methods based on the modified YOLOv5 with support vector machine or random forest algorithms, thereby extending the pet detection capabilities
Claim 4 is rejected the same as claim 21 except claim 4 is directed to a method claim. All the limitations of claim 4 are addressed in claim 21. Thus, argument analogous to that presented above for claim 4 is applicable to claim 21.
As claim 24, Xiaolei teaches image which has 3280 × 2464 pixels image (see section 2.4.1) but fila to teach a gigapixel image
On the Other hand GEORGESCU teaches gigapixel image([112], [117])
A person of ordinary skill in the art would have been motivated to combine Xiaolei with the gigapixel image methods taught by Georgescu before the filing date. This combination would allow Xiaolei users to achieve extreme high resolution, making it easy to zoom in deeply on tiny details without losing clarity.
Claim 7 is rejected the same as claim 24 except claim 7 is directed to a method claim. All the limitations of claim 7 are addressed in claim 24. Thus, argument analogous to that presented above for claim 24 is applicable to claim 7.
As claim 28, Xiaolei teaches image the report (Figs. 7,8, 11 Tables 1-3) but fails to teach “provides a weblink to a webpage that displays the image”
On the other hand GEORGESCU provides a weblink to a webpage that displays the image([239], For example, a website, as a stand-alone product or as an add-in package for installation in an existing software application. It may also be available as a client- server software application, as a web-enabled software application, and/or as a mobile application)
A person of ordinary skill would have been motivated to add the stand-alone website from Georgescu to Xiaolei before the invention date. This combination allows users to share documents with clients over the internet.
Claim 12 is rejected the same as claim 28 except claim 12 is directed to a method claim. All the limitations of claim 12 are addressed in claim 28. Thus, argument analogous to that presented above for claim 28 is applicable to claim 8.
Prior art not used in rejections but pertinent to the claims or disclosure
“Classification and identification of citrus pests based on InceptionV3 convolutional
neural network and migration learning”, pub. 2019, , to ZHOU Dongmei et al., disclosed:
As one of the origins of citrus in the world, China has a large number of excellent citrus resources and mature cultivation techniques. Pests and diseases have become an important constraint on citrus harvest and quality. At present, deep learning has been widely used in many fields, and its application in agricultural research is gradually becoming mature. The use of deep learning convolutional neural networks to identify citrus pests is an effective and high-discrimination recognition technology. In this paper, based on a small amount of self-collected citrus pests dataset, including Blowing scale, Moth, Star scream, Star beetle, Citrus fruit fly, a total of 5 common pests and diseases, and propose a combination of Inceptionv3 network feature extraction model and migration learning According to the classification and recognition method, the final recognition accuracy can reach 96.81% ( see Abstract)
Contact Information
Any inquiry concerning this communication or earlier communication from the examiner should be directed to Mekonen Bekele whose telephone number is (469) 295-9077.The examiner can normally be reached on Monday-Friday from 9:00AM to 6:50 PM Eastern Time.
If attempt to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Eng, George can be reached on (571) 272-7495.The fax phone number for the organization where the application or proceeding is assigned is 571-237-8300. Information regarding the status of an application may be obtained from the patent Application Information Retrieval (PAIR) system. Status information for published application may be obtained from either Private PAIR or Public PAIR. Status information for unpublished application is available through Privet PAIR only.
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have question on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866.217-919 (tool-free)
/MEKONEN T BEKELE/Primary Examiner, Art Unit 2699
Prosecution Insights