DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the claim states “A computer-readable medium having code stored thereon…”, which can read on a signal per se. A potential correction would be to amend the claim to state “A non-transitory computer-readable medium having code stored thereon…”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 5-17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Khait et al., US 2022/0092705 A1 (Khait).
Regarding claim 1, Khait teaches a computer-implemented method (computing device 104) (Fig. 1; [0051]) of sensor input processing (processing from each imaging and treatment arrangement 108 that includes one or more sensors) (Fig. 1; [0055]), implemented by an agricultural system (implemented on a system 100, with agricultural machine 110) (Fig. 1; [0053]) comprising a processor (processor(s) 102) (Fig. 1; [0052]) and one or more sensors (imaging and treatment arrangement 108 that includes one or more sensors) (Fig. 1; [0055]), comprising:
receiving a sensor input from the one or more sensors (wherein computing device 104 receives the image(s) from image sensor(s) 112) (Fig. 1; [0055-0056]);
accessing a first image from the sensor input (an input image depicting a portion of the agricultural field is obtained) ([0141]);
applying a first image processing scheme on the first image, wherein the first image processing scheme implements a first machine learning (ML) algorithm (wherein the input image is fed into an object detector component (e.g. a neural network) that generates bounding boxes) ([0143]) configured to detect plant objects or target objects on a portion of the first image (configured to detect weeds on a portion (bounding box within the first image) ([0143]);
detecting a first target object in a first portion of the first image (detecting an object, such as a potential weed, within the bounding box, of the image, with a probability of the weed being there) ([0143-0144]), the detection comprising identifying at least a first class associated with the first target object (wherein the first class can represent a suspicious object proposal; i.e., not a clear if an object is depicted therein or not) ([0144-0145]);
accessing the first portion of the first image for further processing (wherein when the class is a suspicious object proposal, it’s fed into a classifier component (e.g., another neural network) for further processing) ([0145]);
applying a second image processing scheme on the first portion of the first image, wherein the second image processing scheme implements a second ML algorithm (wherein the bounding box is fed into a classifier component (e.g. a neural network) that generates another probability for the bounding box portion) ([0145]) configured to classify a received image (wherein the portion of the bounding box is classified as a species of weed) ([0145-0146]), the received image including the first portion of the first image (wherein the received input includes the bounding box portion of the image) ([0145]);
identifying at least a second class associated with the first portion of the first image (wherein the classifier (second neural network) classifies the bounding box as a weed species) ([0145-0146]);
comparing results of the first class with the second class (wherein a result for the first class of a suspicious object proposal is compared with the classifier result; which if the classifier has a high probability the bounding box is reclassified) ([0146]); and
performing a subsequent action based on the comparison (performing further processing on the bounding box to determine a final probability that is above the threshold for a specific weed species, and then when a weed species is identified a herbicide is selected for spraying) ([0146-0153]).
Although Khait does not explicitly state a “first class” and “second class” it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention that the object detector component ([0143]) can output that a bounding box represents “suspicious object proposal” ([0143-0145]) or a specific “weed species” ([0143-0144]) and that the classifier component ([0145]) outputs a “weed species”, which can be different then the first weed species, that these are each types of classes (suspicious object, first weed species, second weed species).
Regarding claim 5, Khait teaches wherein detecting a first target object in the first portion of the first image comprises generating a bounding box associated with the first portion of the first image (wherein the input image is fed into the object detector (e.g., neural network), and detecting an object, such as a potential weed, within a generated bounding box, with a probability of the weed being there) ([0143-0144]).
Regarding claim 6, Khait teaches wherein the first portion of the first image is an image patch (wherein the first portion of the first image is an image patch in the form of a bounding box) ([0143-0145]).
Regarding claim 7, Khait teaches wherein identifying at least a second class comprises classifying the image patch (wherein the second neural network classifies the bounding box image patch) ([0145-0146]).
Regrading claim 8, Khait teaches wherein the first class and the second class can be classifications from a plurality classifications including a crop class, a weed class, a background class, or a soil class (wherein the classifications can be of different weed species) ([0019] and [0140]).
Regarding claim 9, Khait teaches wherein the first ML algorithm (object detector component; e.g. neural network) ([0143]) is configured to assign more than one class of a plurality of classifications to one or more detected objects, including plant objects and target objects (wherein the object detector component can classify the object, such as a weed, into one of a plurality of classifications; i.e. from a plurality of weed species and/or a suspicious object) ([0143-0144] and [0146]).
Regarding claim 10, Khait teaches wherein each class, including the first class and second class, has a confidence level associated with the class (wherein the output from the first and second neural networks includes a probability of the class) ([0144-0146]).
Regarding claim 11, Khait teaches wherein the second ML algorithm (classifier component; e.g. another neural network) ([0145]) is configured to assign more than one class of a plurality of classifications to one or more detected objects, including plant objects and target objects (wherein the classifier can classify the object, such as a weed, into one of a plurality of classifications; i.e. from a plurality of weed species) ([0145-0146]).
Regarding claim 12, Khait teaches wherein each class, including the first class and second class, has a confidence level associated with the class (wherein the output from the first and second neural networks includes a probability of the class) ([0144-0146]).
Regarding claim 13, Khait teaches wherein each of the first class and second class has a confidence level (wherein the output from the first and second neural networks includes a probability of the class) ([0144-0146]).
Regarding claim 14, Khait teaches wherein comparing results of the first class with the second class (results from the first and second neural networks) ([0143-0145]) comprises analyzing and comparing each confidence level of the first class and the second class with each other (comparing the probability score of the second classification with the probability score of the first classification) ([0143-0146]) .
Regarding claim 15, Khait teaches wherein performing a subsequent action includes performing a treatment action on a real-world object associated with the detected first target object (wherein based on the final probability a herbicide is selected for a real-world weed) ([0153]) (and applying a treatment to the weeds, such as a herbicide) ([0020], [0060], and [0161]).
Regarding claim 16, Khait teaches wherein the subsequent action comprises determining to treat the real-world object associated with the detected first target object (wherein based on the final probability a herbicide is selected for a real-world weed) ([0153]) (and applying a treatment to the weeds, such as a herbicide) ([0020], [0060], and [0161]) based on comparing the results of the first class and the second class (based on the comparison and determining the class/species of weed) ([0146] and [0152-0153]).
Regarding claim 17, Khait teaches wherein, upon determining that the first class and the second class are not a match (wherein a result for the first class of a suspicious object proposal is compared with the classifier result of a weed species; i.e. they don’t match) ([0146]), sending the results of the first class and the second class (view data) ([0064]), including the first portion of the first image (wherein the user interface can include a client terminal 128 which can be used to remotely monitor imaging and treatment arrangements) ([0062-0065]) to a user interface for further review (wherein a physical user interface can be used to enter and/or view data) ([0064-0065]).
Regarding claim 20, see the rejection made to claim 1, as well as Khait for a computer-readable medium (computer readable medium) ([0044]) having code stored thereon (wherein computer readable program instructions are stored on the computer readable storage medium) ([0045]), the code (computer readable program instructions) ([0046]), upon execution by a processor (executed by a processor) ([0043] and [0048]), causing the processor to implement a method (the processor to execute the instructions to carry out aspects of the present invention) ([0043] and [0048]) of sensor input processing (processing from each imaging and treatment arrangement 108 that includes one or more sensors) (Fig. 1; [0055]), for they teach all the limitations within this claim.
Claim(s) 2-4 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Khait et al., US 2022/0092705 A1 (Khait), and further in view of Redden et al., US 2018/0330166 A1 (Redden).
Regarding claim 2, Khait teaches the detected first target object in the first portion of the first image (detecting an object, such as a potential weed, within the bounding box, of the image, with a probability of the weed being there) ([0143-0144]) is a smaller portion of the first portion (wherein the bounding box is a smaller portion; with the weed being even smaller inside the bounding box) ([0143]), the smaller portion represented by a bounding box associated with pixels of the first image (the bounding box associated with pixels in the first image that indicate a likelihood of a weed and/or suspicious object) ([0143-0145]).
However, Khait does not explicitly teach “wherein the first portion of the first image is a tile”.
Redden teaches a plant treatment platform that uses a plant detection model to detect plants as the plant treatment platform travels through a field (Abstract); and wherein the first portion of the first image is a tile (wherein the image data can be split into multiple images with respect to the physical region of the field captured within those images; a technique referred to as “tiling”) ([0051-0052]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Khait to include tiling since it helps ensure consistency between the images used to train the model and the images captured by the camera for use in performing the task at hand (Redden; [0052])
Regarding claim 3, Khait teaches wherein the second ML algorithm (second neural network) ([0145]) classifies the smaller portion of the first portion of the first image (classifies the smaller weed portion within the bounding box) ([0145-0146]).
Regarding claim 4, Khait teaches wherein the second ML algorithm (second neural network) ([0145]) classifies the smaller portion represented by the bounding box (classifies the smaller weed portion within the bounding box based on a probability) ([0145-0146]).
Regarding claim 19, Redden teaches wherein the first class is a weed species (wherein model tasks from the plant detection model can be trained to perform one or more tasks using one or more submodels; such as to identify bounding boxes that specify where plants/crops/weeds/species are physically located) ([0053-0055]) and the second class is a soil pattern (another task is to detect with a spray box detection model similar to the plant detection model, bounding boxes regarding soil and/or plants) ([0056]).
Claim(s) 18 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Khait et al., US 2022/0092705 A1 (Khait) and further in view of Polzounov et al., US 2019/0362146 A1 (Polzounov).
Regarding claim 18, Khait teaches wherein a selection by a user of the user interface (wherein the user on the user interface can make changes by entering data) ([0062] and [0064]).
However, Khait does not explicitly teach wherein a selection by a user of the user interface “to verify a correct classification of the detected first target object is indexed for training of a first ML model associated with the first ML algorithm”.
Polzounov teaches a system for customized application of herbicides (Abstract); wherein a selection by a user of the user interface (wherein the computer system can include a graphics or liquid crystal display with input devices) ([0126]) to verify a correct classification of the detected first target object (sending the labeled image to users for verification) ([0077]) is indexed for training of a first ML model associated with the first ML algorithm (wherein users verify the labels for using as pre-identification for model 800; a plant identification model) (Fig. 8; [0072-0073] and [0077]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Khait to include user verification for training the model since it reduces the costs of training the semantic segmentation model (Polzounov; [0077]).
Regarding claim 19, Polzounov teaches wherein the first class is a weed species and the second class is a soil pattern (wherein the model can be configured to identify a category of a plan (e.g. weed or a crop); and the identification layer of the model can identify soil) (Fig. 8; [0085]) (wherein the identification layer is configured to identify weeds, cotton, and soil) ([0096]).
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL J VANCHY JR whose telephone number is (571)270-1193. The examiner can normally be reached Monday - Friday 9am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571) 270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL J VANCHY JR/Primary Examiner, Art Unit 2666 Michael.Vanchy@uspto.gov