DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This communication is in response to the Application No. 18/272,728 filed 07/17/2023. Claims 1-16 and 18-21 are pending.
Claim Objections
Claim(s) 7 and 10 are objected to because of the following informalities:
Claim 7 should recite, in part, “The method of claim 1, including: analyzing samples”.
Claim 10 should recite, in part, “The method of claim 7, wherein the analyzing of the samples”.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
Claim 21 and its dependent claim(s) are rejected under 35 U.S.C. 101 because the claimed invention is directed to a “machine-readable storage media” that is non-statutory subject matter.
A “machine-readable storage media” is defined in the specification to include “Exemplary machine readable instructions include instructions compiled from the code in the computer program listing” found at paragraph [0090] of PGPUB.
The broadest reasonable interpretation of a claim drawn to a computer-readable medium (also called machine readable medium and other such variations) typically covers forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media, particularly when the specification is silent. See MPEP 2111.01.
When the broadest reasonable interpretation of a claim covers a signal per se, the claim must be rejected under 35 U.S.C 101 as covering non-statutory subject matter. The claims, as defined in the specification, cover both non-statutory subject matter and statutory subject matter. A claim drawn to such a computer-readable medium that covers both transitory and non-transitory embodiments may be amended to narrow the claim to cover only statutory embodiments to avoid a rejection under 35 U.S.C. § 101 by adding the limitation “non-transitory” to the claim.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 7-9, 13, 16, 18 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Gao et al. (“Recognising weeds in a maize crop using a random forest machine-learning algorithm and near-infrared snapshot mosaic hyperspectral imagery”, 2018) in view of Su et al. (“Wheat yellow rust monitoring by learning from multispectral UAV aerial imagery”, 2018).
Regarding claim 1, Gao discloses a method for automated hyperspectral vegetation index (VI) determination (“A total set of 185 spectral features including reflectance and vegetation index features” Gao, abstract, i.e., using automated techniques such as machine learning; additionally, see Fig. 2 description “Through image preprocessing and calibration formula, the reflectance was obtained as band features, then [Normalized Difference Vegetation Index] NDVI and [Ratio Vegetation Index] RVI features were constructed by reflectance in [Visual Band] VB and [Near Infrared] NIR regions”), the method including: accessing measured spectra and respective measured ground truth values of a selected vegetation trait (“Every pixel from hyperspectral image has complete spectrum information” Gao, pg. 40 Col 2; where ground truth is accessed and used “Cross validation (CV) was employed for evaluation of the RF. The original data were split into 5 folds, using the folds one by one for testing and the remaining folds as training set” Gao, pg. 44, Col 1); accessing a library of VI models (“The normalized difference vegetation index (NVDI) and ratio vegetation index (RVI) by Eqs. (2) and (3)” Gao, pg. 42 Col 2), wherein each VI model includes a relationship defining an index value for the vegetation trait by mathematically combining spectral measurement values at a plurality of wavebands ("model wavebands"), optionally with one or more coefficients ("model coefficients") (see equations (2) and (3) which involve mathematically combining spectral measured values of different Near Infrared (NIR) wavebands and visual bands (VB), Gao, pg. 42 Col 2); a model selection step, including selecting a VI model from the library of VI models (“[R]andomly choose a bootstrap set Xi which contains two thirds of the instances in the original data set” Gao, pg. 43 Col 1; i.e., using random forests (RF) algorithm, each decision tree is constructed by randomly selecting a subset of features using a different bootstrap sample from original data, see Gao pg. 41 Col 1); a model parameter generation step, including: generating a hyperparameter for each of the spectral measurement values of the selected VI model, wherein the hyperparameter includes a selected waveband for each of the plurality of model wavebands (“Table 4 gives the optimal hyperparameters of these three RF models. Running the RF model three times with these parameters, the mean and standard deviation of the model metrics for each plant species are compared in Table 5” Gao, pg. 46 Col 1) , and generating a hyperparameter for each of the model coefficients of the selected VI model if the selected VI model has any coefficients, wherein the hyperparameter includes a selected coefficient value for each of the model coefficients (“the goal of this search is to build a RF model with optimal hyperparameters” Gao, pg. 42, Fig. 2 caption; “the number of trees (m) and the number of randomly selected features (n) to split the tree nodes are two hyperparameters which need to be optimized for obtaining a minimal random forest error” Gao, pg. 43 Col 1; Additionally see Table 4 – Optimal hyperparameters for RF with different feature combinations); a model evaluation step, including evaluating the selected VI model with the selected wavebands and optionally selected coefficient values with an objective function score, wherein the objective function score quantifies a closeness of fit between the ground truth values and calculated VI values from the selected VI model with the generated hyperparameters and the respective measured spectra (“RF allows the importance of every feature to be evaluated, based on OOB errors. The importance score of each feature is displayed in Fig. 8. Based on these importance scores, the features were ranked, and accuracy-oriented feature reduction was performed in a CV loop to select the optimal number of features” Gao, pg. 45 Col 2; “RF also provides valuable information for estimating the importance of a feature by calculating how much the OOB error increases when OOB data for the feature are permuted while all other features are left unchanged” Gao, Pg. 43 Col 1; where “out-of-bag samples (OOB), can be used to evaluate the OOB errors as well as to determine the importance of features” Gao, pg. 43 Col 1); a model parameter tuning step, including using an optimizer to select the waveband for each of the at least two wavebands ("optimum wavebands") (“A grid search approach was used to search for the optimal parameters (m, n) for building RF” Gao, pg. 44 Col 2), and optionally to select the coefficient values for each of the coefficients ("optimum coefficient values"); and repeating the model selection step, the model parameter generation step, the model evaluation step and the model parameter tuning step (together referred to as the "optimization steps") for a plurality of iterations (“The three different combinations of features... were tested by building optimal RF” Gao, pg. 45 Col 2 – pg. 46 Col 1; i.e., the three different combination of RF models are generated in iterations/repetition for building optimal RF models with optimal hyperparameters).
Gao discloses all of the subject matter as described above except for specifically teaching selecting... based on sequential model- based optimization (SMBO). However, Su in the same field of endeavor teaches selecting... based on sequential model- based optimization (SMBO) (3.3 Random forest classifier with Bayesian optimization, Su pg. 160 Col 1-2 “Bayesian optimization optimally suggests new parameters by sequentially performing 1. Fitting a Gaussian process model Q for data points {λi, oobErr(λi)}, and updating it with new data points; 2. Finding new point for evaluation which maximizes the acquisition function based on the posterior distribution function Q. The introduction of acquisition function can efficiently trade off exploration and exploitation of parameter space”).
Therefore, it would have been obvious to one of ordinary skill in the art to combine Gao and Su before the effective filing date of the claimed invention. The motivation for this combination of references would have been to automatically tune hyperparameters of random forest classifiers (Su pg. 160 Col 2). This motivation for the combination of Gao and Su is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim 2, Gao and Su disclose the method of claim 1, including: selecting the VI model from the plurality of iterations with the selected optimum wavebands and optimum coefficient values , which is the VI model with model parameters that generates the highest objective function score over all iterations (“Specifically, the most important feature was first used to build the model, and then the ranked features were added one by one to build the models, respectively. This procedure was repeated until the least important feature used to build the model” Gao, pg. 45 Col 2; i.e., as shown in Table 3, the model selects the hyperparameters that generate the highest objective score).
Regarding claim 3, Gao and Su disclose the method of claim 1, including: a grouping step, including grouping VI models from the library according the number (Nwb) of the model wavebands, including a first group with a plurality of two-waveband models (Nwb = 2) and a second group with a plurality of three-waveband models (Nwb = 3) (Gao, pg. 46 Table 3 discloses groupings in two and three-waveband models); a running step, including determining the best-performing VI model within each group by performing the plurality of the iterations of the optimization steps for each group; and a cross-group comparison step, including selecting an overall best VI model from the best-performing VI models based on their respective objective function scores (“Specifically, the most important feature was first used to build the model, and then the ranked features were added one by one to build the models, respectively. This procedure was repeated until the least important feature used to build the model” Gao, pg. 45 Col 2; wherein three different combination of RF models are generated in iterations/repetition for building optimal RF models with optimal hyperparameters, Gao pg. 46 Col 1).
Regarding claim 4, Gao and Su disclose the method of claim 1, including: creating the library of VI models (“The normalized difference vegetation index (NVDI) and ratio vegetation index (RVI) by Eqs. (2) and (3)” Gao, pg. 42 Col 2).
Regarding claim 5, Gao and Su disclose the method of claim 1, wherein the SMBO is Bayesian SMBO and the optimizer is a Bayesian optimizer (3.3 Random forest classifier with Bayesian optimization, Su pg. 160 Col 1-2).
Therefore, combining Gao and Su would meet the claim limitations for the same reasons as previously discussed in claim 1.
Regarding claim 7, Gao and Su disclose the method of claim 1, including: analysing samples of the plant to generate the measured spectra and the ground truth values of the plant (“Every pixel from hyperspectral image has complete spectrum information” Gao, pg. 40 Col 2; where ground truth is accessed and used “Cross validation (CV) was employed for evaluation of the RF. The original data were split into 5 folds, using the folds one by one for testing and the remaining folds as training set” Gao, pg. 44, Col 1).
Regarding claim 8, Gao and Su disclose the method of claim 7, wherein the measured spectra include reflectance spectra (“process these snapshot hyperspectral images to obtain the spectral reflectance” Gao, pg. 41 Col 1; See also Section 2.3 Reflectance calibration).
Regarding claim 9, Gao and Su disclose the method of claim 7, including: using a hyperspectral imaging sensor or spectrometer to generate the measured spectra (“a snapshot mosaic hyperspectral imaging sensor was applied” Gao, pg. 41 Col 1).
Regarding claim 13, Gao and Su disclose the method of claim 1, wherein the model wavebands include a plurality of wavebands in one or more of: a visible region with wavelengths 400 - 700 nm;
a near infrared region with wavelengths 700 - 1000 nm (“where γ1 represents one NIR band from 724 nm, 738 nm, 750 nm, 764 nm, 776 nm, 790 nm, 802 nm, 814 nm and γ2 represents one visual band (VB) from 601 nm, 605 nm, 614 nm, 627 nm, 636 nm, 644 nm, 652 nm, 660 nm, 669 nm, 677 nm” Gao, pg. 42 Col 2; Table 3); a shortwave infrared region with wavelengths 1000 - 2500 nm; a shortwave infrared region with wavelengths 1200 - 1700 nm; a region with wavelengths 1410 - 1430 nm; a region with wavelengths 1550 - 1680 nm; a near infrared region with wavelengths 800 - 900 nm; and a region with wavelengths 400 - 5,400 nm.
Regarding claim 16, Gao and Su disclose a system configured to perform the method of claim 1 (See Gao, Fig. 2 Key steps of weed and crop recognition by snapshot hyperspectral imaging), the system including: an optimizer module configured to perform the optimization steps, including the model selection step, the model parameter generation step, the model parameter tuning step and the model evaluation step (See rejection of Claim 1 of Gao and Su above for “optimization steps”); and optionally one or more hyperspectral sensors (Gao, Fig. 1 hyperspectral camera(s)).
Regarding claim 18, Gao and Su disclose the system of claim 16, including: an unmanned aerial vehicle (UAV) system with the one or more hyperspectral sensors (“supported through other snapshot hyperspectral sensor applications like scouting early growth stage field weeds using unmanned aerial vehicles or specialised designed field vehicles” Gao, pg. 47 Col 2).
Regarding claim 21, Gao and Su disclose Machine-readable storage media including machine readable instructions that, when executed by a computing system, perform data-processing steps of the method of claim 1 (“image processing, feature engineering and machine learning techniques” Gao, abstract; storage media is inherent to image processing techniques), including one or more of the accessing steps, the model selection step, the model parameter generation step, the model parameter tuning step, the model selection step (See Gao, Fig. 2 Key steps of weed and crop recognition by snapshot hyperspectral imaging; See rejection of Claim 1 of Gao and Su above for “optimization steps”), the grouping step, the running step, and the cross-group comparison step (“Specifically, the most important feature was first used to build the model, and then the ranked features were added one by one to build the models, respectively. This procedure was repeated until the least important feature used to build the model” Gao, pg. 45 Col 2; wherein three different combination of RF models are generated in iterations/repetition for building optimal RF models with optimal hyperparameters, Gao pg. 46 Col 1).
Claim(s) 10-12, 14-15 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Gao et al. in view of Su et al. and in further view of Zhang et al. (US 20200272817).
Regarding claim 10, the combination of Gao and Su as a whole does not expressly disclose wherein the analysing of the samples of the plant includes: imaging the plants at a plurality of mutually different angles. However, Zhang in the same field of endeavor teaches wherein the analysing of the samples of the plant includes: imaging the plants at a plurality of mutually different angles (“the rotating sample bracket 1 is fixed to the bottom of the detection sample chamber by screws at the four corners of a base, a rotating shaft is mounted at the geometrical center of the base of the rotating sample bracket 1, and a round sample bracket is mounted and fixed at the tail end of the rotating shaft; during the detection, the rotating shaft drives the rotating sample bracket 1 to rotate within 360° angle range” Zhang, [0050]; See Fig. 1 and Fig. 2, i.e., the rotating base allows for imaging at different angles).
Therefore, it would have been obvious to one of ordinary skill in the art to combine Gao and Su with Zhang before the effective filing date of the claimed invention. The motivation for this combination of references would have been to displace the plant sample in the horizontal direction and the vertical direction at a constant speed, so that it works with an image acquisition control system to realize a push-broom polarization-hyperspectral imaging (Zhang, [0055]). This motivation for the combination of Gao, Su and Zhang is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim 11, Gao, Su and Zhang disclose the method of claim 10, wherein the plurality of mutually different angles includes 0°, 120° and 240° (“the rotating sample bracket 1 is fixed to the bottom of the detection sample chamber by screws at the four corners of a base, a rotating shaft is mounted at the geometrical center of the base of the rotating sample bracket 1, and a round sample bracket is mounted and fixed at the tail end of the rotating shaft; during the detection, the rotating shaft drives the rotating sample bracket 1 to rotate within 360° angle range” Zhang, [0050]; See Fig. 1 and Fig. 2).
Therefore, combining Gao, Su and Zhang would meet the claim limitations for the same reasons as previously discussed in claim 10.
Regarding claim 12, Gao, Su and Zhang disclose the method of claim 10, including: rotating the plants to the plurality of mutually different angles using a lifter and turner assembly (“the rotating sample bracket 1 is fixed to the bottom of the detection sample chamber by screws at the four corners of a base, a rotating shaft is mounted at the geometrical center of the base of the rotating sample bracket 1, and a round sample bracket is mounted and fixed at the tail end of the rotating shaft; during the detection, the rotating shaft drives the rotating sample bracket 1 to rotate within 360° angle range” Zhang, [0050]; See Fig. 1 and Fig. 2).
Therefore, combining Gao, Su and Zhang would meet the claim limitations for the same reasons as previously discussed in claim 10.
Regarding claim 14, Gao, Su and Zhang disclose the method of claim 1, wherein the model wavebands include: over 1,000 wavebands, over 2,000 wavebands, over 3,000 wavebands, over 4,000 wavebands, or over 5,000 wavebands (“the wavelength range of a visible light-near infrared light source system 11 to 300 to 2,200 nm” Zhang, [0073]).
Therefore, combining Gao, Su and Zhang would meet the claim limitations for the same reasons as previously discussed in claim 10.
Regarding claim 15, Gao, Su and Zhang disclose the method of claim 14, wherein a number of the wavebands is selected based on a number of the wavebands measured by a hyperspectral imaging sensor or spectrometer (“polarization-hyperspectral imaging system, setting the wavelength range of a visible light-near infrared light source system 11 to 300 to 2,200 nm” Zhang, [0073]).
Therefore, combining Gao, Su and Zhang would meet the claim limitations for the same reasons as previously discussed in claim 10.
Regarding claim 19, Gao, Su and Zhang disclose the system of claim 16, including: a hyperspectral imaging station to generate the spectrum (Gao, Fig. 1 hyperspectral camera set up station); and a lifter and turner assembly for imaging plants at a plurality of mutually different angles to generate the measured spectra of the plant (“the rotating sample bracket 1 is fixed to the bottom of the detection sample chamber by screws at the four corners of a base, a rotating shaft is mounted at the geometrical center of the base of the rotating sample bracket 1, and a round sample bracket is mounted and fixed at the tail end of the rotating shaft; during the detection, the rotating shaft drives the rotating sample bracket 1 to rotate within 360° angle range” Zhang, [0050]; See Fig. 1 and Fig. 2).
Therefore, combining Gao, Su and Zhang would meet the claim limitations for the same reasons as previously discussed in claim 10.
Regarding claim 20, Gao, Su and Zhang disclose the system of claim 19, wherein the hyperspectral imaging station includes a pushbroom-type imaging spectrometer, optionally operational over a spectral range of 475-1710 nm and a spectral resolution of less than 10 nm (“using hyperspectral pre-filters with 1,450 nm transmission wavelength, and performing push-broom scanning and imaging” Zhang, [0074]).
Therefore, combining Gao, Su and Zhang would meet the claim limitations for the same reasons as previously discussed in claim 10.
Claim(s) 6 is rejected under 35 U.S.C. 103 as being unpatentable over Gao et al. in view of Su et al. and in further view of Bergstra et al. (“Algorithms for hyper-parameter optimization”, 2011).
Regarding claim 6, the combination of Gao and Su as a whole does not expressly disclose wherein the Bayesian optimizer is a Tree-Structured Parzen Estimator (TPE). However, Bergstra in the same field of endeavor teaches wherein the Bayesian optimizer is a Tree-Structured Parzen Estimator (TPE) (Bergstra pg. 4 section 4 Tree-structured Parzen Estimator Approach (TPE) discloses the TPE model hyper-parameter optimization scheme for SMBO algorithms).
Therefore, it would have been obvious to one of ordinary skill in the art to combine Gao and Su with Bergstra before the effective filing date of the claimed invention. The motivation for this combination of references would have been to obtain a hyper-parameter optimization algorithm over variables which are discrete, ordinal, and continuous while simultaneously choosing which variables to optimize (Bergstra, top of pg. 2). This motivation for the combination of Gao, Su and Bergstra is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Rozenstein et al. (US 20210042523 A1) discloses system and method for determining at least one vegetation index of crop based on multispectral image data.
Inquiries
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EMMANUEL SILVA-AVINA whose telephone number is (571)270-0729. The examiner can normally be reached Monday - Friday 11 AM - 8 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EMMANUEL SILVA-AVINA/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673