DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. This action is responsive to the following communication: Original claims filed 06/13/2013. This action is made non-final.
3. Claims 1-13 are pending in the case. Claims 1, 7 and 13 are independent claims.
Claim Rejections - 35 USC § 102
4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
5. Claims 1-13 are rejected under 35 U.S.C. 102(a)(1) as being rejected by anticipated by Bengston (US 20200342226).
Regarding claim 1, Bengston discloses a processor-implemented method comprising:
receiving, via one or more hardware processors, a first set of input data associated with a target crop in a specific area, the first set of input data comprises a plurality of satellite data (a multitude of data and observations from the ground and satellites should be utilized to develop a more accurate estimate for the yield of a crop, paragraph 0007), a plurality of weather data (the pre-season model uses features selected from field and weather data acquired, paragraph 0010), a plurality of soil data (using crop-specific information, weather data, and soil characteristics, paragraph 0010), and a plurality of district level crop yield data (he plurality of yield predictions may include each of the pre-season model crop yield prediction, the in-season model crop yield prediction, the statistical imagery model crop yield prediction, the histogram-based image model crop yield prediction, and the at least one crop-specific model crop yield prediction, paragraph 0010), wherein the plurality of satellite data comprises a plurality of images (imagery data 111 includes data from satellite images (e.g., PlanetScope™, RapidEye™, PlanetScope Mission 2™, SkySat™, LandSat™ 7, 8, and Sentinel™), paragraph 0039), one or more satellite indices (e.g., PlanetScope™, RapidEye™, PlanetScope Mission 2™, SkySat™, LandSat™ 7, 8, and Sentinel™), paragraph 0039), and one or more satellite indicators (Satellite imagery data is gathered over the growing season (e.g., for North America it typically is April 1.sup.st to September 1.sup.st). This ingest of imagery data is needed and used to train the Histogram-Based Image model. The images are first converted into histograms depending on the type of satellite. For example, PlanetScope™ images have four channels: blue (B), green (G), red (R), and near infrared (NIR), and the pixel values range from 0 to 10000, paragraph 0109);
preprocessing, via the one or more hardware processors, the first set of input data based on one or more pre-processing techniques to obtain a plurality of time-series representations of the first set of input data (All the data is retrieved from the image repository through a data pipeline. This contains all the useful images, which are spread over time for the same spatial grid (e.g., field). The “raw” data is then aggregated into a new temporal resolution (currently about four to seven days) that helps to reduce noise (e.g., clouds and shadow) in the images. The aggregated images are then subjected to feature engineering and then feature selection, as described above, paragraph 0103);
training, via the one or more hardware processors, a first crop yield forecasting model using the plurality of time-series representations of the first set of input data at a plurality of time instances of a growing season of the target crop, for determining an optimum time for crop yield forecasting for the target crop in the specific area (FIG. 3 illustrates an overview of the final yield forecasting model 300 used by the yield forecasting method. In particular, a pre-season model 302, an in-season model 304, a statistical imagery model 306, a histogram-based image model 308, and one or more crop specific models 310 are shown, which may be combined through step 312 of ensembling or stacking to produce a final yield prediction 320, paragraph 0032);
generating, via the one or more hardware processors, a plurality of coarse resolution crop yield maps of the target crop in the specific area based on an optimal performance of the first crop yield forecasting model, wherein the optimal performance of the first crop yield forecasting model is achieved by dynamically performing one or more feature selection techniques on the first crop yield forecasting model, wherein each coarse resolution crop yield map from the plurality of coarse resolution crop yield maps comprises a plurality of pixels, and wherein each pixel from amongst the plurality of pixels is associated with a location and a spatial resolution (As shown in FIG. 7, the Histogram-Based Image Model uses satellite imagery and converts the images into histograms containing pixel information, which are then stacked into a time-channel. The Histogram-Based Image model is developed on this temporal information to predict an estimate of the crop yield, paragraph 0107, The Histogram-Based Image model uses a deep convolutional neural network (CNN) for the supervised learning. The use of deep convolutional neural networks is known to a person skilled in the art of machine learning, however, necessary modifications in model structure, time step setting, cutoff edges, etc., are needed to adapt to this model's dataset. The CNN is trained using a dataset, consisting of thousands of satellite images, that is stored on a cloud storage network. All the data pre-processing (i.e., from raw image to histogram representation) is performed on the cloud. These satellite images contain pixels that represent a field. During the training phase, the network learns the relationship between image representations of the field and crop yield. After the network is trained, information is uploaded into a server and used to detect pixels in query images. These queries might be sent from components to provide yield prediction, paragraph 0115);
dynamically selecting, via the one or more hardware processors, a set of pixels from the plurality of pixels of each coarse resolution crop yield map from the plurality of coarse resolution crop yield maps by applying a stratified random sampling based technique on a second set of input data, wherein the second set of input data comprises the plurality of coarse resolution crop yield maps of the target crop in the specific area, a plurality of crop maps, the plurality of soil data, and a plurality of data related to agro-ecological zones available for the specific area ()Satellite imagery data is gathered over the growing season (e.g., for North America it typically is April 1.sup.st to September 1.sup.st). This ingest of imagery data is needed and used to train the Histogram-Based Image model. The images are first converted into histograms depending on the type of satellite. For example, PlanetScope™ images have four channels: blue (B), green (G), red (R), and near infrared (NIR), and the pixel values range from 0 to 10000. The pixel values are rescaled, and for any abnormal values, which are too small (area that is too dark) or too big (area that is too bright), are cut-off. However, it is important to note that each channel has different cut-off edges to define a normal value range. In our example, we use 14˜46 in both B and G channels, 8˜40 in R channel, and 10˜136 in NIR channel. The scaled and cut pixel values are placed into a histogram with 32 bins in which a 32-dimensional vector is derived for each channel of the image. This results in an image that has been converted into a 32*4 matrix. A temporal image set over the growing season can provide a more valuable and accurate representation of the final crop yield rather than a single image; to achieve this, it is necessary to aggregate all 32*4 matrices of all the images of a field within a growing season into one single representation, paragraph 0109, 0110) ; and
generating, via the one or more hardware processors, a plurality of high resolution crop yield forecast maps using a second crop yield forecasting model trained with a third set of data, wherein the third set of data comprises (i) a set of satellite data and (ii) a plurality of crop yield data corresponding to the set of dynamically selected pixels (Once each of the five models has provided a prediction for the crop yield, the predictions will be ensembled or stacked together to generate a final crop yield prediction. This methodology is used in machine learning to correct overfitting and bias within a single model. Ensembling can be a simple average mean of all the predictions for example, while stacking uses the results of the predictions as features in a new model—a “second layer” prediction. FIG. 3 illustrates an overview of the final yield forecasting model 300 used by the yield forecasting method. Ensembling is used to help improve machine learning results by combining several models. This approach allows the production of better predictive performance compared to a single model and helps prevent overfitting. Each of the five models may employ different techniques that utilize different features to generate their respective sub-predictions. For example, one model may be a convolutional neural network, another model may be a deep learning algorithm, while another model may be a multi-layer predictor. Ensembling of the model can take place in other ways. For example, the non-imagery-based models can be combined either through ensembling or through combining the training of the data into a different model, which can be stacked or ensembled with the other models. This combined model can be a neural network or other deep learning algorithm, tree-based algorithm, or other machine learning method. This combined model can then be ensembled with the imagery models to create a final prediction or used separately as required, paragraph 0125-0126, see also paragraph 0115 and satellite images containing pixels).
Regarding claim 2, Bengston discloses wherein the stratified random sampling based technique comprises at least one of (i) a probability sampling or (ii) a non-probability sampling depending upon variability of a region of crop (Crop yield may be denoted in a variety of ways, some of which may include aggregate production values (e.g., bushels/acre across a field), spatial variations (i.e., relative yield differences between two zones in a field), and crop variations (e.g., yield associated with specific crop types in different zones). In regard to the ideal dataset, ultimately the yield would be a true two-dimensional (2D) yield surface over a particular area (i.e., yield as a function of x y coordinates), such as within a field boundary, see paragraph 0004).
Regarding claim 3, Bengston discloses wherein the first crop yield forecasting model and the second crop yield forecasting model comprises at least one of (i) a multiple linear regression model and (ii) a non-liner regression model depending upon type of crop, region of crop and amount of data availability (in order to train the model and determine the values for the model parameters (i.e., for the calibration equations), certain data may be collected as inputs for training the model. The type of modeling function may vary by implementation. In one embodiment, regression techniques such as: ElasticNet, linear, logistic, or otherwise may be used. Other techniques may also be used, some examples of which include Random Forest Classifiers, Neural Nets, Support Vector Machines, and so on. Once trained, the resulting prediction model can then be used to predict a specific type of data. To validate that the model is working, the predictions versus the benchmark datasets are validated by taking the test cases and relating them back to the true values of a traditional model. Validation involves relating the predictions of the model to the true values that were collected, paragraph 0086).
Regarding claim 4, Bengston discloses further comprises quantifying one or more crop yield losses based on information comprised in a dynamically updated database, wherein the dynamically updated database comprises domain knowledge about crop growth stages, economically important crop growth stages and real time weather based adverse event triggers (weather feeds 113 could be provided, for example, by establishing interfaces with established weather data providers, such as NOAA. Public soil data 114 is acquired from public soil databases, such as SSURGO which provide access to soil sampling data, including chemistry records for various types of soil. This information may be collected and managed by states, counties, and/or farmers. Crop and variety information 115 may be retrieved from a seed variety database. There are several different chemical companies that provide hybrid seeds and the genetic engineering services. As required by governance, every bag of seed is required to have a seed variety number on it, the precision agricultural system of the present description will use this interface to track seed variety numbers for the seed varieties that is being used for farming operation, paragraph 0059).
Regarding claim 5, Bengston discloses wherein the plurality of high resolution crop yield forecast maps are obtained by adjusting the quantified one or more crop yield losses (Data may be retrieved from storage or from a server using an operating system or one or more application programs or systems. An example of an application program used is FarmCommand™ which is commercially available from Farmers Edge Inc.®, Winnipeg, Alberta, Canada. FarmCommand™ features computer software platforms for cloud-based services in the field of agriculture. Furthermore, data may be procured from external sources, which may also be publicly available databases. Stored data may be tagged with several attributes such as, but not limited to a start date, end date, latitude, longitude, and an optional period (for example, daily, monthly, etc.). Growth stages are different depending on the type of crop. For crop-specific models, the growth stages are grouped into different bins to deal with any overlaps between the growth stages and to reduce the number of different growth stages. The weather data during specific growth stages of the crops may include precipitation, temperatures, solar radiation, wind, relative humidity, etc. The crop information may include variety, previous crops, and seeding date, paragraph 0119).
Regarding claim 6, Bengston discloses wherein the plurality of high resolution crop yield forecast maps are scalable (satellite imagery data is gathered over the growing season (e.g., for North America it typically is April 1.sup.st to September 1.sup.st). This ingest of imagery data is needed and used to train the Histogram-Based Image model. The images are first converted into histograms depending on the type of satellite. For example, PlanetScope™ images have four channels: blue (B), green (G), red (R), and near infrared (NIR), and the pixel values range from 0 to 10000. The pixel values are rescaled, and for any abnormal values, which are too small (area that is too dark) or too big (area that is too bright), are cut-off. However, it is important to note that each channel has different cut-off edges to define a normal value range. In our example, we use 14˜46 in both B and G channels, 8˜40 in R channel, and 10˜136 in NIR channel. The scaled and cut pixel values are placed into a histogram with 32 bins in which a 32-dimensional vector is derived for each channel of the image. This results in an image that has been converted into a 32*4 matrix, paragraph 0109).
Regarding claim 7, Bengston discloses a system comprising: a memory storing instructions; one or more communication interfaces; and one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors (see FIG. 10) are configured by the instructions to:
receive, a first set of input data associated with a target crop in a specific area, the first set of input data comprises a plurality of satellite data (a multitude of data and observations from the ground and satellites should be utilized to develop a more accurate estimate for the yield of a crop, paragraph 0007), a plurality of weather data (the pre-season model uses features selected from field and weather data acquired, paragraph 0010), a plurality of soil data (using crop-specific information, weather data, and soil characteristics, paragraph 0010), and a plurality of district level crop yield data (he plurality of yield predictions may include each of the pre-season model crop yield prediction, the in-season model crop yield prediction, the statistical imagery model crop yield prediction, the histogram-based image model crop yield prediction, and the at least one crop-specific model crop yield prediction, paragraph 0010), wherein the plurality of satellite data comprises a plurality of images (imagery data 111 includes data from satellite images (e.g., PlanetScope™, RapidEye™, PlanetScope Mission 2™, SkySat™, LandSat™ 7, 8, and Sentinel™), paragraph 0039), one or more satellite indices (e.g., PlanetScope™, RapidEye™, PlanetScope Mission 2™, SkySat™, LandSat™ 7, 8, and Sentinel™), paragraph 0039), and one or more satellite indicators (Satellite imagery data is gathered over the growing season (e.g., for North America it typically is April 1.sup.st to September 1.sup.st). This ingest of imagery data is needed and used to train the Histogram-Based Image model. The images are first converted into histograms depending on the type of satellite. For example, PlanetScope™ images have four channels: blue (B), green (G), red (R), and near infrared (NIR), and the pixel values range from 0 to 10000, paragraph 0109);
preprocess, the first set of input data based on one or more pre-processing techniques to obtain a plurality of time-series representations of the first set of input data (All the data is retrieved from the image repository through a data pipeline. This contains all the useful images, which are spread over time for the same spatial grid (e.g., field). The “raw” data is then aggregated into a new temporal resolution (currently about four to seven days) that helps to reduce noise (e.g., clouds and shadow) in the images. The aggregated images are then subjected to feature engineering and then feature selection, as described above, paragraph 0103);
train, a first crop yield forecasting model using the plurality of time-series representations of the first set of input data at a plurality of time instances of a growing season of the target crop, for determining an optimum time for crop yield forecasting for the target crop in the specific area (FIG. 3 illustrates an overview of the final yield forecasting model 300 used by the yield forecasting method. In particular, a pre-season model 302, an in-season model 304, a statistical imagery model 306, a histogram-based image model 308, and one or more crop specific models 310 are shown, which may be combined through step 312 of ensembling or stacking to produce a final yield prediction 320, paragraph 0032);
generate, a plurality of coarse resolution crop yield maps of the target crop in the specific area based on an optimal performance of the first crop yield forecasting model, wherein the optimal performance of the first crop yield forecasting model is achieved by dynamically performing one or more feature selection techniques on the first crop yield forecasting model, wherein each coarse resolution crop yield map from the plurality of coarse resolution crop yield maps comprises a plurality of pixels, and wherein each pixel from amongst the plurality of pixels is associated with a location and a spatial resolution (As shown in FIG. 7, the Histogram-Based Image Model uses satellite imagery and converts the images into histograms containing pixel information, which are then stacked into a time-channel. The Histogram-Based Image model is developed on this temporal information to predict an estimate of the crop yield, paragraph 0107, The Histogram-Based Image model uses a deep convolutional neural network (CNN) for the supervised learning. The use of deep convolutional neural networks is known to a person skilled in the art of machine learning, however, necessary modifications in model structure, time step setting, cutoff edges, etc., are needed to adapt to this model's dataset. The CNN is trained using a dataset, consisting of thousands of satellite images, that is stored on a cloud storage network. All the data pre-processing (i.e., from raw image to histogram representation) is performed on the cloud. These satellite images contain pixels that represent a field. During the training phase, the network learns the relationship between image representations of the field and crop yield. After the network is trained, information is uploaded into a server and used to detect pixels in query images. These queries might be sent from components to provide yield prediction, paragraph 0115);
dynamically select, a set of pixels from the plurality of pixels of each coarse resolution crop yield map from the plurality of coarse resolution crop yield maps by applying a stratified random sampling based technique on a second set of input data, wherein the second set of input data comprises the plurality of coarse resolution crop yield maps of the target crop in the specific area, a plurality of crop maps, the plurality of soil data, and a plurality of data related to agro-ecological zones available for the specific area ()Satellite imagery data is gathered over the growing season (e.g., for North America it typically is April 1.sup.st to September 1.sup.st). This ingest of imagery data is needed and used to train the Histogram-Based Image model. The images are first converted into histograms depending on the type of satellite. For example, PlanetScope™ images have four channels: blue (B), green (G), red (R), and near infrared (NIR), and the pixel values range from 0 to 10000. The pixel values are rescaled, and for any abnormal values, which are too small (area that is too dark) or too big (area that is too bright), are cut-off. However, it is important to note that each channel has different cut-off edges to define a normal value range. In our example, we use 14˜46 in both B and G channels, 8˜40 in R channel, and 10˜136 in NIR channel. The scaled and cut pixel values are placed into a histogram with 32 bins in which a 32-dimensional vector is derived for each channel of the image. This results in an image that has been converted into a 32*4 matrix. A temporal image set over the growing season can provide a more valuable and accurate representation of the final crop yield rather than a single image; to achieve this, it is necessary to aggregate all 32*4 matrices of all the images of a field within a growing season into one single representation, paragraph 0109, 0110) ; and
generate, a plurality of high resolution crop yield forecast maps using a second crop yield forecasting model trained with a third set of data, wherein the third set of data comprises (i) a set of satellite data and (ii) a plurality of crop yield data corresponding to the set of dynamically selected pixels (Once each of the five models has provided a prediction for the crop yield, the predictions will be ensembled or stacked together to generate a final crop yield prediction. This methodology is used in machine learning to correct overfitting and bias within a single model. Ensembling can be a simple average mean of all the predictions for example, while stacking uses the results of the predictions as features in a new model—a “second layer” prediction. FIG. 3 illustrates an overview of the final yield forecasting model 300 used by the yield forecasting method. Ensembling is used to help improve machine learning results by combining several models. This approach allows the production of better predictive performance compared to a single model and helps prevent overfitting. Each of the five models may employ different techniques that utilize different features to generate their respective sub-predictions. For example, one model may be a convolutional neural network, another model may be a deep learning algorithm, while another model may be a multi-layer predictor. Ensembling of the model can take place in other ways. For example, the non-imagery-based models can be combined either through ensembling or through combining the training of the data into a different model, which can be stacked or ensembled with the other models. This combined model can be a neural network or other deep learning algorithm, tree-based algorithm, or other machine learning method. This combined model can then be ensembled with the imagery models to create a final prediction or used separately as required, paragraph 0125-0126, see also paragraph 0115 and satellite images containing pixels).
Regarding claim 8, the subject matter of the claim is substantially similar to claim 2 and as such the same rationale of rejection applies.
Regarding claim 9, the subject matter of the claim is substantially similar to claim 3 and as such the same rationale of rejection applies.
Regarding claim 10, the subject matter of the claim is substantially similar to claim 4 and as such the same rationale of rejection applies.
Regarding claim 11, the subject matter of the claim is substantially similar to claim 5 and as such the same rationale of rejection applies.
Regarding claim 12, the subject matter of the claim is substantially similar to claim 6 and as such the same rationale of rejection applies.
Regarding claim 13, the subject matter of the claim is substantially similar to claim 1 and as such the same rationale of rejection applies.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID E CHOI whose telephone number is (571)270-3780. The examiner can normally be reached on M-F: 7-2, 7-10 (PST). If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bechtold, Michelle T. can be reached on (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID E CHOI/Primary Examiner, Art Unit 2148