DETAILED ACTION
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, and 15-21 are rejected under 35 U.S.C. 103 as being unpatentable over Froloff (US 20200117897 A1), and in view of Ampatzidis (US 20200019765 A1).
Re Claim 1, Froloff discloses a computer-implemented method of insect monitoring (see Froloff: e.g., Fig. 1, and, -- AI analytic trained automated crop or plant monitoring system, accumulating wireless sensor image data into identifiable labeled image objects for training AI analytics. A plurality of integrated sensors network wirelessly collecting plant and insect primary sensor data have logic for primary sensor data transfer with associated sensor metadata onto a database, and logic for partitioning primary sensor image data into insect pestilent and plant images. Non-expert monitoring demark suspected objects in the data by digitally bounding border demarking an object in a partitioned data image, view display comparison of exemplar images of specific identified insects and plant harms with the demarked image data objects…. Preset minimum numerical count for threshold of training image data for a specified label of positive and negative data image set counts are used as thresholds for the minimum number of images needed accumulated before forwarding the data sets reaching threshold specific image label count as input training data to an AI machine learning program for creating an AI analytic capable of identifying the specific labeled image object in primary sensor data images. The trained executing AI analytic scanning a plurality wireless sensor data images for pestilence and plant objects matching positive trained identified labeled objects, where the analytic program is responsive to monitoring for specifically trained positive identified labeled positive label trained objects identified in sensor data images so that a system with a plurality of wireless integrated sensor network continuously monitoring plants for pestilence and other plant harms can timely raise alerts of found labeled positively identified insects or plant harm without human visual intervention, alerts retaining sensor meta data, time and location of sensor data triggering image alert.--, in [0030], and, -- inspection is automated through wireless sensor networks and variable, image data taken with frequency required. Monitoring insects on integrated sensors can also be done programmatically from remote sensors 101 with wireless networks sending image data to a wireless network hub 105. Alternatively, government pestilence watcher programs set up to inspect for specific and particularly proven harmful pests will have a completely different geographic placement of sensors with wireless network hubs perhaps situated in citizen's homes, and not necessarily for food production. Based on the jurisdiction boundaries, residential plants and trees in need of protection with early warning alerts, state and local governments and associations can set up wireless network sensors with embodiments of the invention as well.--, in [0055]), said method comprising:
capturing at least one image of one or more insects (see Froloff: e.g., Fig. 7, --Placement of sensor traps is optimal for identifying characteristics of insect/bug/pest trapped on sticky traps 207 view as well as plant or foliage view area 209. The optical sensors 211 placement provides a way to capture image data with view an integrated sticky card 207 with and area having foliage or plant 209 view area adjacent. Imaging both sticky area and crop areas increases the integrated sticky sensor's capability in delivering more comprehensive image data for later processing. Wire, fiber or twisted filament material can be used for the stays 213 providing a sensor position having the requisite view area 209 of the plant as well as the insects.--, in [0059]);
transmitting the at least one image to a computing device (see Froloff: e.g., --[0067] It is conceivable that a build up of trapped insect showing up over a time period on the traps and on the images, making it more difficult to distinguish individual insects. Where the insect information is not alarming, and alerting candidate insects can potentially show up in traps too crowded to distinguish them, in some embodiments successive sensor image transmission redundancies can encoded out, ie only the differences encoded into a sensor upload transmission, such that only the newly arrived insects or trapped bugs are reported by the sensors. Furthermore compression, such as the Discrete Cosine Transform used in MPEG and JPEG formats, techniques embedded in sensors can be used where sensor image transmission bandwidths are limited…., [0076] Acoustic, vibration and other type sensor data 403 can apply bandpass, threshold and other filters 405 to isolate potential actionable data for particular pests, convert by ADC 407, digitize 409 and frame data for transmission to a sensor hub 413 for forwarding and synchronization with meta data before transmission to an image/data/metadata database 415. The sensors can generally have multiple site locations as well as multiple human monitors which will in short period of time develop stored data. All the data and metadata captured at this point is potentially valuable for a machine learning AI in future data mining use regardless of what is found upstream. This data can be stored, in the cloud or other space, for future sale or use above and beyond the current accumulation of training data and crop monitoring.
[0077] The forwarded image data or local server 417 stored image data 415 are partitioned, separating the bug images views from the plant image view data, forming at least two main data image streams. Where there is no bug image part or plant image part, then partitioning is cropping existing other part The image data is programmatically tagged with available surrounding or associated data and metadata available from the sensors as well as server programmable information such as: [0078] date, time, location, [0079] environmental conditions, [0080] temperature, humidity, water, [0081] region, season, climate cycle time, [0082] crop known pest infesters, larvae images, [0083] biofix or mating initiation cycle/activities, [0084] associated pest mating activity parameters, [0085] etc,--, in [0067]-[0085]; and, --0100] In an embodiment of the invention, a wireless sensor network 501 having low-power integrated circuits, and wireless communication is at the heart of a wireless sensor network 501, with sensor 503 monitoring in real-time or programmed delay updates of integrated sensed data and transmission through a wireless network. Intelligent plant sensors are integrated into a wireless sensor network for early detection of plant or crop conditions. Plant growth and plant disease or damage rates do not generally impair normal processing activities. The sensor information is transmitted wirelessly to an external processing unit at program set or programmed delay rates, ostensibly in application of power saving strategies. Data from acoustic or vibration sensors 503 are collected similarly where data can become graphical and then image data.--, in [0100], and [0105]);
wherein the computing device comprises an artificial intelligence model operable to identify insects (see Froloff: e.g., Fig. 1, and, -- AI analytic trained automated crop or plant monitoring system, accumulating wireless sensor image data into identifiable labeled image objects for training AI analytics. A plurality of integrated sensors network wirelessly collecting plant and insect primary sensor data have logic for primary sensor data transfer with associated sensor metadata onto a database, and logic for partitioning primary sensor image data into insect pestilent and plant images. Non-expert monitoring demark suspected objects in the data by digitally bounding border demarking an object in a partitioned data image, view display comparison of exemplar images of specific identified insects and plant harms with the demarked image data objects…. Preset minimum numerical count for threshold of training image data for a specified label of positive and negative data image set counts are used as thresholds for the minimum number of images needed accumulated before forwarding the data sets reaching threshold specific image label count as input training data to an AI machine learning program for creating an AI analytic capable of identifying the specific labeled image object in primary sensor data images. The trained executing AI analytic scanning a plurality wireless sensor data images for pestilence and plant objects matching positive trained identified labeled objects, where the analytic program is responsive to monitoring for specifically trained positive identified labeled positive label trained objects identified in sensor data images so that a system with a plurality of wireless integrated sensor network continuously monitoring plants for pestilence and other plant harms can timely raise alerts of found labeled positively identified insects or plant harm without human visual intervention, alerts retaining sensor meta data, time and location of sensor data triggering image alert.--, in [0030], and, -- inspection is automated through wireless sensor networks and variable, image data taken with frequency required. Monitoring insects on integrated sensors can also be done programmatically from remote sensors 101 with wireless networks sending image data to a wireless network hub 105. Alternatively, government pestilence watcher programs set up to inspect for specific and particularly proven harmful pests will have a completely different geographic placement of sensors with wireless network hubs perhaps situated in citizen's homes, and not necessarily for food production. Based on the jurisdiction boundaries, residential plants and trees in need of protection with early warning alerts, state and local governments and associations can set up wireless network sensors with embodiments of the invention as well.--, in [0055]; and, --Thus adapting a trained executing AI analytic will scan the plurality wireless sensor data images for pestilence and plant objects matching positive trained identified labeled objects with logic responsive to monitoring for positive identified labeled objects raising alerts for positive label trained objects identified in sensor data images and timely raise alerts of found labeled positively identified insects or plant harm without human visual intervention, alerts retaining sensor meta data, time and location of sensor data triggering image alert.),--, in [0105], and, --The image data will have bug pest view area and plant view area. Trained AI analytics for identifying specific image objects as pest or plant pestilence with a set degree of certainty 519 will label the object and signal an alert 533 to the local monitor warning them of the possible intrusion, providing the object identified and associated meta data and confirming data from any associations.--, in [0107]);
wherein the artificial intelligence model is trained on previously collected insect images via an unsupervised domain adaptation technique (see Froloff: e.g., --[0019] Machine learning is a field of computer science that uses statistical techniques to give computer systems the ability to “learn”, progressively improve its ability to identify the target labeled image entity sought. These can come from Supervised or Unsupervised methods.--, in [0019], and,
--Thus adapting a trained executing AI analytic will scan the plurality wireless sensor data images for pestilence and plant objects matching positive trained identified labeled objects with logic responsive to monitoring for positive identified labeled objects raising alerts for positive label trained objects identified in sensor data images and timely raise alerts of found labeled positively identified insects or plant harm without human visual intervention, alerts retaining sensor meta data, time and location of sensor data triggering image alert.),--, in [0105], and, --The image data will have bug pest view area and plant view area. Trained AI analytics for identifying specific image objects as pest or plant pestilence with a set degree of certainty 519 will label the object and signal an alert 533 to the local monitor warning them of the possible intrusion, providing the object identified and associated meta data and confirming data from any associations.--, in [0107]; and, -- The trainer can use any of the typical AI machine learning implemented algorithms for identifying image objects including Dimensionality reduction, Ensemble learning, Meta learning, Reinforcement learning, Supervised learning, Unsupervised learning, Semi-supervised learning, and Deep learning. Next, Export inference graph 611, copy the image sets to the training directory and the text directory.
[0123] Lastly, test the inference on the test or validating data set 613. Over fitting/overtraining in supervised learning is determined as follows. Training error is graphed against validation error, both as a function of the number of training cycles. If the validation error increases, positive slope, while the training error steadily decreases, negative slope, then a situation of over fitting may have occurred. The best predictive and fitted model would be where the validation error has its global minimum.
[0124] Any machine learning methods including but not limited to Dimensionality reduction, Ensemble learning, Meta learning, Reinforcement learning, Supervised learning, Unsupervised learning, Semi-supervised learning, Deep learning, and other machine learning methods can be used where applicable to a candidate pest or plant harm object identification.--, in [0122]-[0124]);
Froloff however does not explicitly teach utilizing the artificial intelligence model to generate insect data related to the one or more insects from the at least one image,
Ampatzidis teaches utilizing the artificial intelligence model to generate insect data related to the one or more insects from the at least one image (see Ampatzidis: e.g., -- The image analysis application 260 may be executed to analyze images 236 captured from the camera(s) 251 (with or without micro lenses) to distinguish, identify, and count the pests from the images using a trained deep-learning convolutional neural network(s) (DL-CNN) or similar machine learning techniques (Artificial Intelligence). The DL-CNNs may be based on models generated by machine learning using a collection of images…. he images 236 can include a collection of images collected by the ATDS 203 and/or images collected by other devices. The images 236 can be used by the image analysis application 260 in the analysis of images of the viewing stage 248 that are captured by the camera(s) 251 for the detection, identification, and counting of the pests deposited on the viewing stage 248. The images 236 can be used in retraining and/or refining the machine learning models implemented by the ATDS computing device 242 and/or any other computing device as can be appreciated.
[0043] The identification rules 269 include rules and/or configuration data for the various algorithms and/or machine learning models used to detect, identify, and/or count the pests in each captured image --, in [0036], [0039], and [0042]-[0043]; and, -- A method, comprising: agitating, via a shaking system of an automated tap and detection system, a vegetation to displace material from the vegetation onto a viewing stage of the automated tap and detection system; capturing, via a camera system of the automated tap and detection system, one or more images of the viewing stage; removing, via a removal system of the automated tap and detection system, the material from the viewing stage; analyzing, via at least one computing device of the automated tap and detection system, the one or more images to detect one or more insects, identify the one or more insects, and count the one or more insects; and generating, via the at least one computing device, insect data comprising an identification of the one or more insects and a number of the one or more insects.--, in claim 11);
Froloff and Ampatzidis are combinable as they are in the same field of endeavor: monitoring insects in the field/wild and using machine learning in image processing for detecting and identifying insects. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify Froloff’s method using Ampatzidis’s teachings by including utilizing the artificial intelligence model to generate insect data related to the one or more insects from the at least one image to Froloff’s identifying determining the insects in order to detect, identify, and/or count the pests in each captured image (see Ampatzidis: e.g. in [0036], [0039], and [0042]-[0043], and claim 11).
Re Claim 15, Froloff as modified by Ampatzidis further disclose wherein the insect data comprises the identity of the one or more insects (see Ampatzidis: e.g., -- The image analysis application 260 may be executed to analyze images 236 captured from the camera(s) 251 (with or without micro lenses) to distinguish, identify, and count the pests from the images using a trained deep-learning convolutional neural network(s) (DL-CNN) or similar machine learning techniques (Artificial Intelligence). The DL-CNNs may be based on models generated by machine learning using a collection of images…. he images 236 can include a collection of images collected by the ATDS 203 and/or images collected by other devices. The images 236 can be used by the image analysis application 260 in the analysis of images of the viewing stage 248 that are captured by the camera(s) 251 for the detection, identification, and counting of the pests deposited on the viewing stage 248. The images 236 can be used in retraining and/or refining the machine learning models implemented by the ATDS computing device 242 and/or any other computing device as can be appreciated.
[0043] The identification rules 269 include rules and/or configuration data for the various algorithms and/or machine learning models used to detect, identify, and/or count the pests in each captured image --, in [0036], [0039], and [0042]-[0043]; and, -- A method, comprising: agitating, via a shaking system of an automated tap and detection system, a vegetation to displace material from the vegetation onto a viewing stage of the automated tap and detection system; capturing, via a camera system of the automated tap and detection system, one or more images of the viewing stage; removing, via a removal system of the automated tap and detection system, the material from the viewing stage; analyzing, via at least one computing device of the automated tap and detection system, the one or more images to detect one or more insects, identify the one or more insects, and count the one or more insects; and generating, via the at least one computing device, insect data comprising an identification of the one or more insects and a number of the one or more insects.--, in claim 11).
1
Re Claim 16, Froloff as modified by Ampatzidis further disclose wherein the identity of the one or more insects comprises a classification of the one or more insects (see Froloff: e.g., --[0019] Machine learning is a field of computer science that uses statistical techniques to give computer systems the ability to “learn”, progressively improve its ability to identify the target labeled image entity sought. These can come from Supervised or Unsupervised methods.--, in [0019], and,
--Thus adapting a trained executing AI analytic will scan the plurality wireless sensor data images for pestilence and plant objects matching positive trained identified labeled objects with logic responsive to monitoring for positive identified labeled objects raising alerts for positive label trained objects identified in sensor data images and timely raise alerts of found labeled positively identified insects or plant harm without human visual intervention, alerts retaining sensor meta data, time and location of sensor data triggering image alert.),--, in [0105], and, --The image data will have bug pest view area and plant view area. Trained AI analytics for identifying specific image objects as pest or plant pestilence with a set degree of certainty 519 will label the object and signal an alert 533 to the local monitor warning them of the possible intrusion, providing the object identified and associated meta data and confirming data from any associations.--, in [0107]; and,
--[0113] In machine learning, algorithms work by making sensor image data-driven predictions or decisions through building a mathematical model from the input input training data. In an embodiment of the invention the data used to build the final program will come from multiple datasets of algorithm sufficient number of images. As mentioned for above embodiments in particular, three data sets of images are commonly used in different stages of the creation of a model.
[0114] Furthermore, an ML knowledgeable individual upon scanning a specific exemplar for features and some candidate training input data will determine the structure of the learned function and corresponding learning algorithm. For example, the selection may algorithm may be support vector machines or decision trees. Learning algorithm selection can address bias-variance tradeoff, function complexity and amount of training data, dimensionality of the input space, noise in the output values and other factors.
[0115] Many Object Detection Classifier algorithms are available to train AIs from training data input. These are installed for executed using the training data sets provided. There are pre-trained object detection models available as well. These or others are used train on embodiments of the invention training data.
[0116] Training data files must be configured and made compatible to the AI program input specs in size and format. There are training data, positive and negative sets, and test data image sets. There are also image labeling programs to help prepare the training data image sets Where there are more than one image of interest in a single data image, multiple labeled images can be saved in independent files for training from a single data image. A preliminary step in the AI process is to create a label map, mapping class names to class ids, and configuring the training data for the object classifier. Once all of the training data labeled, mapped and classified we can train the object detector.--, in [0113]-[0116]; also see Ampatzidis: e.g., -- The image analysis application 260 may be executed to analyze images 236 captured from the camera(s) 251 (with or without micro lenses) to distinguish, identify, and count the pests from the images using a trained deep-learning convolutional neural network(s) (DL-CNN) or similar machine learning techniques (Artificial Intelligence). The DL-CNNs may be based on models generated by machine learning using a collection of images…. he images 236 can include a collection of images collected by the ATDS 203 and/or images collected by other devices. The images 236 can be used by the image analysis application 260 in the analysis of images of the viewing stage 248 that are captured by the camera(s) 251 for the detection, identification, and counting of the pests deposited on the viewing stage 248. The images 236 can be used in retraining and/or refining the machine learning models implemented by the ATDS computing device 242 and/or any other computing device as can be appreciated.
[0043] The identification rules 269 include rules and/or configuration data for the various algorithms and/or machine learning models used to detect, identify, and/or count the pests in each captured image --, in [0036], [0039], and [0042]-[0043]; and, -- A method, comprising: agitating, via a shaking system of an automated tap and detection system, a vegetation to displace material from the vegetation onto a viewing stage of the automated tap and detection system; capturing, via a camera system of the automated tap and detection system, one or more images of the viewing stage; removing, via a removal system of the automated tap and detection system, the material from the viewing stage; analyzing, via at least one computing device of the automated tap and detection system, the one or more images to detect one or more insects, identify the one or more insects, and count the one or more insects; and generating, via the at least one computing device, insect data comprising an identification of the one or more insects and a number of the one or more insects.--, in claim 11).
Re Claim 17, Froloff as modified by Ampatzidis further disclose wherein the classification is based on population-level variation of the one or more insects (see Froloff: e.g., -- [0028] Sticky traps provide an easy method for estimating pest population densities. When the timing of pest control actions is based on these relative estimates along with plant sample data from visual inspections, there is generally a reduction in pesticide use. As a result, there are fewer problems with pesticide resistance, less worker exposure to pesticides, reduced pesticide runoff and improved plant quality with less pesticide-induced phytotoxicity symptoms.--, in [0027]-[0028]; and, -- [0056] In indoor cultivation embodiments, placement of sensors are done differently. Some greenhouse pests are usually not caught on sticky traps. So the integrated sensor sticky trap placement must account. For these situations, sticky traps must be used in combination with plant images to confirm the presence of pest populations.--, in [0056]).
Re Claim 18, Froloff as modified by Ampatzidis further disclose wherein the classification is based on the species of the one or more insects (see Froloff: e.g., --[0019] Machine learning is a field of computer science that uses statistical techniques to give computer systems the ability to “learn”, progressively improve its ability to identify the target labeled image entity sought. These can come from Supervised or Unsupervised methods.--, in [0019], and,
--Thus adapting a trained executing AI analytic will scan the plurality wireless sensor data images for pestilence and plant objects matching positive trained identified labeled objects with logic responsive to monitoring for positive identified labeled objects raising alerts for positive label trained objects identified in sensor data images and timely raise alerts of found labeled positively identified insects or plant harm without human visual intervention, alerts retaining sensor meta data, time and location of sensor data triggering image alert.),--, in [0105], and, --The image data will have bug pest view area and plant view area. Trained AI analytics for identifying specific image objects as pest or plant pestilence with a set degree of certainty 519 will label the object and signal an alert 533 to the local monitor warning them of the possible intrusion, providing the object identified and associated meta data and confirming data from any associations.--, in [0107]; and,
--[0113] In machine learning, algorithms work by making sensor image data-driven predictions or decisions through building a mathematical model from the input input training data. In an embodiment of the invention the data used to build the final program will come from multiple datasets of algorithm sufficient number of images. As mentioned for above embodiments in particular, three data sets of images are commonly used in different stages of the creation of a model.
[0114] Furthermore, an ML knowledgeable individual upon scanning a specific exemplar for features and some candidate training input data will determine the structure of the learned function and corresponding learning algorithm. For example, the selection may algorithm may be support vector machines or decision trees. Learning algorithm selection can address bias-variance tradeoff, function complexity and amount of training data, dimensionality of the input space, noise in the output values and other factors.
[0115] Many Object Detection Classifier algorithms are available to train AIs from training data input. These are installed for executed using the training data sets provided. There are pre-trained object detection models available as well. These or others are used train on embodiments of the invention training data.
[0116] Training data files must be configured and made compatible to the AI program input specs in size and format. There are training data, positive and negative sets, and test data image sets. There are also image labeling programs to help prepare the training data image sets Where there are more than one image of interest in a single data image, multiple labeled images can be saved in independent files for training from a single data image. A preliminary step in the AI process is to create a label map, mapping class names to class ids, and configuring the training data for the object classifier. Once all of the training data labeled, mapped and classified we can train the object detector.--, in [0113]-[0116]; also see Ampatzidis: e.g., -- The image analysis application 260 may be executed to analyze images 236 captured from the camera(s) 251 (with or without micro lenses) to distinguish, identify, and count the pests from the images using a trained deep-learning convolutional neural network(s) (DL-CNN) or similar machine learning techniques (Artificial Intelligence). The DL-CNNs may be based on models generated by machine learning using a collection of images…. he images 236 can include a collection of images collected by the ATDS 203 and/or images collected by other devices. The images 236 can be used by the image analysis application 260 in the analysis of images of the viewing stage 248 that are captured by the camera(s) 251 for the detection, identification, and counting of the pests deposited on the viewing stage 248. The images 236 can be used in retraining and/or refining the machine learning models implemented by the ATDS computing device 242 and/or any other computing device as can be appreciated.
[0043] The identification rules 269 include rules and/or configuration data for the various algorithms and/or machine learning models used to detect, identify, and/or count the pests in each captured image --, in [0036], [0039], and [0042]-[0043]; and, -- A method, comprising: agitating, via a shaking system of an automated tap and detection system, a vegetation to displace material from the vegetation onto a viewing stage of the automated tap and detection system; capturing, via a camera system of the automated tap and detection system, one or more images of the viewing stage; removing, via a removal system of the automated tap and detection system, the material from the viewing stage; analyzing, via at least one computing device of the automated tap and detection system, the one or more images to detect one or more insects, identify the one or more insects, and count the one or more insects; and generating, via the at least one computing device, insect data comprising an identification of the one or more insects and a number of the one or more insects.--, in claim 11).
Re Claim 19, Froloff as modified by Ampatzidis further disclose a step of recommending a course of action, implementing a course of action, or combinations thereof (see Froloff: e.g., --Sticky traps can provide warning of pest presence before plant damage becomes devastating, alerting owners, farmers, counties, cities, regional entities, to step up visual inspections. Some entire counties fearing infestation will implement programs that sparsely located data from sticky traps in residential communities as part of a community service. Once pests are confirmed in to be in the area, slower acting control strategies can be utilized that are more environmentally-friendly and safer for people.
[0006] Sticky traps can also indicate greenhouse hot spots and can document pestilence migration patterns when placed near doors and vents. Data collected from sticky traps can be used to evaluate pest management control actions, including the use of natural enemies.
[0007] However, installing and replacing traps is labor intensive. The number of traps needed depends on many factors, including the main target pest. Inspection of sticky traps is very labor intensive and recommended at least once or twice weekly and replaced after inspection unless the traps are reused. Reusing traps saves on the cost but counting insects on reused traps is more labor-intensive. Typically traps are not be left up for long periods because they become caked with insects, making it difficult to make accurate and quick counts. What is needed are ways to continually monitor the traps taking into account only the latest insects captured.--, in [0005]-[0007]; and, --[0026] What is needed are smart sensors and reporting of relevant data that can be processed to eventually automate the process of human analysis of the data. What is needed are integrated sensors and technology that relieve humans from the arduous and labor intensive tasks of scanning images and data to determine threats to plants and raise early alerts so that these can be managed before they become significant and costly.--, in [0026]; and, --The trained executing AI analytic scanning a plurality wireless sensor data images for pestilence and plant objects matching positive trained identified labeled objects, where the analytic program is responsive to monitoring for specifically trained positive identified labeled positive label trained objects identified in sensor data images so that a system with a plurality of wireless integrated sensor network continuously monitoring plants for pestilence and other plant harms can timely raise alerts of found labeled positively identified insects or plant harm without human visual intervention, alerts retaining sensor meta data, time and location of sensor data triggering image alert.--, in [0030]).
Re Claim 20, Froloff as modified by Ampatzidis further disclose wherein the course of action comprises fumigation, extermination, insect capturing, insect elimination, insect preservation, release of insect repellants, release of insect mating disruption pheromones, or combinations thereof (see Froloff: e.g., --Sticky traps can provide warning of pest presence before plant damage becomes devastating, alerting owners, farmers, counties, cities, regional entities, to step up visual inspections. Some entire counties fearing infestation will implement programs that sparsely located data from sticky traps in residential communities as part of a community service. Once pests are confirmed in to be in the area, slower acting control strategies can be utilized that are more environmentally-friendly and safer for people.
[0006] Sticky traps can also indicate greenhouse hot spots and can document pestilence migration patterns when placed near doors and vents. Data collected from sticky traps can be used to evaluate pest management control actions, including the use of natural enemies.
[0007] However, installing and replacing traps is labor intensive. The number of traps needed depends on many factors, including the main target pest. Inspection of sticky traps is very labor intensive and recommended at least once or twice weekly and replaced after inspection unless the traps are reused. Reusing traps saves on the cost but counting insects on reused traps is more labor-intensive. Typically traps are not be left up for long periods because they become caked with insects, making it difficult to make accurate and quick counts. What is needed are ways to continually monitor the traps taking into account only the latest insects captured.--, in [0005]-[0007]; and, --[0026] What is needed are smart sensors and reporting of relevant data that can be processed to eventually automate the process of human analysis of the data. What is needed are integrated sensors and technology that relieve humans from the arduous and labor intensive tasks of scanning images and data to determine threats to plants and raise early alerts so that these can be managed before they become significant and costly.--, in [0026]; and, --The trained executing AI analytic scanning a plurality wireless sensor data images for pestilence and plant objects matching positive trained identified labeled objects, where the analytic program is responsive to monitoring for specifically trained positive identified labeled positive label trained objects identified in sensor data images so that a system with a plurality of wireless integrated sensor network continuously monitoring plants for pestilence and other plant harms can timely raise alerts of found labeled positively identified insects or plant harm without human visual intervention, alerts retaining sensor meta data, time and location of sensor data triggering image alert.--, in [0030]; also see Ampatzidis: e.g., -- The image analysis application 260 may be executed to analyze images 236 captured from the camera(s) 251 (with or without micro lenses) to distinguish, identify, and count the pests from the images using a trained deep-learning convolutional neural network(s) (DL-CNN) or similar machine learning techniques (Artificial Intelligence). The DL-CNNs may be based on models generated by machine learning using a collection of images…. he images 236 can include a collection of images collected by the ATDS 203 and/or images collected by other devices. The images 236 can be used by the image analysis application 260 in the analysis of images of the viewing stage 248 that are captured by the camera(s) 251 for the detection, identification, and counting of the pests deposited on the viewing stage 248. The images 236 can be used in retraining and/or refining the machine learning models implemented by the ATDS computing device 242 and/or any other computing device as can be appreciated.
[0043] The identification rules 269 include rules and/or configuration data for the various algorithms and/or machine learning models used to detect, identify, and/or count the pests in each captured image --, in [0036], [0039], and [0042]-[0043]; and, -- A method, comprising: agitating, via a shaking system of an automated tap and detection system, a vegetation to displace material from the vegetation onto a viewing stage of the automated tap and detection system; capturing, via a camera system of the automated tap and detection system, one or more images of the viewing stage; removing, via a removal system of the automated tap and detection system, the material from the viewing stage; analyzing, via at least one computing device of the automated tap and detection system, the one or more images to detect one or more insects, identify the one or more insects, and count the one or more insects; and generating, via the at least one computing device, insect data comprising an identification of the one or more insects and a number of the one or more insects.--, in claim 11).
.
Re Claim 21, Froloff as modified by Ampatzidis further comprising a step of repeating the method after implementing the course of action (see -- [0076] A first deep learning convolutional neural network was trained using YOLOv3 (You Only Look Once), a state-of-the-art object detection system. YOLOv3 is a single-stage method for object detection consisting of 106 fully connected neural layers. A training set of 800 labeled images 236 was prepared by capturing images of ACPs on a white background. The network was trained for 10,000 iterations using a common learning rate of 0.01.-- , in [0076], [0080]).
Claims 2-7, and 22-29 are rejected under 35 U.S.C. 103 as being unpatentable over Froloff as modified by Ampatzidis, and in view of Patch (US 20220361471 A1, claims the priority of US-Provisional-Application US 63187356 20210511).
Re Claim 2, Froloff as modified by Ampatzidis however do not explicitly disclose a step of detecting insect movement prior to capturing the at least one image of the one or more insects;
Patch discloses a step of detecting insect movement prior to capturing the at least one image of the one or more insects (see Patch: e.g., --The example insect imaging chamber can represent a minimal baseline for collection of environmental and video data. The example insect imaging chamber can include a tube with two imaging devices (e.g., 12.3 megapixel HD cameras or any other suitable cameras) positioned such that they will capture the top and side views of insects as they crawl through the trap. [0036] To maximize the amount of images with insects, a motion detection module can be implemented. Thus, images that have insects in the frame or image can be only collected and saved to minimize erroneous storage usage.--, in [0035]-[0036]; and, --In some scenarios, the N-frame history buffer can check if a proposed bounding box could be an insect based on movement patterns while the predetermined second window can be a window if there is actually an insect crawling through an insect guide tube 108 in an insect imaging chamber 100 in FIGS. 1-3.--, in [0076]-[0077]);
Froloff (as modified by Ampatzidis) and Patch are combinable as they are in the same field of endeavor: trap device and monitoring insects and using machine learning in image processing for detecting and identifying insects. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify Froloff (as modified by Ampatzidis)’s method using Patch’s teachings by including a step of detecting insect movement prior to capturing the at least one image of the one or more insects to Froloff (as modified by Ampatzidis)’s identifying the insects in order to detect, identify the insects in captured image and video (see Patch: e.g. in [0036], and [0076]-[0077]).
Re Claim 3, Froloff as modified by Ampatzidis and Patch further disclose the detecting occurs during insect migration into an insect imaging zone (see Patch: e.g., --The example insect imaging chamber can represent a minimal baseline for collection of environmental and video data. The example insect imaging chamber can include a tube with two imaging devices (e.g., 12.3 megapixel HD cameras or any other suitable cameras) positioned such that they will capture the top and side views of insects as they crawl through the trap. [0036] To maximize the amount of images with insects, a motion detection module can be implemented. Thus, images that have insects in the frame or image can be only collected and saved to minimize erroneous storage usage.--, in [0035]-[0036]; and, --In some scenarios, the N-frame history buffer can check if a proposed bounding box could be an insect based on movement patterns while the predetermined second window can be a window if there is actually an insect crawling through an insect guide tube 108 in an insect imaging chamber 100 in FIGS. 1-3.--, in [0076]-[0077]).
Re Claim 4, Froloff as modified by Ampatzidis and Patch further disclose wherein the detecting occurs by a motion sensor (see Patch: e.g., --The example insect imaging chamber can represent a minimal baseline for collection of environmental and video data. The example insect imaging chamber can include a tube with two imaging devices (e.g., 12.3 megapixel HD cameras or any other suitable cameras) positioned such that they will capture the top and side views of insects as they crawl through the trap. [0036] To maximize the amount of images with insects, a motion detection module can be implemented. Thus, images that have insects in the frame or image can be only collected and saved to minimize erroneous storage usage.--, in [0035]-[0036]; and, --In some scenarios, the N-frame history buffer can check if a proposed bounding box could be an insect based on movement patterns while the predetermined second window can be a window if there is actually an insect crawling through an insect guide tube 108 in an insect imaging chamber 100 in FIGS. 1-3.--, in [0076]-[0077]).
Re Claim 5, Froloff as modified by Ampatzidis and Patch further disclose wherein the capturing of the at least one image occurs after the motion sensor detects insect movement and signals one or more camcras to initiate the capturing of images in response to the detected insect movement, and wherein the one or more cameras capture at least one image in the insect imaging zone in response to the signaling (see Patch: e.g., --The example insect imaging chamber can represent a minimal baseline for collection of environmental and video data. The example insect imaging chamber can include a tube with two imaging devices (e.g., 12.3 megapixel HD cameras or any other suitable cameras) positioned such that they will capture the top and side views of insects as they crawl through the trap. [0036] To maximize the amount of images with insects, a motion detection module can be implemented. Thus, images that have insects in the frame or image can be only collected and saved to minimize erroneous storage usage.--, in [0035]-[0036]; and, --In some scenarios, the N-frame history buffer can check if a proposed bounding box could be an insect based on movement patterns while the predetermined second window can be a window if there is actually an insect crawling through an insect guide tube 108 in an insect imaging chamber 100 in FIGS. 1-3.--, in [0076]-[0077]).
Re Claim 6, Froloff as modified by Ampatzidis and Patch further disclose wherein the one or more cameras comprise a first camera positioned to capture a top view of insects and a second camera positioned to capture a lateral view of insects; and wherein the at least one image comprises a top-view image captured by the first camera and a lateral-view image captured by the second camera (see Patch: e.g., --The example insect imaging chamber can represent a minimal baseline for collection of environmental and video data. The example insect imaging chamber can include a tube with two imaging devices (e.g., 12.3 megapixel HD cameras or any other suitable cameras) positioned such that they will capture the top and side views of insects as they crawl through the trap. [0036] To maximize the amount of images with insects, a motion detection module can be implemented. Thus, images that have insects in the frame or image can be only collected and saved to minimize erroneous storage usage.--, in [0035]-[0036]; and, --In some scenarios, the N-frame history buffer can check if a proposed bounding box could be an insect based on movement patterns while the predetermined second window can be a window if there is actually an insect crawling through an insect guide tube 108 in an insect imaging chamber 100 in FIGS. 1-3.--, in [0076]-[0077]).
Re Claim 7, Froloff as modified by Ampatzidis and Patch further disclose wherein the one or more cameras transmit the at least one image to the computing device for processing (see Froloff: e.g., --[0067] It is conceivable that a build up of trapped insect showing up over a time period on the traps and on the images, making it more difficult to distinguish individual insects. Where the insect information is not alarming, and alerting candidate insects can potentially show up in traps too crowded to distinguish them, in some embodiments successive sensor image transmission redundancies can encoded out, ie only the differences encoded into a sensor upload transmission, such that only the newly arrived insects or trapped bugs are reported by the sensors. Furthermore compression, such as the Discrete Cosine Transform used in MPEG and JPEG formats, techniques embedded in sensors can be used where sensor image transmission bandwidths are limited…., [0076] Acoustic, vibration and other type sensor data 403 can apply bandpass, threshold and other filters 405 to isolate potential actionable data for particular pests, convert by ADC 407, digitize 409 and frame data for transmission to a sensor hub 413 for forwarding and synchronization with meta data before transmission to an image/data/metadata database 415. The sensors can generally have multiple site locations as well as multiple human monitors which will in short period of time develop stored data. All the data and metadata captured at this point is potentially valuable for a machine learning AI in future data mining use regardless of what is found upstream. This data can be stored, in the cloud or other space, for future sale or use above and beyond the current accumulation of training data and crop monitoring.
[0077] The forwarded image data or local server 417 stored image data 415 are partitioned, separating the bug images views from the plant image view data, forming at least two main data image streams. Where there is no bug image part or plant image part, then partitioning is cropping existing other part The image data is programmatically tagged with available surrounding or associated data and metadata available from the sensors as well as server programmable information such as: [0078] date, time, location, [0079] environmental conditions, [0080] temperature, humidity, water, [0081] region, season, climate cycle time, [0082] crop known pest infesters, larvae images, [0083] biofix or mating initiation cycle/activities, [0084] associated pest mating activity parameters, [0085] etc,--, in [0067]-[0085]; and, --0100] In an embodiment of the invention, a wireless sensor network 501 having low-power integrated circuits, and wireless communication is at the heart of a wireless sensor network 501, with sensor 503 monitoring in real-time or programmed delay updates of integrated sensed data and transmission through a wireless network. Intelligent plant sensors are integrated into a wireless sensor network for early detection of plant or crop conditions. Plant growth and plant disease or damage rates do not generally impair normal processing activities. The sensor information is transmitted wirelessly to an external processing unit at program set or programmed delay rates, ostensibly in application of power saving strategies. Data from acoustic or vibration sensors 503 are collected similarly where data can become graphical and then image data.--, in [0100], and [0105]).
Re Claims 22-23, claims 22-23 are the corresponding system claim to claims 1-2, and 5, 6 respectively. Claims 22-23 thus are rejected for the similar reasons for claims 1-2, 5, and 6. See above discussions with regard to claim 1-2, 5, and 6 respectively. Further, Froloff as modified by Ampatzidis and Patch further disclose system for monitoring insects comprising: one or more cameras operable to perform image capture in an insect imaging zone;
a motion sensor communicably coupled to the one or more cameras and operable to signal the one or more cameras to initiate image capture in response to detection of insect movement into the insect imaging zone; and
a computing device communicably coupled to the one or more cameras,
wherein the computing device comprises an artificial intelligence model operable to identify insects (see Froloff: e.g., Fig. 1, and, -- AI analytic trained automated crop or plant monitoring system, accumulating wireless sensor image data into identifiable labeled image objects for training AI analytics. A plurality of integrated sensors network wirelessly collecting plant and insect primary sensor data have logic for primary sensor data transfer with associated sensor metadata onto a database, and logic for partitioning primary sensor image data into insect pestilent and plant images. Non-expert monitoring demark suspected objects in the data by digitally bounding border demarking an object in a partitioned data image, view display comparison of exemplar images of specific identified insects and plant harms with the demarked image data objects…. Preset minimum numerical count for threshold of training image data for a specified label of positive and negative data image set counts are used as thresholds for the minimum number of images needed accumulated before forwarding the data sets reaching threshold specific image label count as input training data to an AI machine learning program for creating an AI analytic capable of identifying the specific labeled image object in primary sensor data images. The trained executing AI analytic scanning a plurality wireless sensor data images for pestilence and plant objects matching positive trained identified labeled objects, where the analytic program is responsive to monitoring for specifically trained positive identified labeled positive label trained objects identified in sensor data images so that a system with a plurality of wireless integrated sensor network continuously monitoring plants for pestilence and other plant harms can timely raise alerts of found labeled positively identified insects or plant harm without human visual intervention, alerts retaining sensor meta data, time and location of sensor data triggering image alert.--, in [0030], and, -- inspection is automated through wireless sensor networks and variable, image data taken with frequency required. Monitoring insects on integrated sensors can also be done programmatically from remote sensors 101 with wireless networks sending image data to a wireless network hub 105. Alternatively, government pestilence watcher programs set up to inspect for specific and particularly proven harmful pests will have a completely different geographic placement of sensors with wireless network hubs perhaps situated in citizen's homes, and not necessarily for food production. Based on the jurisdiction boundaries, residential plants and trees in need of protection with early warning alerts, state and local governments and associations can set up wireless network sensors with embodiments of the invention as well.--, in [0055]; also see Patch: e.g., --The example insect imaging chamber can represent a minimal baseline for collection of environmental and video data. The example insect imaging chamber can include a tube with two imaging devices (e.g., 12.3 megapixel HD cameras or any other suitable cameras) positioned such that they will capture the top and side views of insects as they crawl through the trap. [0036] To maximize the amount of images with insects, a motion detection module can be implemented. Thus, images that have insects in the frame or image can be only collected and saved to minimize erroneous storage usage.--, in [0035]-[0036]; and, --In some scenarios, the N-frame history buffer can check if a proposed bounding box could be an insect based on movement patterns while the predetermined second window can be a window if there is actually an insect crawling through an insect guide tube 108 in an insect imaging chamber 100 in FIGS. 1-3.--, in [0076]-[0077]).
Re Claim 24, Froloff as modified by Ampatzidis and Patch further disclose comprising a lighting system (see Patch: e.g., -- [0048] In some examples, the first cell 102 is configured to accept an insect. In some examples, the insect can indicate one or more insects of the same and/or different species. For example, the first cell 102 can include a chamber opening 106 such that the one or more insects can enter and/or leave the first cell 102 through the chamber opening 106. The chamber opening 106 can be an empty space or a door on a surface of the first cell 102. In some examples, the chamber opening 106 can further include a step (e.g., a flat or curved surface) for the insect to land on the step before entering the imaging chamber 100. In further examples, the chamber opening 106 may further include a light source (e.g., blue and green lights) to encourage the insect to enter or exit the imaging chamber 100. However, the light source can be disposed at any place on the imaging chamber 100 to attract the insect to enter or leave from the first cell 102 through the chamber opening 106.--, in [0048]).
Re Claim 25, Froloff as modified by Ampatzidis and Patch further disclose wherein the lighting system comprises one or more lights, wherein the one or more cameras and the one or more lights are timed via a hardware trigger such that the one or more cameras capture the at least one image at approximately the same time as the one or more lights flash (see Patch: e.g., -- [0048] In some examples, the first cell 102 is configured to accept an insect. In some examples, the insect can indicate one or more insects of the same and/or different species. For example, the first cell 102 can include a chamber opening 106 such that the one or more insects can enter and/or leave the first cell 102 through the chamber opening 106. The chamber opening 106 can be an empty space or a door on a surface of the first cell 102. In some examples, the chamber opening 106 can further include a step (e.g., a flat or curved surface) for the insect to land on the step before entering the imaging chamber 100. In further examples, the chamber opening 106 may further include a light source (e.g., blue and green lights) to encourage the insect to enter or exit the imaging chamber 100. However, the light source can be disposed at any place on the imaging chamber 100 to attract the insect to enter or leave from the first cell 102 through the chamber opening 106.--, in [0048]).
Re Claim 26, Froloff as modified by Ampatzidis and Patch further disclose insect attracting system (see Froloff: e.g., Fig. 7, ., --Sticky traps can provide warning of pest presence before plant damage becomes devastating, alerting owners, farmers, counties, cities, regional entities, to step up visual inspections. Some entire counties fearing infestation will implement programs that sparsely located data from sticky traps in residential communities as part of a community service. Once pests are confirmed in to be in the area, slower acting control strategies can be utilized that are more environmentally-friendly and safer for people.
[0006] Sticky traps can also indicate greenhouse hot spots and can document pestilence migration patterns when placed near doors and vents. Data collected from sticky traps can be used to evaluate pest management control actions, including the use of natural enemies.--, in [0005]-[0006]; and, --Placement of sensor traps is optimal for identifying characteristics of insect/bug/pest trapped on sticky traps 207 view as well as plant or foliage view area 209. The optical sensors 211 placement provides a way to capture image data with view an integrated sticky card 207 with and area having foliage or plant 209 view area adjacent. Imaging both sticky area and crop areas increases the integrated sticky sensor's capability in delivering more comprehensive image data for later processing. Wire, fiber or twisted filament material can be used for the stays 213 providing a sensor position having the requisite view area 209 of the plant as well as the insects.--, in [0059]).
Re Claim 27, Froloff as modified by Ampatzidis and Patch further disclose wherein the insect attracting system comprises a light trap (see Froloff: e.g., Fig. 7, ., --Sticky traps can provide warning of pest presence before plant damage becomes devastating, alerting owners, farmers, counties, cities, regional entities, to step up visual inspections. Some entire counties fearing infestation will implement programs that sparsely located data from sticky traps in residential communities as part of a community service. Once pests are confirmed in to be in the area, slower acting control strategies can be utilized that are more environmentally-friendly and safer for people.
[0006] Sticky traps can also indicate greenhouse hot spots and can document pestilence migration patterns when placed near doors and vents. Data collected from sticky traps can be used to evaluate pest management control actions, including the use of natural enemies.--, in [0005]-[0006]; and, --Placement of sensor traps is optimal for identifying characteristics of insect/bug/pest trapped on sticky traps 207 view as well as plant or foliage view area 209. The optical sensors 211 placement provides a way to capture image data with view an integrated sticky card 207 with and area having foliage or plant 209 view area adjacent. Imaging both sticky area and crop areas increases the integrated sticky sensor's capability in delivering more comprehensive image data for later processing. Wire, fiber or twisted filament material can be used for the stays 213 providing a sensor position having the requisite view area 209 of the plant as well as the insects.--, in [0059]; also see Patch: e.g., -- [0048] In some examples, the first cell 102 is configured to accept an insect. In some examples, the insect can indicate one or more insects of the same and/or different species. For example, the first cell 102 can include a chamber opening 106 such that the one or more insects can enter and/or leave the first cell 102 through the chamber opening 106. The chamber opening 106 can be an empty space or a door on a surface of the first cell 102. In some examples, the chamber opening 106 can further include a step (e.g., a flat or curved surface) for the insect to land on the step before entering the imaging chamber 100. In further examples, the chamber opening 106 may further include a light source (e.g., blue and green lights) to encourage the insect to enter or exit the imaging chamber 100. However, the light source can be disposed at any place on the imaging chamber 100 to attract the insect to enter or leave from the first cell 102 through the chamber opening 106.--, in [0048]).
Re Claim 28, Froloff as modified by Ampatzidis and Patch further disclose wherein the insect attracting system further comprises one or more semiochemicals to attract insects (see Froloff: e.g., Fig. 7, ., --Sticky traps can provide warning of pest presence before plant damage becomes devastating, alerting owners, farmers, counties, cities, regional entities, to step up visual inspections. Some entire counties fearing infestation will implement programs that sparsely located data from sticky traps in residential communities as part of a community service. Once pests are confirmed in to be in the area, slower acting control strategies can be utilized that are more environmentally-friendly and safer for people.
[0006] Sticky traps can also indicate greenhouse hot spots and can document pestilence migration patterns when placed near doors and vents. Data collected from sticky traps can be used to evaluate pest management control actions, including the use of natural enemies.--, in [0005]-[0006]; and, --Placement of sensor traps is optimal for identifying characteristics of insect/bug/pest trapped on sticky traps 207 view as well as plant or foliage view area 209. The optical sensors 211 placement provides a way to capture image data with view an integrated sticky card 207 with and area having foliage or plant 209 view area adjacent. Imaging both sticky area and crop areas increases the integrated sticky sensor's capability in delivering more comprehensive image data for later processing. Wire, fiber or twisted filament material can be used for the stays 213 providing a sensor position having the requisite view area 209 of the plant as well as the insects.--, in [0059]; also see Patch: e.g., -- [0048] In some examples, the first cell 102 is configured to accept an insect. In some examples, the insect can indicate one or more insects of the same and/or different species. For example, the first cell 102 can include a chamber opening 106 such that the one or more insects can enter and/or leave the first cell 102 through the chamber opening 106. The chamber opening 106 can be an empty space or a door on a surface of the first cell 102. In some examples, the chamber opening 106 can further include a step (e.g., a flat or curved surface) for the insect to land on the step before entering the imaging chamber 100. In further examples, the chamber opening 106 may further include a light source (e.g., blue and green lights) to encourage the insect to enter or exit the imaging chamber 100. However, the light source can be disposed at any place on the imaging chamber 100 to attract the insect to enter or leave from the first cell 102 through the chamber opening 106.--, in [0048]).
Re Claim 29, Froloff as modified by Ampatzidis and Patch further disclose a power supply, wherein the power supply is operable to provide energy to the system (see Froloff: e.g., Fig. 7, ., --Sticky traps can provide warning of pest presence before plant damage becomes devastating, alerting owners, farmers, counties, cities, regional entities, to step up visual inspections. Some entire counties fearing infestation will implement programs that sparsely located data from sticky traps in residential communities as part of a community service. Once pests are confirmed in to be in the area, slower acting control strategies can be utilized that are more environmentally-friendly and safer for people.
[0006] Sticky traps can also indicate greenhouse hot spots and can document pestilence migration patterns when placed near doors and vents. Data collected from sticky traps can be used to evaluate pest management control actions, including the use of natural enemies.--, in [0005]-[0006]; and, --Placement of sensor traps is optimal for identifying characteristics of insect/bug/pest trapped on sticky traps 207 view as well as plant or foliage view area 209. The optical sensors 211 placement provides a way to capture image data with view an integrated sticky card 207 with and area having foliage or plant 209 view area adjacent. Imaging both sticky area and crop areas increases the integrated sticky sensor's capability in delivering more comprehensive image data for later processing. Wire, fiber or twisted filament material can be used for the stays 213 providing a sensor position having the requisite view area 209 of the plant as well as the insects.--, in [0059]; also see Patch: e.g., -- [0048] In some examples, the first cell 102 is configured to accept an insect. In some examples, the insect can indicate one or more insects of the same and/or different species. For example, the first cell 102 can include a chamber opening 106 such that the one or more insects can enter and/or leave the first cell 102 through the chamber opening 106. The chamber opening 106 can be an empty space or a door on a surface of the first cell 102. In some examples, the chamber opening 106 can further include a step (e.g., a flat or curved surface) for the insect to land on the step before entering the imaging chamber 100. In further examples, the chamber opening 106 may further include a light source (e.g., blue and green lights) to encourage the insect to enter or exit the imaging chamber 100. However, the light source can be disposed at any place on the imaging chamber 100 to attract the insect to enter or leave from the first cell 102 through the chamber opening 106.--, in [0048]).
Claims 8-14 are rejected under 35 U.S.C. 103 as being unpatentable over Froloff as modified by Ampatzidis, and in view of Ghazvinian Zanjani (US 20220383114 A1, claims the priority of US-Provisional-Application US 63194323 20210528).
Re Claim 8, Froloff as modified by Ampatzidis wherein the unsupervised domain adaptation technique for training the artificial intelligence model comprises:
training the artificial intelligence model and a classifier on a source dataset in a source domain (see Froloff: e.g., --[0019] Machine learning is a field of computer science that uses statistical techniques to give computer systems the ability to “learn”, progressively improve its ability to identify the target labeled image entity sought. These can come from Supervised or Unsupervised methods.--, in [0019], and, --0094] A training set positive, negative and test minimum required for adequate training set numbers 431 are obtained from ML experts for gathering a sufficient number of labeled member image data for each particular exemplar category sought. This can also be described as collected training set number of images to be submitted by the non-experts 421 to be labeled and representative of what would be expected to be found in the sensor placed setting, as above in the exemplar images identified/classified by domain experts.--, in [0094]; and,
--Thus adapting a trained executing AI analytic will scan the plurality wireless sensor data images for pestilence and plant objects matching positive trained identified labeled objects with logic responsive to monitoring for positive identified labeled objects raising alerts for positive label trained objects identified in sensor data images and timely raise alerts of found labeled positively identified insects or plant harm without human visual intervention, alerts retaining sensor meta data, time and location of sensor data triggering image alert.),--, in [0105], [0107], and [0110]; and,
--[0115] Many Object Detection Classifier algorithms are available to train AIs from training data input. These are installed for executed using the training data sets provided. There are pre-trained object detection models available as well. These or others are used train on embodiments of the invention training data.
[0116] Training data files must be configured and made compatible to the AI program input specs in size and format. There are training data, positive and negative sets, and test data image sets. There are also image labeling programs to help prepare the training data image sets Where there are more than one image of interest in a single data image, multiple labeled images can be saved in independent files for training from a single data image. A preliminary step in the AI process is to create a label map, mapping class names to class ids, and configuring the training data for the object classifier. Once all of the training data labeled, mapped and classified we can train the object detector.--, in [0115]-[0116]; and,
--The image data will have bug pest view area and plant view area. Trained AI analytics for identifying specific image objects as pest or plant pestilence with a set degree of certainty 519 will label the object and signal an alert 533 to the local monitor warning them of the possible intrusion, providing the object identified and associated meta data and confirming data from any associations.--, in [0107]; and, -- The trainer can use any of the typical AI machine learning implemented algorithms for identifying image objects including Dimensionality reduction, Ensemble learning, Meta learning, Reinforcement learning, Supervised learning, Unsupervised learning, Semi-supervised learning, and Deep learning. Next, Export inference graph 611, copy the image sets to the training directory and the text directory.
[0123] Lastly, test the inference on the test or validating data set 613. Over fitting/overtraining in supervised learning is determined as follows. Training error is graphed against validation error, both as a function of the number of training cycles. If the validation error increases, positive slope, while the training error steadily decreases, negative slope, then a situation of over fitting may have occurred. The best predictive and fitted model would be where the validation error has its global minimum.
[0124] Any machine learning methods including but not limited to Dimensionality reduction, Ensemble learning, Meta learning, Reinforcement learning, Supervised learning, Unsupervised learning, Semi-supervised learning, Deep learning, and other machine learning methods can be used where applicable to a candidate pest or plant harm object identification.--, in [0122]-[0124]);
adapting knowledge learned on the source domain to a target domain via unsupervised domain adaptive training, wherein the unsupervised adaptive training (see Froloff: e.g., --[0019] Machine learning is a field of computer science that uses statistical techniques to give computer systems the ability to “learn”, progressively improve its ability to identify the target labeled image entity sought. These can come from Supervised or Unsupervised methods.--, in [0019], and,
--Thus adapting a trained executing AI analytic will scan the plurality wireless sensor data images for pestilence and plant objects matching positive trained identified labeled objects with logic responsive to monitoring for positive identified labeled objects raising alerts for positive label trained objects identified in sensor data images and timely raise alerts of found labeled positively identified insects or plant harm without human visual intervention, alerts retaining sensor meta data, time and location of sensor data triggering image alert.),--, in [0105], and, --The image data will have bug pest view area and plant view area. Trained AI analytics for identifying specific image objects as pest or plant pestilence with a set degree of certainty 519 will label the object and signal an alert 533 to the local monitor warning them of the possible intrusion, providing the object identified and associated meta data and confirming data from any associations.--, in [0107]; and,
--[0113] In machine learning, algorithms work by making sensor image data-driven predictions or decisions through building a mathematical model from the input input training data. In an embodiment of the invention the data used to build the final program will come from multiple datasets of algorithm sufficient number of images. As mentioned for above embodiments in particular, three data sets of images are commonly used in different stages of the creation of a model.
[0114] Furthermore, an ML knowledgeable individual upon scanning a specific exemplar for features and some candidate training input data will determine the structure of the learned function and corresponding learning algorithm. For example, the selection may algorithm may be support vector machines or decision trees. Learning algorithm selection can address bias-variance tradeoff, function complexity and amount of training data, dimensionality of the input space, noise in the output values and other factors.
[0115] Many Object Detection Classifier algorithms are available to train AIs from training data input. These are installed for executed using the training data sets provided. There are pre-trained object detection models available as well. These or others are used train on embodiments of the invention training data.
[0116] Training data files must be configured and made compatible to the AI program input specs in size and format. There are training data, positive and negative sets, and test data image sets. There are also image labeling programs to help prepare the training data image sets Where there are more than one image of interest in a single data image, multiple labeled images can be saved in independent files for training from a single data image. A preliminary step in the AI process is to create a label map, mapping class names to class ids, and configuring the training data for the object classifier. Once all of the training data labeled, mapped and classified we can train the object detector.--, in [0113]-[0116]; and,
-- The trainer can use any of the typical AI machine learning implemented algorithms for identifying image objects including Dimensionality reduction, Ensemble learning, Meta learning, Reinforcement learning, Supervised learning, Unsupervised learning, Semi-supervised learning, and Deep learning. Next, Export inference graph 611, copy the image sets to the training directory and the text directory.
[0123] Lastly, test the inference on the test or validating data set 613. Over fitting/overtraining in supervised learning is determined as follows. Training error is graphed against validation error, both as a function of the number of training cycles. If the validation error increases, positive slope, while the training error steadily decreases, negative slope, then a situation of over fitting may have occurred. The best predictive and fitted model would be where the validation error has its global minimum.
[0124] Any machine learning methods including but not limited to Dimensionality reduction, Ensemble learning, Meta learning, Reinforcement learning, Supervised learning, Unsupervised learning, Semi-supervised learning, Deep learning, and other machine learning methods can be used where applicable to a candidate pest or plant harm object identification.--, in [0122]-[0124]) comprises:
projecting features that are on at least two domains into one-dimensional space (see Froloff: e.g., --[0019] Machine learning is a field of computer science that uses statistical techniques to give computer systems the ability to “learn”, progressively improve its ability to identify the target labeled image entity sought. These can come from Supervised or Unsupervised methods.--, in [0019], and,
--Thus adapting a trained executing AI analytic will scan the plurality wireless sensor data images for pestilence and plant objects matching positive trained identified labeled objects with logic responsive to monitoring for positive identified labeled objects raising alerts for positive label trained objects identified in sensor data images and timely raise alerts of found labeled positively identified insects or plant harm without human visual intervention, alerts retaining sensor meta data, time and location of sensor data triggering image alert.),--, in [0105], and, --The image data will have bug pest view area and plant view area. Trained AI analytics for identifying specific image objects as pest or plant pestilence with a set degree of certainty 519 will label the object and signal an alert 533 to the local monitor warning them of the possible intrusion, providing the object identified and associated meta data and confirming data from any associations.--, in [0107]; and,
--[0113] In machine learning, algorithms work by making sensor image data-driven predictions or decisions through building a mathematical model from the input input training data. In an embodiment of the invention the data used to build the final program will come from multiple datasets of algorithm sufficient number of images. As mentioned for above embodiments in particular, three data sets of images are commonly used in different stages of the creation of a model.
[0114] Furthermore, an ML knowledgeable individual upon scanning a specific exemplar for features and some candidate training input data will determine the structure of the learned function and corresponding learning algorithm. For example, the selection may algorithm may be support vector machines or decision trees. Learning algorithm selection can address bias-variance tradeoff, function complexity and amount of training data, dimensionality of the input space, noise in the output values and other factors.
[0115] Many Object Detection Classifier algorithms are available to train AIs from training data input. These are installed for executed using the training data sets provided. There are pre-trained object detection models available as well. These or others are used train on embodiments of the invention training data.
[0116] Training data files must be configured and made compatible to the AI program input specs in size and format. There are training data, positive and negative sets, and test data image sets. There are also image labeling programs to help prepare the training data image sets Where there are more than one image of interest in a single data image, multiple labeled images can be saved in independent files for training from a single data image. A preliminary step in the AI process is to create a label map, mapping class names to class ids, and configuring the training data for the object classifier. Once all of the training data labeled, mapped and classified we can train the object detector.--, in [0113]-[0116]; and,
-- The trainer can use any of the typical AI machine learning implemented algorithms for identifying image objects including Dimensionality reduction, Ensemble learning, Meta learning, Reinforcement learning, Supervised learning, Unsupervised learning, Semi-supervised learning, and Deep learning. Next, Export inference graph 611, copy the image sets to the training directory and the text directory.
[0123] Lastly, test the inference on the test or validating data set 613. Over fitting/overtraining in supervised learning is determined as follows. Training error is graphed against validation error, both as a function of the number of training cycles. If the validation error increases, positive slope, while the training error steadily decreases, negative slope, then a situation of over fitting may have occurred. The best predictive and fitted model would be where the validation error has its global minimum.
[0124] Any machine learning methods including but not limited to Dimensionality reduction, Ensemble learning, Meta learning, Reinforcement learning, Supervised learning, Unsupervised learning, Semi-supervised learning, Deep learning, and other machine learning methods can be used where applicable to a candidate pest or plant harm object identification.--, in [0122]-[0124]);
Froloff as modified by Ampatzidis however do not explicitly disclose computing a plurality of Gromov-Wasserstein distances on the one-dimensional space, and determining a sliced Gromov-Wasserstein distance based at least partly on an average of the plurality of Gromov-Wasserstein distances;
Ghazvinian Zanjani discloses computing a plurality of Gromov-Wasserstein distances on the one-dimensional space, and determining a sliced Gromov-Wasserstein distance based at least partly on an average of the plurality of Gromov-Wasserstein distances (see Ghazvinian Zanjani: e.g., -- [0040] Training the map Φ by minimizing the ∥D.sub.s−D.sub.i∥.sup.2 and using gradient descent optimization, leads to a parametric approximation of the multidimensional scaling (MDS) algorithm. However, this formulation is ill-posed when X.sub.s is a non-convex set since comparing the geodesic distances with Euclidean distances by using ∥.∥.sup.2 (i.e., the L2 norm or root mean-squared error) is only valid inside a convex region. Unfortunately, this is not the case in many real applications, such as indoor localization, where the sample set is, for example, collected from several zones/rooms that are partitioned with walls and other obstacles.
[0041] In a localization task, finding the map Φ for representing the input samples in their intrinsic space is not sufficient by itself; a transformation between the embedding in the intrinsic space and the target space Ω.sub.t (e.g., the target topological map) needs to be found. This can be a challenge since the correspondences between these two domains are unknown. However, the Gromov-Wasserstein discrepancy for measuring the dissimilarity between two distance matrices may be used for solving the correspondence problem. In this sense, the correspondences (coupling) between the entries of two distance matrices are found by performing a regularized optimal transport between these two spaces.--, in [0040]-[0041]; and, -- 0085] In some aspects, determining parameters of the neural network configured to map samples in the input space based on the input data to samples in the intrinsic space comprises minimizing a difference between a distance matrix associated with the input space and a distance matrix associated with the intrinsic space.
[0086] In some aspects, minimizing a difference between the distance matrix associated with the input space and the distance matrix associated with the intrinsic space comprises minimizing a dissimilarity measure between the distance matrix associated with the input space and the distance matrix associated with the intrinsic space via an optimal transport coupling matrix. In some aspects, the dissimilarity measure comprises a Gromov-Wasserstein discrepancy measure.--, in [0086]-[0090]; also see: --[0030] When data are associated with geometrical properties, optimal transport metrics (also called Wasserstein distance or Earth Mover distance) measure the spatial variations between probability distributions of source and target domains. Correspondent matching is one example application of optimal transport. Given a transport cost function, the Wasserstein distance computes the optimal transportation plan between two measures. Recent progress on efficient computing of optimal transport by introducing entropy regularization and the Sinkhorn's matrix scaling algorithm reduced the computational cost of optimal transport several orders of magnitude compared to the original transport solver. In particular, it has been shown that computing the optimal transportation loss and its gradient can be tractable by using Sinkhorn fixed-point iterations.--, in [0030]);
Froloff (as modified by Ampatzidis) and Ghazvinian Zanjani are combinable as they are in the same field of endeavor: using machine learning in image processing for detecting and identifying target objects. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify Froloff (as modified by Ampatzidis)’s method using Ghazvinian Zanjani’s teachings by including computing a plurality of Gromov-Wasserstein distances on the one-dimensional space, and determining a sliced Gromov-Wasserstein distance based at least partly on an average of the plurality of Gromov-Wasserstein distances to Froloff (as modified by Ampatzidis)’s optimizing parameters of machine learning models for identifying the objects in order to determining parameters of the neural network configured to map samples in the input space based on the input data to samples in the intrinsic space comprises minimizing a difference between a distance matrix associated with the input space and a distance matrix associated with the intrinsic space (see Ghazvinian Zanjani: e.g. in [0030], [0040]-[0041], and [0086]-[0090]);
deploying the artificial intelligence model in the target domain in response to the adapting (see Ghazvinian Zanjani: e.g., --training a machine learning model based on input data for performing localization of an object in a target space, including: determining parameters of a neural network configured to map samples in an input space based on the input data to samples in an intrinsic space; and determining parameters of a coupling matrix configured to transport the samples in the intrinsic space to the target space.--, in abstract, and, --[0030] When data are associated with geometrical properties, optimal transport metrics (also called Wasserstein distance or Earth Mover distance) measure the spatial variations between probability distributions of source and target domains. Correspondent matching is one example application of optimal transport. Given a transport cost function, the Wasserstein distance computes the optimal transportation plan between two measures. Recent progress on efficient computing of optimal transport by introducing entropy regularization and the Sinkhorn's matrix scaling algorithm reduced the computational cost of optimal transport several orders of magnitude compared to the original transport solver. In particular, it has been shown that computing the optimal transportation loss and its gradient can be tractable by using Sinkhorn fixed-point iterations.--, in [0030]).
Re Claim 9, Froloff as modified by Ampatzidis and Ghazvinian Zanjani further disclose wherein the determined Gromov-Wasserstein distance aligns and associates features between the source domain and the target domain (see Ghazvinian Zanjani: e.g., --training a machine learning model based on input data for performing localization of an object in a target space, including: determining parameters of a neural network configured to map samples in an input space based on the input data to samples in an intrinsic space; and determining parameters of a coupling matrix configured to transport the samples in the intrinsic space to the target space.--, in abstract, and, --[0030] When data are associated with geometrical properties, optimal transport metrics (also called Wasserstein distance or Earth Mover distance) measure the spatial variations between probability distributions of source and target domains. Correspondent matching is one example application of optimal transport. Given a transport cost function, the Wasserstein distance computes the optimal transportation plan between two measures. Recent progress on efficient computing of optimal transport by introducing entropy regularization and the Sinkhorn's matrix scaling algorithm reduced the computational cost of optimal transport several orders of magnitude compared to the original transport solver. In particular, it has been shown that computing the optimal transportation loss and its gradient can be tractable by using Sinkhorn fixed-point iterations.--, in [0030]; -- [0040] Training the map Φ by minimizing the ∥D.sub.s−D.sub.i∥.sup.2 and using gradient descent optimization, leads to a parametric approximation of the multidimensional scaling (MDS) algorithm. However, this formulation is ill-posed when X.sub.s is a non-convex set since comparing the geodesic distances with Euclidean distances by using ∥.∥.sup.2 (i.e., the L2 norm or root mean-squared error) is only valid inside a convex region. Unfortunately, this is not the case in many real applications, such as indoor localization, where the sample set is, for example, collected from several zones/rooms that are partitioned with walls and other obstacles.
[0041] In a localization task, finding the map Φ for representing the input samples in their intrinsic space is not sufficient by itself; a transformation between the embedding in the intrinsic space and the target space Ω.sub.t (e.g., the target topological map) needs to be found. This can be a challenge since the correspondences between these two domains are unknown. However, the Gromov-Wasserstein discrepancy for measuring the dissimilarity between two distance matrices may be used for solving the correspondence problem. In this sense, the correspondences (coupling) between the entries of two distance matrices are found by performing a regularized optimal transport between these two spaces.--, in [0040]-[0041]; and, -- 0085] In some aspects, determining parameters of the neural network configured to map samples in the input space based on the input data to samples in the intrinsic space comprises minimizing a difference between a distance matrix associated with the input space and a distance matrix associated with the intrinsic space.
[0086] In some aspects, minimizing a difference between the distance matrix associated with the input space and the distance matrix associated with the intrinsic space comprises minimizing a dissimilarity measure between the distance matrix associated with the input space and the distance matrix associated with the intrinsic space via an optimal transport coupling matrix. In some aspects, the dissimilarity measure comprises a Gromov-Wasserstein discrepancy measure.--, in [0086]-[0090]).
Re Claim 10, Froloff as modified by Ampatzidis and Ghazvinian Zanjaniwherein further disclose the alignment reduces topological differences of feature distributions between the source domain and the target domain (see Ghazvinian Zanjani: e.g., -- aligning the intrinsic embedding with the target domain in an unsupervised style. Consequently, it is instead desirable to learn the intrinsic embedding and the transformation jointly without having access to the two-dimensional positions (e.g., (x,y)) of the object on the map, which may not be known. Notably, finding the transformation as a solution of the optimal transport problem and minimizing its cost constrains the topology of the embedding to resemble the target space Ω.sub.t (e.g., the topological map).--, in [0037]).
Re Claim 11, Froloff as modified by Ampatzidis and Ghazvinian Zanjani further disclose wherein the unsupervised domain adaptation technique for training the artificial intelligence model comprises training the artificial intelligence model on labeled data from the source domain to achieve better performance on data from the target domain with access to only unlabeled data in the target domain (see Froloff: e.g., --[0019] Machine learning is a field of computer science that uses statistical techniques to give computer systems the ability to “learn”, progressively improve its ability to identify the target labeled image entity sought. These can come from Supervised or Unsupervised methods.--, in [0019], and,
--Thus adapting a trained executing AI analytic will scan the plurality wireless sensor data images for pestilence and plant objects matching positive trained identified labeled objects with logic responsive to monitoring for positive identified labeled objects raising alerts for positive label trained objects identified in sensor data images and timely raise alerts of found labeled positively identified insects or plant harm without human visual intervention, alerts retaining sensor meta data, time and location of sensor data triggering image alert.),--, in [0105], and, --The image data will have bug pest view area and plant view area. Trained AI analytics for identifying specific image objects as pest or plant pestilence with a set degree of certainty 519 will label the object and signal an alert 533 to the local monitor warning them of the possible intrusion, providing the object identified and associated meta data and confirming data from any associations.--, in [0107]; and,
--[0113] In machine learning, algorithms work by making sensor image data-driven predictions or decisions through building a mathematical model from the input input training data. In an embodiment of the invention the data used to build the final program will come from multiple datasets of algorithm sufficient number of images. As mentioned for above embodiments in particular, three data sets of images are commonly used in different stages of the creation of a model.
[0114] Furthermore, an ML knowledgeable individual upon scanning a specific exemplar for features and some candidate training input data will determine the structure of the learned function and corresponding learning algorithm. For example, the selection may algorithm may be support vector machines or decision trees. Learning algorithm selection can address bias-variance tradeoff, function complexity and amount of training data, dimensionality of the input space, noise in the output values and other factors.
[0115] Many Object Detection Classifier algorithms are available to train AIs from training data input. These are installed for executed using the training data sets provided. There are pre-trained object detection models available as well. These or others are used train on embodiments of the invention training data.
[0116] Training data files must be configured and made compatible to the AI program input specs in size and format. There are training data, positive and negative sets, and test data image sets. There are also image labeling programs to help prepare the training data image sets Where there are more than one image of interest in a single data image, multiple labeled images can be saved in independent files for training from a single data image. A preliminary step in the AI process is to create a label map, mapping class names to class ids, and configuring the training data for the object classifier. Once all of the training data labeled, mapped and classified we can train the object detector.--, in [0113]-[0116]; also see Ghazvinian Zanjani: e.g., --training a machine learning model based on input data for performing localization of an object in a target space, including: determining parameters of a neural network configured to map samples in an input space based on the input data to samples in an intrinsic space; and determining parameters of a coupling matrix configured to transport the samples in the intrinsic space to the target space.--, in abstract, and, --[0030] When data are associated with geometrical properties, optimal transport metrics (also called Wasserstein distance or Earth Mover distance) measure the spatial variations between probability distributions of source and target domains. Correspondent matching is one example application of optimal transport. Given a transport cost function, the Wasserstein distance computes the optimal transportation plan between two measures. Recent progress on efficient computing of optimal transport by introducing entropy regularization and the Sinkhorn's matrix scaling algorithm reduced the computational cost of optimal transport several orders of magnitude compared to the original transport solver. In particular, it has been shown that computing the optimal transportation loss and its gradient can be tractable by using Sinkhorn fixed-point iterations.--, in [0030], and [0037]; -- [0040] Training the map Φ by minimizing the ∥D.sub.s−D.sub.i∥.sup.2 and using gradient descent optimization, leads to a parametric approximation of the multidimensional scaling (MDS) algorithm. However, this formulation is ill-posed when X.sub.s is a non-convex set since comparing the geodesic distances with Euclidean distances by using ∥.∥.sup.2 (i.e., the L2 norm or root mean-squared error) is only valid inside a convex region. Unfortunately, this is not the case in many real applications, such as indoor localization, where the sample set is, for example, collected from several zones/rooms that are partitioned with walls and other obstacles.
[0041] In a localization task, finding the map Φ for representing the input samples in their intrinsic space is not sufficient by itself; a transformation between the embedding in the intrinsic space and the target space Ω.sub.t (e.g., the target topological map) needs to be found. This can be a challenge since the correspondences between these two domains are unknown. However, the Gromov-Wasserstein discrepancy for measuring the dissimilarity between two distance matrices may be used for solving the correspondence problem. In this sense, the correspondences (coupling) between the entries of two distance matrices are found by performing a regularized optimal transport between these two spaces.--, in [0040]-[0041]; and, -- 0085] In some aspects, determining parameters of the neural network configured to map samples in the input space based on the input data to samples in the intrinsic space comprises minimizing a difference between a distance matrix associated with the input space and a distance matrix associated with the intrinsic space.
[0086] In some aspects, minimizing a difference between the distance matrix associated with the input space and the distance matrix associated with the intrinsic space comprises minimizing a dissimilarity measure between the distance matrix associated with the input space and the distance matrix associated with the intrinsic space via an optimal transport coupling matrix. In some aspects, the dissimilarity measure comprises a Gromov-Wasserstein discrepancy measure.--, in [0086]-[0090])
Re Claim 12, Froloff as modified by Ampatzidis and Ghazvinian Zanjani further disclose wherein the classifier is a convolutional neural network (CNN) algorithm (see Ampatzidis: e.g., -- The image analysis application 260 may be executed to analyze images 236 captured from the camera(s) 251 (with or without micro lenses) to distinguish, identify, and count the pests from the images using a trained deep-learning convolutional neural network(s) (DL-CNN) or similar machine learning techniques (Artificial Intelligence). The DL-CNNs may be based on models generated by machine learning using a collection of images…. he images 236 can include a collection of images collected by the ATDS 203 and/or images collected by other devices. The images 236 can be used by the image analysis application 260 in the analysis of images of the viewing stage 248 that are captured by the camera(s) 251 for the detection, identification, and counting of the pests deposited on the viewing stage 248. The images 236 can be used in retraining and/or refining the machine learning models implemented by the ATDS computing device 242 and/or any other computing device as can be appreciated.
[0043] The identification rules 269 include rules and/or configuration data for the various algorithms and/or machine learning models used to detect, identify, and/or count the pests in each captured image --, in [0036], [0039], and [0042]-[0043]; and, -- A method, comprising: agitating, via a shaking system of an automated tap and detection system, a vegetation to displace material from the vegetation onto a viewing stage of the automated tap and detection system; capturing, via a camera system of the automated tap and detection system, one or more images of the viewing stage; removing, via a removal system of the automated tap and detection system, the material from the viewing stage; analyzing, via at least one computing device of the automated tap and detection system, the one or more images to detect one or more insects, identify the one or more insects, and count the one or more insects; and generating, via the at least one computing device, insect data comprising an identification of the one or more insects and a number of the one or more insects.--, in claim 11).
Re Claim 13, Froloff as modified by Ampatzidis and Ghazvinian Zanjaniwherein further disclose the CNN algorithm is selected from the group consisting of Region-based CNN (R-CNN) algorithms, Fast R-CNN algorithms, rotated CNN algorithms, mask CNN algorithms, and combinations thereof (see Ampatzidis: e.g., -- The image analysis application 260 may be executed to analyze images 236 captured from the camera(s) 251 (with or without micro lenses) to distinguish, identify, and count the pests from the images using a trained deep-learning convolutional neural network(s) (DL-CNN) or similar machine learning techniques (Artificial Intelligence). The DL-CNNs may be based on models generated by machine learning using a collection of images…. he images 236 can include a collection of images collected by the ATDS 203 and/or images collected by other devices. The images 236 can be used by the image analysis application 260 in the analysis of images of the viewing stage 248 that are captured by the camera(s) 251 for the detection, identification, and counting of the pests deposited on the viewing stage 248. The images 236 can be used in retraining and/or refining the machine learning models implemented by the ATDS computing device 242 and/or any other computing device as can be appreciated.
[0043] The identification rules 269 include rules and/or configuration data for the various algorithms and/or machine learning models used to detect, identify, and/or count the pests in each captured image --, in [0036], [0039], and [0042]-[0043]; and, -- A method, comprising: agitating, via a shaking system of an automated tap and detection system, a vegetation to displace material from the vegetation onto a viewing stage of the automated tap and detection system; capturing, via a camera system of the automated tap and detection system, one or more images of the viewing stage; removing, via a removal system of the automated tap and detection system, the material from the viewing stage; analyzing, via at least one computing device of the automated tap and detection system, the one or more images to detect one or more insects, identify the one or more insects, and count the one or more insects; and generating, via the at least one computing device, insect data comprising an identification of the one or more insects and a number of the one or more insects.--, in claim 11).
Re Claim 14, Froloff as modified by Ampatzidis further disclose wherein the insect data comprises the identity of the one or more insects, the number of the one or more insects, the gender of the one or more insects, or combinations thereof (see Ampatzidis: e.g., -- The image analysis application 260 may be executed to analyze images 236 captured from the camera(s) 251 (with or without micro lenses) to distinguish, identify, and count the pests from the images using a trained deep-learning convolutional neural network(s) (DL-CNN) or similar machine learning techniques (Artificial Intelligence). The DL-CNNs may be based on models generated by machine learning using a collection of images…. he images 236 can include a collection of images collected by the ATDS 203 and/or images collected by other devices. The images 236 can be used by the image analysis application 260 in the analysis of images of the viewing stage 248 that are captured by the camera(s) 251 for the detection, identification, and counting of the pests deposited on the viewing stage 248. The images 236 can be used in retraining and/or refining the machine learning models implemented by the ATDS computing device 242 and/or any other computing device as can be appreciated.
[0043] The identification rules 269 include rules and/or configuration data for the various algorithms and/or machine learning models used to detect, identify, and/or count the pests in each captured image --, in [0036], [0039], and [0042]-[0043]; and, -- A method, comprising: agitating, via a shaking system of an automated tap and detection system, a vegetation to displace material from the vegetation onto a viewing stage of the automated tap and detection system; capturing, via a camera system of the automated tap and detection system, one or more images of the viewing stage; removing, via a removal system of the automated tap and detection system, the material from the viewing stage; analyzing, via at least one computing device of the automated tap and detection system, the one or more images to detect one or more insects, identify the one or more insects, and count the one or more insects; and generating, via the at least one computing device, insect data comprising an identification of the one or more insects and a number of the one or more insects.--, in claim 11).
Claims 30-37 are rejected under 35 U.S.C. 103 as being unpatentable over Froloff as modified by Ampatzidis, Patch and Ghazvinian Zanjani.
Re Claims 30-35, claims 30-35 are the corresponding system claim to claims 8-13 respectively. Claims 30-35 thus are rejected for the similar reasons for claims 8-13. See above discussions with regard to claim 8-13 respectively. Further, Froloff as modified by Ampatzidis, Patch, and Ghazvinian Zanjani further disclose system for monitoring insects.
Re Claim 36, Froloff as modified by Ampatzidis, Patch, and Ghazvinian Zanjani further disclose wherein the system further comprises a dispenser comprising one or more chemicals, wherein the dispenser is in electrical communication with the computing device and operable to dispense the one or more chemicals upon receiving instructions from the computing device (see Froloff: e.g., -- [0027] Using sticky traps to monitor pests is a standard greenhouse practice that can helps reduce overall pest management costs when coupled with plant inspections. Sticky traps are also a tool for identifying adult insect pests present in a greenhouse, including whiteflies, thrips, fungus gnats, shore flies and leafminers. Sticky traps can also be used to monitor adult parasitoids released in biological control programs.
[0028] Sticky traps provide an easy method for estimating pest population densities. When the timing of pest control actions is based on these relative estimates along with plant sample data from visual inspections, there is generally a reduction in pesticide use. As a result, there are fewer problems with pesticide resistance, less worker exposure to pesticides, reduced pesticide runoff and improved plant quality with less pesticide-induced phytotoxicity symptoms.
[0029] What is needed are automated insect and plant sensors that can capture data for analysis and early warning, data that can be captured by non-professionals and converted into automated pestilence and crop disease sentinels.--, in [0027]-[0029]).
Re Claim 37, Froloff as modified by Ampatzidis, Patch, and Ghazvinian Zanjani further disclose wherein the one or more chemicals are selected from the group consisting of fumigators, exterminators, insect repellants, insect mating disruption hormones, or combinations thereof (see Froloff: e.g., -- [0027] Using sticky traps to monitor pests is a standard greenhouse practice that can helps reduce overall pest management costs when coupled with plant inspections. Sticky traps are also a tool for identifying adult insect pests present in a greenhouse, including whiteflies, thrips, fungus gnats, shore flies and leafminers. Sticky traps can also be used to monitor adult parasitoids released in biological control programs.
[0028] Sticky traps provide an easy method for estimating pest population densities. When the timing of pest control actions is based on these relative estimates along with plant sample data from visual inspections, there is generally a reduction in pesticide use. As a result, there are fewer problems with pesticide resistance, less worker exposure to pesticides, reduced pesticide runoff and improved plant quality with less pesticide-induced phytotoxicity symptoms.
[0029] What is needed are automated insect and plant sensors that can capture data for analysis and early warning, data that can be captured by non-professionals and converted into automated pestilence and crop disease sentinels.--, in [0027]-[0029]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEI WEN YANG whose telephone number is (571)270-5670. The examiner can normally be reached on 8:00 - 5:00 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on 571-272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEI WEN YANG/Primary Examiner, Art Unit 2662
Prosecution Insights