DETAILED ACTION
Claim Objections
Claim 31 is objected to because of the following informalities: The abbreviation PCB and CAN should be spelled out at least for the first time use. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 25-27 are rejected under 35 U.S.C. 103 as being unpatentable over MIZUSHIMA et al. (Pub. No. US 2022/0192084 hereinafter “MIZ”) in view of Grimm et al. (Pub. No. US 2020/0221682) further in view of Fu et al. (Pub. No. US 2022/0101554).
Regarding claim 25, MIZ teaches a system (target identification and control system) for plant identification (identify targets) and spray control (applicator functionality) [Para 60 “The target identification and control system captures an image of an area ahead of the agricultural machine, in the direction of travel, and processes that image to identify targets in time for applicator functionality on the agricultural machine to apply a material to those targets.”]; comprising a device installed on a spray boom of an agricultural vehicle (agricultural machine) that actuates valves (controllable valves) of spray nozzles (nozzle bodies 120) [Para. 305 “wherein the agricultural machine includes a plurality of image processing modules mounted to the boom and a plurality of optical sensors mounted to the boom, each image processing module receiving images from a different set of optical sensors in the plurality of optical sensors, and wherein the material application mechanism includes a plurality of different sets of controllable valves, each of the controllable valves …..”; Para. 78 “FIG. 2 shows that each image processing module 124 processes images from a corresponding set of sensors 122 for control of a corresponding set of nozzle bodies 120. For instance, image processing module 124A can receive inputs from a plurality of image sensors 122A and generate output signals that are used to control the open/close state of valves in a subset of nozzle bodies 120A”]; wherein the device communicates by a connection with the cloud server which, in turn, communicates with a platform of a user interface module [Para. 231 “FIG. 38 is a block diagram of machine 100, shown in FIG. 1, except that it communicates with elements in a remote server architecture 930. In one example, remote server architecture 930 can provide computation, software, data access, and storage services that do not require end-user knowledge of the physical location or configuration of the system that delivers the services.”. Para. 231 “For instance, remote servers can deliver applications over a wide area network and they can be accessed through a web browser or any other computing component”];
However, MIZ doesn’t explicitly teach having a WiFi connection.
Grimm teaches Controller may include a wireless transceiver that enables controller to connect to devices on a wireless network, e.g., Wi-Fi. [Para. 48].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify MIZ’s machine-to-remote-server communication system by incorporating Grimm’s teaching of WiFi connection through a wireless transceiver so that Mizushima’s MIZ’s agricultural machine accesses the remote server architecture (remote server architecture 930) through that wireless network. This modification improves MIZ by providing a known local wireless data path for sprayer equipment, thereby simplifying data exchange when the machine uploads material application data or accesses remote server services.
MIZ also teaches an operator interface (operator interface mechanisms 154) installed in the sprayer vehicle cabin (operator compartment), with a positioning module of the sprayer vehicle and with a sensing module (boom sensor 126) of the spray boom [Para. 70 “FIG. 1C is a block diagram showing some portions of agricultural machine 100 in more detail. Some of the items shown in FIG. 1C are similar to those shown in FIGS. 1A and 1B and they are similarly numbered. FIG. 1C shows that agricultural machine 100 can also include one or more processors or servers 150, data store 151, a communication system 152, one or more operator interface mechanisms 154 that an operator 156 can interact with in order to control and manipulate agricultural machine 100, target identification system 158, control system 160, controllable subsystems 162, and agricultural machine 100 can include a wide variety of other agricultural machine functionality 164”; Para. 235 “For instance, a mobile device can be deployed in the operator compartment of machine 100 for use in generating, processing, or displaying the material application data.”; Para. 75 “Double knock processing system 165 receives the stored map of weed locations that was generated during the first pass and a geographic position sensor senses a geographic position of agricultural machine 100”; Para. 162 “The boom height sensor data can be communicated from boom sensor 126 to image processing module 124 via CAN directly or indirectly through another controller”].
MIZ doesn’t explicitly teach the device further communicating with an operator interface.
Grimm teaches the device (controller 62) further communicating (send signals to and receives signals from) with an operator interface [Para. 47 “controller 62 is connected to and configured to send signals to and receive signals from any components of spray system 10. For example, controller 62 may be connected to and configured to send signals to and receive signals from pump 68, spray boom 24, fluid storage tank 22, and/or valve assemblies 36”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify MIZ’s boom-mounted image processing module and agricultural-machine communication system by incorporating Grimm’s teaching that a device (controller 62) further communicates (send signals) with sprayer-system components so that MIZ’s device exchanges information with the operator interface (operator interface mechanisms 154), positioning module (geographic position sensor), and sensing module (booms sensor). This modification improves MIZ by centralizing boom sensor, position and operation status exchanges at the module that controls nozzle timing, thereby improving coordinated real time spray controls.
MIZ also teaches said device further comprises an image capture module (image sensor 122) [Para. 162 “One or more of the image sensors 122 first capture images of the ground ahead of the boom 118”].
However, MIZ in view of Grimm doesn’t explicitly teach having an RGB camera and an infrared camera.
Fu teaches having an RGB camera and an infrared camera [Para. 103, “Here, the camera 310 is an image sensor (e.g., RGB camera, near infrared camera, ultraviolet camera, multi-spectral camera), but could be other types of image sensors suitable for capturing an image of plants in a field.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify MIZ’s image sensors (image sensors 122), modified by Grimm, by incorporating Fu’s teaching of an RGB camera and an infrared camera as different camera types mounted along the machine mounting mechanism so that MIZ’s boom-mounted image capture arrangement captures visible-color and near-infrared plant information. This modification improves MIZ by adding complementary inputs for crop and weed imaging, thereby improving target identification and spray-control decisions under varying plant and soil conditions.
Regarding claim 26, MIZ teaches wherein the device captures the images while the vehicle (agricultural machine 100) is operating, processes them (processes that image), decides in real time whether spraying is necessary at a given location and controls the spraying through the solenoid valves installed in each spray nozzle, and stores in memory the images and other information acquired in the field and sends them to the cloud server (remote server architecture 930) [Para. 60, 76 and 231].
However, MIZ doesn’t explicitly teach having a solenoid valve and a Wi-Fi connection.
Grimm teaches having solenoid valves and WiFi connection [Para. 5 and 48].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify MIZ’s agricultural spray control system by incorporating Grimm’s solenoid valves and Wi-fi connection so that MIZ’s nozzle/valve control and remote server communication operate through known sprayer nozzle valve hardware and a known wires network. This modification improves MIZ by providing recognized individual nozzle actuation and local wireless connectivity, thereby supporting selective spray control and remote data transfer.
Regarding claim 27, MIZ teaches wherein the device comprises a lighting module and a processing module that communicates with the cloud server (Remote server) [Para. 60, 76, and 231].
Claims 28-29 are rejected under 35 U.S.C. 103 as being unpatentable over MIZUSHIMA et al. (Pub. No. US 2022/0192084 hereinafter “MIZ”) in view of Grimm et al. (Pub. No. US 2020/0221682) further in view of Fu et al. (Pub. No. US 2022/0101554) and further in view of AMPATZIDIS et al. (Pub. No. US 2023/0124398).
Regarding claim 28, MIZ in view of Grimm further in view of Fu, doesn’t explicitly teach the claim limitations.
However, AMPATZIDIS teaches wherein the cloud server (cloud environment 150) processes and analyzes the images and data (collected data 190) received from the device (smart sprayer system 180) and sends them to the user interface module [Para. 47].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify MIZ’s remote server architecture, modified by Grimm and Fu, by incorporating Ampatzidis’s cloud processing of collected data and access through a user interface so that Miz’s uploaded field images and sprayer data are processed and made viewable to the user. This modification improves MIZ by moving higher level analysis and visualization to scalable cloud resource, thereby improving field data interpretation and operator decision support.
Regarding claim 29, MIZ in view of Grimm further in view of Fu doesn’t explicitly teach the claim limitation.
However, Ampatzidies teaches wherein the platform of the user interface module is an online platform that receives the data processed by the cloud server and presents them to the user in the form of maps with crop information (field analytics) [Para. 97].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify MIZ’s remote server architecture, modified by Grimm and Fu, by incorporating Ampatzidis’s cloud processing of collected data and access through a user interface so that Miz’s uploaded field images and sprayer data are processed and made viewable to the user. This modification improves MIZ by moving higher level analysis and visualization to scalable cloud resource, thereby improving field data interpretation and operator decision support.
Claims 30-32 are rejected under 35 U.S.C. 103 as being unpatentable over MIZUSHIMA et al. (Pub. No. US 2022/0192084 hereinafter “MIZ”) in view of Grimm et al. (Pub. No. US 2020/0221682) further in view of Fu et al. (Pub. No. US 2022/0101554) and further in view of McCann et al. (Pub. No. US 2022/0211026).
Regarding claims 30 and 32, MIZ in view of Grimm further in view of Fu, doesn’t explicitly teach the claim limitation.
However, McCann teaches wherein the lighting module comprising light emitting diodes (LEDs) with visible light and infrared light [Para. 104].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify MIZ in view of Grimm and Fu, by incorporating McCann cloud processing of collected data and access through a user interface so that Miz’s uploaded field images and sprayer data are processed and made viewable to the user. This modification improves MIZ by moving higher level analysis and visualization to scalable cloud resource, thereby improving field data interpretation’
Regarding claim 31, MIZ in view of Grimm further in view of Fu, doesn’t explicitly teach the claim limitation.
However, McCann teaches characterized by comprising a PCB board where at least two cameras (image sensors 1910 and 1912) are mounted for capturing images of the crop, wherein the two cameras are an RGB camera and an infrared camera, a processor, two LEDs for field lighting, an input for gyroscope and accelerometer sensors (gyroscopes an accelerometers), a CAN communication driver with a communication module for vehicle sensing and for the vehicle's operator interface, a valve actuation driver for the valves, and an adapter input (socket) for the Wi-Fi connection for communication with the cloud server [Para. 141, 142, 164, and 104].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify MIZ in view of Grimm and Fu, by incorporating McCann cloud processing of collected data and access through a user interface so that Miz’s uploaded field images and sprayer data are processed and made viewable to the user. This modification improves MIZ by moving higher level analysis and visualization to scalable cloud resource, thereby improving field data interpretation’
Claims 33-34 are rejected under 35 U.S.C. 103 as being unpatentable over MIZUSHIMA et al. (Pub. No. US 2022/0192084 hereinafter “MIZ”) in view of Grimm et al. (Pub. No. US 2020/0221682) further in view of Fu et al. (Pub. No. US 2022/0101554) and further in view of Wu et al. (Pub. No. US 2019/0150357).
Regarding claim 33, Fu teaches A. Generation of images by means of the device's RGB and Infrared cameras [Para. 103]
However, MIZ in view of Grimm further in view of Fu doesn’t explicitly teach the rest of claim limitations.
Wu teaches B. Real-time image acquisition on the processor [Para. 87]; C. Images are stored in the memory of the processor [Para. 157]; D. Extraction of information from the images by the processor's algorithms [Para. 97]; E. Algorithm performs image analysis and treatment and data extraction in the processor [Para. 150]; F. From the extracted information, the processor is able to perform the following analyses for selective spraying in real time: Plant present/absent, presence and type of pest, presence and type of disease, and classification of plant species [Para. 97, 150 and 190]; G. The server-side flow follows through the images and information received by the device [Para. 154]; H. New training of the plant identification and spray control method on the server [Para. 86, 189 and 191]; I. Plant stand analysis, biomass analysis, and map generation on the server; J. The data from the server is sent to the user interface module platform where the maps with the images and the result of the evaluation of each image are presented [Para. 83, 85, and 135].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify MIZ in view of Grimm and Fu’s RGB/NIR image analysis process, by incorporating Wu’s use of intensity signals, reflected light and absorbed light to analyze plant image data. This modification improves MIZ by grounding plant detection in calibrated spectral intensity behavior, thereby improving discrimination of plants, weeds, and crop health conditions under changing field light.
Regarding claim 34, Fu teaches analysis and treatment of images referring to red, green, blue and infrared through RGB and infrared imaging [Para. 103].
However, MIZ in view Grimm further Fu doesn’t explicitly teach the rest of claim limitations.
Wu teaches wherein the analysis and treatment of the images of step E being based on the intensity of reflection and absorption of electromagnetic waves referring to red, green, blue, and infrared [Para. 89, and 108].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify MIZ in view of Grimm and Fu’s RGB/NIR image analysis process, by incorporating Wu’s use of intensity signals, reflected light and absorbed light to analyze plant image data. This modification improves MIZ by grounding plant detection in calibrated spectral intensity behavior, thereby improving discrimination of plants, weeds, and crop health conditions under changing field light.
Claim 35 is rejected under 35 U.S.C. 103 as being unpatentable over MIZUSHIMA et al. (Pub. No. US 2022/0192084 hereinafter “MIZ”) in view of Grimm et al. (Pub. No. US 2020/0221682) further in view of Fu et al. (Pub. No. US 2022/0101554) and Wu et al. (Pub. No. US 2019/0150357), and further in view of Chowdhary et al. (Pub. No. US 2021/0158041).
Regarding claim 35, MIZ in view of Grimm further in view of Fu and Wu, doesn’t explicitly teach the claim limitation.
However, Chowdhary teaches wherein the fact that, from the analysis and interpretation of the images, the processing module counts the number of white pixels to determine the amount of biomass [Para. 11, and 108].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify MIZ in view of Grimm and Fu and Wu’s RGB/NIR image analysis process, by incorporating Chowdhary’s use of intensity signals, reflected light and absorbed light to analyze plant image data. This modification improves MIZ by adding a quantified biomass proxy derived from image pixel, thereby supporting variable dose spraying and crop condition reporting.
Claim 36 is rejected under 35 U.S.C. 103 as being unpatentable over MIZUSHIMA et al. (Pub. No. US 2022/0192084 hereinafter “MIZ”) in view of Grimm et al. (Pub. No. US 2020/0221682) further in view of Fu et al. (Pub. No. US 2022/0101554) and Wu et al. (Pub. No. US 2019/0150357), and further in view of Redden et al. (Pub. No. US 2018/0330166).
Regarding claim 36, MIZ in view of Grimm further in view of Fu and Wu, doesn’t explicitly teach the claim limitation.
However, Redden teaches wherein the images stored in the database are evaluated by a technician or expert who generates an annotation for each image that retrains the plant identification and spray control method [Para. 110].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify MIZ in view of Grimm and Fu and Wu’s stored image and target identification system by incorporating Redden’s human labeling process using labeled and used to improve the plant identification model used for spray control. This modification improves MZ by using post field labeled images to update detection performance, thereby improving later selective spraying accuracy.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOLOMON G BEZUAYEHU whose telephone number is (571)270-7452. The examiner can normally be reached on Monday-Friday 10 AM-7 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’Neal Mistry can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-0101 (IN USA OR CANADA) or 571-272-1000.
/SOLOMON G BEZUAYEHU/ Primary Examiner, Art Unit 2666
Prosecution Insights