DETAILED ACTION
Response to Arguments
Applicant's arguments filed with respect to claims 1-20 have been fully considered but are moot in view of the new ground(s) of rejection. The rejections are necessitated due to claim amendments.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 10, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Sibley (Pub. NO. US 2021/0192754) in view of Mikesell et al. (Pub. No. US 2021/0076662).
Regarding claim 1, Sibley teaches a method for treating a plant (agricultural object) in a field (agricultural environment) with a farming machine (agricultural treatment delivery vehicle), comprising: treating, while the farming machine is operating (traverses) in an initial configuration (emitter altitude) of mechanized components (emitters), a first plant of a plurality of plants (agricultural objects) in a field by identifying the first plant in an initial active region (optical sight) of a first image of the field [Para. 47 “Emitter 112c may be configured emit a treatment 112b, for example, via a trajectory 112d in any direction to intercept a target (“T”) 112a as vehicle 110 traverses path portions 119 at a velocity, v.”; Para. 120 “Based on a velocity of an agricultural treatment delivery vehicle, a time to emit an agricultural projectile may be computed and tracked. Hence, tracking an optical sight relative to an agricultural object may be a function of a rate of displacement of one or more emitters or a vehicle (e.g., relative to the soil or the agricultural environment)”; Para. 121 “For example, if a change in emitter altitude changes relative to the ground, an initial optical sight may be misaligned. Thus, another optical sight may be selected at 626. But if there is no variance, flow 600 moves from 622 to”; Para. 118 “In particular, an optical sight may be selected to align with a target relative to other optical sights, the optical sight being associated with an emitter for applying a treatment to a corresponding identified agricultural object”; Para. 124 “FIGS. 7A and 7B depict examples of data generated to identify, track, and perform an action for one or more agricultural objects in an agricultural environment, according to some examples”; Para. 125 “In agricultural projectile delivery system (not shown) may be configured to identify and select optical sight 726a (and corresponding emitter) to apply a treatment to target 722a.”; and Para. 163], wherein:
the initial active region is based on a configuration (emitter altitude) of mechanized components (emitters) of the farming machine, [Para. 118 “At 616, each optical sight may be predicted to align with an associated agricultural object at 616, the optical sight being associated with an emitter”; “Based on a velocity of an agricultural treatment delivery vehicle, a time to emit an agricultural projectile may be computed and tracked. Hence, tracking an optical sight relative to an agricultural object may be a function of a rate of displacement of one or more emitters or a vehicle (e.g., relative to the soil or the agricultural environment)” and Para. 121 “For example, if a change in emitter altitude changes relative to the ground, an initial optical sight may be misaligned. Thus, another optical sight may be selected at 626. But if there is no variance, flow 600 moves from 622 to 628”];
and the initial active region comprises pixels representing the first plant in the first image [para. 118 “In some cases, an emitter is oriented to emit an emission parallel (e.g., coaxially) with an optical ray extending from an optical sight to a target, the optical sight being associated with one or more pixels of an image capture device. Further, one or more agricultural objects may be tracked relative to one or more optical sights. For example, reflective light from one or more of the agricultural objects may be tracked in a field of view of an image captured by a camera.”; Para. 123 “In one example, an emission time may specify a time at which a pixel associated with an optical sight is aligned with an optical ray that extends from the pixel to at least a portion of a targeted agricultural object”; Para. 163];
responsive (responsive to a change in elevation) to the farming machine modifying an initial configuration (emitter altitude) of the mechanized components (emitters) to a modified configuration of the mechanized components determining (selected) a modified active region (optical sight) based on the modified configuration (emitter altitude) [Para. 121 “At 622, a determination is made as to whether any of sensor data detects a variance, such as a change in emitter altitude (e.g., a bump or raised elevation, or dip or depression) or any other change in sensor data, such as a variation in vehicle speed. If there is a variance, a trajectory may be recomputed at 624 (e.g., recomputing an emission parameter associated with the trajectory). For example, if a change in emitter altitude changes relative to the ground, an initial optical sight may be misaligned. Thus, another optical sight may be selected at 626”; and Para. 164 “At time, t2, initial positions of an array of emitters 2291 at time, t1, is changed to another position (e.g., relative to positions of targets 2222a to 2222c) and is depicted as an array of emitters 2292 at time, t2. For example, one or more sensors (e.g., accelerometers and the like) may detect a change in elevation 2239 as vehicle 2210 traverses uneven soil topology 2290. Responsive to a change in elevation, a second subset of optical sights may be selected to align with targets 2222a to 2222c. For example, optical sight 2227a may be selected to align with target 2222a, optical sight 2227b may be selected to align with target 2222b, and optical sight 2227c may be selected to align with target 2222c”] and
treating, while the farming machine is operating in the modified configuration, a second plant of the plurality of plants in the field by identifying the second plant in the modified active region of a second image of the field, wherein the modified active region [fig. 6, steps 622, 624, 626, 628 and 630. Para. 122 “At 628, an agricultural object [second agricultural object in the new optical sight] can be predicted to align with an optical sight to form a predicted emission parameter, which may be monitored to detect alignment of an optical sight and a target”; Para. 123 “At 630, an emitter is activated to apply an action based on a predicted emission parameter”].
However, Siblye doesn’t explicitly teach pixels representing the second plant in the second image.
Mikesell teaches treating (eradicating weeds), while the farming machine (vehicle 601) is operating in the modified configuration (actuator pan and tilt positions), a second plant (weed) of the plurality of plants (weeds) in the field by identifying the second plant (identifies the weed) in the modified active region (region in the image comprising a target) of a second image of the field, wherein the modified active region comprises pixels (pixel location of the weed in the target image)representing the second plant in the second image [Para. 58, 66, and 68].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Sibley’s agricultural treatment delivery system so that it performs Mikesell’s complete per weed process: for each second plant (weed), capture a second image (target image of the field), identify the modified active region (region in the image comprising a target) and it’s pixels (pixel location of the weed in the target image), translate that pixel location into the modified configuration (actuator pan and tilt positions), and activate the emitter after the actuators reach that positions. This medication improves Sibley by maintaining treatment beam alignment across successive weeds and actuator positions despite vehicle motion, thereby reducing treatment placement error.
Claim 10 and claim 19 are rejected for the same reasons as claim 1 above.
Claims 1, 2, 10, 11, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Slaughter et al. (Pub. No. US 2019/0104722) in view of REES (Pub. No. US 2022/0076387) in view of SERRAT (Pub. No. US 2022/0408715).
Regarding claim 1, Slaughter teaches a method for treating (plant care) a plant in a field with a farming machine (plant care assembly), comprising: treating (forming plant care), while the farming machine (plant care assembly) is operating in an initial configuration (extends the pneumatic cylinders 108) of mechanized components, a first plant of a plurality of plants in a field by identifying the first plant in an initial active region (region corresponding to an image segment) of a first image (digital color image) of the field [Para. 138 “FIG. 14A shows a white light illuminated image of field-grown tomato plants with the crop signaling plant label viewed via a crop identification system 30 in accordance with the present description”. Para. 115 “As the plant care assembly 100 travels along the row, a continuous panoramic view of the row is created by acquiring a sequence of images, processing them as described above in real-time to detect the crop plants and recording their geolocation using the centroid information within each image for the crop plus the odometry geolocation for the image itself”. Para. and “When the robotic weed hoes 110 (FIG. 10) are located between crop plants, the computer 40 extends the pneumatic cylinders 108 to cause the two hoes 110 to touch and kill all weeds in their path.”; Para. 151 “acquiring one or more digital color images of the target plant; and segmenting the one or more digital color images into image segments representing crop, weed, and background objects as a function of the optical crop signal”; Para. 154 “performing plant care comprises activating a weed removal apparatus based on identification of an image segment representing a weed, and selectively performing weed removal in a region corresponding to an image segment identified as containing the weed.”], wherein: the initial active region (crop detection zone) is based on a configuration (position and oriented) of mechanized components (support structure) of the farming machine [Para. 92 “The detector 112 is located inside the support structure 102, positioned and oriented to sense objects in the crop detection zone.]” and the initial active region comprises pixels (pixel level data) representing the first plant (weeds) in the first image [Para. 35 and 36]; responsive to determining the configuration of mechanized components of the farming machine is modified to a modified configuration (retracts the pneumatic cylinders 108) [Para. 115 “When the robotic hoes 110 approach a crop plant location stored in computer memory 44, the computer 40 retracts the pneumatic cylinders 108 to cause the two hoes 110 to separate, leaving a gap between them and allowing the crop plant 32 to pass by unharmed”], and treating, while the farming machine (plant care assembly) is operating in the modified configuration (retracts the pneumatic cylinders 108), a second plant of the plurality of plants in the field by identifying the second plant, wherein the modified active region comprises pixels (pixel Level) representing the second plant in the second image [Para. 115 “As the plant care assembly 100 travels along the row, a continuous panoramic view of the row is created by acquiring a sequence of images, processing them as described above in real-time to detect the crop plants and recording their geolocation using the centroid information within each image for the crop plus the odometry geolocation for the image itself”, “the computer 40 retracts the pneumatic cylinders 108 to cause the two hoes 110 to separate, leaving a gap between them and allowing the crop plant 32 to pass by unharmed.”; and Para. 35 “FIG. 12A is a processed image showing pixel level segmentation of a section of a row of tomato plants surrounded by purslane weeds, where a crop signaling treatment to the tomato plants prior to planting was used to distinguish tomato from weeds.”].
However, Slaughter does not explicitly teach determining a modified active region based on the modified configuration; and identifying the second plant in the modified active region of a second image of the field.
REES teaches determining a modified active region (ROI) based on the modified configuration (boom tilt) [Para. 12 “tilt of the boom as it varies in height with respect to the surface of the field moves the center of the base image data received from the image sensor and correspondingly moves the ROI of the base image data”; Para. 13 “the correcting performing remapping of the base image data into corrected image data to place the region of interest (ROI) in a desired location of the corrected image data”. Para. 25 and 26]; and identifying the second plant (weeds) in the modified active region (Region of interest) of a second image (successive images) of the field, wherein the modified active region comprises pixels representing the second plant in the second image [Para. 67 “In order to adapt machine vision to effectively operate on such a freestanding boom sprayer, Applicant has determined that the analysis to be performed to identify weeds or the like such as described in the aforementioned copending application, should be performed on a region of interest that is substantially sized invariant. Applicant has determined that regularizing the image information derived from each camera to develop a regularly sized and shaped region of interest for processing to identify desired features is a preferred approach for implementing the system disclosed in the aforementioned copending application. This both increases accuracy and reduces processing time for processing of the region of interest”. Para. 10 “The present application further provides a method of treating as described above wherein said correcting may optionally correct the base image data of successive images produced by each said image sensor into corrected image data that is size invariant so that each image represents the ROI with the same pixels of the corrected image data”; Para. 11 “corrects the size and/or perspective of the base image data to produce said corrected image data”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to apply REES’s specified evaluation portion (ROI) concept to Slaughter as image-based plant identification so that identification and treatment occur with the configuration dependent active region, reducing computation while preserving correct targeting.
Slaughter in view of REES don’t explicitly teach responsive to the farming machine modifying an initial configuration of the mechanized components to a modified configuration of the mechanized components determining [automatically, as applicant’s argument] a modified active region based on the modified configuration.
However, SERRAT teaches responsive (in this case) to the farming machine modifying (movement of the boom) an initial configuration (geometric parameter of each camera relative to the ground) of the mechanized components to a modified configuration of the mechanized components determining (recalculate in real time) a modified active region (grid G) based on (taking into account correction factors) the modified configuration (geometric parameter of each camera relative to the ground) [Para. 12, 38, 49, and 50].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Slaughter’s plant care treatment map logic, modified by REES, so that it performs Serrat’s complete teaching of the limitations. “In this case”, when movement of the boom change the geometric parameter of each camera relative to the ground, the system could recalculate (automatically) in real time the grid G by taking into account correction factors. This medication improves Slaughter by maintaining correspondence between image-grid cells and treatment locations as boom geometry changes thereby reducing misplaced plant treatment on uneven ground.
Claims 10 and 19 are also rejected for the same reasons as claim 1. Furthermore, Slaughter teaches a farming machine and non-transitory computer readable storage medium [fig. 2 and related description].
Regarding claims 2, 11 and 20, Slaughter doesn’t explicitly teach the claim limitations.
However, REES teaches accessing the first image (base image data) from a plurality of images (successive images) of the field from a plurality of image sensors (cameras) tilted towards the field at a tilt angle (Angle A), each of the plurality of images including a plurality of active pixels (pixels of the corrected image data) representing an active region (region of interest) for treating the plant (weeds) in the field and a plurality of remaining pixels (pixels of the base image data outside the region of interest) [Para. 4, 72, and 79]; wherein the initial active region (region of interest) is the active region (region of interest) for the first image (base image data) [Para. 9].
It would have been obvious to one of ordinary skill in the art before the effective filing date to apply REES’s specified evaluation portion (ROI) concept to Slaughter as image-based plant identification so that identification and treatment occur with the configuration dependent active region, reducing computation while preserving correct targeting.
Claims 3 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Slaughter et al. (Pub. No. US 2019/0104722) in view of REES (Pub. No. US 2022/0076387) in view of SERRAT (Pub. No. US 2022/0408715).and further in view of Saed (Pub. No. US 2005/0147322).
Regarding claims 3 and 12, Slaughter in view of REES in view of SERRAT doesn’t explicitly teach the claim limitations.
However, Saed teaches generating the first image by: identifying (selecting) the active region (inner or outer boundary) in each of the plurality of images (raw source image) [Para. 50]; generating a plurality of cropped images (scatter-cropped tile images), each of the plurality of cropped images including the active pixels (pixels values) corresponding to the identified active region for the cropped image [Para. 51, and 56]; and tiling (arranged) the plurality of cropped images (tile images) into the first image (mosaic) such that the initial active region comprises the active pixels in the plurality of cropped images [Para. 38, and 42].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Slaughter in view of REES in view of SERRAT to teach the claim limitations, feature as taught by Saed; because the modification enables the system to improve the visual quality and flexibility of photo-mosaic construction by letting each source image be cropped using selectable inner/outer boundaries so that resulting tile images better fit the mosaic regions and preserve more meaningful subject matter.
Claims 4, 5, 7, 13, 14, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Slaughter et al. (Pub. No. US 2019/0104722) in view of REES (Pub. No. US 2022/0076387) in view of SERRAT (Pub. No. US 2022/0408715). and further in view of Peake et al. (Pub. No. US 2022/0058770).
Regarding claims 4 and 13, Slaughter in view of REES in view of SERRAT doesn’t explicitly teach the claim limitations.
However, Peake teaches wherein the initial active region comprises: a near boundary representing a location in the field where the plant can be identified before the farming machine travels past the plant in the field [Para. 8 “For example, the active region may comprise a near boundary representing a location in the field where the plant can be identified by the controller before the farming machine travels past the plant in the field, and a far boundary representing a location in the field where an identification accuracy of the plant is below a threshold accuracy level”]; and a far boundary representing a location in the field where an identification accuracy of the plant is below a threshold accuracy level [Para. 8 “For example, the active region may comprise a near boundary representing a location in the field where the plant can be identified by the controller before the farming machine travels past the plant in the field, and a far boundary representing a location in the field where an identification accuracy of the plant is below a threshold accuracy level”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Slaughter in view of REES in view of SERRAT to teach the claim limitations, feature as taught by Peake; because the modification enables the system to improve a farming machine’s plant identification timing and reliability by defining the field region where a plant can still be identified accurately enough before the machine passes it.
Regarding claims 5 and 14, Slaughter in view of REES in view of SERRAT doesn’t explicitly teach the claim limitations.
However, Peake teaches wherein the far boundary is determined based on an occlusion area associated with an object within the initial active region, the occlusion area corresponding to an area behind the object that is not visible to image sensors of the farming machine [Para. 8 “For example, the active region may comprise a near boundary representing a location in the field where the plant can be identified by the controller before the farming machine travels past the plant in the field, and a far boundary representing a location in the field where an identification accuracy of the plant is below a threshold accuracy level”; and Para. 5 “the farming machine uses a plurality of image sensors to capture images of the field for detecting plants to be treated.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Slaughter in view of REES in view of SERRAT to teach the claim limitations, feature as taught by Peake; because the modification enables the system to improve a farming machine’s plant identification timing and reliability by defining the field region where a plant can still be identified accurately enough before the machine passes it.
Regarding claim 7 and 16, Slaughter in view of REES in view of SERRAT doesn’t explicitly teach the claim limitations.
However, Peake teaches wherein identifying the initial active region comprises: calculating the initial active region for the farming machine as it travels through the field based on a configuration of the mechanized components (image sensor; mounting mechanism; treatment mechanism) of the farming machine [Para. 8, 89 and 108].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Slaughter in view of REES in view of SERRAT to teach the claim limitations, feature as taught by Peake; because the modification enables the system to improve a farming machine’s plant identification timing and reliability by defining the field region where a plant can still be identified accurately enough before the machine passes it.
Claims 6 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Slaughter et al. (Pub. No. US 2019/0104722) in view of REES (Pub. No. US 2022/0076387) in view of SERRAT (Pub. No. US 2022/0408715). and further in view of Redden et al. (Pub. NO. US 2019/0090408).
Regarding claims 6 and 15, Slaughter in view of REES in view of SERRAT doesn’t explicitly teach the claim limitations.
However, Redden teaches wherein the initial active region comprises: a near boundary (target treatment boundary location) representing a location (geographic location) in the field at a lower threshold distance from a plurality of treatment mechanisms (multiple treatment mechanisms) of the farming machine [Para. 27, 57, 59, and 65]; and a far boundary representing a location in the field at an upper threshold distance from the plurality of treatment mechanisms [Para. 27, 57, 59, and 65].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Slaughter in view of REES in view of SERRAT to teach the claim limitations, feature as taught by Redden; because the modification enables the system to improve the precision of where and when a framing machine applies or switches treatment in the field by using near and far distance boundaries relative to its treatment mechanisms.
Claims 8 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Slaughter et al. (Pub. No. US 2019/0104722) in view of REES (Pub. No. US 2022/0076387) in view of SERRAT (Pub. No. US 2022/0408715). and further in view of Doepke et al. (Pub. No. US 2012/0293610).
Regarding claims 8 and 17 Slaughter in view of REES’ in view of SERRAT doesn’t explicitly teach the claim limitations.
However, Doepke teaches generating the first image by: identifying an overlapping region in a first image and a second image of a plurality of images, the overlapping region comprising one or more active pixels in each of the first image and second image representing one or more same objects (feature of interest) [Para. 25]; and applying one or more image processing functions (selecting; blending) to the one or more active pixels in the overlapping region such that the one or more same objects (feature of interest) are accurately represented in the first image [Para. 25]. ‘
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Slaughter in view of REES in view of SERRAT to teach the claim limitations, feature as taught by Doepke; because the modification enables the system to improve panoramic image stitching by detecting regions of interest in overlap areas and selecting one image’s representation there so that prior art seam blending artifacts, including inconsistent facial features, are avoided or diminished.
Claims 9 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Slaughter et al. (Pub. No. US 2019/0104722) in view of REES (Pub. No. US 2022/0076387) in view of SERRAT (Pub. No. US 2022/0408715). and further in view of Cote et al. (Pub. No. US 2012/0044372).
Regarding claim 9 and 18. Slaughter in view of REES in view of SERRAT doesn’t explicitly teach the claim limitations.
However, Cote teaches applying one or more image processing functions (image processing operation) to the initial active region that modifies one or more characteristics of the first image (source frame), the one or more image processing functions comprising one or more of: cropping, key-stoning, scaling, shifting, stabilizing, delayering, white balancing, resizing, controlling exposure (auto exposure), correcting color, adjusting reflectance, or normalizing values [Para. 161, 162, 196, 260 and 261].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Slaughter in view of REES in view of SERRAT to teach the claim limitations, feature as taught by Cote; because the modification enables the system to improve digital image processing in cameras by better correcting image errors and distortions introduced by the imaging device or image sensor, including in systems that use multiple image sensors.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOLOMON G BEZUAYEHU whose telephone number is (571)270-7452. The examiner can normally be reached on Monday-Friday 10 AM-8 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 888-786-0101 (IN USA OR CANADA) or 571-272-4000.
/SOLOMON G BEZUAYEHU/
Primary Examiner, Art Unit 2666
Prosecution Insights