DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to independent claims 1 and 16 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. The arguments focus on Bettenhausen which is no longer applied to reject the claims. Instead, see the OverDriveTM light series which is newly applied and necessitated by the extensive amendments to the independent claims.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: High-Intensity, Overdriven, Strobed Light Emitting Diode Illumination And Stereo Camera Imaging of Thin Canopy Crops For Yield Estimation
Claim Objections
Claims 1, 11, and 16 are objected to because of the following informalities: Amended claim 11 refers to “tall plant” when “thin-canopy” crop was almost certainly intended particularly given the parallel case directed to tall plants (see ODB rejection below) and claim 2 further specifying that the thin canopy crop is a vineyard or orchard. This appears to be a minor typo and these claims are being treated as if they recited thin-canopy crop. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1-3, 7, 11-14, 16-17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Nuske {Nuske, Stephen, et al. "Automated visual yield estimation in vineyards." Journal of Field Robotics 31.5 (2014): 837-860}; Noguchi (US 20170118925 A1); Underwood (WO 2021/051150), Official Notice;
Haron {N. M. A. N. Haron, et al, Fadzilah, "White LED illumination for improving outdoor image analysis for palm oil fruit ripeness detection," 2012 International Conference on Advanced Computer Science and Information Systems (ICACSIS), Depok, West Java, Indonesia, 2012, pp. 95-99}; Underwood (WO 2021/051150); and
OverDriveTM lighting series including on-sale product catalog, https://smartvisionlights.com/resources/lighting-basics-resources/overdrive-series-for-machine-vision-lighting/, 26 November 2019, downloaded 20 May 2026; and associated product data sheets including the ODR130-16 product data sheet, https://smartvisionlights.com/products/odr130/, revised 03 October 2022, downloaded 20 May 2026; ODS75 product data sheet, https://smartvisionlights.com/wp-content/uploads/ODS75_Datasheet.pdf, revised 13 January 2022, downloaded 20 May 2026; and ODRL300 product data sheet, https://smartvisionlights.com/wp-content/uploads/ODRL300_Datasheet-1.pdf, revised 13 January 2022, downloaded 20 May 2026.
It is noted that the OverDriveTM lighting series including the product catalog and specific models cited above are being applied under the printed publications and/or the on-sale or otherwise available to the public sections of 35 USC 102(a)(1).
Further as to the Datasheet portion of the evidence note that these Datasheets list the specifications of the exact same product models listed for sale in the product catalog, thus, the on-sale date from the product catalog of 26 November 2019 provides evidence that each of the specific products listed therein was on-sale and included the specifications listed in the datasheets. Moreover, MPEP 2124 provides an exception to the rule that a reference, such as the Datasheets, need not antedate the filing date including when such references are utilized to demonstrate that characteristics of the prior art product were known which is the purpose of the Datasheets.
Claim 1
In regards to claim 1, Nuske discloses a method comprising:
activating an illumination source
{see sections 3.1.2; Figs. 1, 6 (copied below), disclosing controlled flash lighting upon the grapes; Section 5 discussing Prosilica GE 400 camera or Nikon D300s and two Einstein 640 flashlights or AlienBees ARB800 ring flash},
polarising the illuminating light in a first polarisation axis {section 5.3.1; 5.3.2, fig. 9(c) discussing cross polarized light detection which includes using two polarizer filters, one on the light source and one on the camera set at 90 degrees (transverse) to eliminate glare by blocking specular reflections from glossy surfaces such as grapes};
illuminating at least part of a thin canopy plant crop with the polarised illuminating light to produce reflected illuminating light {see above cites. See also Fig. 6 illustrating illumination of a grape vine which is a thin canopy plant as per claim 2 and the specification};
polarising the reflected illuminating light in a second polarisation axis transverse to the first polarisation axis to produce cross-polarised reflected illuminating light {see above cites discussing cross-polarized light detection};
capturing images of at least part of the thin canopy plant crop, using the cross-polarised reflected illuminating light {Section 3, Fig. 2 Berry Detection includes capturing an image of the grape vine using a sideways facing camera mounted on a utility vehicle, Section 5 specifies, e.g., Prosilica GE 400 camera used in device}; and
analysing the captured images to determine a condition of the thin canopy plant crop, the condition of the thing canopy plant crop comprising yield, and
{Fig, 2, yield estimation process, sections 3.1-4 including image-based measurements for yield predictions while noting that yield is a “condition” as per claim 6 and the specification}.
tall plant crop (thin-canopy crop) and polarised in the second polarisation axis transverse to the first polarisation axis.
Haron is an analogous reference from the same field of illuminating and imaging plants. Haron also teaches wherein the active illumination source comprises one or more light-emitting diodes and the image capture system is configured to strobe the light-emitting diodes {see section II White LED for The Improvement of Image Analysis which also teaches the advantages of using LEDs in this role}. Haron further teaches applying super bright LEDs to improve outdoor image analysis against outdoor conditions in Section V. Lastly, Haron suggests using “higher power sources” for LED arrays in Section V.
OverDrive TM lighting series is an analogous reference because it is reasonably pertinent to the problem faced by the inventor, namely high-intensity lighting for imaging large-scale objects. See the product catalog and Datasheets. This lighting series includes models specifically adapted for high-intensity strobe lighting (SafeStrobe) of various objects such that camera(s) may capture images of the objects for machine vision purposes. Some are also particularly adapted to and are described as “large area, long-distance” lighting such as the ODRL300 large ring lights which are disclosed as being easily integrated with cameras for machine vision. Such lighting applications logically include illumination sources for agricultural applications including thin-canopy plant crops due to their high intensity output, large illumination area, long working distances, and integration with a camera for machine vision purposes. Moreover, the product data sheets discuss radial, bright field, projector, diffuse panel, and direct illumination modes thus making these lights further applicable to agriculture.
Still further, the OverDrive TM lighting series includes “one or more light-emitting diodes to produce illuminating light”, a variety of lens optics for wide and narrow beams, and, significantly, polarizers to reduce reflections from specular surfaces (e.g. such as thin-canopy crops), and strobing or pulsing the LEDs at an intermittent voltage above a rated LED voltage. See the OverDriveTM and “SafeStrobe technology”.
Even further, the total output in lumens for the ODRL 300 is 1.83M lumens, the output of the ODR130-16 is 22M lumens, and the total output of the OD575 is 3.2M lumens. These values are calculated using the lighting pattern data from the datasheets using the values at a distance of 500mm and converting lux to lumens.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Nuske’s very bright flash illumination and cross-polarized detection, vehicle mounted apparatus having high-intensity illumination sources and image capture device such that wherein the active illumination source comprises one or more light-emitting diodes and the image capture system is configured to strobe the light-emitting diodes as taught by Haron and to strobe the LEDs with an intermittent voltage above a rated LED voltage of the LED diodes as taught by the OverDrive TM lighting series on-sale product catalog and associated ODR130-16; ODS75, and ODRL300 datasheets because Haron motivates using higher power sources for super bright LEDS for agricultural imagery, because LEDs are more energy efficient than the Xenon tubes of Nuske, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Nuske discloses a stereo camera (section 3.3) but for odometry and vehicle navigation not for plant imaging.
Noguchi is an analogous reference from the same field of machine vision for agriculture including a plant information obtainment unit 410 which is a field vehicle for illuminating and imaging plants including wherein the capturing an image comprises capturing stereo images {stereo camera 430, Fig. 3 copied below, [0088], [0091]
PNG
media_image1.png
546
692
media_image1.png
Greyscale
}
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Nuske’s very bright flash illumination and cross-polarized detection such that wherein the capturing an image comprises capturing stereo images as taught by Noguchi because stereo imagery advantageously generates three-dimensional images thus increasing the accuracy and capabilities of Nuske’s crop monitoring device and method such as being able to estimate plant size in three dimensions thus increasing accuracy of harvest predictions while Noguchi’s stereo camera would also benefit from cross-polarized imagery to reduce specular reflections and increase S/N ratio which is particularly useful and advantageous for downstream machine vision tasks such as crop monitoring, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Nuske discloses wherein the condition of the thin canopy plant crop comprises yield {see above cites} due does not analyze captured stereo images to determine a condition of the thin canopy plant crop, the condition of the thing canopy plant crop comprising yield, and the yield being determined by estimating berry and/or bunch volume/weight from the stereo images
Noguchi also teaches that yield is determined from the stereo imaging to estimating berry and/or bunch volume/weight from the stereo images {see Fig. 26 including NDVI, predicted harvest date and crop yield [0078], [0091], [0129], [0184], [0189]-[0194] determining plant size/volume}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Nuske’s very bright flash illumination and cross-polarized detection such that wherein the capturing an image comprises capturing stereo images as taught by Noguchi and wherein yield is determined from the stereo imaging to estimating berry and/or bunch volume/weight as also taught by Noguchi because stereo imagery advantageously generates three-dimensional images thus increasing the accuracy and capabilities of Nuske’s crop monitoring device and method such as being able to estimate plant size in three dimensions thus increasing accuracy of harvest predictions while Noguchi’s stereo camera would also benefit from cross-polarized imagery to reduce specular reflections and increase S/N ratio which is particularly useful and advantageous for downstream machine vision tasks such as crop monitoring, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
As detailed above, the OverDriveTM Series of lighting meets the claim limitations of “:wherein the activating the illumination sources comprises strobing or pulsing the light- emitting diodes at an intermittent voltage above a rated LED voltage of the light-emitting diodes”. Further in regards to the effect of such high-intensity overdriven lighting “such that the cross-polarised reflected illuminating light has a greater intensity than sunlight that is reflected from the tall plant crop and polarised in the second polarisation axis transverse to the first polarisation axis” first note that the cross-polarization and overdriven LEDs are met by the applied art above including Nuske and the OverDriveTM Series of lights. As such, the remaining issue is the “greater intensity than sunlight” limitation.
The total output in lumens for the ODRL 300 is calculated as 1.83M lumens, the output of the ODR130-16 is 22M lumens, and the total output of the OD575 is 3.2M lumens.
The disclosed output in Lux for the ODRL 300 is 9200 Lux, the output of the ODR130-16 is 140k Lux, and the total output of the OD575 is 49k Lux.
Given a typical polarizer loss of 50%, which is suffered twice due to the cross-polarization from light and lens, the total cross polarized light reaching the camera sensor from these extremely bright LED units is about 140k lux *(0.5 polarization loss in light)*(0.5 polarization loss in camera) = 35k lux.
Considering that plants typically reflect about 10% of visible light, the final value of imaged light for plants illuminated by the ODR130-16 is approximately 3.5k lux.
Official Notice is taken that sunlight varies from 400 lux (sunrise/sunset) to 100k lux for direct sunlight at mid-day. The sunlight suffers from one polarizer loss at the camera as well as reflection loss from the plant. As such the claimed range works out to be 400lux (0.5)(0.1) = 20 lux to (100k lux)(0.5)(0.1) = 5k lux. The cross-polarized reflected light received by the camera from the ODR13-16 is therefore brighter than sunlight, reflected from the plant and polarized by the camera at least during much of the day. Similar conclusions are reach for the ODRL 300 and OD575 lights.
Underwood is an analogous reference from the same field of machine vision for agriculture and also solves the same problem as the instant invention which is dealing with poor S/N ratio when imaging crops during the daytime such that downstream machine vision tasks suffer from poor quality imagery. See in particular page 10 in which “the use of very powerful illumination/lighting system (e.g. short pulse high energy strobes) … has been found to overpower the illumination provided by sunlight to allow for consistent scanning of a crop at any time of day or night as well as operation in sunny or cloudy conditions with repeatable results” (emphasis added).
Moreover, Underwood’s use of high-powered artificial lighting also provides other advantages including the ability to use very short camera exposure times thus giving rise to clear sharp images despite motion of the vehicle-mounted equipment travelling across uneven field terrain to image the orchard crops as further discussed on pg. 10.
In more detail, Underwood teaches a vehicle-mounted imager (cameras) with two strobe light units 14, 15 having an “extremely high level of brightness with a combined output of 1.2 x 10-4 joules/cm2 at two feet, pg. 5. Moreover, the flash duration is 50 to 100 microseconds.
While it is difficult to calculate the intensity of the output of Underwood’s “extremely bright” artificial lights, it is fair to conclude that it is sufficiently bright that it teaches or at least suggests the claimed result of “such that the cross-polarised reflected illuminating light has a greater intensity than sunlight that is reflected from the tall plant crop and polarised in the second polarisation axis transverse to the first polarisation axis”.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Nuske’s very bright flash illumination and cross-polarized detection such that the cross-polarised reflected illuminating light has a greater intensity than sunlight that is reflected from the tall plant crop and polarised in the second polarisation axis transverse to the first polarisation axis as taught by OverDriveTM lighting series in view of Underwood and/or Official Notice regarding well-known physical values including typical sunlight illumination levels, polarization loss, reflection loss, and identified hardware light output values because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Furthermore, Underwood demonstrates that light intensity/brightness is a result effective variable that achieves the recognized and disclosed result of improving agricultural imagery by “overpower[ing] the illumination provided by sunlight to allow for consistent scanning of a crop at any time of day”, pg. 10. In other words, light intensity is a result effective variable that achieves a recognized result (improving outdoor plant imagery) wholly consistent with In re Antonie, 559 F.2d 618, 195 USPQ 6 (CCPA 1977) and MPEP 2144.05II(B).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified modified Nuske’s very bright flash illumination and cross-polarized detection such that the cross-polarised reflected illuminating light has a greater intensity than sunlight that is reflected from the tall plant crop and polarised in the second polarisation axis transverse to the first polarisation axis because light intensity has been taught by Undersood as a result effective variable consistent with In re Antonie, 559 F.2d 618, 195 USPQ 6 (CCPA 1977) and MPEP 2144.05II(B) to overcome sunlight conditions when imaging plants outdoors.
Claim 2
In regards to claim 2, Nuske discloses wherein the thin canopy plant crop is a vineyard or orchard crop {see above including application to grape vine crops}.
Claim 3
In regards to claim 3, Nuske discloses wherein the illumination source and/or the capturing the stereo image is directed upward to the underside of the thin canopy plant crop {see above cites for claim 1. Figs. 1, 6 copied below graphically illustrating light and camera directed upward to the underside of the grape vines. Note the claim recites either illumination source or stereo capturing is directed upwards. Consistent with Ex Parte Gross, Appeal No. 2011-004811, “‘and/or’ covers embodiments having element A alone, element B alone, or elements A and B taken together.” Appeal No. 2011-004811, Decision on Appeal, at 4 (Jan. 3, 2014, emphasis added). Therefore, this interpretation will be applied to claim 3 such that only the illumination source is directed upwards. Should Applicant revise to include stereo imaging upwards then Noguchi would be applied.
PNG
media_image2.png
711
961
media_image2.png
Greyscale
Claim 7
In regards to claim 7, Nuske is not relied upon to disclose but Noguchi teaches wherein the condition of the thin canopy plant crop is also determined using input from one or more of:
temperature sensors, relative humidity sensors, soil moisture sensors, barometric pressure sensors, wind sensors, UV sensors, light sensors, depth sensors and/or rain sensors {see Fig. 24, [0095] teaching a sensor suite including infrared, CO2, water, windspeed, nutrition, temperature and humidity sensors gathering data that may be used by the server 704 to determine condition of the plants such as predicting harvest time}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Nuske’s very bright flash illumination and cross-polarized detection such that wherein the capturing an image comprises capturing stereo images as taught by Noguchi and wherein the condition of the thin canopy plant crop is also determined using input from one or more of temperature sensors, relative humidity sensors, soil moisture sensors, barometric pressure sensors, wind sensors, UV sensors, light sensors, depth sensors and/or rain sensors as also taught by Noguchi because a more robust sensor suite increases the accuracy and range of plant condition monitoring functions and harvest predictions, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 11
Nuske discloses wherein the intensity of the cross-polarised reflected light is more than 10 times greater than the intensity of the reflected sunlight that is reflected from the tall (thin-canopy) plant crop and polarized in the second polarization axis transverse to the first polarization axis.
{Note that “sunlight” is a very broadly recited term that includes a large range of values as noted above. See Figs 1 and 6 illustrating these broadly recited conditions in which a very bright flash (e.g. two high intensity Einstein 640 flashlights) are used to illuminate the grape vines in a cross-polarized configuration where the sunlight condition is very dark (nighttime, sun reflects from moon). Official Notice is taken that outdoor night time ambient illumination may be near zero to about 1 lumen/m2 for a full moon reflecting sunlight.
One may calculate the lumen output from the Einstein 640 based on the 640 Watt-second (W-sec) output. Let’s assume a typical 100 to 200 lumens/Watt (Lm/W) luminous efficacy (official notice). Using the lower value of 100 Lm/W results in (640 W-sec)(100Lm/W) = 6,400 Lm-sec. With the minimum flash duration at full power of 1/2000 sec, we have (6,400 L-s/0.0005sec) = 1.28M lumens at full power and minimum flash duration for one Einstein 640 flash unit. Since Nuske employs two of them, the total output value for both is 2.56M lumens.
Given a typical polarizer loss of 50%, which is suffered twice due to the cross-polarization from light and lens, the total cross polarized light reaching the camera sensor from these extremely bright flash units is about 2.56M lumens*(0.5 polarization loss in light)*(0.5 polarization loss in camera) = 640,000 lumens.
Compare this value with nighttime conditions {e.g. 10 lumens * (0.5 polarization loss in camera’s polarizer) = 5 lumens ambient light received by the camera.
Considering that plants typically reflect about 10% of visible light, the final values are approximately 64,000 lumens versus 0.5 lumens. As such, the Nuske’s very bright flash illumination easily satisfies the broadly recited claim language given well known and officially noticed values and straightforward math.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Nuske’s very bright flash illumination and cross-polarized detection such that (claim 11) wherein the intensity of the cross-polarised reflected light is more than 10 times greater than the intensity of the reflected sunlight that is reflected from the tall (thin-canopy) plant crop and polarized in the second polarization axis transverse to the first polarization axis in view of Official Notice regarding well-known physical values including typical ambient illumination levels, polarization loss, reflection loss, and identified hardware light output values because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Underwood is an analogous reference from the same field of machine vision for agriculture and also solves the same problem as the instant invention which is dealing with poor S/N ratio when imaging crops during the daytime such that downstream machine vision tasks suffer from poor quality imagery. See in particular page 10 in which “the use of very powerful illumination/lighting system (e.g. short pulse high energy strobes) … has been found to overpower the illumination provided by sunlight to allow for consistent scanning of a crop at any time of day or night as well as operation in sunny or cloudy conditions with repeatable results” (emphasis added).
Moreover, Underwood’s use of high-powered artificial lighting also provides other advantages including the ability to use very short camera exposure times thus giving rise to clear sharp images despite motion of the vehicle-mounted equipment travelling across uneven field terrain to image the orchard crops as further discussed on pg. 10.
In more detail, Underwood teaches a vehicle-mounted imager (cameras) with two strobe light units 14, 15 having an “extremely high level of brightness with a combined output of 1.2 x 10-4 joules/cm2 at two feet, pg. 5. Moreover, the flash duration is 50 to 100 microseconds.
As such and particularly in reference to pg. 10 disclosure, Underwood clearly teaches wherein the cross-polarised reflected light has a greater intensity than polarised ambient light that has reflected from the plant and been captured along with the image even during daylight conditions.
While it is difficult to calculate the intensity of the output of Underwood’s “extremely bright” artificial lights, it is fair to conclude that it is sufficiently bright that it teaches or at least suggests (claim 11) wherein the intensity of the cross-polarised reflected light is more than 10 times greater than the intensity of the reflected sunlight that is reflected from the tall (thin-canopy) plant crop and polarized in the second polarization axis transverse to the first polarization axis.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Nuske’s very bright flash illumination and cross-polarized detection such that (claim 11) wherein the intensity of the cross-polarised reflected light is more than 10 times greater than the intensity of the reflected sunlight that is reflected from the tall (thin-canopy) plant crop and polarized in the second polarization axis transverse to the first polarization axis in view of Underwood and/or Official Notice regarding well-known physical values including typical ambient illumination levels, polarization loss, reflection loss, and identified hardware light output values because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Further in regards to claim 11, Underwood demonstrates that light intensity/brightness is a result effective variable that achieves the recognized and disclosed result of improving agricultural imagery by “overpower[ing] the illumination provided by sunlight to allow for consistent scanning of a crop at any time of day”, pg. 10. In other words, light intensity is a result effective variable that achieves a recognized result (improving outdoor plant imagery) wholly consistent with In re Antonie, 559 F.2d 618, 195 USPQ 6 (CCPA 1977) and MPEP 2144.05II(B).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified modified Nuske’s very bright flash illumination and cross-polarized detection such that wherein the intensity of the cross-polarised reflected light is more than 10 times greater than the intensity of the reflected sunlight that is reflected from the tall (thin-canopy) plant crop and polarized in the second polarization axis transverse to the first polarization axis because light intensity has been taught by Undersood as a result effective variable consistent with In re Antonie, 559 F.2d 618, 195 USPQ 6 (CCPA 1977) and MPEP 2144.05II(B) to overcome ambient light conditions when imaging plants outdoors.
Claim 12
In regards to claim 12, Nuske discloses wherein analysing the captured
Nuske discloses a stereo camera (section 3.3) but for odometry and vehicle navigation not for plant imaging.
Noguchi is an analogous reference from the same field of machine vision for agriculture including a plant information obtainment unit 410 which is a field vehicle for illuminating and imaging plants including wherein the capturing an image comprises capturing stereo images {stereo camera 430, Fig. 3 copied below, [0088], [0091], wherein analysing the captured stereo images comprises performing surface analysis of an imaged part of the thin canopy plant crop {see Fig. 26 including NDVI, predicted harvest date and crop yield [0078], [0091], [0129], [0184], [0189]-[0194] determining plant size/volume}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Nuske’s very bright flash illumination and cross-polarized detection and which already analyzes the captured images comprises performing surface analysis of an imaged part of the thin canopy plant crop such that such that wherein the capturing an image comprises capturing stereo images and analysing the captured stereo images comprises performing surface analysis of an imaged part of the thin canopy plant crop as taught by Noguchi because stereo imagery advantageously generates three-dimensional images thus increasing the accuracy and capabilities of Nuske’s crop monitoring device and method such as being able to estimate plant size in three dimensions thus increasing accuracy of harvest predictions while Noguchi’s stereo camera would also benefit from cross-polarized imagery to reduce specular reflections and increase S/N ratio which is particularly useful and advantageous for downstream machine vision tasks such as crop monitoring, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 13
In regards to claim 13, Nuske discloses wherein performing the surface analysis comprises segmenting the
Noguchi is an analogous reference from the same field of machine vision for agriculture including a plant information obtainment unit 410 which is a field vehicle for illuminating and imaging plants including wherein the capturing an image comprises capturing stereo images {stereo camera 430, Fig. 3 copied below, [0088], [0091]} and
performing the surface analysis comprises segmenting the stereo images based on visual symptoms of the condition detected on a surface of the thin canopy plant crop, or features of the thin canopy plant crop {see Fig. 26 including NDVI, predicted harvest date and crop yield [0078], [0091], [0129], [0184], [0189]-[0194] determining plant size/volume}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Nuske’s very bright flash illumination and cross-polarized detection and which already performing the surface analysis comprises segmenting the images based on visual symptoms of the condition detected on a surface of the thin canopy plant crop, or features of the thin canopy plant crop such that such that wherein the capturing an image comprises capturing stereo images and analysing the captured stereo images comprises performing the surface analysis comprises segmenting the stereo images based on visual symptoms of the condition detected on a surface of the thin canopy plant crop, or features of the thin canopy plant crop as taught by Noguchi because stereo imagery advantageously generates three-dimensional images thus increasing the accuracy and capabilities of Nuske’s crop monitoring device and method such as being able to estimate plant size in three dimensions thus increasing accuracy of harvest predictions while Noguchi’s stereo camera would also benefit from cross-polarized imagery to reduce specular reflections and increase S/N ratio which is particularly useful and advantageous for downstream machine vision tasks such as crop monitoring, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 14
In regards to claim 14, Nuske discloses wherein the first polarisation axis is at approximately 900 to the second polarisation axis {see above cites for claim 1 for cross-polarized illumination and detection in which “cross” means that the first polarisation axis is 900 to the second polarisation axis. It is noted that the term “approximately” is considered definite because it refers to unintentional or otherwise slightly uncalibrated alignment with respect to the preferred 90 degree cross polarization that most effectively blocks specular reflections}.
Claim 16
The rejection of method claim 1 above applies mutatis mutandis to the corresponding limitations of system claim 16 while noting that the rejection above cites to both device and method disclosures
Claim 17
In regards to claim 17, Nuske discloses a single mounting system to a vehicle
{Nuske’s image capture system (see above mapping of claims 1 and 16) employs a digital camera such as the Prosilica GE 400 camera which includes a microprocessor to process the digital images. As such and as shown in Figs. 1 and 6 the image capturing system including the lights, camera (with internal processor) and polarizers are integrated onto a boom-like structure that has a single mounting system for the illustrated vehicle}.
Claim 20
In regards to claim 20, Nuske discloses wherein the illumination source has a total output intensity of at least 1,100,000 lumens.
{One may calculate the lumen output from the Einstein 640 based on the 640 Watt-second (W-sec) output. Let’s assume a typical 100 to 200 lumens/Watt (Lm/W) luminous efficacy (official notice). Using the lower value of 100 Lm/W results in (640 W-sec)(100Lm/W) = 6,400 Lm-sec. With the minimum flash duration at full power of 1/2000 sec, we have (6,400 L-s/0.0005sec) = 1.28M lumens at full power and minimum flash duration for one Einstein 640 flash unit. Since Nuske employs two of them, the total output value for both is 2.56M lumens}.
Although Nuske discloses the total output intensity above 1.1M lumens, Nuske does not employ LEDs. As noted above, the total output in lumens for the ODRL 300 is calculated as 1.83M lumens, the output of the ODR130-16 is 22M lumens, and the total output of the OD575 is 3.2M lumens and these lights employs LEDs.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Nuske’s very bright flash illumination and cross-polarized detection, vehicle mounted apparatus having high-intensity illumination sources and image capture device such that wherein the active illumination source comprises one or more light-emitting diodes wherein the illumination source has a total output intensity of at least 1,100,000 lumens as taught by the OverDrive TM lighting series on-sale product catalog and associated ODR130-16; ODS75, and ODRL300 datasheets because Haron motivates using higher power sources for super bright LEDS for agricultural imagery, because LEDs are more energy efficient than the Xenon tubes of Nuske, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Nuske, Official Notice; Noguchi, Underwood, and OverDriveTM lighting series including the on-sale product catalog and associated product data sheets for ODR130-16, ODS75; and ODRL300 as applied to claim 1 above, and further in view of Fahey {Fahey T, Pham H, Gardi A, Sabatini R, Stefanelli D, Goodwin I, Lamb DW. Active and Passive Electro-Optical Sensors for Health Assessment in Food Crops. Sensors (Basel). 2020 Dec 29;21(1):171. doi: 10.3390/s21010171. PMID: 33383831; PMCID: PMC7795220}
Claim 6
In regards to claim 6, Nuske discloses wherein the condition of the thin canopy plant crop is selected from the group consisting of:
{see Fig, 2, yield estimation process, sections 3.1-4 including image-based measurements for yield predictions but which feature has been moved to claim 1)
Fahey is an analogous reference from the same field of crop imaging and monitoring. See abstract. Fahey also teaches
wherein the condition of the thin canopy plant crop is selected from the group consisting of: at least pests or disease and, leaf area index {see plant health and disese assessments in abstract, Introduction, sections 2.4, 2.5, 2.6, Table 7, Sections 5 and 7.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Nuske’s very bright flash illumination and cross-polarized detection, vehicle mounted apparatus having high-intensity illumination sources and image capture device and which also determines condition of the thin canopy crop in terms of yield is extended to at least pests or disease and, leaf area index as taughy by Fahey because doing so provides a richer data set of plant crop conditions useful for better managing and growing crops, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Nuske, Official Notice; Noguchi, Underwood, and OverDriveTM lighting series including the on-sale product catalog and associated product data sheets for ODR130-16, ODS75; and ODRL300 as applied to claim 1 above, and further in view of Nuske ‘894 US 2018/0129894 A1) and Official Notice.
Claim 23
In regards to claim 23, Nuske discloses a vehicle mounted image capture system, first polarizer, second polarizer, and image capture device but is not relied upon to disclose a housing integrating the illumination source, the first polarizer, the second polarizer, the image capture device and a controller {Figs. 1 and 6 illustrate a mounting system for mounting at least the light and cameras to the vehicle but does not appear to mount a housing the vehicle. See 3.1.1 and 3.2, 5.3.3 quad core CPU with RAN for controller}.
Nuske ‘894 teaches an on-board computation unit(s) to process images of the illuminated foliage and/or fruit in real-time {see [0016]- [0018] and Figs. 1, 2 in which a processor processing the digital images of the illuminate foliage in real time}; and
a housing integrating the illumination source, the first polarizer, the second polarizer, the image capture device and a controller {See Fig. 8 showing housings appropriate for housing the illumination sources 801, image capture devices 802 and computation unit, particularly when combined with Nuske’s cross-polarizers for camera and lights. Official notice is also taken that processors (computation unit) are commonly used to execute the methods of Figs. 1, 2 and are routinely located within a common housing integrating components such as those disclosed by Nuske and Nuske ‘894.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Nuske’s very bright flash illumination and cross-polarized detection vehicle mounted apparatus having high-intensity illumination sources, image capture device and computation unit to process images of the illuminated foliage and fruit such that the system includes a housing integrating the illumination source, the first polarizer, the second polarizer, the image capture device and a controller as taught by Nuske ‘894 and Official notice because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Allowable Subject Matter
Claim 8 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 8
In regards to claim 8, Nuske discloses wherein yield is determined directly by imaging and counting fruits, berries, bunches, or buds, blossoms, inflorescences which will later turn into fruits, or imaging and counting fruits unsuitable for harvest due to pest, disease, or other damages {see above cites for claim 1 which includes imaging and counting grapes (fruits)} but none of the prior art of record discloses or fairly suggests the additional indirect yield determination that includes “wherein the yield is determined indirectly by estimating the yield from counting shoots, identifying the growth stage of the plant or fruits over time to forecast how much of the thin canopy will reach maturity, depth imaging and estimating the berry and/or bunch volume/weight, and using pest and disease information to forecast how much of the thin-canopy crop may be affected” in combination with base claim 1 and intervening claim 6.
Double Patenting
Claims 1-3, 6, 7, 8, 11-14, 16-17, 20, and 23 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over reference claims of copending Application No. 17/800,817 (also corresponds to Delkovic above) in view of the prior art references applied above. The prior art mappings for each of the secondary prior art references indicated below as well as the corresponding obviousness rationales are hereby incorporated by reference and account for any differences between the claim sets. Indeed, each of these prior art references and obviousness rationales have been extensively discussed above in relation to the corresponding prior art rejections.
The remaining and persistent difference between the instant claims and the reference claims is the field of use (thin canopy plant crop as opposed to tall plant crop). Nuske also teaches the instant claims’ thin canopy crop field of use as per rejection of claim 1 and it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified the reference claims’ field of use of tall plant crops for illuminating and imaging thin canopy plant crops as taught by Nuske particularly because the claims do not recite details of such a modified field of use such as, for example, specially adapted lights, cameras, and/or supporting frames for thin canopy plants as opposed to tall plant crops, because there is a reasonable expectation of success given such a general field of use and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
This is a provisional nonstatutory double patenting rejection.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
In the interests of compact prosecution, with respect to the other conditions in this Markush group see the cited NPL in the 892 form including Fahey {Fahey T, Pham H, Gardi A, Sabatini R, Stefanelli D, Goodwin I, Lamb DW. Active and Passive Electro-Optical Sensors for Health Assessment in Food Crops. Sensors (Basel). 2020 Dec 29;21(1):171. doi: 10.3390/s21010171. PMID: 33383831; PMCID: PMC7795220} and Ding {Ding Y, Jiang Y, Yu H, Yang C, Wu X, Sun G, Fu X, Dou X. Measurement Method for Height-Independent Vegetation Indices Based on an Active Light Source. Sensors (Basel). 2020 Mar 25;20(7):1830. doi: 10.3390/s20071830. PMID: 32218359; PMCID: PMC7180979}.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael R Cammarata whose telephone number is (571)272-0113. The examiner can normally be reached M-Th 7am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL ROBERT CAMMARATA/Primary Examiner, Art Unit 2667
Prosecution Insights