DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This action is in reply to the amendments filed on 07/01/2026.
Claims 1-20 are currently pending and have been examined.
Claims 1, 5, 8, 11, 13, 16, and 18 are amended.
Claims 1-20 are currently rejected.
This action is made FINAL.
Response to Arguments
Applicant’s arguments filed 07/01/2026 have been fully considered but they are not persuasive.
Applicant’s arguments with regards to the art rejections have been considered and appear to be directed solely to the instant amendments to the claims. Accordingly, the claims are addressed in the body of the updated rejections below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-8, 11-13, and 16-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fu et. al. (US 2022/0101554), herein Fu in view of Wu et. al. (US 2019/0150357), herein Wu, Redden (US 2020/0187406), herein Redden, Lin et. al. (CN 116152562), herein Lin, and Ascherl (US 2022/0125032), herein Ascherl.
Regarding claim 1:
Fu teaches:
A method (methods of identifying and treating plants [0063]) comprising:
Detecting (The farming machine 100, illustrated in FIGS. 1A-1E, includes a detection mechanism 110 [0079]), by an [autonomous] farming machine (fig. 1c, farming machine 100b), [a brightness level] at each of a plurality of points of a ground surface in front of a boom (The farming machine 100 can additionally include a mounting mechanism 140 [0079]) of the [autonomous] farming machine (a farming machine may utilize depth information obtained by a depth sensor coupled to identify and treat plants. For example, a farming machine may employ a light detection and ranging system (LIDAR) to identify and treat plants that are too tall, too short, too close, etc. [0070]), the boom comprising one or more lights (a light detection and ranging system (LIDAR)) configured to illuminate the ground surface in front of the boom when the autonomous farming machine operates (see at least fig. 3a showing the detecting means capturing the surface ahead of the machine.) [at night] (although not explicitly taught LIDAR would be able to capture data and work at night.);
Identifying, by the [autonomous] farming machine, a region of interest (The field of view 315, herein, is the angular extent of an area captured by a camera 310 [0104])
Detecting (The control system 130 may identify a plant in the image based on the depth information using a plant identification module (e.g., plant identification module 232) [0112]), by the [autonomous] farming machine, a plant within the identified region of interest (The field of view 315a includes several plants: crops 302a, 302b, 302c, and weed 350 [0106]);
Classifying (control system 130 can determine whether a plant is a weed or a crop [0112]), by the autonomous farming machine, the detected plant (The control system 130 may identify a plant in the image based on the depth information using a plant identification module (e.g., plant identification module 232) [0112])
Selecting, by the [autonomous] farming machine, an action based on the re-classified detected plant (the control system 130 may be configured to generate and take a treatment action for the identified plant based on the extracted depth information. For example, the control system 130 can determine whether a plant is a weed or a crop based on the height of the plant, and treat the plant accordingly [0112]); and
Performing, by the [autonomous] farming machine, the selected action (Operating parameters may include, for example, speed of the farming machine 200, direction of the farming machine 200, etc. Treatment parameters may include, for example, height of the treatment mechanism 120 (e.g., distance between the treatment mechanism 120 and the ground), type of treatment (e.g., spray, mechanical manipulation, etc.), time of treatment (e.g., at selected times, periodic intervals, length of treatment, a time delay between an image capture and a treatment, etc.), location of treatment (e.g., near the stem, treatment area 122), and/or other parameters related to treatment of the one or more plants [0200]) the measured location of the detected plant (Treatment parameters may include, for example, height of the treatment mechanism 120 (e.g., distance between the treatment mechanism 120 and the ground), type of treatment (e.g., spray, mechanical manipulation, etc.), time of treatment (e.g., at selected times, periodic intervals, length of treatment, a time delay between an image capture and a treatment, etc.), location of treatment (e.g., near the stem, treatment area 122), and/or other parameters related to treatment of the one or more plants [0200]).
While Fu teaches the ability to use lidar as a means to identify plants, Fu does not explicitly teach, however Wu teaches:
Detecting (If the differential value is past a threshold value or, for example, four sigma deviation, this is characterized as an anomaly and a spray is applied to treat the anomalous area [0105]),by an autonomous farming machine (an autonomous agricultural vehicle [0005]), a brightness level (triggering off of distant objects in the image versus nearby objects in the image is equalized by scaling the threshold for each pixel or grid member to constitute a “hit” [0089]) at each of a plurality of points of a ground surface in front of a boom of the autonomous farming machine (For night spraying, a pre-run calibration is performed to eliminate the shadow effect of the different lights shining on the ground (e.g. the headlights of the vehicle and boom lights illuminating the ground). The pre-run calibration also sets a threshold of what is considered “green” during the nighttime conditions. The night-time color that is associated with “green” is contrasted against the expected background color. When there is a large enough contrast signal past a pre-determined threshold value, a candidate object would trigger the herbicide to be released in the area of the candidate object [0115]), the boom comprising one or more lights configured to illuminate the ground surface in front of the boom when the autonomous farming machine operates (boom lights illuminating the ground [0115]) at night (For night spraying [0115]);
Identifying, by the autonomous farming machine, a region of interest (In some embodiments, to reduce the likelihood of false triggers, multiple pixels or at least a selected number of pixels are each required to satisfy one or more of the conditions stored or programmed into the computer processor in order to fully trigger the spray nozzle to actually release herbicide to kill the weeds. As an example, the selected number of pixels include adjacent pixels within a region of interest or distance among the pixels (e.g. all pixels within 10 pixel distance), and a peak value can be taken as a maximum average value among these pixels within such region of interest [0098]) corresponding to a portion of the ground surface in front of the boom corresponding to an above-threshold detected brightness level (When there is a large enough contrast signal past a pre-determined threshold value, a candidate object would trigger the herbicide to be released in the area of the candidate object [0115]; the magnitude of contrast instead of absolute color is another procedure or an additional procedure. The background color (e.g. ground or residue) is contrasted with the candidate signal color. When the contrast passes a threshold, herbicide is released in the area where the candidate object is found. [0150]; examiner notes that contrast is the difference between brightness levels which would therefore require some threshold brightness level in order to satisfy a contrast threshold.);
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Fu to include the teachings as taught by Wu with a reasonable expectation of success. Fu teaches a machine able to identify and treat weeds in a field but does not teach the ability to perform this task at night. Wu teaches the ability to perform this task at night and provides the benefits of “For night spraying, a pre-run calibration is performed to eliminate the shadow effect of the different lights shining on the ground (e.g. the headlights of the vehicle and boom lights illuminating the ground). The pre-run calibration also sets a threshold of what is considered “green” during the nighttime conditions. The night-time color that is associated with “green” is contrasted against the expected background color. When there is a large enough contrast signal past a pre-determined threshold value, a candidate object would trigger the herbicide to be released in the area of the candidate object [Wu, 0115].”
Fu and Wu do not explicitly teach, however Redden teaches:
classifying, by the autonomous farming machine, the detected plant by comparing the modified plant classification threshold (Each extracted point of interest preferably includes one or more features that a plant center is expected to exhibit. The points of interest can be dark regions surrounded by one or more colors associated with a plant (e.g. particularly when low incidence lighting is used to capture the image) [0020]) to a classification confidence value representing a likelihood that the detected plant belongs to a particular plant class (The point of interest is preferably classified as a plant center when the associated confidence level exceeds a predetermined threshold [0022])
re-analyzing, by the autonomous farming machine, the detected plant based on the measured location, at a subsequent time (The confidence level for the point of interest is preferably increased if the point of interest is identified or extracted from a predetermined area or pixel-neighborhood (after accounting for movement of the system) in a subsequent image S134 (as shown in FIG. 3B), and preferably decreased otherwise. The confidence level for each point of interest is preferably updated with each new image, wherein the decision whether retain or remove the plant associated with the point of interest is preferably made prior to the point of interest exiting the images. Alternatively, the confidence level of the point of interest can be proportional to the inverse of the square of the distance to the nearest point of interest to the estimated position of the first point of interest in the subsequent frame, after accounting for movement of the system [0022]) at which an amount of light satisfies a threshold brightness level, to re-classify the detected plant (The frequency, intensity, and/or any other suitable radiation parameter can be adjusted in response to a change in the imaging conditions (e.g. wherein more visible light needs to be provided to meet a brightness threshold in response to a decrease in visible light provided by the sun). [0037]);
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Fu and Wu to include the teachings as taught by Redden with a reasonable expectation of success. Redden teaches the ability to improve over the state of the art of automated plant removal by identifying that “these systems fail to offer the plant removal flexibility in plant selection and removal that human labor offers. In one example, a conventional crop thinning system removes plants at fixed intervals, whether or not the plant removal was necessary. In another example, a conventional crop thinning system removes plants using system vision, but fails to identify multiple close-packed plants as individual plants and treats the close-packed plants as a single plant. [Redden, 0003]”.
Fu, Wu, and Redden do not explicitly teach, however Lin teaches:
modifying, [by the autonomous farming machine], a plant classification threshold for the detected plant (dynamically setting threshold value, forming adaptive adjustment of threshold value, and performing colour classification according to the adjusted threshold value [page 6]) based on a brightness level of one or more image pixels depicting the detected plant (the preset dynamic threshold value is dynamically set according to the central point gray value range of the color stripe. It should be noted that the preset dynamic threshold value set based on the color corresponding to the angle of the color ring based on the actual color structure to be classified color structure light image in the grey value of the central point, namely the brightness of the setting, not a fixed threshold [page 6]);
comparing the modified plant classification threshold to a classification confidence value (performing colour classification according to the adjusted threshold value [page 6])
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Fu, Wu, and Redden to include the teachings as taught by Lin with a reasonable expectation of success. Lin teaches the benefit of “performing colour classification according to the adjusted threshold value, which is helpful for improving the accuracy of the classification. In practical application, the larger the brightness is easier to identify color, the smaller the influence is, the threshold value can be set more wide; the lower the brightness, the more it is affected by the environment light and object surface color, the threshold range is set to be smaller [Lin, page 6]”.
Ascherl also teaches:
re-analyzing, by the autonomous farming machine, the detected plant based on the measured location, at a subsequent time (the areas of the field can be stored and referenced later, such as, when analyzing yield values for different areas of the field [0117])
Fu, Wu, Redden, and Lin do not explicitly teach, however Ascherl teaches:
generating, by the autonomous farming machine, an analysis flag for the detected plant in response to the classification confidence value being below a reliability threshold (FIG. 6 also shows that map display 470 includes confidence level indicators 482, illustratively shown as flags placed at various locations along the traveled path of mobile machine 101, indicate confidence level values at various locations across a worksite. In some examples, confidence level indicators 482 indicate areas of the field where the confidence level value did not satisfy the confidence level value threshold. [0117]), the analysis flag including a measured location of the detected plant (the areas of the field can be stored and referenced later, such as, when analyzing yield values for different areas of the field. These are merely examples, indicators 482 can indicate a variety of characteristics or conditions. [0117]);
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Fu, Wu, Redden, and Lin to include the teachings as taught by Lin with a reasonable expectation of success. Ascherl teaches the benefit of “a confidence system configured to receive the stored data and generate a confidence level value indicative of a confidence in the ability of the mobile agricultural machine to perform the operation based on the stored data; and an action signal generator configured to generate an action signal to control an action of the mobile agricultural machine based on the confidence level value. [Ascherl, 0006]”.
Regarding claim 2:
Fu, Wu, Redden, Lin, and Ascherl teach all the limitations of claim 1, upon which this claim is dependent.
Wu further teaches:
wherein detecting the brightness level at each of the plurality of points of the ground surface in front of the boom (If the differential value is past a threshold value or, for example, four sigma deviation, this is characterized as an anomaly and a spray is applied to treat the anomalous area [0105]) comprises:
for a plurality of partitions of an image of the ground surface along a first dimension of the image (see at least fig. 4 showing grid 100 extending in a horizontal direction.):
determining the brightness level of a plurality of image pixels along a second dimension of the image (If the coarse grid or data indicates a “hit” or a possible activity (threshold detect), then a fine grid's data centered on the coarse grid may be stored or analyzed instantaneously. [0087]; the difference in values among the grid elements provides a trigger whether an anomaly is observed or detected [0105]), wherein the plurality of points correspond to the plurality of image pixels, and wherein the first dimension is orthogonal to the second dimension (see at least fig. 4 showing grid 100 extending in a vertical direction direction.).
Regarding claim 3:
Fu, Wu, Redden, Lin, and Ascherl teach all the limitations of claim 2, upon which this claim is dependent.
Wu further teaches:
wherein identifying the region of interest comprises determining a region within the image comprising pixels having the above-threshold detected brightness level (When there is a large enough contrast signal past a pre-determined threshold value, a candidate object would trigger the herbicide to be released in the area of the candidate object [0115]; the magnitude of contrast instead of absolute color is another procedure or an additional procedure. The background color (e.g. ground or residue) is contrasted with the candidate signal color. When the contrast passes a threshold, herbicide is released in the area where the candidate object is found. [0150]).
Regarding claim 4:
Fu, Wu, Redden, Lin, and Ascherl teach all the limitations of claim 3, upon which this claim is dependent.
Wu further teaches:
determining, along the second dimension, two or more boundary lines (fig. 4, crop rows 12), wherein the region comprises pixels having values in the second dimension that are bounded (see at least fig. 4 showing anomalies bounded by the crop rows.) by the two or more boundary lines (the image grid 100 is overlaid on or be associated with the soil, but can also be overlaid on the crop rows 12, or whatever image is captured by the image sensor units 50 [0096]).
Regarding claim 5:
Fu, Wu, Redden, and Lin teach all the limitations of claim 1, upon which this claim is dependent.
Fu further teaches:
wherein performing the selected action comprises performing the selected action at a delayed time (a time delay between an image capture and a treatment [0200])
determining the location of the region of interest (the control system 130 determines a treatment direction for the farming machine that allows the farming machine to correctly actuate a treat mechanism 120 to treat the weed 250 as it travels past the weed 250 in the field. Accordingly, the control system 130 actuates the systems of the farming machine such that it travels at the treatment velocity. In other examples, the farming machine may modify a sprayer timing or sprayer height to correctly actuate a treatment mechanism to treat the plant. Similarly, the control system 130 may determine a distance between each plant (e.g., a proximity) based on the feature values. The control system 130 can then determine an inter-treatment velocity that allows the farming machine to efficiently treat plants based on the proximity between plants (e.g., spray less if the plants are proximate). The control system 130 may then actuate systems of the farming machine 200 to apply the treatments as needed. [0232]); and
determining the delayed time (a time delay between an image capture and a treatment [0200]) based on a speed of the autonomous farming machine (speed of the farming machine 200 [0200]) and the location of the region of interest (location of treatment [0200]).
Regarding claim 6:
Fu, Wu, Redden, Lin, and Ascherl teach all the limitations of claim 1, upon which this claim is dependent.
Wu further teaches:
wherein region of interest comprises a fixed area (three weeds happen to be in the area between crop rows 12 where ideally they would not be expected. In some programs, the found unexpected objects are sprayed with herbicide, represented by a dashed circle [0097]).
Regarding claim 7:
Fu, Wu, Redden, Lin, and Ascherl teach all the limitations of claim 1, upon which this claim is dependent.
Fu further teaches:
for each of a plurality of heights at which the boom is configurable (a height of the treatment mechanism relative to the ground [0006]):
Wu further teaches:
wherein the region of interest is a first region of interest associated with the boom configured at a first height (the sensor units facing the corn cobs are tilted downwards by 10 to 40 degrees depending on the height of the back horizontal bar and depending on whether broader forward image capture is desired. [0134]), further comprising:
for each of a plurality of heights at which the boom is configurable (to adjust the height of the boom during field operation [0090]):
identifying a respective region of interest associated with the boom configured at the height (Equalize or normalize the elements (farther image elements are multiplied by some factor based on height at which the sensor is mounted, pointing angle, distance to the point of view [0185]), wherein the respective region of interest corresponds to a respective portion of the ground surface in front of the boom, the respective portion corresponding to the above-threshold detected brightness level (When there is a large enough contrast signal past a pre-determined threshold value, a candidate object would trigger the herbicide to be released in the area of the candidate object [0115]; the magnitude of contrast instead of absolute color is another procedure or an additional procedure. The background color (e.g. ground or residue) is contrasted with the candidate signal color. When the contrast passes a threshold, herbicide is released in the area where the candidate object is found. [0150]).
Regarding claim 8:
Fu, Wu, Redden, Lin, and Ascherl teach all the limitations of claim 1, upon which this claim is dependent.
Wu further teaches:
in response to determining, based on a first image, a first brightness level of a first point of the ground surface, determining the plant classification threshold for the detected plant (For night spraying, a pre-run calibration is performed to eliminate the shadow effect of the different lights shining on the ground (e.g. the headlights of the vehicle and boom lights illuminating the ground). The pre-run calibration also sets a threshold of what is considered “green” during the nighttime conditions. The night-time color that is associated with “green” is contrasted against the expected background color. [0115]), wherein the first brightness level satisfies the above-threshold detected brightness level (When there is a large enough contrast signal past a pre-determined threshold value, a candidate object would trigger the herbicide to be released in the area of the candidate object [0115]); and
in response to determining, based on a second image captured subsequent to the first image (Adjust real-time pattern recognition and calibration when major discrepancies are found [0185]), a second brightness level of a second point of the ground surface, modifying the plant classification threshold for the detected plant, wherein the second brightness level falls below the above-threshold detected brightness level (there is automatic calibration for in situ conditions such as color to adjust for detected lighting or time of day, to adjust the height of the boom during field operation based on calibrated crop height, and so on. [0090]).
Regarding claim 11:
Fu teaches:
A farming machine (A farming machine [abstract]) comprising:
a sensor configured for capturing images of plants in a field as the farming machine travels through the field (The detection mechanism 110 is configured to identify a plant for treatment. As such, the detection mechanism 110 can include one or more sensors for identifying a plant. For example, the detection mechanism 110 can include a multispectral camera, a stereo camera, a CCD camera, a single lens camera, a CMOS camera, hyperspectral imaging system, LIDAR system (light detection and ranging system), a depth sensing system, dynamometer, IR camera, thermal camera, humidity sensor, light sensor, temperature sensor, or any other suitable sensor. In one embodiment, and described in greater detail below, the detection mechanism 110 includes an array of image sensors configured to capture an image of a plant [0083]);
a treatment array configured to treat plants (The treatment mechanism 120 functions to apply a treatment to an identified plant 102 [0084]);
a boom (fig. 1c mounting mechanism 140) comprising one or more lights configured to illuminate a ground surface in front of the boom when the farming machine (a farming machine may utilize depth information obtained by a depth sensor coupled to identify and treat plants. For example, a farming machine may employ a light detection and ranging system (LIDAR) to identify and treat plants that are too tall, too short, too close, etc. [0070])
a processor (The processor 1602 is, for example, a central processing unit (CPU) [0241]);
a non-transitory computer readable storage medium (The computer system 1600 also includes a main memory 1604. The computer system may include a storage unit 1616 [0241]) storing computer program instructions (a computer program may be stored in a computer readable storage medium [0248]), the computer program instructions, when executed by the processor, causing the processor (The instructions 1624 may also reside, completely or at least partially, within the main memory 1604 or within the processor 1602 (e.g., within a processor's cache memory) during execution thereof by the computer system 1600 [0243]) to:
detect (The farming machine 100, illustrated in FIGS. 1A-1E, includes a detection mechanism 110 [0079]) a [brightness level] at each of a plurality of points of the ground surface in front of a boom of the farming machine (The farming machine 100 can additionally include a mounting mechanism 140 [0079]);
identify a region of interest corresponding to a portion of the ground surface in front of the boom (The field of view 315, herein, is the angular extent of an area captured by a camera 310 [0104])
detect (The control system 130 may identify a plant in the image based on the depth information using a plant identification module (e.g., plant identification module 232) [0112]) a plant within the identified region of interest (The field of view 315a includes several plants: crops 302a, 302b, 302c, and weed 350 [0106]);
select an action based on the re-classified detected plant (the control system 130 may be configured to generate and take a treatment action for the identified plant based on the extracted depth information. For example, the control system 130 can determine whether a plant is a weed or a crop based on the height of the plant, and treat the plant accordingly [0112]); and
perform the selected action (Operating parameters may include, for example, speed of the farming machine 200, direction of the farming machine 200, etc. Treatment parameters may include, for example, height of the treatment mechanism 120 (e.g., distance between the treatment mechanism 120 and the ground), type of treatment (e.g., spray, mechanical manipulation, etc.), time of treatment (e.g., at selected times, periodic intervals, length of treatment, a time delay between an image capture and a treatment, etc.), location of treatment (e.g., near the stem, treatment area 122), and/or other parameters related to treatment of the one or more plants [0200]) the measured location of the detected plant (Treatment parameters may include, for example, height of the treatment mechanism 120 (e.g., distance between the treatment mechanism 120 and the ground), type of treatment (e.g., spray, mechanical manipulation, etc.), time of treatment (e.g., at selected times, periodic intervals, length of treatment, a time delay between an image capture and a treatment, etc.), location of treatment (e.g., near the stem, treatment area 122), and/or other parameters related to treatment of the one or more plants [0200]).
While Fu teaches the ability to use lidar as a means to identify plants, Fu does not explicitly teach, however Wu teaches:
a boom comprising one or more lights (boom lights illuminating the ground [0115]) configured to illuminate a ground surface in front of the boom when the farming machine operates at night (For night spraying, a pre-run calibration is performed to eliminate the shadow effect of the different lights shining on the ground (e.g. the headlights of the vehicle and boom lights illuminating the ground). The pre-run calibration also sets a threshold of what is considered “green” during the nighttime conditions. The night-time color that is associated with “green” is contrasted against the expected background color. When there is a large enough contrast signal past a pre-determined threshold value, a candidate object would trigger the herbicide to be released in the area of the candidate object [0115]);
detect (If the differential value is past a threshold value or, for example, four sigma deviation, this is characterized as an anomaly and a spray is applied to treat the anomalous area [0105]) a brightness level (triggering off of distant objects in the image versus nearby objects in the image is equalized by scaling the threshold for each pixel or grid member to constitute a “hit” [0089]) at each of a plurality of points of the ground surface in front of a boom of the farming machine (an autonomous agricultural vehicle [0005]);
identify a region of interest (In some embodiments, to reduce the likelihood of false triggers, multiple pixels or at least a selected number of pixels are each required to satisfy one or more of the conditions stored or programmed into the computer processor in order to fully trigger the spray nozzle to actually release herbicide to kill the weeds. As an example, the selected number of pixels include adjacent pixels within a region of interest or distance among the pixels (e.g. all pixels within 10 pixel distance), and a peak value can be taken as a maximum average value among these pixels within such region of interest [0098]) corresponding to a portion of the ground surface in front of the boom corresponding to an above-threshold detected brightness level (When there is a large enough contrast signal past a pre-determined threshold value, a candidate object would trigger the herbicide to be released in the area of the candidate object [0115]; the magnitude of contrast instead of absolute color is another procedure or an additional procedure. The background color (e.g. ground or residue) is contrasted with the candidate signal color. When the contrast passes a threshold, herbicide is released in the area where the candidate object is found. [0150]; examiner notes that contrast is the difference between brightness levels which would therefore require some threshold brightness level in order to satisfy a contrast threshold.);
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Fu to include the teachings as taught by Wu with a reasonable expectation of success. Fu teaches a machine able to identify and treat weeds in a field but does not teach the ability to perform this task at night. Wu teaches the ability to perform this task at night and provides the benefits of “For night spraying, a pre-run calibration is performed to eliminate the shadow effect of the different lights shining on the ground (e.g. the headlights of the vehicle and boom lights illuminating the ground). The pre-run calibration also sets a threshold of what is considered “green” during the nighttime conditions. The night-time color that is associated with “green” is contrasted against the expected background color. When there is a large enough contrast signal past a pre-determined threshold value, a candidate object would trigger the herbicide to be released in the area of the candidate object [Wu, 0115].”
Fu and Wu do not explicitly teach, however Redden teaches:
classifying the detected plant by comparing the modified plant classification threshold (Each extracted point of interest preferably includes one or more features that a plant center is expected to exhibit. The points of interest can be dark regions surrounded by one or more colors associated with a plant (e.g. particularly when low incidence lighting is used to capture the image) [0020]) to a classification confidence value representing a likelihood that the detected plant belongs to a particular plant class (The point of interest is preferably classified as a plant center when the associated confidence level exceeds a predetermined threshold [0022])
re-analyzing, by the autonomous farming machine, the detected plant based on the measured location, at a subsequent time (The confidence level for the point of interest is preferably increased if the point of interest is identified or extracted from a predetermined area or pixel-neighborhood (after accounting for movement of the system) in a subsequent image S134 (as shown in FIG. 3B), and preferably decreased otherwise. The confidence level for each point of interest is preferably updated with each new image, wherein the decision whether retain or remove the plant associated with the point of interest is preferably made prior to the point of interest exiting the images. Alternatively, the confidence level of the point of interest can be proportional to the inverse of the square of the distance to the nearest point of interest to the estimated position of the first point of interest in the subsequent frame, after accounting for movement of the system [0022]) at which an amount of light satisfies a threshold brightness level, to re-classify the detected plant (The frequency, intensity, and/or any other suitable radiation parameter can be adjusted in response to a change in the imaging conditions (e.g. wherein more visible light needs to be provided to meet a brightness threshold in response to a decrease in visible light provided by the sun). [0037]);
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Fu and Wu to include the teachings as taught by Redden with a reasonable expectation of success. Redden teaches the ability to improve over the state of the art of automated plant removal by identifying that “these systems fail to offer the plant removal flexibility in plant selection and removal that human labor offers. In one example, a conventional crop thinning system removes plants at fixed intervals, whether or not the plant removal was necessary. In another example, a conventional crop thinning system removes plants using system vision, but fails to identify multiple close-packed plants as individual plants and treats the close-packed plants as a single plant. [Redden, 0003]”.
Fu, Wu, and Redden do not explicitly teach, however Lin teaches:
modifying a plant classification threshold for the detected plant (dynamically setting threshold value, forming adaptive adjustment of threshold value, and performing colour classification according to the adjusted threshold value [page 6]) based on a brightness level of one or more image pixels depicting the detected plant (the preset dynamic threshold value is dynamically set according to the central point gray value range of the color stripe. It should be noted that the preset dynamic threshold value set based on the color corresponding to the angle of the color ring based on the actual color structure to be classified color structure light image in the grey value of the central point, namely the brightness of the setting, not a fixed threshold [page 6]);
comparing the modified plant classification threshold to a classification confidence value (performing colour classification according to the adjusted threshold value [page 6])
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Fu, Wu, and Redden to include the teachings as taught by Lin with a reasonable expectation of success. Lin teaches the benefit of “performing colour classification according to the adjusted threshold value, which is helpful for improving the accuracy of the classification. In practical application, the larger the brightness is easier to identify color, the smaller the influence is, the threshold value can be set more wide; the lower the brightness, the more it is affected by the environment light and object surface color, the threshold range is set to be smaller [Lin, page 6]”.
Ascherl also teaches:
re-analyzing, by the autonomous farming machine, the detected plant based on the measured location, at a subsequent time (the areas of the field can be stored and referenced later, such as, when analyzing yield values for different areas of the field [0117])
Fu, Wu, Redden, and Lin do not explicitly teach, however Ascherl teaches:
generating, by the autonomous farming machine, an analysis flag for the detected plant in response to the classification confidence value being below a reliability threshold (FIG. 6 also shows that map display 470 includes confidence level indicators 482, illustratively shown as flags placed at various locations along the traveled path of mobile machine 101, indicate confidence level values at various locations across a worksite. In some examples, confidence level indicators 482 indicate areas of the field where the confidence level value did not satisfy the confidence level value threshold. [0117]), the analysis flag including a measured location of the detected plant (the areas of the field can be stored and referenced later, such as, when analyzing yield values for different areas of the field. These are merely examples, indicators 482 can indicate a variety of characteristics or conditions. [0117]);
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Fu, Wu, Redden, and Lin to include the teachings as taught by Lin with a reasonable expectation of success. Ascherl teaches the benefit of “a confidence system configured to receive the stored data and generate a confidence level value indicative of a confidence in the ability of the mobile agricultural machine to perform the operation based on the stored data; and an action signal generator configured to generate an action signal to control an action of the mobile agricultural machine based on the confidence level value. [Ascherl, 0006]”.
Regarding claim 12:
Fu, Wu, Redden, Lin, and Ascherl teach all the limitations of claim 11, upon which this claim is dependent.
Fu further teaches:
for each of a plurality of heights at which the boom is configurable (a height of the treatment mechanism relative to the ground [0006]):
Wu further teaches:
wherein the region of interest is a first region of interest associated with the boom configured at a first height (the sensor units facing the corn cobs are tilted downwards by 10 to 40 degrees depending on the height of the back horizontal bar and depending on whether broader forward image capture is desired. [0134]),
for each of a plurality of heights at which the boom is configurable (to adjust the height of the boom during field operation [0090]):
identifying a respective region of interest associated with the boom configured at the height (Equalize or normalize the elements (farther image elements are multiplied by some factor based on height at which the sensor is mounted, pointing angle, distance to the point of view [0185]), wherein the respective region of interest corresponds to a respective portion of the ground surface in front of the boom, the respective portion corresponding to the above-threshold detected brightness level (When there is a large enough contrast signal past a pre-determined threshold value, a candidate object would trigger the herbicide to be released in the area of the candidate object [0115]; the magnitude of contrast instead of absolute color is another procedure or an additional procedure. The background color (e.g. ground or residue) is contrasted with the candidate signal color. When the contrast passes a threshold, herbicide is released in the area where the candidate object is found. [0150]).
Regarding claim 13:
Fu, Wu, Redden, Lin, and Ascherl teach all the limitations of claim 11, upon which this claim is dependent.
Wu further teaches:
in response to determining, based on a first image, a first brightness level of a first point of the ground surface, determining the plant classification threshold for the detected plant (For night spraying, a pre-run calibration is performed to eliminate the shadow effect of the different lights shining on the ground (e.g. the headlights of the vehicle and boom lights illuminating the ground). The pre-run calibration also sets a threshold of what is considered “green” during the nighttime conditions. The night-time color that is associated with “green” is contrasted against the expected background color. [0115]), wherein the first brightness level satisfies the above-threshold detected brightness level (When there is a large enough contrast signal past a pre-determined threshold value, a candidate object would trigger the herbicide to be released in the area of the candidate object [0115]); and
in response to determining, based on a second image captured subsequent to the first image (Adjust real-time pattern recognition and calibration when major discrepancies are found [0185]), a second brightness level of a second point of the ground surface, modifying the plant classification threshold for the detected plant, wherein the second brightness level falls below the above-threshold detected brightness level (there is automatic calibration for in situ conditions such as color to adjust for detected lighting or time of day, to adjust the height of the boom during field operation based on calibrated crop height, and so on. [0090]).
Regarding claim 16:
Fu teaches:
A non-transitory computer readable storage medium (The computer system 1600 also includes a main memory 1604. The computer system may include a storage unit 1616 [0241]) storing computer program storing instructions (a computer program may be stored in a computer readable storage medium [0248]), the computer program instructions, when executed by a processor (The instructions 1624 may also reside, completely or at least partially, within the main memory 1604 or within the processor 1602 (e.g., within a processor's cache memory) during execution thereof by the computer system 1600 [0243]), causing the processor to:
detect (The farming machine 100, illustrated in FIGS. 1A-1E, includes a detection mechanism 110 [0079]) a [brightness level] at each of a plurality of points of the ground surface in front of a boom of the farming machine (The farming machine 100 can additionally include a mounting mechanism 140 [0079]);
identify a region of interest corresponding to a portion of the ground surface in front of the boom (The field of view 315, herein, is the angular extent of an area captured by a camera 310 [0104])
detect (The control system 130 may identify a plant in the image based on the depth information using a plant identification module (e.g., plant identification module 232) [0112]) a plant within the identified region of interest (The field of view 315a includes several plants: crops 302a, 302b, 302c, and weed 350 [0106]);
select an action based on the re-classified detected plant (the control system 130 may be configured to generate and take a treatment action for the identified plant based on the extracted depth information. For example, the control system 130 can determine whether a plant is a weed or a crop based on the height of the plant, and treat the plant accordingly [0112]); and
perform the selected action (Operating parameters may include, for example, speed of the farming machine 200, direction of the farming machine 200, etc. Treatment parameters may include, for example, height of the treatment mechanism 120 (e.g., distance between the treatment mechanism 120 and the ground), type of treatment (e.g., spray, mechanical manipulation, etc.), time of treatment (e.g., at selected times, periodic intervals, length of treatment, a time delay between an image capture and a treatment, etc.), location of treatment (e.g., near the stem, treatment area 122), and/or other parameters related to treatment of the one or more plants [0200]) the measured location of the detected plant (Treatment parameters may include, for example, height of the treatment mechanism 120 (e.g., distance between the treatment mechanism 120 and the ground), type of treatment (e.g., spray, mechanical manipulation, etc.), time of treatment (e.g., at selected times, periodic intervals, length of treatment, a time delay between an image capture and a treatment, etc.), location of treatment (e.g., near the stem, treatment area 122), and/or other parameters related to treatment of the one or more plants [0200]).
While Fu teaches the ability to use lidar as a means to identify plants, Fu does not explicitly teach, however Wu teaches:
a boom comprising one or more lights (boom lights illuminating the ground [0115]) configured to illuminate a ground surface in front of the boom when the farming machine operates at night (For night spraying, a pre-run calibration is performed to eliminate the shadow effect of the different lights shining on the ground (e.g. the headlights of the vehicle and boom lights illuminating the ground). The pre-run calibration also sets a threshold of what is considered “green” during the nighttime conditions. The night-time color that is associated with “green” is contrasted against the expected background color. When there is a large enough contrast signal past a pre-determined threshold value, a candidate object would trigger the herbicide to be released in the area of the candidate object [0115]);
detect (If the differential value is past a threshold value or, for example, four sigma deviation, this is characterized as an anomaly and a spray is applied to treat the anomalous area [0105]) a brightness level (triggering off of distant objects in the image versus nearby objects in the image is equalized by scaling the threshold for each pixel or grid member to constitute a “hit” [0089]) at each of a plurality of points of the ground surface in front of a boom of the farming machine (an autonomous agricultural vehicle [0005]);
identify a region of interest (In some embodiments, to reduce the likelihood of false triggers, multiple pixels or at least a selected number of pixels are each required to satisfy one or more of the conditions stored or programmed into the computer processor in order to fully trigger the spray nozzle to actually release herbicide to kill the weeds. As an example, the selected number of pixels include adjacent pixels within a region of interest or distance among the pixels (e.g. all pixels within 10 pixel distance), and a peak value can be taken as a maximum average value among these pixels within such region of interest [0098]) corresponding to a portion of the ground surface in front of the boom corresponding to an above-threshold detected brightness level (When there is a large enough contrast signal past a pre-determined threshold value, a candidate object would trigger the herbicide to be released in the area of the candidate object [0115]; the magnitude of contrast instead of absolute color is another procedure or an additional procedure. The background color (e.g. ground or residue) is contrasted with the candidate signal color. When the contrast passes a threshold, herbicide is released in the area where the candidate object is found. [0150]; examiner notes that contrast is the difference between brightness levels which would therefore require some threshold brightness level in order to satisfy a contrast threshold.);
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Fu to include the teachings as taught by Wu with a reasonable expectation of success. Fu teaches a machine able to identify and treat weeds in a field but does not teach the ability to perform this task at night. Wu teaches the ability to perform this task at night and provides the benefits of “For night spraying, a pre-run calibration is performed to eliminate the shadow effect of the different lights shining on the ground (e.g. the headlights of the vehicle and boom lights illuminating the ground). The pre-run calibration also sets a threshold of what is considered “green” during the nighttime conditions. The night-time color that is associated with “green” is contrasted against the expected background color. When there is a large enough contrast signal past a pre-determined threshold value, a candidate object would trigger the herbicide to be released in the area of the candidate object [Wu, 0115].”
Fu and Wu do not explicitly teach, however Redden teaches:
classifying the detected plant by comparing the modified plant classification threshold (Each extracted point of interest preferably includes one or more features that a plant center is expected to exhibit. The points of interest can be dark regions surrounded by one or more colors associated with a plant (e.g. particularly when low incidence lighting is used to capture the image) [0020]) to a classification confidence value representing a likelihood that the detected plant belongs to a particular plant class (The point of interest is preferably classified as a plant center when the associated confidence level exceeds a predetermined threshold [0022])
re-analyzing, by the autonomous farming machine, the detected plant based on the measured location, at a subsequent time (The confidence level for the point of interest is preferably increased if the point of interest is identified or extracted from a predetermined area or pixel-neighborhood (after accounting for movement of the system) in a subsequent image S134 (as shown in FIG. 3B), and preferably decreased otherwise. The confidence level for each point of interest is preferably updated with each new image, wherein the decision whether retain or remove the plant associated with the point of interest is preferably made prior to the point of interest exiting the images. Alternatively, the confidence level of the point of interest can be proportional to the inverse of the square of the distance to the nearest point of interest to the estimated position of the first point of interest in the subsequent frame, after accounting for movement of the system [0022]) at which an amount of light satisfies a threshold brightness level, to re-classify the detected plant (The frequency, intensity, and/or any other suitable radiation parameter can be adjusted in response to a change in the imaging conditions (e.g. wherein more visible light needs to be provided to meet a brightness threshold in response to a decrease in visible light provided by the sun). [0037]);
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Fu and Wu to include the teachings as taught by Redden with a reasonable expectation of success. Redden teaches the ability to improve over the state of the art of automated plant removal by identifying that “these systems fail to offer the plant removal flexibility in plant selection and removal that human labor offers. In one example, a conventional crop thinning system removes plants at fixed intervals, whether or not the plant removal was necessary. In another example, a conventional crop thinning system removes plants using system vision, but fails to identify multiple close-packed plants as individual plants and treats the close-packed plants as a single plant. [Redden, 0003]”.
Fu, Wu, and Redden do not explicitly teach, however Lin teaches:
modifying a plant classification threshold for the detected plant (dynamically setting threshold value, forming adaptive adjustment of threshold value, and performing colour classification according to the adjusted threshold value [page 6]) based on a brightness level of one or more image pixels depicting the detected plant (the preset dynamic threshold value is dynamically set according to the central point gray value range of the color stripe. It should be noted that the preset dynamic threshold value set based on the color corresponding to the angle of the color ring based on the actual color structure to be classified color structure light image in the grey value of the central point, namely the brightness of the setting, not a fixed threshold [page 6]);
comparing the modified plant classification threshold to a classification confidence value (performing colour classification according to the adjusted threshold value [page 6])
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Fu, Wu, and Redden to include the teachings as taught by Lin with a reasonable expectation of success. Lin teaches the benefit of “performing colour classification according to the adjusted threshold value, which is helpful for improving the accuracy of the classification. In practical application, the larger the brightness is easier to identify color, the smaller the influence is, the threshold value can be set more wide; the lower the brightness, the more it is affected by the environment light and object surface color, the threshold range is set to be smaller [Lin, page 6]”.
Ascherl also teaches:
re-analyzing, by the autonomous farming machine, the detected plant based on the measured location, at a subsequent time (the areas of the field can be stored and referenced later, such as, when analyzing yield values for different areas of the field [0117])
Fu, Wu, Redden, and Lin do not explicitly teach, however Ascherl teaches:
generating, by the autonomous farming machine, an analysis flag for the detected plant in response to the classification confidence value being below a reliability threshold (FIG. 6 also shows that map display 470 includes confidence level indicators 482, illustratively shown as flags placed at various locations along the traveled path of mobile machine 101, indicate confidence level values at various locations across a worksite. In some examples, confidence level indicators 482 indicate areas of the field where the confidence level value did not satisfy the confidence level value threshold. [0117]), the analysis flag including a measured location of the detected plant (the areas of the field can be stored and referenced later, such as, when analyzing yield values for different areas of the field. These are merely examples, indicators 482 can indicate a variety of characteristics or conditions. [0117]);
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Fu, Wu, Redden, and Lin to include the teachings as taught by Lin with a reasonable expectation of success. Ascherl teaches the benefit of “a confidence system configured to receive the stored data and generate a confidence level value indicative of a confidence in the ability of the mobile agricultural machine to perform the operation based on the stored data; and an action signal generator configured to generate an action signal to control an action of the mobile agricultural machine based on the confidence level value. [Ascherl, 0006]”.
Regarding claim 17:
Fu, Wu, Redden, Lin, and Ascherl teach all the limitations of claim 16, upon which this claim is dependent.
Fu further teaches:
for each of a plurality of heights at which the boom is configurable (a height of the treatment mechanism relative to the ground [0006]):
Wu further teaches:
wherein the region of interest is a first region of interest associated with the boom configured at a first height (the sensor units facing the corn cobs are tilted downwards by 10 to 40 degrees depending on the height of the back horizontal bar and depending on whether broader forward image capture is desired. [0134]), further comprising:
for each of a plurality of heights at which the boom is configurable (to adjust the height of the boom during field operation [0090]):
identifying a respective region of interest associated with the boom configured at the height (Equalize or normalize the elements (farther image elements are multiplied by some factor based on height at which the sensor is mounted, pointing angle, distance to the point of view [0185]), wherein the respective region of interest corresponds to a respective portion of the ground surface in front of the boom, the respective portion corresponding to the above-threshold detected brightness level (When there is a large enough contrast signal past a pre-determined threshold value, a candidate object would trigger the herbicide to be released in the area of the candidate object [0115]; the magnitude of contrast instead of absolute color is another procedure or an additional procedure. The background color (e.g. ground or residue) is contrasted with the candidate signal color. When the contrast passes a threshold, herbicide is released in the area where the candidate object is found. [0150]).
Regarding claim 18:
Fu, Wu, Redden, Lin, and Ascherl teach all the limitations of claim 16, upon which this claim is dependent.
Wu further teaches:
in response to determining, based on a first image, a first brightness level of a first point of the ground surface, determining the classification threshold for the detected plant (For night spraying, a pre-run calibration is performed to eliminate the shadow effect of the different lights shining on the ground (e.g. the headlights of the vehicle and boom lights illuminating the ground). The pre-run calibration also sets a threshold of what is considered “green” during the nighttime conditions. The night-time color that is associated with “green” is contrasted against the expected background color. [0115]), wherein the first brightness level satisfies the above-threshold detected brightness level (When there is a large enough contrast signal past a pre-determined threshold value, a candidate object would trigger the herbicide to be released in the area of the candidate object [0115]); and
in response to determining, based on a second image captured subsequent to the first image (Adjust real-time pattern recognition and calibration when major discrepancies are found [0185]), a second brightness level of a second point of the ground surface, modifying the plant classification threshold for the detected plant, wherein the second brightness level falls below the above-threshold detected brightness level (there is automatic calibration for in situ conditions such as color to adjust for detected lighting or time of day, to adjust the height of the boom during field operation based on calibrated crop height, and so on. [0090]).
Claim(s) 9-10, 14-15, and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fu et. al. (US 2022/0101554), herein Fu in view of Wu et. al. (US 2019/0150357), herein Wu, Redden (US 2020/0187406), herein Redden, Lin et. al. (CN 116152562), herein Lin, and Ascherl (US 2022/0125032), herein Ascherl. in further view of Hu et. al. (CN 115278099), herein Hu.
Regarding claim 9:
Fu, Wu, Redden, Lin, and Ascherl teach all the limitations of claim 1, upon which this claim is dependent.
Wu further teaches:
determining a number of points of the ground surface in front of the boom having a brightness level at the above-threshold detected brightness level (fig. 4, anomalies 11);
Fu, Wu, Redden, Lin, and Ascherl do not explicitly teach, however Hu teaches:
determining a number of points of the ground surface in front of the boom having a brightness level at the above-threshold detected brightness level (determining the brightness value of each pixel point in the initial image [Hu]);
determining, based on the number of points, whether the one or more lights is illuminating the ground surface in front of the boom at the above-threshold detected brightness level (determining the area composed of the pixel points less than the preset brightness threshold in all the brightness values as the dark area; determining the external rectangle of the dark area; and determining the area included in the external rectangle as the target area. [Hu]).
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Fu, Wu, Redden, Lin, and Ascherl to include the teachings as taught by Hu with a reasonable expectation of success. Hu teaches “when the user uses the rear camera of the terminal device to shoot the content of the book in parallel, if the whole environment light is dark, then the image quality of shooting can be improved turn on the light supplementing lamp. However, if the whole ambient light is not dark when shooting, but the part of the light above the book is shielded to form a local shadow, if not turn on the shooting effect of the book shadow area will not be good. Therefore, how to intelligently and accurately control the fill light of the camera to fill light into the problem urgently needs to be solved [Hu, background]”.
Regarding claim 10:
Fu, Wu, Redden, Lin, Ascherl and Hu teach all the limitations of claim 9, upon which this claim is dependent.
Hu further teaches:
in response to determining that the one or more lights is illuminating the ground surface in front of the boom below the above-threshold detected brightness level (when detecting that the target image in the brightness value is less than the preset brightness threshold value of the area, prompt message,),
generating a notification that the one or more lights are mounted at an incorrect angle (the prompt message is used for prompting the user to adjust the angle of the terminal device, so that the light to the brightness value is less than the preset brightness threshold value of the area by the fill light.).
Regarding claim 14:
Fu, Wu, Redden, Lin, and Ascherl teach all the limitations of claim 11, upon which this claim is dependent.
Wu further teaches:
determine a number of points of the ground surface in front of the boom having a brightness level at the above-threshold detected brightness level (fig. 4, anomalies 11);
Fu, Wu, Redden, Lin, and Ascherl do not explicitly teach, however Hu teaches:
determine a number of points of the ground surface in front of the boom having a brightness level at the above-threshold detected brightness level (determining the brightness value of each pixel point in the initial image [Hu]);
determine, based on the number of points, whether the one or more lights is illuminating the ground surface in front of the boom at the above-threshold detected brightness level (determining the area composed of the pixel points less than the preset brightness threshold in all the brightness values as the dark area; determining the external rectangle of the dark area; and determining the area included in the external rectangle as the target area. [Hu]).
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Fu, Wu, Redden, Lin, and Ascherl to include the teachings as taught by Hu with a reasonable expectation of success. Hu teaches “when the user uses the rear camera of the terminal device to shoot the content of the book in parallel, if the whole environment light is dark, then the image quality of shooting can be improved turn on the light supplementing lamp. However, if the whole ambient light is not dark when shooting, but the part of the light above the book is shielded to form a local shadow, if not turn on the shooting effect of the book shadow area will not be good. Therefore, how to intelligently and accurately control the fill light of the camera to fill light into the problem urgently needs to be solved [Hu, background]”.
Regarding claim 15:
Fu, Wu, Redden, Lin, Ascherl and Hu teach all the limitations of claim 14, upon which this claim is dependent.
Hu further teaches:
in response to determining that the one or more lights is illuminating the ground surface in front of the boom below the above-threshold detected brightness level (when detecting that the target image in the brightness value is less than the preset brightness threshold value of the area, prompt message,),
generate a notification that the one or more lights are mounted at an incorrect angle (the prompt message is used for prompting the user to adjust the angle of the terminal device, so that the light to the brightness value is less than the preset brightness threshold value of the area by the fill light.).
Regarding claim 19:
Fu, Wu, Redden, Lin, and Ascherl teach all the limitations of claim 16, upon which this claim is dependent.
Wu further teaches:
determine a number of points of the ground surface in front of the boom having a brightness level at the above-threshold detected brightness level (fig. 4, anomalies 11);
Fu, Wu, Redden, Lin, and Ascherl do not explicitly teach, however Hu teaches:
determine a number of points of the ground surface in front of the boom having a brightness level at the above-threshold detected brightness level (determining the brightness value of each pixel point in the initial image [Hu]);
determine, based on the number of points, whether the one or more lights is illuminating the ground surface in front of the boom at the above-threshold detected brightness level (determining the area composed of the pixel points less than the preset brightness threshold in all the brightness values as the dark area; determining the external rectangle of the dark area; and determining the area included in the external rectangle as the target area. [Hu]).
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Fu, Wu, Redden, Lin, and Ascherl to include the teachings as taught by Hu with a reasonable expectation of success. Hu teaches “when the user uses the rear camera of the terminal device to shoot the content of the book in parallel, if the whole environment light is dark, then the image quality of shooting can be improved turn on the light supplementing lamp. However, if the whole ambient light is not dark when shooting, but the part of the light above the book is shielded to form a local shadow, if not turn on the shooting effect of the book shadow area will not be good. Therefore, how to intelligently and accurately control the fill light of the camera to fill light into the problem urgently needs to be solved [Hu, background]”.
Regarding claim 20:
Fu, Wu, Redden, Lin, Ascherl and Hu teach all the limitations of claim 19, upon which this claim is dependent.
Hu further teaches:
in response to determining that the one or more lights is illuminating the ground surface in front of the boom below the above-threshold detected brightness level (when detecting that the target image in the brightness value is less than the preset brightness threshold value of the area, prompt message,),
generate a notification that the one or more lights are mounted at an incorrect angle (the prompt message is used for prompting the user to adjust the angle of the terminal device, so that the light to the brightness value is less than the preset brightness threshold value of the area by the fill light.).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Faers (WO 2021/180474) discloses an unmanned aerial vehicle (UAV) for agricultural weed management. The UAV comprises a control and processing unit (20), and a camera (30). The control and processing unit is configured to control the UAV to fly to a location inside the canopy of a crop and below the vertical height of the crop and/or between a row of a plurality of crops and below the vertical height of the plurality of crops. The control and processing unit is configured to control the camera to acquire at least one image relating to the ground at the location inside the canopy of a crop and below the vertical height of the crop and/or between a row of a plurality of crops and below the vertical height of the plurality of crops. The control and processing unit is configured to analyse the at least one image to determine the presence of at least one weed and its location on the ground.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Scott R Jagolinzer whose telephone number is (571)272-4180. The examiner can normally be reached M-Th 8AM - 4PM Eastern.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christian Chace can be reached at (571)272-4190. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Scott R. Jagolinzer
Examiner
Art Unit 3665
/S.R.J./Examiner, Art Unit 3665 /CHRISTIAN CHACE/Supervisory Patent Examiner, Art Unit 3665
Prosecution Insights