DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Amended claims 1 thru 5 and 7 thru 21 have been entered into the record. Claim 6 has been cancelled.
Response to Amendment
The amendments to Figure 13 and to the specification overcome the drawing objections from the previous office action (11/26/2025). The drawing objections are withdrawn.
The amendment to claim 4 overcomes the claim objection from the previous office action (11/26/2025). The claim objection is withdrawn.
The amendments to the claims overcome the 35 U.S.C. 112(b) rejections from the previous office action (11/26/2025). The 35 U.S.C. 112(b) rejections are withdrawn.
The amendments to the claims overcome the 35 U.S.C. 101 rejections from the previous office action (11/26/2025). The 35 U.S.C. 101 rejections are withdrawn.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 thru 5 and 7 thru 14 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Claims 15 thru 21 are new claims without any currently pending rejections.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 16 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 16 recites “a trigger” in line 3, while “a trigger” is also recited in line 6 of claim 15. It is unclear if this is a new trigger or the same trigger. The examiner assumes it is the same trigger for continued examination.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 6, 7, 10, 13 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al Patent Number 9,234,618 B1 in view of Wilson et al Patent Application Publication Number 2021/0382490 A1.
Regarding claim 1 Zhu et al teach the claimed obstacle detection system, “The computer vision system 140 can process and analyze images captured by camera 130 to identify objects and/or features in the environment surrounding vehicle 100. The detected features/objects can include traffic signals, road way boundaries, other vehicles, pedestrians, and/or obstacles, etc.” (column 9 lines 4 thru 8), for the claimed agricultural machine, “an example system may also be implemented in or take the form of other vehicles, such as cars, trucks, motorcycles, buses, boats, airplanes, helicopters, lawn mowers, earth movers, boats, snowmobiles, aircraft, recreational vehicles, amusement park vehicles, farm equipment, construction equipment, trams, golf carts, trains, and trolleys” (column 6 lines 39 thru 44), to perform the claimed self-driving, “Example embodiments relate to an autonomous vehicle, such as a driverless automobile” (column 4 lines 12 and 13), while the claimed sensing a surrounding environment with a LIDAR sensor and a camera, “that includes a light detection and ranging (LIDAR) sensor for actively detecting reflective features in the environment surrounding the vehicle” (column 4 lines 13 thru 15), “the laser rangefinder or LIDAR unit 128 can be any sensor configured to sense objects in the environment in which the vehicle 100 is located” (column 8 lines 14 thru 16), and “The camera 130 can include one or more devices configured to capture a plurality of images of the environment surrounding the vehicle 100.” (column 7 lines 22 thru 24), the obstacle detection system comprising:
the claimed controller, “a computer system 112 can control the vehicle 100 while in an autonomous mode via control instructions to a control system 106 for the vehicle 100” (column 6 lines 50 thru 52), configured or programmed to:
the claimed cause the camera to acquire an image of an obstacle candidate upon detecting the obstacle candidate based on data output from the LIDAR sensor, “FIG. 7 is a flowchart 700 of a process for navigating an autonomous vehicle according to real time environmental feedback information from both the LIDAR device 302 and the hyperspectral sensor 620. The LIDAR device 302 is scanned through a scanning zone surrounding the vehicle (702). Information from reflected light signals provided by the LIDAR device 302 and/or associated optical detectors is analyzed to generate a 3-D point cloud of positions of reflective features in the scanning zone (704). The 3-D point cloud is analyzed via the controller 610, the sensor fusion algorithm 138, the computer vision system 140, and/or the object detection module described above, etc. to identify regions of the scanning zone for spectral analysis (706). The region identified for spectral analysis (706) can be a region including a LIDAR-indicated reflective feature/object.” (column 20 lines 48 thru 62, and Figure 7) (claimed data output from the LIDAR sensor), “The identified region is identified with the hyperspectral sensor 620 to characterize the region according to its spectral properties (708). The spectral information is used to determine whether the region includes a solid material (710).” (column 21 lines 6 thru 9 and Figure 7) (claimed camera to acquire an image of an obstacle candidate), and “the hyperspectral sensor 620 can include both imaging optics and a spectral selectivity module. The imaging optics can be one or more lenses, mirrors, shutters, and/or apertures arranged to focus received radiation on an imaging plane that includes a photo sensitive detector, such as a charge coupled device array, or a similar detector for generating electrical signals related to an intensity pattern in the imaging plane.” (column 18 lines 53 thru 60), the hyperspectral sensor equates to the claimed camera, and the solid material equates to the claimed obstacle candidate;
the claimed determine whether or not to change a traveling status of the agricultural machine based on the image of the obstacle candidate acquired with the camera, “The 3-D point cloud information from the LIDAR device can be combined with the indications of whether reflective features are solid or not solid to map solid objects in the scanning zone (712).” (column 21 lines 30 thru 34, and Figure 7), and “The autonomous vehicle is navigated to avoid interference with objects in the map of solid objects 630 (714). In some embodiments, non-solid LIDAR-indicated features, such as water spray patterns and/or exhaust plumes are substantially ignored by the navigation control systems operating the autonomous vehicle. For example, the object avoidance system 144 and/or navigation/pathing system 142 can be provided with the map of solid objects 630 to automatically determine a path for the autonomous vehicle that avoids solid objects without avoiding splashes of water or exhaust plumes even if optically opaque. Thus, in some embodiments of the present disclosure, information from both the LIDAR device 302 and the hyperspectral sensor 620 is combined generate the map of solid objects 630.” (column 21 lines 40 thru 53, and Figure 7); and
the claimed cause the agricultural machine to stop traveling or to decelerate upon detecting the obstacle candidate as the trigger, the obstacle avoidance system 144 can effect changes in the navigation of the vehicle by operating one or more subsystems in the control system 106 to undertake braking maneuvers (column 9 lines 36 thru 40).
Zhu et al do not teach the claimed cause the agricultural machine to stop or decelerate upon detecting the obstacle before determining whether or not to change the traveling status of the agricultural machine. The examiner interprets this limitation as an ordering of the control/program steps. Persons having ordinary skill in the art understand how to implement changes to the order of steps.
Wilson et al teach, the autonomy controller 210 reduce speed when there is a probable object and the machine controller 220 drives the mower at reduce speed along the specified path (Figure 6B), the autonomy controller 210 proceeds to send an alert, identify a region of interest, and tighten tolerance on the ground plane algorithm (Figure 6B), the autonomy controller 210 proceeds to process the buffered region of interest and confirm the object (Figure 6C), and when the object is confirmed, a path adjustment is determined and performed by the machine controller 220 (Figure 6C). The path adjustment equates to the claimed change of traveling status, and the reduced speed of Wilson et al occurs before the path adjustment.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the computer vision system of Zhu et al with the reducing speed before the path adjustment of Willson et al in order to, with a reasonable expectation of success, avoid running over objects (Wilson et al P[0040]).
Regarding claim 2 Zhu et al and Wilson et al teach the claimed system of claim 1 (see above). Zhu et al further teach, wherein the controller is further configured or programmed to:
the claimed detect an object within the surrounding environment based on data that is output from the LIDAR sensor, “The LIDAR device 302 is scanned through a scanning zone surrounding the vehicle (702). Information from reflected light signals provided by the LIDAR device 302 and/or associated optical detectors is analyzed to generate a 3-D point cloud of positions of reflective features in the scanning zone (704). The 3-D point cloud is analyzed via the controller 610, the sensor fusion algorithm 138, the computer vision system 140, and/or the object detection module described above, etc. to identify regions of the scanning zone for spectral analysis (706). The region identified for spectral analysis (706) can be a region including a LIDAR-indicated reflective feature/object.” (column 20 lines 51 thru 62, and Figure 7); and
the claimed when the detected object is the obstacle candidate, cause the camera to acquire the image of the obstacle candidate and determine whether or not to change the traveling status of the agricultural machine based on the image, “The identified region is identified with the hyperspectral sensor 620 to characterize the region according to its spectral properties (708). The spectral information is used to determine whether the region includes a solid material (710).” (column 21 lines 6 thru 9 and Figure 7), and “The 3-D point cloud information from the LIDAR device can be combined with the indications of whether reflective features are solid or not solid to map solid objects in the scanning zone (712).” (column 21 lines 30 thru 34, and Figure 7), and “The autonomous vehicle is navigated to avoid interference with objects in the map of solid objects 630 (714). In some embodiments, non-solid LIDAR-indicated features, such as water spray patterns and/or exhaust plumes are substantially ignored by the navigation control systems operating the autonomous vehicle. For example, the object avoidance system 144 and/or navigation/pathing system 142 can be provided with the map of solid objects 630 to automatically determine a path for the autonomous vehicle that avoids solid objects without avoiding splashes of water or exhaust plumes even if optically opaque. Thus, in some embodiments of the present disclosure, information from both the LIDAR device 302 and the hyperspectral sensor 620 is combined generate the map of solid objects 630.” (column 21 lines 40 thru 53, and Figure 7).
Regarding claim 7 Zhu et al and Wilson et al teach the claimed system of claim 1 (see above). Zhu et al further teach, wherein the controller is further configured or programmed to: the claimed cause the agricultural machine to resume travel or accelerate when determining not to change the traveling status, “the obstacle avoidance system 144 can be configured such that a swerving maneuver is not undertaken when other sensor systems detect vehicles, construction barriers, other obstacles, etc.” (column 9 lines 44 thru 48), and “Non-solid features to not avoid (i.e., to ignore) can be, for example, LIDAR-indicated reflective features with corresponding spectral information associated with a non-solid material.” (column 20 lines 41 thru 44).
Regarding claim 10 Zhu et al and Wilson et al teach the claimed system of claim 1 (see above). Zhu et al further teach, wherein, the claimed if a field of view of the camera does not include the obstacle candidate when the obstacle candidate is detected, the controller changes an orientation of the camera so that the field of view of the camera includes the obstacle candidate, “sensor unit 202 can include any combination of cameras, RADARs, LIDARs, range finders, and acoustic sensors. The sensor unit 202 can include one or more movable mounts that could be operable to adjust the orientation of one or more sensors in the sensor unit 202. In one embodiment, the movable mount could include a rotating platform that could scan sensors so as to obtain information from each direction around the vehicle 200. In another embodiment, the movable mount of the sensor unit 202 could be moveable in a scanning fashion within a particular range of angles and/or azimuths. The sensor unit 202 could be mounted atop the roof of a car, for instance, however other mounting locations are possible. Additionally, the sensors of sensor unit 202 could be distributed in different locations and need not be collocated in a single location. Some possible sensor types and mounting locations include LIDAR unit 206 and laser rangefinder unit 208. Furthermore, each sensor of sensor unit 202 could be configured to be moved or scanned independently of other sensors of sensor unit 202.” (column 12 lines 14 thru 33), and “The camera 210 can have associated optics operable to provide an adjustable field of view. Further, the camera 210 can be mounted to vehicle 200 with a movable mount to vary a pointing angle of the camera 210” (column 13 lines 13 thru 16).
Regarding claim 13 Zhu et al teach the claimed agricultural machine “an example system may also be implemented in or take the form of other vehicles, such as cars, trucks, motorcycles, buses, boats, airplanes, helicopters, lawn mowers, earth movers, boats, snowmobiles, aircraft, recreational vehicles, amusement park vehicles, farm equipment, construction equipment, trams, golf carts, trains, and trolleys” (column 6 lines 39 thru 44), comprising, the claimed obstacle detection system (see above rejection of claim 1), the claimed LIDAR sensor, the sensor system includes a laser rangefinder/LIDAR unit 128 (Figure 1), and the claimed camera, the sensor system includes a camera 30 (Figure 1), and the system 600 for employs a hyperspectral sensor 620 (Figure 6A).
Regarding claim 14 Zhu et al teach the claimed obstacle detection method, (Figures 6B and 7), for the claimed agricultural machine, “an example system may also be implemented in or take the form of other vehicles, such as cars, trucks, motorcycles, buses, boats, airplanes, helicopters, lawn mowers, earth movers, boats, snowmobiles, aircraft, recreational vehicles, amusement park vehicles, farm equipment, construction equipment, trams, golf carts, trains, and trolleys” (column 6 lines 39 thru 44), to perform the claimed self-driving, “Example embodiments relate to an autonomous vehicle, such as a driverless automobile” (column 4 lines 12 and 13), while the claimed sensing a surrounding environment with a LIDAR sensor and a camera, “that includes a light detection and ranging (LIDAR) sensor for actively detecting reflective features in the environment surrounding the vehicle” (column 4 lines 13 thru 15), “the laser rangefinder or LIDAR unit 128 can be any sensor configured to sense objects in the environment in which the vehicle 100 is located” (column 8 lines 14 thru 16), and “The camera 130 can include one or more devices configured to capture a plurality of images of the environment surrounding the vehicle 100.” (column 7 lines 22 thru 24), the obstacle detection method comprising:
the claimed detecting an obstacle candidate based on data that is output from the LIDAR sensor, “FIG. 7 is a flowchart 700 of a process for navigating an autonomous vehicle according to real time environmental feedback information from both the LIDAR device 302 and the hyperspectral sensor 620. The LIDAR device 302 is scanned through a scanning zone surrounding the vehicle (702). Information from reflected light signals provided by the LIDAR device 302 and/or associated optical detectors is analyzed to generate a 3-D point cloud of positions of reflective features in the scanning zone (704). The 3-D point cloud is analyzed via the controller 610, the sensor fusion algorithm 138, the computer vision system 140, and/or the object detection module described above, etc. to identify regions of the scanning zone for spectral analysis (706). The region identified for spectral analysis (706) can be a region including a LIDAR-indicated reflective feature/object.” (column 20 lines 48 thru 62, and Figure 7);
the claimed acquiring an image of the obstacle candidate with the camera upon detecting the obstacle candidate as a trigger, “The identified region is identified with the hyperspectral sensor 620 to characterize the region according to its spectral properties (708). The spectral information is used to determine whether the region includes a solid material (710).” (column 21 lines 6 thru 9 and Figure 7) (claimed camera acquiring an image of an obstacle candidate), and “the hyperspectral sensor 620 can include both imaging optics and a spectral selectivity module. The imaging optics can be one or more lenses, mirrors, shutters, and/or apertures arranged to focus received radiation on an imaging plane that includes a photo sensitive detector, such as a charge coupled device array, or a similar detector for generating electrical signals related to an intensity pattern in the imaging plane.” (column 18 lines 53 thru 60), the hyperspectral sensor equates to the claimed camera, and the solid material equates to the claimed obstacle candidate;
the claimed determining whether or not to change a traveling status of the agricultural machine based on the image of the obstacle candidate acquired with the camera, “The 3-D point cloud information from the LIDAR device can be combined with the indications of whether reflective features are solid or not solid to map solid objects in the scanning zone (712).” (column 21 lines 30 thru 34, and Figure 7), and “The autonomous vehicle is navigated to avoid interference with objects in the map of solid objects 630 (714). In some embodiments, non-solid LIDAR-indicated features, such as water spray patterns and/or exhaust plumes are substantially ignored by the navigation control systems operating the autonomous vehicle. For example, the object avoidance system 144 and/or navigation/pathing system 142 can be provided with the map of solid objects 630 to automatically determine a path for the autonomous vehicle that avoids solid objects without avoiding splashes of water or exhaust plumes even if optically opaque. Thus, in some embodiments of the present disclosure, information from both the LIDAR device 302 and the hyperspectral sensor 620 is combined generate the map of solid objects 630.” (column 21 lines 40 thru 53, and Figure 7); and
the claimed causing the agricultural machine to stop traveling or to decelerate upon detecting the obstacle candidate as the trigger, the obstacle avoidance system 144 can effect changes in the navigation of the vehicle by operating one or more subsystems in the control system 106 to undertake braking maneuvers (column 9 lines 36 thru 40).
Zhu et al do not teach the claimed causing the agricultural machine to stop or decelerate upon detecting the obstacle before determining whether or not to change the traveling status of the agricultural machine. The examiner interprets this limitation as an ordering of the control/program steps. Persons having ordinary skill in the art understand how to implement changes to the order of steps.
Wilson et al teach, the autonomy controller 210 reduce speed when there is a probable object and the machine controller 220 drives the mower at reduce speed along the specified path (Figure 6B), the autonomy controller 210 proceeds to send an alert, identify a region of interest, and tighten tolerance on the ground plane algorithm (Figure 6B), the autonomy controller 210 proceeds to process the buffered region of interest and confirm the object (Figure 6C), and when the object is confirmed, a path adjustment is determined and performed by the machine controller 220 (Figure 6C). The path adjustment equates to the claimed change of traveling status, and the reduced speed of Wilson et al occurs before the path adjustment.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the computer vision system of Zhu et al with the reducing speed before the path adjustment of Willson et al in order to, with a reasonable expectation of success, avoid running over objects (Wilson et al P[0040]).
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al Patent Number 9,234,618 B1 and Wilson et al Patent Application Publication Number 2021/0382490 A1 as applied to claims 1 and 2 above, and in further view of Kuroda Patent Application Publication Number 2018/0267170 A1.
Regarding claim 3 Zhu et al and Wilson et al teach the claimed system of claims 1 and 2 (see above). Zhu et al further teach, wherein the controller is further configured or programmed to:
the claimed identify a type of object based on the image acquired by the camera, “The spectral information is used to determine whether the region includes a solid material (710).” (column 21 lines 8 and 9, and Figure 7), the material being solid or not equates to the claimed type of object.
Zhu et al (and Wilson et al) do not teach the claimed when the obstacle candidate is a person or animal cause the agricultural machine to stop traveling or change a traveling path to avoid the obstacle candidate. Zhu et al adjust the travel path to avoid solid material (obstacles) (Figure 7 step 714). A person is a solid object and would be avoided by Zhu et al. Zhu et al lack the teaching of detecting that the obstacle is a person or animal. Kuroda teaches, “The information processing unit 340 characteristically includes a person determination unit 43 in addition to a recognition processing unit 41 and an obstacle determination unit 42. The person determination unit 43 determines whether an obstacle is a person” P[0090], and “If a person is detected within the detection regions for the ultrasonic sensor 30 and the LIDAR sensor 31 at the start of operation of the traveling apparatus 301, it is determined that the person is in the vicinity of the traveling apparatus 301, and the operation of the traveling apparatus 301 is stopped until the person exits from the detection regions.” P[0093]. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the computer vision system of Zhu et al and the reducing speed before the path adjustment of Willson et al with the person determination unit of Kuroda in order to, with a reasonable expectation of success, inhibit erroneous sensing and implement stable sensing (Kuroda P[0008]).
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al Patent Number 9,234,618 B1 and Wilson et al Patent Application Publication Number 2021/0382490 A1 as applied to claims 1 and 2 above, and in further view of Neitemeier et al Patent Application Publication Number 2019/0098825 A1.
Regarding claim 4 Zhu et al and Wilson et al teach the claimed system of claims 1 and 2 (see above). Zhu et al (and Wilson et al) do not teach the claimed when the obstacle candidate is detected, cause the camera to acquire a color image of the obstacle candidate, and the claimed identify a type of object based on the RGB values of the color image of the obstacle candidate. The use of color imaging is common and well known in the art, and in the general use of camera images.
Neitemeier et al teach,
the claimed when the obstacle candidate is detected, cause the camera to acquire a color image of the obstacle candidate, “The camera-based sensor system 12 preferably comprises at least one camera, in particular at least one color image camera, for generating the starting camera images 14.” P[0027], and “the segmentation can be carried out based on the color distribution in the particular starting camera image” P[0028]; and
the claimed identify a type of object based on the RGB values of the color image of the obstacle candidate, “The color scheme of the image segments and the selective display of the image segments of predetermined classes form the basis for a particularly intuitive and clear display of the characteristics of the relevant surroundings area.” P[0015], and “Within the scope of the subsequent classification, it is inferred, from factors such as the shape, the volume, or the color of the segment 27 in combination with the factor of the piece of height information “3”, that the image segment 27 is to be allocated to the class of an obstacle.” P[0037].
The color imaging and class identification of Neitemeier et al would be included as color imaging of Zhu et al. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the computer vision system of Zhu et al and the reducing speed before the path adjustment of Willson et al with the color imaging classification of Neitemeier et al in order to, with a reasonable expectation of success, reduce the volume of data to be processed (Neitemeier et al P[0011]).
Claim(s) 5 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al Patent Number 9,234,618 B1 and Wilson et al Patent Application Publication Number 2021/0382490 A1 as applied to claim 1 above, and in further view of Isawe et al Patent Application Publication Number 2023/0221728 A1.
Regarding claim 5 Zhu et al and Wilson et al teach the claimed system of claim 1 (see above). Zhu et al (and Wilson et al) do not teach the claimed transmit the image of the obstacle candidate acquired with the camera to an external device, and the claimed based on a signal received from the external device indicating whether or not to change the traveling status of the agricultural machine, determine whether or not to change the traveling status.
Isawe et al teach,
the claimed transmit the image of the obstacle candidate acquired with the camera to an external device, “the tractor 1 and the mobile communication terminal 5 are provided with communication modules 28 and 52, respectively, that enable wireless communication of information including positioning information between the in-vehicle control unit 23 and the terminal control unit 3B [51]” (P[0049] and Figure 6), and “the image processing device 85 performs image transmission processing to transmit the generated all-around image and the images from the cameras 81 to 84 to the display control section 23E on the tractor side and the display control section 51A on a mobile communication terminal side (step #2)” (P[0083] and Figure 11); and
the claimed based on a signal received from the external device indicating whether or not to change the traveling status of the agricultural machine, determine whether or not to change the traveling status, “If the obstacle is detected to be located in the deceleration control range Rdc of the first detection range Rd1 in the sixth determination processing, the automatic travel control section 23F performs second notification command processing to issue a notification command for notifying about the obstacle being located in the deceleration control range Rdc on the liquid crystal monitor 27 of the tractor 1 or the display device 50 of the mobile communication terminal 5 to the display control section 23E of the in-vehicle control unit 23 and the display control section 51A of the terminal control unit 51 (step #25). In addition, the automatic travel control section 23F performs deceleration command processing to issue a deceleration command for decreasing the vehicle speed of the tractor 1 as the obstacle located in the deceleration control range Rdc approaches the tractor 1, to the vehicle speed control section 23B (step #26). In this way, it is possible to notify the user, such as the occupant in the driving unit 12 or the administrator on the outside of the vehicle, of the presence of the obstacle in the deceleration control range Rdc of the first detection range Rd1 for the tractor 1. In addition, by the control actuation of the vehicle speed control section 23B, the vehicle speed of the tractor 1 can be appropriately reduced as the tractor 1 approaches the obstacle.” (P[0105] and Figure 22).
The mobile communication of Isawe et al would be combined with Zhu et al as a further control means for the vehicle. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the computer vision system of Zhu et al and the reducing speed before the path adjustment of Willson et al with the mobile communications for remote control of the vehicle of Isawe et al in order to, with a reasonable expectation of success, avoid performing unnecessary collision avoidance operations (Isawe et al P[0006]).
Regarding claim 12 Zhu et al and Wilson et al teach the claimed system of claim 1 (see above). Zhu et al (and Wilson et al) do not explicitly teach the claimed agricultural machine is a tractor with an attachable implement, and the claimed when the implement is attached to the tractor, the controller sets a region to be scanned by the LIDAR sensor when traveling based on the type of implement.
Isawe et al teach,
the claimed agricultural machine is a tractor with an attachable implement, tractor 1 may include rotary tiller 3, a plow, a disc harrow, a cultivator, a subsoiler, a seeder, a spraying device, and a mowing device coupled to the rear portion of the tractor 1 (Figure 1 and P[0034]); and
the claimed when the implement is attached to the tractor, the controller sets a region to be scanned by the LIDAR sensor when traveling based on the type of implement, “the LiDAR control sections 86B and 87B perform cut processing and masking processing, which are based on the vehicle body information and the like, for the measurement ranges Rm1 and Rm2 of the measuring sections 86A and 87A, and thereby set a first detection range Rd1 and a second detection range Rd2 for the above-described obstacle candidate as a detection target, respectively. In the cut processing, the LiDAR control sections 86B and 87B acquire a maximum left-right width of the vehicle body including the rotary tiller 3 (a left-right width of the rotary tiller 3 in the present embodiment) by the communication with the in-vehicle control unit 23, add a predetermined safety range to this maximum left-right width of the vehicle body, and thereby set a detection target width Wd of the obstacle candidate.” P[0077].
The rotary tiller information of Isawe et al would be combined with Zhu et al as vehicle information to assist in automatic control of the vehicle (i.e. the tiller would be part of the vehicle system). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the computer vision system of Zhu et al and the reducing speed before the path adjustment of Willson et al with the LIDAR processing of the vehicle operation to include the rotary tiller information of Isawe et al in order to, with a reasonable expectation of success, avoid performing unnecessary collision avoidance operations (Isawe et al P[0006]).
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al Patent Number 9,234,618 B1 and Wilson et al Patent Application Publication Number 2021/0382490 A1 as applied to claim 1 above, and in further view of Nishi et al Patent Application Publication Number 2019/382005 A1.
Regarding claim 8 Zhu et al and Wilson et al teach the claimed system of claim 1 (see above). Zhu et al (and Wilson et al) do not teach the claimed detect an object that is higher than a predetermined height as the obstacle candidate based on output data from the LIDAR sensor while the agricultural machine is traveling. Nishi et al teach, “the areas indicated by thin long dash-dotted lines in FIG. 24 are areas in which an obstacle that is present at a position lower than the predetermined height cannot be detected by the obstacle detectors 65” (P[0386] and Figure 24), and each obstacle detector 65 employs a laser scanner P[0261]. The obstacle detectors not detecting obstacles lower than a predetermined height equates to the claimed detect an object that is higher than a predetermined height, and would be combined with Zhu et al by limiting the objects detected. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the computer vision system of Zhu et al and the reducing speed before the path adjustment of Willson et al with the obstacle at a position lower than the predetermined height not detected by the obstacle detectors of Nishi et al in order to, with a reasonable expectation of success, avoid a reduction in work efficiency based on misdetection (Nishi et al P[0043]).
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al Patent Number 9,234,618 B1 and Wilson et al Patent Application Publication Number 2021/0382490 A1 as applied to claim 1 above, and in further view of Omoto et al Japanese Patent Application Publication Number JP-2017050829-A (provided translation from previous office action of 11/26/2025 cited in rejection).
Regarding claim 9 Zhu et al and Wilson et al teach the claimed system of claim 1 (see above). Zhu et al teach the claimed cause the camera to acquire an image of the surrounding environment before the obstacle candidate is detected, “The camera 130 can include one or more devices configured to capture a plurality of images of the environment surrounding the vehicle 100.” (column 8 lines 22 thru 24), and “the camera 130 can capture a plurality of images that represent information about an environment of the vehicle 100 while operating in an autonomous mode. The environment may include other vehicles, traffic lights, traffic signs, road markers, pedestrians, etc.” (column 11 lines 41 thru 45).
Zhu et al (and Wilson et al) do not teach the claimed when the obstacle candidate is detected, cause the camera to capture the obstacle candidate at a higher magnification than the image of the surrounding environment, but the zooming of a camera (claimed magnification) is a common and well known feature of cameras. Omoto et al teach, “A camera control system comprises: an object detection part for detecting a position of an object by a laser sensor at a fixed time interval; a positional information measuring part for measuring positional information of the object; a camera angle-of-view/magnification discrimination part which calculates an angle-of-view width and a magnification of the object on imaging by a camera and discriminates a trigger of camera control in accordance with the condition; and a camera operation instruction part for performing panning/tilting control and zooming magnification control on the camera.” (abstract), and “The camera 300 is a device that acquires an image of the target object 10 in the target area 110 by rotating a lens of the camera or adjusting a shooting magnification by a zoom function based on a command from the control device 200.” (translation page 2 paragraph 9). The control of the zoom function of Omoto et al would be used in the system of Zhu et al for the identified solid material (when it is identified). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the computer vision system of Zhu et al and the reducing speed before the path adjustment of Willson et al with the control of the zoom function of Omoto et al in order to, with a reasonable expectation of success, accurately track an object with less image blur (Omoto et al translation page 5 paragraph 13).
Claim(s) 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al Patent Number 9,234,618 B1 and Wilson et al Patent Application Publication Number 2021/0382490 A1 as applied to claim 1 above, and in further view of Paxman et al Patent Application Publication Number 2022/0146982 A1.
Regarding claim 11 Zhu et al and Wilson et al teach the claimed system of claim 1 (see above). Zhu et al teach the claimed agricultural machine includes a light to illuminate the surrounding environment, the vehicle headlights (Figure 2). Zhu et al (and Wilson et al) do not teach the claimed controller turns on the light or increases an illumination of the light when the obstacle candidate is detected. Paxman et al teach, “the processing system 112 also controls the light source 108 and/or the detector plane 110 during the acquisition of interference data from the detector plane 110. Control of the light source may include controlling the light source or an optical arrangement to change the illumination of the target object.” P[0033]. The change in illumination of Paxman et al would be applied to the headlights of Zhu et al to further illuminate the solid material as needed to identify whether it is solid or non-solid material. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the computer vision system of Zhu et al and the reducing speed before the path adjustment of Willson et al with the light source control to change illumination of the target object of Paxman et al in order to, with a reasonable expectation of success, improve reception for signals in the presence of multipath effects (Paxman et al P[0008]).
Allowable Subject Matter
Claims 15 and 17 thru 21 have been indicated as allowable.
Claim 16 would be allowable if rewritten or amended to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action.
The following is a statement of reasons for the indication of allowable subject matter: The reasons for indicating allowable subject matter over the prior art of record are based on the combined limitations of claim 15. The closest prior art of record is Zhu et al Patent Number 9,234,618 B1. Zhu et al disclose a light detection and ranging device associated with an autonomous vehicle scans through a scanning zone while emitting light pulses and receives reflected signals corresponding to the light pulses. The reflected signals indicate a three-dimensional point map of the distribution of reflective points in the scanning zone. A hyperspectral sensor images a region of the scanning zone corresponding to a reflective feature indicated by the three-dimensional point map. The output from the hyperspectral sensor includes spectral information characterizing a spectral distribution of radiation received from the reflective feature. The spectral characteristics of the reflective feature allow for distinguishing solid objects from non-solid reflective features, and a map of solid objects is provided to inform real time navigation decisions.
In regards to claim 15, Zhu et al taken either individually or in combination with other prior art, fails to teach or render obvious an obstacle detection system for an agricultural machine to perform self-driving while sensing a surrounding environment with a LiDAR sensor and a camera. The obstacle detection system comprising a controller configured or programmed to cause the camera, upon detecting an obstacle candidate based on data that is output from the LiDAR sensor, as a trigger, to acquire an image of the obstacle candidate. The controller is further configured or programmed to determine whether or not to change a traveling status of the agricultural machine based on the image of the obstacle candidate acquired with the camera, and to detect an object within the surrounding environment based on data that is output from the LiDAR sensor. When the detected object is the obstacle candidate, the controller is further configured or programmed to cause the camera to acquire the image of the obstacle candidate and determine whether or not to change the traveling status of the agricultural machine based on the image. The controller is further configured or programmed to identify, based on the image acquired with the camera, a type of the object that is the obstacle candidate. When the obstacle candidate is a person or animal, the controller is further configured or programmed to cause the agricultural machine to stop traveling or to change a traveling path of the agricultural machine to avoid the obstacle candidate. When the obstacle candidate is detected, the controller is further configured or programmed to cause the camera to acquire a color image of the obstacle candidate. The controller is further configured or programmed to exclude, from data of the color image, green pixels from a subject of identifying the type of the object. The first six steps of the controller process are the same as claims 1 thru 4, but the combination along with the last exclusion step makes the combination of limitations non-obvious.
Related Art
The examiner points to McArthur et al PGPub 2020/0019160 A1 as related art, but not relied upon for any rejection. McArthur et al is directed to the detection of a sensor triggering event, obtaining sensor data from a vehicle sensor based on the trigger event of a region of the environment using first and second sensors of a vehicle, and determining a sensor offset based on differences between the first and second sensor data (see Figures 4 thru 6).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DALE W HILGENDORF whose telephone number is (571)272-9635. The examiner can normally be reached Monday - Friday 9-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jelani Smith can be reached at 571-270-3969. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DALE W HILGENDORF/Primary Examiner, Art Unit 3662