DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This Office Action is in response to the application filed on 06/23/2025. Claim(s) 1 - 15 are presently pending and are examined in this first action on the merits (FAOM).
Priority
Examiner acknowledges Applicant’s claim to priority based on Application JP2022-208091 filed 12/26/2022.
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on 06/23/2025 has been considered by the Examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2-4, 8-12 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Ronald M. Taylor et. al. US 20180203113 (“Taylor”) in view of David E. Bertucci et. al. US 20220272890 (“Bertucci”).
As per Claim 1,
Taylor discloses,
An area recognition system comprising:
a first processor configured or programmed to discriminate whether an area is a road based on first information obtained by setting a surrounding area of a work vehicle as a detection target
and
discriminate between a road and a non-road area (see at least [0009] As part of operating an automated vehicle, for example a host-vehicle 14, the system 10 classifies a ground-cover 12 (FIG. 2) proximate to (i.e. surrounding, nearby, or in view of sensors used by the system 10) the host-vehicle 14, and [0009] the ground-cover 12 may be, but is not limited to, asphalt, concrete, a lane-marking, grass, gravel, dirt, snow, and the like).
a second discrimination processor configured or programmed to discriminate between a road and a non-road area by using identification information on a height or a width based on second information obtained for a range overlapping a range discriminated by using the first information (see at least [0015] While more complicated than the fixed or predetermined instance of the lidar-grid 38, this implementation may be able to better determine the relative position of the edge of the roadway where the ground-cover 12 transitions from, for example, asphalt to gravel, and [0016] the ground may first be segmented using the intensity 28 and/or information in the image 32. The segmentation of the ground-cover 12 may then be partitioned and then classified sequentially to better separate the areas of the ground-cover 12 that have the same classification)
Taylor does not disclose,
discriminate between a road and a non-road area by using identification information on a height or a width
Bertucci teaches,
discriminate between a road and a non-road area by using identification information on a height or a width (see at least [0083] when the system has insufficient confidence in classification of a detected obstacle, obstacle facial size is the maximum of width and height estimated from both three-dimensional and two-dimensional data, and [0108] The three-dimensional data may be used to quickly detect whether there is an obstacle on the way and act safely as it takes a certain amount of time for the vehicle to fully stop. The three-dimensional based obstacle detection technique may be fundamentally based on the aforementioned Euclidean cluster extraction algorithm, but it may aim for objects right in front of the vehicle instead).
Thus, Taylor discloses a ground-classifier that classifies ground-cover proximate to an automated vehicle that includes a camera, a lidar and a controller and Bertucci teaches a vehicle controller for agricultural and industrial applications using motion sensors, image sensors and distance sensors.
As a result, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to provide the inventions as disclosed by Taylor with use of localization data based on motions sensors (ex. GPS, IMU), image sensor (ex. camera) and distance sensor (ex. LIDAR) as taught by Bertucci, with a reasonable expectation of success, to control one or more of the actuators to cause the vehicle to move from the current location of the vehicle to a target location (0007).
As per Claim 2,
Taylor discloses,
An area wherein the first discrimination processor is configured or programmed to discriminate whether the area is a road by using a trained model based on the first information (see at least [0021] the classification 36 is done by comparing the values of various characteristics (e.g. lidar-characteristic 42, camera-characteristic 60) indicated by the lidar 16 and the camera 30 to various characteristic threshold/range values 72 to determine the classification 36 of the ground-cover 12 in question. It is contemplated that empirical testing, possibly in combination with supervised machine learning, will be used to ‘train’ the controller 34 to determine the classification 36. Machine learning algorithms build a model from an examples training set and use the model to make predications on new data set).
As per Claim 3,
Taylor discloses,
wherein the area recognition system according to wherein the first information and the second information are different from each other (see at least [0015] the lidar-grid 38 may be dynamically determined based on a lidar-characteristic 42 (e.g. the range 24, the direction 26, and/or the intensity 28) of each of the cloud-points 20 in the point-cloud 18. For example, the controller 34 may form an irregularly shaped patch by selecting instances of the cloud-points 20 that are adjacent to each other and have the same or about the same value of the intensity 28. While more complicated than the fixed or predetermined instance of the lidar-grid 38, this implementation may be able to better determine the relative position of the edge of the roadway where the ground-cover 12 transitions from, for example, asphalt to gravel, and [0020] The controller 34 is also configured to determine a camera-characteristic 60 (e.g. the hue 84, the brightness 88, the saturation 78, and/or the temperature 96) of the pixels 70 in the image 32 that are located within the cell 52. It was recognized that the intensity 28 alone was insufficient to distinguish the classification 36 of certain instances of the ground-cover 12, and the camera 30 alone was also insufficient).
As per Claim 4,
Taylor discloses,
an information acquisition processor configured or programmed to acquire the identification information; wherein
when the identification information satisfies a discrimination condition related to a height or a width (see at least [0018] the controller 34 is configured to determine a height 54 of the instances of cloud-points within the patch 50. Those in the art will recognize that the height 54 of a cloud-point can be determine based on the range 24 and the direction 26, which may be expressed in terms of azimuth-angle and elevation-angle. The patch 50 may be determined to be ground 56 when the height 54 is less than a height-threshold 58, ten centimeters (0.01 m) for example. The patch 50 may be determined to be non-ground if some or all of the cloud-points 20 within the patch 50 are not less than the height-threshold 58).
the second discrimination processor is configured or programmed to discriminate that a portion discriminated as a road by the first discrimination processor is a non-road area (see at least [0020] The controller 34 is also configured to determine a camera-characteristic 60 (e.g. the hue 84, the brightness 88, the saturation 78, and/or the temperature 96) of the pixels 70 in the image 32 that are located within the cell 52, [0020] It was recognized that the intensity 28 alone was insufficient to distinguish the classification 36 of certain instances of the ground-cover 12, and the camera 30 alone was also insufficient, and [0020] it was discovered that the combination of the lidar-characteristic 42 and the camera-characteristic 60 was effective to distinguish the classification 36 of many instances of the ground-cover 12. Accordingly, the controller 34 is further configured to determine the classification 36 of the patch 50 when the patch is determined to be ground 56, where the classification 36 of the patch 50 is determined based on the lidar-characteristic 42 and the camera-characteristic 60.)
As per Claim 8,
Taylor discloses,
wherein the second information includes a three-dimensional point group data set in a portion discriminated as a road by the first discrimination processor (see at least [0004] the controller is configured to define a lidar-grid that segregates the point-cloud into an array of patches, and define a camera-grid that segregates the image into an array of cells. The point-cloud and the image are aligned such that a patch is aligned with a cell. The controller is further configured to determine a height of cloud-points within the patch. The patch is determined to be ground when the height is less than a height-threshold. The controller is configured to determine a lidar-characteristic of cloud-points within the patch, determine a camera-characteristic of pixels within the cell, and determine a classification of the patch when the patch is determined to be ground, wherein the classification of the patch is determined based on the lidar-characteristic and the camera-characteristic).
the identification information includes information of a value indicating variation in a height component of a three-dimensional point group data set (see at least [0004] The controller is further configured to determine a height of cloud-points within the patch).
the discrimination condition includes a condition that the value indicating variation exceeds a range of a third threshold (see at least [0004] The patch is determined to be ground when the height is less than a height-threshold).
As per Claim 9,
Taylor discloses,
the discrimination condition includes a condition that the width dimension exceeds a reference dimension.
Taylor does not explicitly disclose,
wherein the identification information includes information of a width dimension of a portion discriminated as a road by the first discrimination processor
Bertucci teaches,
wherein the identification information includes information of a width dimension of a portion discriminated as a road by the first discrimination processor (see at least [0083] a sensing algorithm to plan an alternate route may include estimating three-dimensional size of the obstacle, calculating width of the route to travel, validating width of the vehicle to the route width, and keeping track of the obstacle. In the first step, three-dimensional obstacle size may be estimated from the stage of object detection and classification. In some implementations, when the system has insufficient confidence in classification of a detected obstacle, obstacle facial size is the maximum of width and height estimated from both three-dimensional and two-dimensional data, and depth of the obstacle is set at infinity until it is figured out)
Thus, Taylor discloses a ground-classifier that classifies ground-cover proximate to an automated vehicle that includes a camera, a lidar and a controller and Bertucci teaches a vehicle controller for agricultural and industrial applications using motion sensors, image sensors and distance sensors.
As a result, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to provide the inventions as disclosed by Taylor with use of localization data based on motions sensors (ex. GPS, IMU), image sensor (ex. camera) and distance sensor (ex. LIDAR) as taught by Bertucci, with a reasonable expectation of success, to control one or more of the actuators to cause the vehicle to move from the current location of the vehicle to a target location (0007).
As per Claim 10,
Taylor discloses,
wherein the first discrimination processor is configured or programmed to discriminate whether the area is a road by using image information acquired by a camera as the first information (see at least [0020] The controller 34 is also configured to determine a camera-characteristic 60 (e.g. the hue 84, the brightness 88, the saturation 78, and/or the temperature 96) of the pixels 70 in the image 32 that are located within the cell 52. It was recognized that the intensity 28 alone was insufficient to distinguish the classification 36 of certain instances of the ground-cover 12).
As per Claim 11,
Taylor discloses,
wherein the information acquisition processor is configured or programmed to:
acquire a three-dimensional point group data set of a target area obtained by a three-dimensional range sensor (see at least [0019] the controller 34, determines the lidar-characteristic 42 (e.g. the range 24, the direction 26, and/or the intensity 28) of the cloud-points 20 within the patch 50).
acquire, as the identification information, a height of a portion discriminated as a road by the first discrimination processor, using the three-dimensional point group data set (see at least [0018] the classification 36 of the patch 50 of the ground-cover 12, the controller 34 is configured to determine a height 54 of the instances of cloud-points within the patch 50, [0020] The controller 34 is also configured to determine a camera-characteristic 60 (e.g. the hue 84, the brightness 88, the saturation 78, and/or the temperature 96) of the pixels 70 in the image 32 that are located within the cell 52, and [0021] he classification 36 is done by comparing the values of various characteristics (e.g. lidar-characteristic 42, camera-characteristic 60) indicated by the lidar 16 and the camera 30 to various characteristic threshold/range values 72 to determine the classification 36 of the ground-cover 12 in question).
As per Claim 12,
Taylor discloses,
wherein the first discrimination processor is configured or programmed to discriminate whether the area is a road by using a three-dimensional point group data set acquired by a three- dimensional range sensor as the first information (see at least [0019] The goal or desire is to determine a reflectivity-value of the ground-cover 12 within the patch 50 and elsewhere in the field-of-view 22 as the reflectivity-value is often a strong indication of the classification 36 of the ground-cover 12).
As per Claim 15,
Taylor discloses,
A work vehicle comprising a vehicle body; a detector configured to detect a surrounding area of the vehicle body as a detection target; and the area recognition system according to claim 1 (see at least [0009] FIG. 1 illustrates a non-limiting example of a ground-classifier system 10, hereafter referred to as the system 10. As part of operating an automated vehicle, for example a host-vehicle 14, the system 10 classifies a ground-cover 12 (FIG. 2) proximate to (i.e. surrounding, nearby, or in view of sensors used by the system 10) the host-vehicle 14. As used herein, the term ground-cover refers to whatever material or substance is exposed on the surface of the ground proximate to the host-vehicle).
Claims 5-7 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Taylor in view of Bertucci as applied to Claim 1 above, and further in view of Mitsunobu Yoshida et. al. US 20110164037 (“Yoshida”).
As per Claim 5,
Taylor discloses,
a confirmation processor configured or programmed to confirm a position where the work vehicle is present is a road; wherein the identification information includes information on a height of a portion discriminated as a road by the first discrimination processor; (see at least [0018] the controller 34 is configured to determine a height 54 of the instances of cloud-points within the patch 50. Those in the art will recognize that the height 54 of a cloud-point can be determine based on the range 24 and the direction 26, which may be expressed in terms of azimuth-angle and elevation-angle. The patch 50 may be determined to be ground 56 when the height 54 is less than a height-threshold 58, ten centimeters (0.01 m) for example. The patch 50 may be determined to be non-ground if some or all of the cloud-points 20 within the patch 50 are not less than the height-threshold 58).
Taylor does not specifically disclose,
the discrimination condition includes a condition that a height of a portion discriminated as a road by the first discrimination processor is lower than a reference position of the work vehicle by more than a first threshold
Yoshida teaches,
the discrimination condition includes a condition that a height of a portion discriminated as a road by the first discrimination processor is lower than a reference position of the work vehicle by more than a first threshold ([0197] The laser scanner 210 performs measurements in the height direction on the sides of the vehicle 202, and therefore a feature having height (hereinafter, referred to as the standing feature), such as a wall surface, a power pole, or a streetlight, is measured at a plurality of points in the height direction. A feature having no height, on the other hand, such as a road surface is measured at one point in the height direction. Therefore, the point density 169 a of a standing feature is higher than the point density 169 a of a road surface. Given this fact, the standing feature image portion 179 a specifies as the standing feature image portion 179 a a minute zone whose point density 169 a is the same or higher than a predetermined number).
Thus, Taylor discloses a ground-classifier that classifies ground-cover proximate to an automated vehicle that includes a camera, a lidar and a controller and Yoshida teaches an orthoimage generating apparatus using lidar, camera and motion sensors to measure height direction on the sides of a vehicle and detect features like road surface based on point density of the cloud and a camera image.
As a result, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to provide the inventions as disclosed by Taylor with the ground height and road surface detection method as taught by Yoshida, with a reasonable expectation of success, to generate an aerial image in which a standing feature is discriminated from a road surface (0014).
As per Claim 6,
Taylor does not specifically disclose,
a confirmation processor configured or programmed to confirm a position where the work vehicle is present is a road;
the identification information includes information on a height of a portion discriminated as a road by the first discrimination processor;
the discrimination condition includes a condition that a height of a portion discriminated as a road by the first discrimination processor is higher than a reference position of the work vehicle by more than a second threshold.
Yoshida teaches,
a confirmation processor configured or programmed to confirm a position where the work vehicle is present is a road (see at least [0074] The position and attitude localizing section 310 calculates the position (latitude, longitude, and height [altitude]) (East, North, and Up) and the attitude angle (a roll angle, a pitch angle, and a yaw angle) of the mobile measuring apparatus 200 at the time of measurement by using Central Processing Unit (CPU) based on the GPS observation information 293, the gyro measurement value 294 and the odometer measurement value 295 acquired from the mobile measuring apparatus 200, [0077] the position and attitude localizing section 310 may calculate the position and attitude angle of the mobile measuring apparatus 200 by dead reckoning based on the gyro measurement value 294 and the odometer measurement value 295, and [0078] the position and attitude angle of the mobile measuring apparatus 200 calculated by the position and attitude localizing section 310 will be referred to as a “position and attitude localized value 391”. The position and attitude localized value 391 indicates the position and attitude angle of the mobile measuring apparatus 200 at each time).
the identification information includes information on a height of a portion discriminated as a road by the first discrimination processor; (see at least [0146] the point cloud extracting section 120 extracts every point whose height from the ground is the same or lower than a predetermined height based on the ground height 139 a specified by the ground height specifying section 130).
the discrimination condition includes a condition that a height of a portion discriminated as a road by the first discrimination processor is higher than a reference position of the work vehicle by more than a second threshold (see at least [0160] if the predetermined height is “50 cm”, then the point cloud extracting section 120 extracts as the predetermined height point cloud 129 a every point at which the height indicated by the 3D coordinates is the same or lower than “(the ground height 139 a) +50 [cm]” from the point cloud 491. If the ground height 139 a is specified for each zone, the point cloud extracting section 120 extracts the predetermined height point cloud 129 a for each zone).
Thus, Taylor discloses a ground-classifier that classifies ground-cover proximate to an automated vehicle that includes a camera, a lidar and a controller and Yoshida teaches an orthoimage generating apparatus using lidar, camera and motion sensors to measure height direction on the sides of a vehicle and detect features like road surface based on point density of the cloud and a camera image.
As a result, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to provide the inventions as disclosed by Taylor with the ground height and road surface detection method as taught by Yoshida, with a reasonable expectation of success, to generate an aerial image in which a standing feature is discriminated from a road surface (0014).
As per Claim 7,
Taylor does not specifically disclose,
wherein the area recognition system according to wherein
the confirmation processor is configured or programmed to confirm that a position where the work vehicle is present is a road by using information different from the first information and the second information or
by using an algorithm different from an algorithm used by the first discrimination processor for discrimination.
Bertucci teaches,
wherein the area recognition system according to wherein
the confirmation processor is configured or programmed to confirm that a position where the work vehicle is present is a road by using information different from the first information and the second information or
by using an algorithm different from an algorithm used by the first discrimination processor for discrimination (see at least [0053] motion sensor data capturing using the one or more motion sensors 142 may be used to estimate a position and/or an orientation of the implement 120. For example, the processing apparatus 130 may be configured to access (e.g., receive via wired or wireless communications or read from a memory) motion sensor data captured using the one or more motion sensors 142, [0054] The sensors 140 include one or more image sensors 144 connected to a vehicle 110. The one or more image sensors 144 are configured to capture images (e.g., RGB images or normalized difference vegetation index images), and [0055] The sensors 140 include one or more distance sensors 146 connected to the vehicle 110. For example, the one or more distance sensors may include a lidar sensor, a radar sensor, a sonar sensor, and/or a structured light sensor).
Thus, Taylor discloses a ground-classifier that classifies ground-cover proximate to an automated vehicle that includes a camera, a lidar and a controller and Bertucci teaches a vehicle controller for agricultural and industrial applications using motion sensors, image sensors and distance sensors.
As a result, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to provide the inventions as disclosed by Taylor with use of localization data based on motions sensors (ex. GPS, IMU), image sensor (ex. camera) and distance sensor (ex. LIDAR) as taught by Bertucci, with a reasonable expectation of success, to control one or more of the actuators to cause the vehicle to move from the current location of the vehicle to a target location (0007).
As per Claim 13,
Taylor does not disclose,
Comprising a position detector configured to detect a height of a reference position of the work vehicle.
Yoshida teaches,
Comprising a position detector configured to detect a height of a reference position of the work vehicle (see at least [Abstract] The position and attitude localizing apparatus may localize the position and attitude of the vehicle based on the GPS observation information, the gyro measurement value and the odometer measurement value, [0110] Then, the position and attitude localizing section 310 of the position and attitude localizing apparatus 300 calculates the position and attitude localized value 391 based on the GPS observation information 293, the gyro measurement value 294, and the odometer measurement value 295 acquired in S110, and [0111] The position and attitude localized value 391 indicates the 3D coordinates and 3D attitude angle of the mobile measuring apparatus 200 at each time when the mobile measuring apparatus 200 moves in the target area).
Thus, Taylor discloses a ground-classifier that classifies ground-cover proximate to an automated vehicle that includes a camera, a lidar and a controller and Yoshida teaches an orthoimage generating apparatus using lidar, camera and motion sensors to measure height direction on the sides of a vehicle and detect features like road surface based on point density of the cloud and a camera image.
As a result, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to provide the inventions as disclosed by Taylor with the ground height and road surface detection method as taught by Yoshida, with a reasonable expectation of success, to generate an aerial image in which a standing feature is discriminated from a road surface (0014).
As per Claim 14,
Taylor does not disclose,
wherein the confirmation processor is configured or programmed to use at least one of output information from an inertial measurement device mounted on the vehicle, information as a combination of position information for the vehicle obtained by a GNSS and map information, or work plan information for the vehicle in which work content and time are associated with each other
Bertucci teaches,
wherein the confirmation processor is configured or programmed to use at least one of output information from an inertial measurement device mounted on the vehicle, information as a combination of position information for the vehicle obtained by a GNSS and map information, or work plan information for the vehicle in which work content and time are associated with each other (see at least [0184] The method 2200 includes sensor data acquisition 2210 that utilizes a reference GPS 2212 (e.g., mounted near the back of a vehicle), an attitude GPS 2214 (e.g., mounted near the front of the vehicle), an inertial measurement unit (IMU) 2216, and a radar speed sensor 2218; vehicle moving state estimation 2220 that includes cross-validation 2222 of GPS-based and radar-based speed; vehicle GPS-based heading estimation 2230 that includes utilization 2232 of reference and attitude positions and utilization 2234 of difference of reference positions; UKF-based sensor fusion 2240 that includes fusing 2242 GPS heading as absolute heading, fusing 2244 GPS positions as absolute positions, fusing 2246 IMU yaw as differential heading, fusing 2248 radar speed as absolute velocity, and fusing 2250 perception-based heading as differential heading; path pre-learning 2260; and distance-to-waypoint minimization 2270 that includes angle-to-steer estimation 2272 and vehicle auto-steering 2274)
Thus, Taylor discloses a ground-classifier that classifies ground-cover proximate to an automated vehicle that includes a camera, a lidar and a controller and Bertucci teaches a vehicle controller for agricultural and industrial applications using motion sensors, image sensors and distance sensors.
As a result, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to provide the inventions as disclosed by Taylor with use of localization data based on motions sensors (ex. GPS, IMU), image sensor (ex. camera) and distance sensor (ex. LIDAR) as taught by Bertucci, with a reasonable expectation of success, to control one or more of the actuators to cause the vehicle to move from the current location of the vehicle to a target location (0007).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Applicants should take note of the prior art in the PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ASHUTOSH PANDE whose telephone number is (571)272-6269. The examiner can normally be reached Monday -Friday 9:00 AM -5:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Fadey Jabr can be reached at 5712721516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.P./Examiner, Art Unit 3668
/Thomas Ingram/Primary Examiner, Art Unit 3668
Prosecution Insights