DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This is a final office action on the merits. Claims 1-20 are currently pending and are addressed below.
The examiner notes that the fundamentals of the rejection are based on the broadest reasonable interpretation of the claim language. Applicant is kindly invited to consider the references as a whole. References are to be interpreted as by one of ordinary skill in the art rather than as by a novice. See MPEP 2141. Therefore, the relevant inquiry when interpreting a reference is not what the reference expressly discloses on its face but what the reference would teach or suggest to one of ordinary skill in the art.
Response to Arguments
Applicant’s arguments with respect to the rejection of claims 1-20 under 35 U.SC 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-21 of U.S. Patent No. 17/606,296. Although the claims at issue are not identical, they are not patentably distinct from each other because the recited limitations would still fall within the claimed limitations of the application. For example, the limitations of claim 1 for the current application can be similarly mapped to that of claims 1, 15, and 18 of the related application.
The breadth of the current application claims would read on the narrow claims as depicted in the table below.
Current Application
Claim 1
A computer-implemented method comprising:
accessing, by a computing system comprising one or more processors, image data and sensor data, wherein the image data comprises a plurality of images of one or more locations and semantic information associated with the one or more locations, and wherein the sensor data comprises sensor information associated with detection of one or more surfaces at the one or more locations by one or more sensors
determining, by the computing system, one or more irregular surfaces based at least in part on the image data and the sensor data, wherein the one or more irregular surfaces comprise the one or more surfaces associated with the image data and the sensor data that satisfy one or more irregular surface criteria at each of the one or more locations respectively
generating, by the computing system, map data comprising information associated with the one or more irregular surfaces
controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data.
Claim 1
A computer-implemented method comprising:
accessing, by a computing system comprising one or more processors, image data and sensor data, wherein the image data comprises a plurality of images of one or more locations and semantic information associated with the one or more locations, and wherein the sensor data comprises sensor information associated with detection of one or more surfaces at the one or more locations by one or more sensors
determining, by the computing system, one or more irregular surfaces based at least in part on the image data and the sensor data, wherein the one or more irregular surfaces comprise the one or more surfaces associated with the image data and the sensor data that satisfy one or more irregular surface criteria at each of the one or more locations respectively
generating, by the computing system, map data comprising information associated with the one or more irregular surfaces
controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data.
Claim 1
A computer-implemented method comprising:
accessing, by a computing system comprising one or more processors, image data and sensor data, wherein the image data comprises a plurality of images of one or more locations and semantic information associated with the one or more locations, and wherein the sensor data comprises sensor information associated with detection of one or more surfaces at the one or more locations by one or more sensors
determining, by the computing system, one or more irregular surfaces based at least in part on the image data and the sensor data, wherein the one or more irregular surfaces comprise the one or more surfaces associated with the image data and the sensor data that satisfy one or more irregular surface criteria at each of the one or more locations respectively
generating, by the computing system, map data comprising information associated with the one or more irregular surfaces
controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data.
Related Application – 17/573,085
Claim 1
A computer-implemented method of mapping, the computer-implemented method comprising:
accessing, by a computing system comprising one or more processors, image data and sensor data, wherein the image data comprises a plurality of images of one or more locations and semantic information associated with the one or more locations, and wherein the sensor data comprises sensor information associated with three-dimensional features of one or more surfaces based on detection of the one or more surfaces at the one or more locations by one or more sensors;
determining, by the computing system, one or more irregular surfaces based at least in part on the image data and the sensor data, wherein the one or more irregular surfaces comprise the one or more surfaces associated with the image data and the sensor data that satisfy one or more irregular surface criteria at each of the one or more locations respectively, wherein satisfying the one or more irregular surface criteria comprises the three-dimensional features indicating that the one or more surfaces exceed a surface area threshold and include a depression that exceeds a depth threshold
generating, by the computing system, map data comprising information associated with the one or more irregular surfaces
controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data, wherein the one or more vehicle systems comprise one or more motor systems.
Claim 15
One or more tangible non-transitory computer-readable media storing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform operations, the operations comprising:
accessing image data and sensor data, wherein the image data comprises semantic information that is descriptive of one or more geographic features of a geographic region, and wherein the sensor data comprises sensor information associated with three- dimensional features of one or more surface elements associated with one or more surfaces in the geographic region
determining one or more irregular surfaces based at least in part on whether the one or more surface elements and the semantic information that is descriptive of the one or more geographic features of the geographic region satisfy one or more irregular surface criteria, wherein satisfying the one or more irregular surface criteria comprises the three-dimensional features indicating that the one or more surface elements exceed a surface area threshold and include a depression that exceeds a depth threshold
generating map data for the geographic region associated with the one or more surface elements based on whether there is the association between the one or more surface elements and the semantic information that is descriptive of the one or more geographic features
controlling one or more vehicle systems of a vehicle based at least in part on the map data, wherein the one or more vehicle systems comprise one or more motor systems.
Claim 18
A computing system comprising: one or more processors; one or more non-transitory computer-readable media storing instructions that when executed by the one or more processors cause the one or more processors to perform operations comprising:
accessing image data and sensor data, wherein the image data comprises semantic information that is descriptive of one or more geographic features of a geographic region, and wherein, the sensor data comprises sensor information associated with three- dimensional features of one or more surface elements associated with one or more surfaces in the geographic region;
determining one or more irregular surfaces based at least in part on whether one or more surface elements and the semantic information that is descriptive of the one or more geographic features of the geographic region satisfy one or more irregular surface criteria, wherein the satisfying the one or more irregular surface criteria comprises the three-dimensional features indicating that the one or more surface elements exceed a surface area threshold and include a depression that exceeds a depth threshold;
generating map data for the geographic region associated with the one or more surface elements based on whether there is the association between the one or more surface elements and the semantic information that is descriptive of the one or more geographic features; and
controlling one or more vehicle systems of a vehicle based at least in part on the map data, wherein the one or more vehicle systems comprise one or more motor systems.
These alterations in view of the related applications would be obvious to one of ordinary skill in the art over the related application and/or secondary references and the corresponding claims they are recited within. A person of ordinary skill in the art at the time of the invention would determine that the claimed limitations in the current application and the related application is not patentably distinct and would render the invention obvious. This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 8-11, 14-16 are rejected under 35 U.S.C. 103 as being unpatentable over Okamoto Hideki et al. (US20180340794A1), hereinafter referred to as Hideki in view of Stein Gideon et al. (US20180194286A1), hereinafter referred to as Gideon, in further view of Bhavsar Parth et al. (WO20190055465A1), hereinafter referred to as Parth.
Regarding claim 1, Hideki discloses: a computer-implemented method (see at least Hideki, Fig.1, ¶¶ [0008]-[0009], [0035]-[0036] which discloses a method for providing obstacle data via a data processing apparatus) comprising:
accessing, by a computing system comprising one or more processors, image data and sensor data, wherein the image data comprises a plurality of images of one or more locations and semantic information associated with the one or more locations (see at least Hideki, Fig.1, discloses an example of image data comprising an image of one or more locations with obstacle data; [0058]-[0059] which discloses sensor data, this means accessing, by a computing system comprising one or more processors, image data and sensor data, wherein the image data comprises a plurality of images of one or more locations and semantic information associated with the one or more locations)
wherein the sensor data comprises sensor information associated with detection of one or more surfaces at the one or more locations by one or more sensors (see at least Hideki, ¶¶ [0031], [0055] discloses sensor information obtained from sensors, such as, a three-axis acceleration, gyro, and azimuth sensor which may provide information associated with characteristics (altitude, azimuth, obstacle, etc.) of a road surface, this means wherein the sensor data comprises sensor information associated with detection of one or more surfaces at the one or more locations by one or more sensors)
determining, by the computing system, the one or more irregular surfaces based at least in part on the image data and the sensor data, wherein the one or more irregular surfaces comprise the one or more surfaces associated with the image data and the sensor data that satisfy one or more irregular surface criteria at each of the one or more locations respectively (see at least Hideki, ¶¶ [0075], which discloses the determination of an irregular surface based in part on image and sensor data; in the disclosure an irregularity is referred to as an obstacle factor; [0077] the determination of attribute data associated with a surface, such as, “lateral inclination” is an angle, and the attribute data of the “irregularity” is a type of a road surface with observed characteristics; Table 2 discloses an example of determining an irregular surface by observed characteristics from sensor and image data; [0098]-[0103] provides the processes for determining different types of irregular surfaces based on predetermined characteristics detected in sensor data)
generating, by the computing system, map data comprising information associated with the one or more irregular surfaces (see at least Hideki, Fig.3, ¶¶ [0078] discloses generating map data that aggregates obstacle factors onto map data, this means generating, by the computing system, map data comprising information associated with the one or more irregular surfaces)
Hideki is silent on, however, in the same field of endeavor, Gideon discloses: controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data (see at least Gideon, ¶¶ [0093] which discloses utilizing the generated map data in order to provide control signals to navigate the vehicle, this means controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data)
It would have been obvious to a person of ordinary skill in the art to modify Hideki to include controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data as taught by Gideon. The examiner would like to note that the disclosure of Hideki provides that the navigation function of the terminal device is capable of displaying the generated electronic map on a display and navigating a route from a starting point to a destination. However, direct control of the vehicle relative to the generated map and surface irregularity data is not as explicitly provided. Incorporating the teachings of Gideon would allow for an improvement to the base device of Hideki that would have provided further vehicle safety and navigation with the detection of irregular surfaces by one or more sensors.
Modified Hideki is silent on, however, in the same field of endeavor, Parth teaches: wherein the image data is based at least in part on use of one or more machine-learned models to detect one or more features of the plurality of images (see at least Parth, pg.7, lines 9-24, which discloses machine learning algorithms to distinguish image frames showing one or more features of good and problematic road segments; assigned priority for processing identified features of the camera images; pg8, pg.10, lines 23-29, which discloses continuously acquiring images and identifying one or more features of road segments through object detection and classification on the features)
and determine the one or more features that are associated with one or more irregular surfaces at the one or more locations (see at least Parth, pg.1, lines 22-29, pg.8, pg.9, lines 1-13, which discloses categorizing the determined features associated with irregular surfaces at one or more locations of the plurality of image data, such as potholes, cracks, depressions, protrusions, etc., this means determine the one or more features that are associated with one or more irregular surfaces at the one or more locations)
It would have been obvious to a person of ordinary skill in the art to further change modified Hideki to include: wherein the image data is based at least in part on use of one or more machine-learned models to detect one or more features of the plurality of images and determine the one or more features that are associated with one or more irregular surfaces at the one or more locations as taught by Parth. Incorporating the teachings would allow for the an improvement to the existing surface detection and mapping by generating road irregularity information from images. The trained models are able to assist in identifying and classifying particular features of the road surface and associate their results with a plurality of locations.
Regarding claim 2, Hideki discloses: the computer-implemented method of claim 1, wherein the one or more vehicle systems comprise one or more motor systems (see at least Hideki, ¶¶ [0048]-[0049] which discloses one or more motor systems associated with the vehicle system)
Hideki is silent on, however, in the same field of endeavor, Gideon teaches: wherein the controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data comprises: controlling, by the computing system, the one or more motor systems to slow a velocity of the vehicle based on the one or more irregular surfaces indicated in the map data (see at least Gideon, ¶¶ [0113]-[0114], [0116], [0122] which discloses based on navigational responses, such as determining a road surface characteristic, transmitting one or more electronic signals to trigger a change in velocity of acceleration by physically depressing the brake (slow) or easing up off the accelerator of the vehicle, this means controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data comprises: controlling, by the computing system, the one or more motor systems to slow a velocity of the vehicle based on the one or more irregular surfaces indicated in the map data)
It would have been obvious to a person of ordinary skill in the art to modify Hideki to include: controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data comprises: controlling, by the computing system, the one or more motor systems to slow a velocity of the vehicle based on the one or more irregular surfaces indicated in the map data as taught by Gideon. An example in Hideki is provided where the integration of angular speeds when an irregular surface such as an incline is detected, however, it is not explicitly provided that a control signal is sent out in response to this. Incorporating the teachings of Gideon would allow for an improvement to the base device of Hideki that would have provided further vehicle safety and navigation with the detection of irregular surfaces by one or more sensors.
Regarding claim 3, Hideki is silent on, however, in the same field of endeavor, Gideon teaches: the computer-implemented method of claim 1, wherein the one or more vehicle systems comprise one or more steering systems, and wherein the controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data comprises: controlling, by the computing system, the one or more steering systems to avoid an irregular surface indicated in the map data (see at least Gideon, ¶¶ [0003], [0113]-[0114], [0116], [0122] which discloses based on navigational responses, such as determining a road surface characteristic, transmitting one or more electronic signals to trigger a change in velocity of acceleration by physically depressing the brake (slow) or easing up off the accelerator of the vehicle, this means controlling, by the computing system, the one or more steering systems to avoid an irregular surface indicated in the map data)
It would have been obvious to a person of ordinary skill in the art to modify Hideki to include wherein the controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data comprises: controlling, by the computing system, the one or more steering systems to avoid an irregular surface indicated in the map data. Incorporating the teachings of Gideon would allow for an improvement to the base device of Hideki that would have provided further vehicle safety and navigation with the detection of irregular surfaces by one or more sensors.
Regarding claim 4, Hideki discloses: the computer-implemented method of claim 1, wherein the one or more vehicle systems comprise one or more notification systems, and wherein the controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data comprises: controlling, by the computing system, the one or more notification systems to generate a notification to indicate the locations of the irregular surfaces (see at least Hideki, Fig.1, [0063]-[0066] which provides a visual representation provided as a notification to the driver of detected irregular surfaces indicative of the area being traversed)
Regarding claim 5, Hideki discloses: the computer-implemented method of claim 4, wherein the notification comprises a visual notification or an auditory notification (see at least Hideki, Fig.1, [0063]-[0066] which provides a visual representation provided as a notification to the driver of detected irregular surfaces indicative of the area being traversed)
Regarding claim 6, Hideki discloses: the computer-implemented method of claim 1, wherein the one or more vehicle systems comprise one or more braking systems, and wherein the controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data comprises: controlling, by the computing system, the one or more braking systems to stop the vehicle before passing over an irregular surface indicated in the map data (see at least Hideki, ¶¶ Fig. 4D; [0049], [0089], which provides an example of a vehicle stop before passing over an irregular surface (narrow road) as indicated by the map data)
Regarding claim 8, Hideki discloses: the computer-implemented method of claim 1, wherein the vehicle is remote from the computing system (see at least Hideki, ¶¶ [0201]-[0203] discloses the terminal device remote from the vehicle)
Regarding claim 9, Hideki discloses: the computer-implemented method of claim 1, further comprising: determining, by the computing system, based on the map data, a navigational route for the vehicle (see at least Hideki, ¶¶ [0063]-[0064] which discloses determining a navigational route for the user)
Regarding claim 10, Hideki discloses: the computer-implemented method of claim 1, wherein the determining, by the computing system, the one or more irregular surfaces based at least in part on the image data and the sensor data, wherein the one or more irregular surfaces comprise the one or more surfaces associated with the image data and the sensor data that satisfy the one or more irregular surface criteria at each of the one or more locations respectively comprises:
determining, by the computing system, one or more vehicle characteristics of the vehicle (see at least Hideki, ¶¶ [0083]-[0087] which discloses the one or more irregular surface criteria are satisfied when the one or more surface characteristics associated with each of the one or more surfaces satisfy one or more surface characteristic criteria based at least in part on the one or more vehicle characteristics of the vehicle, such as, height of the vehicle relative to a surface and the vehicle’s clearance)
determining, by the computing system, one or more surface characteristics associated with each of the one or more surfaces, wherein the one or more surface characteristics comprise one or more gradients associated with the one or more surfaces or a surface height associated with the one or more surfaces (see at least Hideki, ¶¶ [0075], which discloses the determination of an irregular surface based in part on image and sensor data; in the disclosure an irregularity is referred to as an obstacle factor; [0077] the determination of attribute data associated with a surface, such as, “lateral inclination” is an angle (gradient), and the attribute data of the “irregularity” is a type of a road surface with observed characteristics which may include “step” which is equivalent to height associated with the surface; Table 2 discloses an example of determining an irregular surface by observed characteristics from sensor and image data; [0098]-[0103] provides the processes for determining different types of irregular surfaces based on predetermined characteristics detected in sensor data)
determining, by the computing system, that the one or more irregular surface criteria are satisfied when the one or more surface characteristics associated with each of the one or more surfaces satisfy one or more surface characteristic criteria based at least in part on the one or more vehicle characteristics of the vehicle (see at least Hideki, ¶¶ [0083]-[0087] which discloses the one or more irregular surface criteria are satisfied when the one or more surface characteristics associated with each of the one or more surfaces satisfy one or more surface characteristic criteria based at least in part on the one or more vehicle characteristics of the vehicle, such as, height of the vehicle relative to a surface and the vehicle’s clearance)
Regarding claim 11, Hideki discloses: the computer-implemented method of claim 10, wherein the one or more vehicle characteristics comprise a ground clearance of the vehicle, a height of the vehicle, a width of the vehicle, a distance between a front wheel of the vehicle and a front bumper of the vehicle, or a firmness of a suspension system of the vehicle (see at least Hideki, ¶¶ [0083]-[0087] which discloses the one or more irregular surface criteria are satisfied when the one or more surface characteristics associated with each of the one or more surfaces satisfy one or more surface characteristic criteria based at least in part on the one or more vehicle characteristics of the vehicle, such as, height of the vehicle relative to a surface and the vehicle’s clearance)
Regarding claim 14, Hideki discloses: the computer-implemented method of claim 1, wherein the one or more irregular surface criteria comprise an irregularity height threshold or an irregularity depth threshold, and further comprising:
accessing, by the computing system, vehicle height data associated with a ground clearance of the vehicle (see at least Hideki, ¶¶ [0083]-[0087] which discloses the one or more irregular surface criteria are satisfied when the one or more surface characteristics associated with each of the one or more surfaces satisfy one or more surface characteristic criteria based at least in part on the one or more vehicle characteristics of the vehicle, such as, height of the vehicle relative to a surface and the vehicle’s clearance)
adjusting, by the computing system, the irregularity height threshold or the irregularity depth threshold based at least in part on the ground clearance of the vehicle (see at least Hideki, ¶¶ [0100]-[0103] which discloses adjusting the height with a threshold value to determine a step (height irregularity) greater than or equal to a threshold value; the attribute data is adjusted based on the absolute value based on the ground clearance of the vehicle, this means adjusting, by the computing system, the irregularity height threshold or the irregularity depth threshold based at least in part on the ground clearance of the vehicle)
Regarding claim 15, Hideki discloses: one or more tangible non-transitory computer-readable media storing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform operations (see at least Hideki, ¶¶ [0009] which discloses a processing apparatus), the operations comprising:
accessing image data and sensor data, wherein the image data comprises a plurality of images of one or more locations and semantic information associated with the one or more locations (see at least Hideki, Fig.1, discloses an example of image data comprising an image of one or more locations with obstacle data; [0058]-[0059] which discloses sensor data, this means accessing, by a computing system comprising one or more processors, image data and sensor data, wherein the image data comprises a plurality of images of one or more locations and semantic information associated with the one or more locations)
wherein the sensor data comprises sensor information associated with detection of one or more surfaces at the one or more locations by one or more sensors (see at least Hideki, ¶¶ [0031], [0055] discloses sensor information obtained from sensors, such as, a three-axis acceleration, gyro, and azimuth sensor which may provide information associated with characteristics (altitude, azimuth, obstacle, etc.) of a road surface, this means wherein the sensor data comprises sensor information associated with detection of one or more surfaces at the one or more locations by one or more sensors)
determining the one or more irregular surfaces based at least in part on the image data and the sensor data, wherein the one or more irregular surfaces comprise the one or more surfaces associated with the image data and the sensor data that satisfy one or more irregular surface criteria at each of the one or more locations respectively (see at least Hideki, ¶¶ [0075], which discloses the determination of an irregular surface based in part on image and sensor data; in the disclosure an irregularity is referred to as an obstacle factor; [0077] the determination of attribute data associated with a surface, such as, “lateral inclination” is an angle, and the attribute data of the “irregularity” is a type of a road surface with observed characteristics; Table 2 discloses an example of determining an irregular surface by observed characteristics from sensor and image data; [0098]-[0103] provides the processes for determining different types of irregular surfaces based on predetermined characteristics detected in sensor data)
generating map data comprising information associated with the one or more irregular surfaces (see at least Hideki, Fig.3, ¶¶ [0078] discloses generating map data that aggregates obstacle factors onto map data, this means generating, by the computing system, map data comprising information associated with the one or more irregular surfaces)
Hideki is silent on, however, in the same field of endeavor, Gideon discloses: controlling one or more vehicle systems of a vehicle based at least in part on the map data (see at least Gideon, ¶¶ [0093] which discloses utilizing the generated map data in order to provide control signals to navigate the vehicle, this means controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data)
It would have been obvious to a person of ordinary skill in the art to modify Hideki to include controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data as taught by Gideon. The examiner would like to note that the disclosure of Hideki provides that the navigation function of the terminal device is capable of displaying the generated electronic map on a display and navigating a route from a starting point to a destination. However, direct control of the vehicle relative to the generated map and surface irregularity data is not as explicitly provided. Incorporating the teachings of Gideon would allow for an improvement to the base device of Hideki that would have provided further vehicle safety and navigation with the detection of irregular surfaces by one or more sensors.
Modified Hideki is silent on, however, in the same field of endeavor, Parth teaches: wherein the image data is based at least in part on use of one or more machine-learned models to detect one or more features of the plurality of images (see at least Parth, pg.7, lines 9-24, which discloses machine learning algorithms to distinguish image frames showing one or more features of good and problematic road segments; assigned priority for processing identified features of the camera images; pg8, pg.10, lines 23-29, which discloses continuously acquiring images and identifying one or more features of road segments through object detection and classification on the features)
and determine the one or more features that are associated with one or more irregular surfaces at the one or more locations (see at least Parth, pg.1, lines 22-29, pg.8, pg.9, lines 1-13, which discloses categorizing the determined features associated with irregular surfaces at one or more locations of the plurality of image data, such as potholes, cracks, depressions, protrusions, etc., this means determine the one or more features that are associated with one or more irregular surfaces at the one or more locations)
It would have been obvious to a person of ordinary skill in the art to further change modified Hideki to include: wherein the image data is based at least in part on use of one or more machine-learned models to detect one or more features of the plurality of images and determine the one or more features that are associated with one or more irregular surfaces at the one or more locations as taught by Parth. Incorporating the teachings would allow for the an improvement to the existing surface detection and mapping by generating road irregularity information from images. The trained models are able to assist in identifying and classifying particular features of the road surface and associate their results with a plurality of locations.
Regarding claim 16, Hideki discloses: the one or more tangible non-transitory computer-readable media of claim 15, wherein the one or more vehicle systems comprise one or more motor systems (see at least Hideki, ¶¶ [0048]-[0049] which discloses one or more motor systems associated with the vehicle system)
Hideki is silent on, however, in the same field of endeavor, Gideon teaches: wherein the operations comprising controlling one or more vehicle systems of a vehicle based at least in part on the map data further comprise: controlling the one or more motor systems to slow a velocity of the vehicle based on the one or more irregular surfaces indicated in the map data (see at least Gideon, ¶¶ [0113]-[0114], [0116], [0122] which discloses based on navigational responses, such as determining a road surface characteristic, transmitting one or more electronic signals to trigger a change in velocity of acceleration by physically depressing the brake (slow) or easing up off the accelerator of the vehicle, this means controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data comprises: controlling, by the computing system, the one or more motor systems to slow a velocity of the vehicle based on the one or more irregular surfaces indicated in the map data)
It would have been obvious to a person of ordinary skill in the art to modify Hideki to include: controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data comprises: controlling, by the computing system, the one or more motor systems to slow a velocity of the vehicle based on the one or more irregular surfaces indicated in the map data as taught by Gideon. An example in Hideki is provided where the integration of angular speeds when an irregular surface such as an incline is detected, however, it is not explicitly provided that a control signal is sent out in response to this. Incorporating the teachings of Gideon would allow for an improvement to the base device of Hideki that would have provided further vehicle safety and navigation with the detection of irregular surfaces by one or more sensors.
9. Claims 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Okamoto Hideki et al. (US20180340794A1), hereinafter referred to as Hideki in view of Stein Gideon et al. (US20180194286A1), hereinafter referred to as Gideon.
Regarding claim 18, Hideki discloses: a computing system comprising:
one or more processors (see at least Hideki, ¶¶ [0009] which discloses a processing apparatus)
one or more non-transitory computer-readable media storing instructions (see at least Hideki, ¶¶ [0009] which discloses one or more non-transitory computer-readable media storing instructions) that when executed by the one or more processors cause the one or more processors to perform operations comprising:
accessing image data and sensor data, wherein the image data comprises a plurality of images of one or more locations and semantic information associated with the one or more locations (see at least Hideki, Fig.1, discloses an example of image data comprising an image of one or more locations with obstacle data; [0058]-[0059] which discloses sensor data, this means accessing, by a computing system comprising one or more processors, image data and sensor data, wherein the image data comprises a plurality of images of one or more locations and semantic information associated with the one or more locations)
wherein the sensor data comprises sensor information associated with detection of one or more surfaces at the one or more locations by one or more sensors (see at least Hideki, ¶¶ [0031], [0055] discloses sensor information obtained from sensors, such as, a three-axis acceleration, gyro, and azimuth sensor which may provide information associated with characteristics (altitude, azimuth, obstacle, etc.) of a road surface, this means wherein the sensor data comprises sensor information associated with detection of one or more surfaces at the one or more locations by one or more sensors)
determining the one or more irregular surfaces based at least in part on the image data and the sensor data, wherein the one or more irregular surfaces comprise the one or more surfaces associated with the image data and the sensor data that satisfy one or more irregular surface criteria at each of the one or more locations respectively (see at least Hideki, ¶¶ [0075], which discloses the determination of an irregular surface based in part on image and sensor data; in the disclosure an irregularity is referred to as an obstacle factor; [0077] the determination of attribute data associated with a surface, such as, “lateral inclination” is an angle, and the attribute data of the “irregularity” is a type of a road surface with observed characteristics; Table 2 discloses an example of determining an irregular surface by observed characteristics from sensor and image data; [0098]-[0103] provides the processes for determining different types of irregular surfaces based on predetermined characteristics detected in sensor data)
generating map data comprising information associated with the one or more irregular surfaces (see at least Hideki, Fig.3, ¶¶ [0078] discloses generating map data that aggregates obstacle factors onto map data, this means generating, by the computing system, map data comprising information associated with the one or more irregular surfaces)
Hideki is silent on, however, in the same field of endeavor, Gideon teaches: controlling one or more vehicle systems of a vehicle based at least in part on the map data (see at least Gideon, ¶¶ [0093] which discloses utilizing the generated map data in order to provide control signals to navigate the vehicle, this means controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data)
It would have been obvious to a person of ordinary skill in the art to modify Hideki to include controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data as taught by Gideon. The examiner would like to note that the disclosure of Hideki provides that the navigation function of the terminal device is capable of displaying the generated electronic map on a display and navigating a route from a starting point to a destination. However, direct control of the vehicle relative to the generated map and surface irregularity data is not as explicitly provided. Incorporating the teachings of Gideon would allow for an improvement to the base device of Hideki that would have provided further vehicle safety and navigation with the detection of irregular surfaces by one or more sensors.
Regarding claim 19, Hideki is silent on, however, in the same field of endeavor, Gideon teaches: the computing system of claim 18, wherein the one or more vehicle systems comprise one or more motor systems, and wherein the operations comprising controlling one or more vehicle systems of a vehicle based at least in part on the map data further comprise: controlling the one or more motor systems to slow a velocity of the vehicle based on the one or more irregular surfaces indicated in the map data (see at least Gideon, ¶¶ [0113]-[0114], [0116], [0122] which discloses based on navigational responses, such as determining a road surface characteristic, transmitting one or more electronic signals to trigger a change in velocity of acceleration by physically depressing the brake (slow) or easing up off the accelerator of the vehicle, this means controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data comprises: controlling, by the computing system, the one or more motor systems to slow a velocity of the vehicle based on the one or more irregular surfaces indicated in the map data)
It would have been obvious to a person of ordinary skill in the art to modify Hideki to include: controlling, by the computing system, one or more vehicle systems of a vehicle based at least in part on the map data comprises: controlling, by the computing system, the one or more motor systems to slow a velocity of the vehicle based on the one or more irregular surfaces indicated in the map data as taught by Gideon. An example in Hideki is provided where the integration of angular speeds when an irregular surface such as an incline is detected, however, it is not explicitly provided that a control signal is sent out in response to this. Incorporating the teachings of Gideon would allow for an improvement to the base device of Hideki that would have provided further vehicle safety and navigation with the detection of irregular surfaces by one or more sensors.
Claims 7, 12-13, 17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Okamoto Hideki et al. (US20180340794A1), hereinafter referred to as Hideki in view of Stein Gideon et al. (US20180194286A1), hereinafter referred to as Gideon, in further view of Bhavsar Parth et al. (WO20190055465A1), hereinafter referred to as Parth, in further view of Velusamy Umashankar et al. (US20160042644A1), hereinafter referred to as Umashankar.
Regarding claim 7, modified Hideki is silent on, however, in the same field of endeavor, Umashankar teaches: the computer-implemented method of claim 1, wherein the one or more vehicle systems comprise one or more lighting systems (see at least Umashankar, ¶¶ [0031], [0091])
It would have been obvious to a person of ordinary skill in the art to further change modified Hideki to include the computer-implemented method of claim 1, wherein the one or more vehicle systems comprise one or more lighting systems as taught by Umashankar. Incorporating the teachings would allow for an improvement to the base invention of Hideki that provides further accuracy and assessment in the determination of irregularities of a road surface and navigation based on that.
Regarding claim 12, modified Hideki is silent on, however, in the same field of endeavor, Umashankar teaches: the computer-implemented method of claim 1, wherein the one or more sensors include one or more light detection and ranging (LiDAR) devices configured to generate the sensor data based at least in part on a LiDAR scan of the one or more surfaces of the one or more locations (see at least Umashankar, ¶¶ [0024] which discloses one or more light detection and ranging (LiDAR) devices configured to generate the sensor data based at least in part on a LiDAR scan of the one or more surfaces of the one or more locations)
It would have been obvious to a person of ordinary skill in the art to further change modified Hideki to include the computer-implemented method of claim 1, wherein the one or more sensors include one or more light detection and ranging (LiDAR) devices configured to generate the sensor data based at least in part on a LiDAR scan of the one or more surfaces of the one or more locations as taught by Umashankar. Incorporating the teachings would allow for an improvement to the base invention of Hideki that provides further accuracy and assessment in the determination of irregularities of a road surface and navigation based on that.
Regarding claim 13, modified Hideki is silent on, however, in the same field of endeavor, Umashankar teaches: the computer-implemented method of claim 1, further comprising:
generating, by the computing system, data associated with implementing one or more indications based at least in part on the vehicle being within a predetermined distance of the one or more irregular surfaces, wherein the one or more indications comprise one or more visual indications, one or more maps comprising the one or more locations of the one or more irregular surfaces, one or more textual descriptions of the one or more irregular surfaces, or one or more auditory indications associated with the one or more irregular surfaces (see at least Umashankar, Fig.4A which depicts the generation of data associated with the implemented indications based in part on the vehicle being within a predetermined distance of the one or more irregular surfaces)
It would have been obvious to a person of ordinary skill in the art to further change modified Hideki to include generating, by the computing system, data associated with implementing one or more indications based at least in part on the vehicle being within a predetermined distance of the one or more irregular surfaces, wherein the one or more indications comprise one or more visual indications, one or more maps comprising the one or more locations of the one or more irregular surfaces, one or more textual descriptions of the one or more irregular surfaces, or one or more auditory indications associated with the one or more irregular surfaces as taught by Umashankar. Incorporating the teachings would allow for an improvement to the base invention of Hideki that provides further accuracy and assessment in the determination of irregularities of a road surface and navigation based on that.
Regarding claim 17, modified Hideki is silent on, however, in the same field of endeavor, Umashankar teaches: the one or more tangible non-transitory computer-readable media of claim 15, wherein the one or more sensors include one or more light detection and ranging (LiDAR) devices configured to generate the sensor data based at least in part on a LiDAR scan of the one or more surfaces of the one or more locations (see at least Umashankar, ¶¶ [0024] which discloses one or more light detection and ranging (LiDAR) devices configured to generate the sensor data based at least in part on a LiDAR scan of the one or more surfaces of the one or more locations)
It would have been obvious to a person of ordinary skill in the art to further change modified Hideki to include the one or more tangible non-transitory computer-readable media of claim 15, wherein the one or more sensors include one or more light detection and ranging (LiDAR) devices configured to generate the sensor data based at least in part on a LiDAR scan of the one or more surfaces of the one or more locations as taught by Umashankar. Incorporating the teachings would allow for an improvement to the base invention of Hideki that provides further accuracy and assessment in the determination of irregularities of a road surface and navigation based on that.
Regarding claim 20, modified Hideki is silent on, however, in the same field of endeavor, Umashankar teaches: the computing system of claim 18, wherein the one or more sensors include one or more light detection and ranging (LiDAR) devices configured to generate the sensor data based at least in part on a LiDAR scan of the one or more surfaces of the one or more locations (see at least Umashankar, ¶¶ [0024] which discloses one or more light detection and ranging (LiDAR) devices configured to generate the sensor data based at least in part on a LiDAR scan of the one or more surfaces of the one or more locations)
It would have been obvious to a person of ordinary skill in the art to further change modified Hideki to include the computing system of claim 18, wherein the one or more sensors include one or more light detection and ranging (LiDAR) devices configured to generate the sensor data based at least in part on a LiDAR scan of the one or more surfaces of the one or more locations as taught by Umashankar. Incorporating the teachings would allow for an improvement to the base invention of Hideki that provides further accuracy and assessment in the determination of irregularities of a road surface and navigation based on that.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KIRSTEN JADE M SANTOS whose telephone number is (571)272-7442. The examiner can normally be reached Monday: 8:00 am - 4:00 pm, 6:00-8:00 pm (+ with flex).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rachid Bendidi can be reached at (571) 272-4896. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KIRSTEN JADE M SANTOS/Examiner, Art Unit 3664
/RACHID BENDIDI/Supervisory Patent Examiner, Art Unit 3664
Prosecution Insights