DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The 11.18.2025 Amendments are entered. Claims 11, 14-15, and 21 are amended. Claims 12 and 19 are canceled. No claims are newly added. Claims 11, 13-18, and 20-21 are pending.
The Claim Objections
The claim objection to claim 14 is withdrawn in light of the amendments made.
The §101 Rejections
The §101 rejections are withdrawn in light of the incorporation of subject matter from claim 12 that represents more than well-understood, routine computer activities.
The Prior Art Rejections
Applicant’s 11.18.2025 arguments with respect to claims 1 and 21 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 11, and 13-14, 18, and 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Quint in view of US 20150348512 A1 to Sabatelli, Alessandro (“Sabatelli”), and further in view of US 20170030722 A1 to Kojo, Naoki (“Naoki”).
Regarding claim 11, Quint teaches a system designed to assist a driver of a motor vehicle during a second journey along a trajectory recorded within a scope of a first journey ([0018]: “ . . . the recorded travel data . . . may be subsequently retrieved . . . to guide a traveler in later repeating the off-road travel.”), comprising:
a driver assistance system operatively configured to:
ascertain elevational change information in relation to an elevational change of the trajectory recorded within the scope of the first journey ([0018]: “ . . . travel data is recorded representing the off-road travel of the vehicle. In some embodiments and as will be described below, the travel data may include positional coordinates, an elevation . . ..”; [0040]: “ . . . once off-road travel ends, the status window 475 is replaced with a save recording window 775 to inquire whether the user wishes to save the travel data for the off-road trip. The user may select a save option 776 to save the data for later retrieval . . . ”);
ascertain a display trajectory for the second journey taking into account the elevational change information (FIG. 14; [0056]: “ . . . the system 100 enables a user to access information about points along . . . the path ahead 1341 along the selected route 1070 (FIG. 11). . . . a user can select a point 1480 to access an information window 1452 . . . the information window includes . . . an elevation 1454, and an orientation 1455 of the vehicle at the point 1480.” Quint inherently teaches ascertaining a display trajectory by teaching that the path ahead is displayed by broadly taking into account previously-recorded elevational change information (used to discern elevation at any selected point).); and
cause the display trajectory to be displayed on a display of the vehicle (FIG. 14; [0056]: “ . . . the system 100 enables a user to access information about points along . . . the path ahead 1341 along the selected route 1070 (FIG. 11). . . . a user can select a point 1480 to access an information window 1452 . . . the information window includes . . . an elevation 1454, and an orientation 1455 of the vehicle at the point 1480.” );
wherein the first journey comprises an outward journey from a first point to a second point (Quint [0055]: “ . . . the location map 1330 shows a path 1341 that is part of the selected route 1070 that the user elected to follow.” Understood that the user can choose to follow recorded routes.).
Quint further teaches the second journey comprises . . . another journey from the first point to the second point (Quint [0055]: “ . . . the location map 1330 shows a path 1341 that is part of the selected route 1070 that the user elected to follow.” Understood that the user can choose to follow recorded routes.).
Quint does not appear to expressly teach the second journey comprises a corresponding return journey from the second point to the first point.
However, Sabatelli teaches the second journey comprises a corresponding return journey from the second point to the first point (Sabatelli [0046]: “ . . . the device has a backtracking mode during which a user can align an orientation indicator with the route representation by rotating the device, in order to travel backwards along a traveled route.” APOSITA would have understood that the combination of Quint and Sabatelli made below would have taught the backtracking mode of Sabatelli being displayed on the display of Quint.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have combined the system that allows for a user to replay a previously-recorded route to follow taught by Quint with a system that allows a user to backtrack along a previously-recorded route taught by Sabatelli. Doing so would have helped the user backtrack along the recorded route, allowing them to more easily return places they previously visited.
While Quint and Sabatelli as combined above appear to teach identification of the vehicle location during the second journey (Quint [0031]: “ . . . as the system 100 continues to move, the vehicle location 435 is maintained at a center 437 of the location map 430.”), this combination does not appear to expressly teach wherein during the second journey, the driver assistance system is configured to:
ascertain image data in relation to a surround of the vehicle located in front of the vehicle in the direction of travel using at least one camera of the vehicle; and
cause, based on the image data, a pictorial representation showing the display trajectory overlaid on the surround of the vehicle to be displayed on the display.
However, Kojo teaches wherein during the second journey, the driver assistance system is configured to:
ascertain image data in relation to a surround of the vehicle located in front of the vehicle in the direction of travel using at least one camera of the vehicle ([0023]: “In this embodiment, the cameras 16, 18, 20,22 include four digital cameras disposed in a front 30 of the vehicle 10 . . . ”).
Kojo further teaches that the camera data may be used to aid in localization as part of the first localization system at [0034], the Abstract, and so forth.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have combined the system that detects vehicle position and uses it to display a trajectory during replay of recorded route data taught by the above combination of Quint and Sabatelli with the system that detects vehicle position using image data taught by Kojo. Doing so would have improved the positioning accuracy by bringing more sensors to aid in localization. Doing so would have also improved the reliability of the system by allowing it to determine vehicle location during times where GPS is unavailable.
One of ordinary skill in the art would have understood that the above combination of Quint, Sabatelli, and Kojo further teaches the system is configured to cause, based on the image data, a pictorial representation showing the display trajectory overlaid on the surround of the vehicle to be displayed on the display (APOSITA would have understood in the above combination that the display in for example FIG. 14 of Quint displays the map with the route based on the location of the vehicle, which is in turn based on the location circuitry which acquires location based on GPS as taught by Quint and image data as taught by Kojo.).
Regarding claim 13, the above combination of Quint, Sabatelli, and Kojo further teaches the system according to claim 11, wherein during the second journey, the driver assistance system is configured to:
repeatedly ascertain a respective actual position of the vehicle (Quint [0031]: “ . . . as the system 100 continues to move, the vehicle location 435 is maintained at a center 437 of the location map 430.”);
repeatedly update the display trajectory in a manner dependent on the respective actual position ([0031]: “ . . . as the system 100 continues to move, the vehicle location 435 is maintained at a center 437 of the location map 430.”) and dependent on the elevational change information ([0056]: “In various embodiments, by using a digit 190 or another input, a user can select a point 1480 to access an information window 1452 about the point 1480.” The system being able to update the display whenever the location changes and display elevational change information when selected taken as repeated updates of the trajectory at least dependent on both of these pieces of information.); and
repeatedly cause the respective updated display trajectory to be displayed on the display of the vehicle ([0031]: “ . . . as the system 100 continues to move, the vehicle location 435 is maintained at a center 437 of the location map 430.”).
Regarding claim 14, Quint teaches the system according to claim 13 .
While Quint teaches determining the location of the vehicle using GPS, it does not appear to expressly teach wherein the respective actual position of the vehicle is a respective actual position of the at least one camera of the vehicle.
However, Kojo teaches determining a vehicle location using a camera ([0034]: “That is, the position of the vehicle can be determined using the first or second localization method.”; [0039]: Localization determined using a camera.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have combined the system that uses GPS to determine its location for recording and replaying off-road routes taught by Quint with the system that uses cameras to determine vehicle location taught by Kojo. Doing so would have improved the reliability of localization by allowing the vehicle to determine its location at times where GPS is unavailable.
A person of ordinary skill in the art would have understood that the above combination of Quint and Kojo further teaches wherein the respective actual position of the vehicle is a respective actual position of the at least one camera of the vehicle (APOSITA would have understood in the above combination that the position determined by the vehicle would have been the position of the camera, as Kojo teaches using a camera attached to a vehicle ([0023]) to determine the vehicle’s position (Abstract).).
Regarding claim 18, Quint discloses the system according to claim 11, wherein the driver assistance system is configured to:
ascertain trajectory data in relation to the trajectory recorded during the first journey, the trajectory data indicating positions of a multiplicity of waypoints on the trajectory recorded during the first journey and the elevational change information indicating a relative elevation of the multiplicity of waypoints relative to one another ([0041]: “Data associated with the route traveled 840 includes a series of data points captured during the off-road travel, such as Point A 850 and Point B 860. In various embodiments, at regular intervals set by time and/or distance traveled along the route traveled 840, travel data is sampled and recorded for potential storage for a number of such points.” As shown above, Quint teaches the travel data includes elevation data. The elevation at each point is relative to the other points’ elevation inherently because each point includes its own elevation relative to the same reference (evidenced by the ability of the system to calculate an elevation change over a route in [0036]).) and/or an elevation difference between the multiplicity of waypoints; and
take into account the trajectory data when providing the driver assistance during the second journey (FIG. 14; [0056]: “ . . . the system 100 enables a user to access information about points along . . . the path ahead 1341 along the selected route 1070 (FIG. 11). . . . a user can select a point 1480 to access an information window 1452 . . . the information window includes . . . an elevation 1454, and an orientation 1455 of the vehicle at the point 1480.” Quint inherently teaches ascertaining a display trajectory by teaching that the path ahead is displayed by broadly taking into account previously-recorded elevational change information (used to discern elevation at any selected point).).
Regarding claim 20, Quint further discloses the system according to claim 11, wherein the driver assistance system is configured to:
read the elevational change information from a memory unit of the vehicle in order to ascertain the elevational change information ([0018]: “ . . . travel data is recorded representing the off-road travel of the vehicle. In some embodiments and as will be described below, the travel data may include positional coordinates, an elevation . . ..”; [0056]: “In various embodiments, the information window includes positional coordinates 1453, an elevation 1454, and an orientation 1455 of the vehicle at the point 1480.”); and
wherein the elevational change information was recorded during the first journey and stored in the memory unit ([0056]: “In various embodiments, the information window includes positional coordinates 1453, an elevation 1454, and an orientation 1455 of the vehicle at the point 1480.” Understood that by disclosing that the elevation at the selected point is the elevation of the vehicle, it is the recorded travel data from the vehicle on the first trip along the route.).
Claim 21 is rejected over similar reasons to claim 1, applied to a method.
Claims 15-17 are rejected under 35 U.S.C. 103 as being unpatentable over Quint in view of US 20150348512 A1 to Sabatelli, Alessandro (“Sabatelli”) and US 20170030722 A1 to Kojo, Naoki (“Naoki”), further in view of US 20160284125 A1 to Bostick, James E. et al. (“Bostick”) and WO 2020149109 A1 to Horihata, Satoshi (“Horihata”).
Regarding claim 15, the above combination of Quint, Sabatelli, and Kojo teaches the system according to claim 14, wherein the driver assistance system is configured to:
determine a projection point of the at least one camera at the respective actual position of the vehicle (APOSITA would have understood in the above combination that the position determined by the vehicle would have been the position of the camera, as Kojo teaches using a camera attached to a vehicle ([0023]) to determine the vehicle’s position (Abstract).).
This combination does not appear to expressly teach the system configured to project a multiplicity of waypoints on the trajectory recorded during the first journey onto a projection plane in a manner dependent on the projection point and dependent on the elevational change information, in order to determine the display trajectory.
However, Bostick teaches a system configured to:
project a line representing a route on the trajectory recorded during the first journey ([0032]: “History 117 is a collection of recorded paths or trails traversed by a user of AR display device 120 connected to computing device 110.”) onto a projection plane in a manner ([0033]: “Virtual overlay program 400 . . . provides a hands-free virtual overlay of a path or trail to a designated destination on a see-through component of the user view . . ..” See-through component of user view taken as projection plane.) dependent on the projection point ([0033]: “In some embodiments of the present invention, virtual overlay program 400 determines the view of the user of AR display device 120 from the input of compass sensor 125 and incline sensor 127, in combination with GPS 115 location information and map data 140. ” APOSITA would have understood in the above combination that the location of the vehicle, derived and represented using camera location on the vehicle, would have been used to derive the view of the display device.) and dependent on the elevational change information (FIG. 2B; [0051]: “Having determined a destination and starting location, virtual overlay program 400 receives map data for the area of the starting and destination locations. The map data may include elevation data . . ..”), in order to determine the display trajectory ([0041]: “FIG. 2B depicts an example of virtual overlay 250 of a near-distance route image over steep rocky terrain as viewed through an augmented reality display device, in accordance with an embodiment of the present invention.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have combined the system that displays a previously recorded route taking elevational change into account when displaying a display trajectory taught by the above combination of Quint, Sabatelli, and Kojo with the system that projects a trajectory onto a projection plane based on camera location and the elevation data taught by Bostick. Doing so would have improved user convenience by providing them with a natural, intuitive view of the spatial layout of the route from their point of view.
This combination does not appear to expressly teach projecting a multiplicity of waypoints that represent the route.
However, Horihata teaches projecting a multiplicity of waypoints that represent the route (FIG. 7: Markers CNT1 used to show where turns need to be made, taken as waypoints.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have combined the system that projects a route based on a series of recorded points onto an overlay taught by the above combination of Quint, Sabatelli, Kojo, and Bostick with the system that projects a route onto a display that is represented by discrete waypoints taught by Horihata. Doing so would have improved the safety of the overlay by representing the route in a fashion that does not cover the entire road surface, improving the driver’s ability to recognize potholes and other hazards on the road surface that would have been covered by a solid line.
Regarding claim 16, the above combination of Quint, Sabatelli, Kojo, Bostick, and Horihata teaches the system according to claim 15, wherein the elevational change information indicates an elevation of the multiplicity of waypoints along an elevation (Quint [0018]: “ . . . travel data is recorded representing the off-road travel of the vehicle. In some embodiments and as will be described below, the travel data may include positional coordinates, an elevation . . ..”; Quint [0040]: “ . . . once off-road travel ends, the status window 475 is replaced with a save recording window 775 to inquire whether the user wishes to save the travel data for the off-road trip. The user may select a save option 776 to save the data for later retrieval . . . ” APOSITA would have understood in the above combination that the recorded elevation data from Quint would have been projected in the manner taught by Bostick and Horihata, at least indicating the elevation of the waypoints because the waypoints represent the route.); and
the driver assistance system is configured to:
project the multiplicity of waypoints onto the projection plane in a manner dependent on the respective elevation, in order to ascertain the display trajectory axis (Bostick FIG. 2B: APOSITA would have understood in the above combination that the overlay of Bostick which is depicted as representing an elevation of the route recorded by Quint on an elevation axis would have been presented as a set of waypoints as taught by Horihata.).
Regarding claim 17, the above combination of Quint, Sabatelli, Kojo, Bostick, and Horihata teaches the system according to claim 16, wherein
the elevation axis is arranged within the projection plane (Bostick FIG. 2B: Elevation axis depicted as being arranged in the overlay, taken as the projection plane.); and/or
the elevation axis corresponds to a vertical axis of the vehicle when the vehicle is aligned horizontally (Usage of the term “and/or” requires consideration of only one of the two options.).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Tarnok, Gabor. US 20130096819 A1. GPS Tuner.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HENRY RICHARD HINTON whose telephone number is (703)756-1051. The examiner can normally be reached Monday-Friday 7:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hunter Lonsberry can be reached at (571) 272-7298. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HENRY R HINTON/Examiner, Art Unit 3665
/HUNTER B LONSBERRY/Supervisory Patent Examiner, Art Unit 3665