Notice of Pre-AIA or AIA Status
Claims 1-20 are currently presented for Examination.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. GB2101235.6, filed on 01/29/2021.
Information Disclosure Statement
All the information disclosure statements (IDS) submitted are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claims 1, 13 and 15 are objected to because of the following informalities: The claims have numerous issues with antecedent basis. The Examiner suggests amending the claims such that the first recitation of each distinct element uses articles such as “a”/”an”, later recitations referring back to the same distinct element uses articles such as “the”/”said”, to use disambiguating modifiers (e.g., first, second, etc.) when there are multiple distinct elements with the same base term, and that the use of modifiers for each distinct element is kept consistent. Below is a non-exhaustive list of examples of these issues:
Such as claims 1, 13 and 15 recite the limitation "the dynamic ego object" in its limitation. There is insufficient antecedent basis for this limitation in the claim. Each claim introduces “an ego vehicle object” not a “dynamic ego object”. Also, claims 1, 13 and 15 recite the limitation "the challenger objects" in its limitation. There is insufficient antecedent basis for this limitation in the claim. Each claim introduces “dynamic challenger objects” not a “challenger objects” previously.
Also, claims 2-4 and 16-18 recite the limitation "the scene" in its limitation. There is insufficient antecedent basis for this limitation in the claim. Claim introduces “scene topology” and “scene objects” not a “scene” previously. Appropriate correction is required.
With respect to claim 1, 13 and 15, it has “…one or more interaction…” which is grammatical incorrect, and it should be amended to recite – one or more interactions--.
With respect to claim 10-11, line 1 should be amended to recite – presenting to the user--.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-2, 4-5, 8-16 and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tuukkanen et al. (PUB NO: US20200041997A1) in view of Amelunxen et al. (PUB NO: US20190295335A1)
Regarding claim 1
Tuukkanen teaches a computer implemented method of generating a scenario to be run in a simulation environment , (see para 0003- According to one embodiment, a computer-implemented method comprises collecting sensor data, contextual data, or a combination thereof during an operation of a vehicle. The method also comprises processing the sensor data, the contextual data, or a combination thereof to predict a future event that is expected to occur proximate to the vehicle. The method further comprises generating a simulation of the future event. The method further comprises presenting the simulation in a user interface. See para 37- In one embodiment, the system 100 generates a simulation of the future event (e.g., a second vehicle crossing the path and route of the passenger's autonomous vehicle. In one embodiment, the system 100 can generate a safe simulation mode for simulating and testing future events and situations by a passenger (e.g., allowing the passenger to “play with time”))
the method comprising: rendering on the display of a computer device an interactive visualisation of a scenario model (See para 40-41- In one embodiment, the system 100 presents the simulation of the future event in a user interface. By way of example, the system 100 can present the simulation in one or more possible rendered views (e.g., two dimensional (2D), three dimensional (3D), AR, VR, etc.) on a window (e.g., a vehicle windshield, a heads-up display, etc.) or in a display (e.g., a handheld display such as a mobile phone or an integrated dashboard or headrest display. In one instance, a passenger can use the interface element to control and manipulate the visualization of the simulated scene over time (i.e., “playing with the time”) by presenting the future in a variety of increments (e.g., 5 second, 10 second, 20 second). In one embodiment, in the scenario where a user has required a projection of 20 seconds ahead, the system 100 can compute and surface this projected future to the user at time t1. As the seconds pass, the “future happens according to the expectations” and the autonomous vehicle happens to be 20 seconds later than where it started (time t2). See para 84- In one embodiment, the visualization platform 107 can enable a passenger to modify the operation of the vehicle 703 in response to the presented future event (e.g., by enabling the front seat passenger to interact with the map user interface 701).
the scenario model comprising one or more interaction between an ego vehicle object and one or more dynamic challenger objects, (see para 36-37- the “urgency” based on time (i.e., how soon the event will take place, 10 seconds versus 45 seconds). As described, the future event may include the autonomous vehicle overtaking another vehicle, performing a passing maneuver, turning at an intersection, entering a complex interchange, performing a U-turn, parallel parking, etc. as well as the interactions of other proximate vehicles, pedestrians, cyclists, etc. (i.e., situations where humans can identify and imagine potential threats and situations that then lead to irrational behavior).see para 81- there is sufficient time for the vehicle 703 to make its turn west on road 707 at t4 (FIG. 7D) before vehicle 717 is even ready to make its turn west on road 707 (i.e., there is no real risk to the safety of the passenger).)
wherein the scenario model comprises a scene topology and the interactive visualisation comprises scene objects including the ego vehicle and the at least one challenger object displayed in the scene topology;(see para 80- FIGS. 7A-7D are diagrams of a map user interface depicting a simulated future event that is expected to occur proximate to a vehicle, according to one embodiment. In this case, at t1 (start), the vehicle 703 is heading north on road 705. In one embodiment, the visualization platform 107 can determine that a passenger in the front seat of the vehicle 703 is worried/concerned about an upcoming left turn from road 705 to road 707 since she/he knows from experience that roads 709 and 711 both feed into road 707 through road 705 and she/he can see that buildings 713 and 715 block her/his view of traffic traveling west of the roads 709 and 711, respectively. See para 81- In one embodiment, the visualization platform 107 processes the sensor data, the contextual data, or a combination thereof to predict a future event that is expected to occur proximate to the vehicle 703. In this instance, the visualization platform 107 can determine from the contextual data of the vehicle 703 and the vehicle 717 (e.g., V2V information) that although both vehicles will be face-to-face at t3 (FIG. 7C),).
wherein the scenario is associated with a timeline extending in a driving direction of the ego vehicle relative to the scene topology; (see para 33- The system 100, for instance, can clearly visually differentiate the current scene form the future events (e.g., through an AR view). In addition, the system 100 can present the system 100's confidence level in the accuracy of a predicted future even (e.g., along a timeline of the future) wherein the further into the future the system 100 looks, the less accurate the prediction may be. The timeline of confidence may be helpful because it is important to understand the psychological limits of each passengers (e.g., how far in the future a passenger normally anticipates and prepares), especially since humans react differently to different events. See para 41- In one instance, a passenger can use the interface element to control and manipulate the visualization of the simulated scene over time (i.e., “playing with the time”) by presenting the future in a variety of increments (e.g., 5 second, 10 second, 20 second. see para 80- FIGS. 7A-7D are diagrams of a map user interface depicting a simulated future event that is expected to occur proximate to a vehicle, according to one embodiment. As shown, the map user interface 701 of FIGS. 7A-7D (e.g., a UE 105 such as a mobile device) depicts a presentation of a future event over time t1 (FIG. 7A), t2 (FIG. 7B), t3 (FIG. 7C), and t4 (FIG. 7D), by the visualization platform 107 to one or more passengers of the vehicle 703 (e.g., an autonomous car). In one embodiment, the visualization platform 107 first collects sensor data, contextual data, or a combination thereof during an operation of the vehicle 703. In this case, at t1 (start), the vehicle 703 is heading north on road 705. Also see fig 8A-8D- interface element 805 to t1-t4 (as shown by the indicator 807, 809, 811 and 813)
Examiner note: Scenario where a predicted future event is visualized along a temporal timeline (t1 through t4) that extends directly forward along the road topology (road 705) in the exact heading/driving direction (heading north) of the ego vehicle (vehicle 703).
rendering on the display a timing control which is responsive to user input to select a time instant along the timeline; (See para 86-89- In this instance, the front seat passenger engages with the system 100 and controls the presentation of the simulated future event by turning the interface element 805 (e.g., a knob-based interface) to control and to manipulate the visualization of the simulated scene over time (i.e., “playing with the time”) Consequently, the passenger first moves the interface element 805 to t1 (e.g., start) as shown by the indicator 807. Referring to FIG. 8B, once the passenger turns the interface element 805 to t2 (e.g., 5 seconds in the future) as shown by the indicator 809 and in the vehicle user interface 801 (e.g., +0:05), the visualization platform 107 can present the passenger with the simulated future event now showing the passenger's vehicle 703 in its proximate position on the road 705 and the proximate position of the vehicle 717, now much closer to the road 705. Once the passenger turns the interface element 805 to t3 (e.g., 10 seconds in the future) as shown by the indicator 811 and in the vehicle user interface 801 in FIG. 8C, the visualization platform 107 can present the proximate position of the passenger's vehicle 703 on the road 705 and the proximate position of the vehicle 717, now on the same road 705 and face-to-face. Referring to FIG. 8D, once the passenger turns the interface element 805 to t4 (e.g., 20 seconds in the future) as shown by the indicator 813 and in the vehicle user interface 801, the visualization platform 107 can present the proximate position of the passenger's vehicle 703, now on road 707 and well ahead of vehicle 717, and the proximate position of the vehicle 717, now just about to turn west from road 705 to road 707. See fig 8A-8D- interface element 805 to t1-t4 (as shown by the indicator 807, 809, 811 and 813)))
and generating on the display an interactive visualisation of the scene topology and scene objects of the scenario displayed at the selected time instant. (see para 82-89-In this instance, the visualization platform can simulate the future event by simulating the paths of the vehicles 703 and 717 over the times t1, t2, t3, and t4 (e.g., 3:00, 3:05, 3:10, 3:20) as depicted in FIGS. 7A-7D, respectively. FIGS. 8A-8D are diagrams illustrating a vehicle user interface depicting a simulated future event that is expected to occur proximate to a vehicle, according to one embodiment. As shown, the vehicle user interface 801 of FIGS. 8A-8D (e.g., a heads-up display or windshield) depicts a presentation of a future event over time t1 (FIG. 8A), t2 (FIG. 8B), t3 (FIG. 8C), and t4 (FIG. 8D), by the visualization platform 107 to one or more passengers of the vehicle 703 (e.g., an autonomous car). In this instance, the front seat passenger engages with the system 100 and controls the presentation of the simulated future event by turning the interface element 805 (e.g., a knob-based interface) to control and to manipulate the visualization of the simulated scene over time (i.e., “playing with the time”))
Tuukkanen does not teach for testing the behaviour of an autonomous vehicle. visualisation of a scenario model for editing and each interaction defined as a set of temporal and/ or relational constraints between the dynamic ego object and at least one of the challenger objects.
In the related field of invention, Amelunxen teaches for testing the behaviour of an autonomous vehicle (see para 56- thereby allowing the sensor system of the test vehicle—including the algorithms to be tested for the autonomous or semiautonomous driving—to be able to handle these movements. The method according to the preferred exemplary embodiment of the invention described in the present case can thus be used to test, record, and evaluate specific scenarios to be tested, namely dangerous situations and possible incorrect behavior, in any plausible situation)
visualisation of a scenario model for editing, (see para 65 With reference to FIG. 4, according to a preferred exemplary embodiment of the invention, predefined traffic situations can be input on a screen using a GUI. FIG. 4 depicts the input of traffic situation entries which correspond to the example discussed hereinabove in connection with FIG. 2. The GUI is part of a piece of software for creating a simulation environment. According to the example of FIG. 4, maneuver definition corresponds to the test vehicle E, denoted by “Ego,” and a plurality of “fellow” vehicles. The quantity of fellows is arbitrarily selectable by the user. For each fellow, a target specification in the form of absolute parameters (or parameters relative to the test vehicle) is storable via a text input field “requirements).
each interaction defined as a set of temporal and/ or relational constraints between the dynamic ego object and at least one of the challenger objects, (see para 57-61-A predefined traffic situation which leads to the abandonment of the completely randomized driving of the other vehicles can, for example, be defined according to the following four criteria, which are to be satisfied at the same time: Criterion 1: a vehicle, which is in front of the test vehicle E and in the same lane as the test vehicle E, is at a distance of less than 50 m from the test vehicle E. Criterion 2: a vehicle, which is in a lane to the right of the test vehicle E, is at a distance of less than 10 m from the test vehicle E. Criterion 3: a vehicle, which is in the lane to the left of the test vehicle E, is at a distance of less than 10 m from the test vehicle E. Criterion 4: the test vehicle is moving at a speed of at least 40 km/h. see para 65- According to the example of FIG. 4, maneuver definition corresponds to the test vehicle E, denoted by “Ego,” and a plurality of “fellow” vehicles. The quantity of fellows is arbitrarily selectable by the user. For each fellow, a target specification in the form of absolute parameters (or parameters relative to the test vehicle) is storable via a text input field “requirements.” For each fellow, there is also a second text input field. This second text input field is labeled “vehicle maneuver.” The “vehicle maneuver” field for a given fellow can be used to define a maneuver that the fellow performs once all of the target specifications, set via the corresponding “requirements” text input field, have been met)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method for generating a simulation of the future events for an autonomous vehicle as disclosed by Tuukkanen to include for testing the behaviour of an autonomous vehicle. visualisation of a scenario model for editing and each interaction defined as a set of temporal and/ or relational constraints between the dynamic ego object and at least one of the challenger objects as taught by Amelunxen in the system of Tuukkanen to simulating different traffic situations for an autonomous or semiautonomous test vehicle. (see para 002, Amelunxen).
Regarding claim 13
Tuukkanen teaches a computer system for generating a scenario to be run in a simulation environment (see para 0003- According to one embodiment, a computer-implemented method comprises collecting sensor data, contextual data, or a combination thereof during an operation of a vehicle. The method also comprises processing the sensor data, the contextual data, or a combination thereof to predict a future event that is expected to occur proximate to the vehicle. The method further comprises generating a simulation of the future event. The method further comprises presenting the simulation in a user interface. See para 37- In one embodiment, the system 100 generates a simulation of the future event (e.g., a second vehicle crossing the path and route of the passenger's autonomous vehicle. In one embodiment, the system 100 can generate a safe simulation mode for simulating and testing future events and situations by a passenger (e.g., allowing the passenger to “play with time”)). see para 65 and fig 2- n one embodiment, the visualization platform 107 includes a collection module 201, a user interface (UI) module 203, an analysis module 205, a simulation module 207, a presentation module 209, and a routing module 211.
Amelunxen teaches for testing the behaviour of an autonomous vehicle (see para 56- thereby allowing the sensor system of the test vehicle—including the algorithms to be tested for the autonomous or semiautonomous driving—to be able to handle these movements. The method according to the preferred exemplary embodiment of the invention described in the present case can thus be used to test, record, and evaluate specific scenarios to be tested, namely dangerous situations and possible incorrect behavior, in any plausible situation)
The rest of the limitation of claim 13 is rejected for the same reasons as Claim 1, as they share the same elements.
Regarding claim 15
Tuukkanen teaches non-transitory computer readable media, on which is stored computer readable instructions which when executed by one or more processor cause the one or more processors to: (see para 0003- According to one embodiment, a computer-implemented method comprises collecting sensor data, contextual data, or a combination thereof during an operation of a vehicle. The method also comprises processing the sensor data, the contextual data, or a combination thereof to predict a future event that is expected to occur proximate to the vehicle. The method further comprises generating a simulation of the future event. The method further comprises presenting the simulation in a user interface. See para 37- In one embodiment, the system 100 generates a simulation of the future event (e.g., a second vehicle crossing the path and route of the passenger's autonomous vehicle. In one embodiment, the system 100 can generate a safe simulation mode for simulating and testing future events and situations by a passenger (e.g., allowing the passenger to “play with time”)). See para 98- The term non-transitory computer-readable medium is used herein to refer to any medium that participates in providing information to processor 902, including instructions for execution. Such a medium may take many forms, including, but not limited to, non-volatile media, volatile media and transmission media).
The rest of the limitation of claim 15 is rejected for the same reasons as Claim 1, as they share the same elements.
Regarding claim 2
The combination of Tuukkanen and Amelunxen teaches the method according to claim 1. Tuukkanen further teaches the method of claim 1, comprising, responsive to selection of the time instant, rendering a dynamic visualisation of the scene according to the scenario model from the selected time instant. (see para 41- In one instance, a passenger can use the interface element to control and manipulate the visualization of the simulated scene over time (i.e., “playing with the time”) by presenting the future in a variety of increments (e.g., 5 second, 10 second, 20 second). See para 80- As shown, the map user interface 701 of FIGS. 7A-7D (e.g., a UE 105 such as a mobile device) depicts a presentation of a future event over time t1 (FIG. 7A), t2 (FIG. 7B), t3 (FIG. 7C), and t4 (FIG. 7D), by the visualization platform 107 to one or more passengers of the vehicle 703 (e.g., an autonomous car). See para 85- FIGS. 8A-8D are diagrams illustrating a vehicle user interface depicting a simulated future event that is expected to occur proximate to a vehicle, according to one embodiment. As shown, the vehicle user interface 801 of FIGS. 8A-8D (e.g., a heads-up display or windshield) depicts a presentation of a future event over time t1 (FIG. 8A), t2 (FIG. 8B), t3 (FIG. 8C), and t4 (FIG. 8D), by the visualization platform 107 to one or more passengers of the vehicle 703 (e.g., an autonomous car). In this instance, the front seat passenger engages with the system 100 and controls the presentation of the simulated future event by turning the interface element 805 (e.g., a knob-based interface) to control and to manipulate the visualization of the simulated scene over time (i.e., “playing with the time”)))
Regarding claim 4
The combination of Tuukkanen and Amelunxen teaches the method according to claim 2. Tuukkanen further teaches wherein the step of rendering the dynamic visualisation of the scene occurs automatically responsive to selection of the time instant. (see para 88- In one instance, because the visualization platform 107 can recognize that the passenger is most concerned about the predicted face-to-face encounter during t3 (e.g., based on an increase in the passenger's heart rate), the visualization platform 107 can prompt the passenger to continue viewing the simulated future event by turning the interface element 805 and/or the visualization platform 107 can automatically play the future for the passenger (e.g., if the visualization platform 107 determines that the passenger has a high level of concern or fear). See para 95- In some embodiments, for example, in embodiments in which the computer system 900 performs all functions automatically without human input, one or more of external input device 912, display device 914 and pointing device 916 is omitted.)
Regarding claim 5
The combination of Tuukkanen and Amelunxen teaches the method according to claim 1. Tuukkanen further teaches regenerating and rendering on the display a new interactive visualisation of the scenario, comprising the one or more edited interaction. see para 84-Consequently, the visualization platform 107 can cause the vehicle 703 to slow down or speed up—respecting of course all limitations and regulations (e.g., speed, traffic lights, etc.) to avoid the face-to-face event, as depicted in FIG. 7C. see para 88- t this time (t3), and now that the visualization platform 107 has collected and processed more contextual data, the visualization platform 107 can change the representation of the vehicle 717 to reflect an increased confidence level regarding the predicted future to make the passenger feel more confident in the vehicle 703's decisions and, therefore, potentially reduce the passenger's fear. For example, whereas the visualization platform 107 initially represented the vehicle 717 as yellow at t1 and t2 (FIGS. 8A and 8B, respectively), the visualization platform 107 can now represent the vehicle in green or gold or any color or manner suggestive of confidence or certainty. see also claim 4)
Tuukkanen does not teach displaying to a user at an editing user interface of the computer device the set of temporal and/or relational constraints defining one or more of the interactions presented in the scenario, and receiving user input which edits one or more of the set of temporal and/or relational constraints for each one or more of the interactions and the one or more edited interaction.
In the related field of invention, Amelunxen teaches displaying to a user at an editing user interface of the computer device the set of temporal and/or relational constraints defining one or more of the interactions presented in the scenario, and receiving user input which edits one or more of the set of temporal and/or relational constraints for each one or more of the interactions;(see para 65- With reference to FIG. 4, according to a preferred exemplary embodiment of the invention, predefined traffic situations can be input on a screen using a GUI. FIG. 4, in particular, depicts the input of traffic situation entries which correspond to the example discussed hereinabove in connection with FIG. 2. The GUI is part of a piece of software for creating a simulation environment. According to the example of FIG. 4, maneuver definition corresponds to the test vehicle E, denoted by “Ego,” and a plurality of “fellow” vehicles. The quantity of fellows is arbitrarily selectable by the user. For each fellow, a target specification in the form of absolute parameters (or parameters relative to the test vehicle) is storable via a text input field “requirements.” For each fellow, there is also a second text input field. This second text input field is labeled “vehicle maneuver.” The “vehicle maneuver” field for a given fellow can be used to define a maneuver that the fellow performs once all of the target specifications, set via the corresponding “requirements” text input field, have been met. A blank “vehicle maneuver” field means that the applicable fellow is not to perform any particular maneuver.)
the one or more edited interaction. (see para 54- By changing various parameters, in particular by changing simulation parameters, the traffic situations can easily be varied. The changed parameters can include the number of vehicles and the starting positions thereof, and the parameters of the vehicles (e.g., the vehicle types—according to the performance and design—such as automobile, truck, with trailer, etc.). It is also possible to vary parameters such as the setpoint speed, safety distances, volume of traffic, average speeds, etc.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method for generating a simulation of the future events for an autonomous vehicle as disclosed by Tuukkanen to include displaying to a user at an editing user interface of the computer device the set of temporal and/or relational constraints defining one or more of the interactions presented in the scenario, and receiving user input which edits one or more of the set of temporal and/or relational constraints for each one or more of the interactions as taught by Amelunxen in the system of Tuukkanen to simulating different traffic situations for an autonomous or semiautonomous test vehicle. (see para 002, Amelunxen).
Regarding claim 8
The combination of Tuukkanen and Amelunxen teaches the method according to claim 1. Tuukkanen further teaches comprising rendering on the display a visual representation of the timeline associated with the timing control. (see para 33-In addition, the system 100 can present the system 100's confidence level in the accuracy of a predicted future even (e.g., along a timeline of the future) wherein the further into the future the system 100 looks, the less accurate the prediction may be. See para 85- FIGS. 8A-8D are diagrams illustrating a vehicle user interface depicting a simulated future event that is expected to occur proximate to a vehicle, according to one embodiment. As shown, the vehicle user interface 801 of FIGS. 8A-8D (e.g., a heads-up display or windshield) depicts a presentation of a future event over time t1 (FIG. 8A), t2 (FIG. 8B), t3 (FIG. 8C), and t4 (FIG. 8D), by the visualization platform 107 to one or more passengers of the vehicle 703 (e.g., an autonomous car). In this instance, the front seat passenger engages with the system 100 and controls the presentation of the simulated future event by turning the interface element 805 (e.g., a knob-based interface) to control and to manipulate the visualization of the simulated scene over time (i.e., “playing with the time”). See fig 8A-8D- interface element 805 to t1-t4 (as shown by the indicator 807, 809, 811 and 813))
Regarding claim 9
The combination of Tuukkanen and Amelunxen teaches the method according to claim 8. Tuukkanen further teaches rendering on the display a first visual indication of a set of time instants prior to the starting constraint. (see para 86-87-Consequently, the passenger first moves the interface element 805 to t1 (e.g., start) as shown by the indicator 807. In response to the passenger's interaction with the interface element 805, the visualization platform 107 can present the passenger with an AR view in the vehicle user interface 801 (e.g., a heads-up display) of the simulated future event including the passenger's vehicle 703 in its proximate position on the road 705 (e.g., at a starting time) and an “x-ray” view through buildings 713 and 715 of any oncoming traffic (e.g., vehicle 717 moving west on road 711) or objects of concern (e.g., pedestrians). Referring to FIG. 8B, once the passenger turns the interface element 805 to t2 (e.g., 5 seconds in the future) as shown by the indicator 809 and in the vehicle user interface 801 (e.g., +0:05), the visualization platform 107 can present the passenger with the simulated future event now showing the passenger's vehicle 703 in its proximate position on the road 705 and the proximate position of the vehicle 717, now much closer to the road 705.)
a second visual indication of a set of time instants during the interaction (s). (see para 88-89-Once the passenger turns the interface element 805 to t3 (e.g., 10 seconds in the future) as shown by the indicator 811 and in the vehicle user interface 801 in FIG. 8C, the visualization platform 107 can present the proximate position of the passenger's vehicle 703 on the road 705 and the proximate position of the vehicle 717, now on the same road 705 and face-to-face. Referring to FIG. 8D, once the passenger turns the interface element 805 to t4 (e.g., 20 seconds in the future) as shown by the indicator 813 and in the vehicle user interface 801, the visualization platform 107 can present the proximate position of the passenger's vehicle 703, now on road 707 and well ahead of vehicle 717, and the proximate position of the vehicle 717, now just about to turn west from road 705 to road 707. See para 84-Consequently, the visualization platform 107 can cause the vehicle 703 to slow down or speed up—respecting of course all limitations and regulations (e.g., speed, traffic lights, etc.) to avoid the face-to-face event, as depicted in FIG. 7C.)
Tuukkanen does not teach defining in the scenario a starting constraint to trigger the interaction(s).
In the related field of invention, Amelunxen teaches defining in the scenario a starting constraint to trigger the interaction(s). (see para 57-61-A predefined traffic situation which leads to the abandonment of the completely randomized driving of the other vehicles can, for example, be defined according to the following four criteria, which are to be satisfied at the same time: Criterion 1: a vehicle, which is in front of the test vehicle E and in the same lane as the test vehicle E, is at a distance of less than 50 m from the test vehicle E. Criterion 2: a vehicle, which is in a lane to the right of the test vehicle E, is at a distance of less than 10 m from the test vehicle E. Criterion 3: a vehicle, which is in the lane to the left of the test vehicle E, is at a distance of less than 10 m from the test vehicle E. Criterion 4: the test vehicle is moving at a speed of at least 40 km/h. see para 65- According to the example of FIG. 4, maneuver definition corresponds to the test vehicle E, denoted by “Ego,” and a plurality of “fellow” vehicles. The quantity of fellows is arbitrarily selectable by the user. For each fellow, a target specification in the form of absolute parameters (or parameters relative to the test vehicle) is storable via a text input field “requirements.” For each fellow, there is also a second text input field. This second text input field is labeled “vehicle maneuver.” The “vehicle maneuver” field for a given fellow can be used to define a maneuver that the fellow performs once all of the target specifications, set via the corresponding “requirements” text input field, have been met)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method for generating a simulation of the future events for an autonomous vehicle as disclosed by Tuukkanen to include defining in the scenario a starting constraint to trigger the interaction(s) as taught by Amelunxen as in the system of Tuukkanen to simulating different traffic situations for an autonomous or semiautonomous test vehicle. (see para 002, Amelunxen).
Regarding claim 10
The combination of Tuukkanen and Amelunxen teaches the method according to claim 1. Tuukkanen further teaches presenting to user on the display a play control which, when selected by a user, causes a dynamic visualisation of the scenario to be played from a currently selected time instant. (see para 38- n one embodiment, in addition to or instead of simulating future events, the system 100 can use the embodiments described herein to simulate and playback past events. See para 85- FIGS. 8A-8D are diagrams illustrating a vehicle user interface depicting a simulated future event that is expected to occur proximate to a vehicle, according to one embodiment. As shown, the vehicle user interface 801 of FIGS. 8A-8D (e.g., a heads-up display or windshield) depicts a presentation of a future event over time t1 (FIG. 8A), t2 (FIG. 8B), t3 (FIG. 8C), and t4 (FIG. 8D), by the visualization platform 107 to one or more passengers of the vehicle 703 (e.g., an autonomous car). In this instance, the front seat passenger engages with the system 100 and controls the presentation of the simulated future event by turning the interface element 805 (e.g., a knob-based interface) to control and to manipulate the visualization of the simulated scene over time (i.e., “playing with the time”) )
Regarding claim 11
The combination of Tuukkanen and Amelunxen teaches the method according to claim 1. Tuukkanen further teaches presenting to user on the display a play control which, when selected by a user, causes a dynamic visualisation of the scenario to be played from an initiating point of the scenario. (see para 41- In one embodiment, the user interface element (e.g., a knob, a roller or trackball-based interface, etc.) may also be used by a passenger to control the presentation of the of the simulated future event (i.e., play the future). See para 86- Consequently, the passenger first moves the interface element 805 to t1 (e.g., start) as shown by the indicator 807. In response to the passenger's interaction with the interface element 805, the visualization platform 107 can present the passenger with an AR view in the vehicle user interface 801 (e.g., a heads-up display) of the simulated future event including the passenger's vehicle 703 in its proximate position on the road 705 (e.g., at a starting time) and an “x-ray” view through buildings 713 and 715 of any oncoming traffic (e.g., vehicle 717 moving west on road 711) or objects of concern (e.g., pedestrians))
Regarding claim 12
The combination of Tuukkanen and Amelunxen teaches the method according to claim 1. Tuukkanen further teaches presenting to the user on the display a map view in which at least one selectable location is rendered in the map corresponding to a selectable time instant. (see para 40- In one embodiment, the system 100 can present or surface the simulation of the future event in multiple interfaces simultaneously (e.g., presenting a 2D map, a 3D map, an AR view, a VR display, or a combination thereof). In one instance, the system 100 can present the simulation of the future event through multiple interfaces within the vehicle based on the location or positioning of the passengers (e.g., a windshield for passengers in the front seats and on side windows for passengers in the back seats). See para 42- By way of example, a car A driving a given route (e.g., known by V2V communication) reaching a location B at time t1 could be represented in a color that suggests confidence or a likelihood of happening (e.g., green). See para 85- FIGS. 8A-8D are diagrams illustrating a vehicle user interface depicting a simulated future event that is expected to occur proximate to a vehicle, according to one embodiment. As shown, the vehicle user interface 801 of FIGS. 8A-8D (e.g., a heads-up display or windshield) depicts a presentation of a future event over time t1 (FIG. 8A), t2 (FIG. 8B), t3 (FIG. 8C), and t4 (FIG. 8D), by the visualization platform 107 to one or more passengers of the vehicle 703 (e.g., an autonomous car). In this instance, the front seat passenger engages with the system 100 and controls the presentation of the simulated future event by turning the interface element 805 (e.g., a knob-based interface) to control and to manipulate the visualization of the simulated scene over time (i.e., “playing with the time”) )
Regarding claim 13
The combination of Tuukkanen and Amelunxen teaches the computer system of claim 13. Tuukkanen further teaches wherein the processor is configured to generate the interactive visualisation from stored parameters of the scenario model. (see para 38- In one embodiment, the system 100 includes a user interface that includes a user interface element configured to receive a user input (e.g., a knob, a joystick, a rollerball or trackball-based interface, a touch screen, etc.) to enable a passenger to specify a parameter of the simulation. In one instance, the system 100 can then generate the simulation based on the one or more specified parameters. By way of example, a parameter can include a time duration (e.g., 5 seconds, 10 seconds, 20 seconds, etc.), a starting time (e.g., a user initiated starting time or a starting time determined by the system 100 based on the one or more sensors), a time direction (e.g., forwards or backwards), starting direction, etc. or a combination thereof. In one embodiment, the past simulation can be based on vehicle telemetry data to provide simulations where other traditional means of recording past events (e.g., recorded video) are not available or would consume more system resources (e.g., more memory and processor time for storing and processing video or other image data). See para 41- In one instance, a passenger can use the interface element to control and manipulate the visualization of the simulated scene over time (i.e., “playing with the time”) by presenting the future in a variety of increments (e.g., 5 second, 10 second, 20 second). )
Claim(s) 3, 6-7 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tuukkanen et al. (PUB NO: US20200041997A1) in view of Amelunxen et al. (PUB NO: US20190295335A1) and further in view of Zhou (PUB NO: US20200348834A1)
Regarding claim 3
The combination of Tuukkanen and Amelunxen teaches the method according to claim 1. Tuukkanen further teaches comprising, prior to selection of the time instant, displaying the interactive visualisation at an initial time instant and, responsive to selection of the selected time instant, rendering a new interactive visualisation on the display of the scene at the selected time instant (see para 86-87-89- Consequently, the passenger first moves the interface element 805 to t1 (e.g., start) as shown by the indicator 807. Referring to FIG. 8B, once the passenger turns the interface element 805 to t2 (e.g., 5 seconds in the future) as shown by the indicator 809 and in the vehicle user interface 801 (e.g., +0:05), the visualization platform 107 can present the passenger with the simulated future event now showing the passenger's vehicle 703 in its proximate position on the road 705 and the proximate position of the vehicle 717, now much closer to the road 705. Once the passenger turns the interface element 805 to t3 (e.g., 10 seconds in the future) as shown by the indicator 811 and in the vehicle user interface 801 in FIG. 8C, the visualization platform 107 can present the proximate position of the passenger's vehicle 703 on the road 705 and the proximate position of the vehicle 717, now on the same road 705 and face-to-face. Referring to FIG. 8D, once the passenger turns the interface element 805 to t4 (e.g., 20 seconds in the future) as shown by the indicator 813 and in the vehicle user interface 801, the visualization platform 107 can present the proximate position of the passenger's vehicle 703, now on road 707 and well ahead of vehicle 717, and the proximate position of the vehicle 717, now just about to turn west from road 705 to road 707. While FIGS. 8A-8D depict a forward representation of time, the passenger could also use the interface element 805 to move backwards in the simulated of the future event (e.g., to “replay” a scene that the passenger is uncertain about such as during t3).)
The combination of Tuukkanen and Amelunxen does not teach without rendering views of the scene at time instants between the initial time instant and the selected time instant
In the related field of invention, Zhou teaches without rendering views of the scene at time instants between the initial time instant and the selected time instant; (See para 58-59-In some cases, the content item display system 120 may visually jump directly to the associated content item(s) so that no intervening content items are scrolled past. In other examples, the content item display system 120 may generate content markers that are associated with a particular time in the user's content item browsing history. For example, content markers can be associated with a specific time such as “three minutes ago,” “five minutes ago,” “twenty minutes ago,” and so on. See para 38- In other example embodiments, the computer system may display the content item associated with the selected content marker without displaying a scrolling animation.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method for generating a simulation of the future events for an autonomous vehicle as disclosed by Tuukkanen to include defining in the scenario a starting constraint to trigger the interaction(s) as taught by Zhou in the system of Tuukkanen and Amelunxen in order to allow a user to immediately jump to a desired content item and thus avoid the need to re-download or re-display any unwanted, intermediate content items from a remote server system. This can result in reduced bandwidth usage, reduced use of processors to display unwanted content items. (see para 0041, Zhou).
Regarding claim 6
The combination of Tuukkanen, Amelunxen and Zhou teaches the method according to claim 3. Tuukkanen further teaches wherein the selected time instant is later in the timeline than the initial time instant. (see para 41- In one instance, a passenger can use the interface element to control and manipulate the visualization of the simulated scene over time (i.e., “playing with the time”) by presenting the future in a variety of increments (e.g., 5 second, 10 second, 20 second). See para 80- As shown, the map user interface 701 of FIGS. 7A-7D (e.g., a UE 105 such as a mobile device) depicts a presentation of a future event over time t1 (FIG. 7A), t2 (FIG. 7B), t3 (FIG. 7C), and t4 (FIG. 7D), by the visualization platform 107 to one or more passengers of the vehicle 703 (e.g., an autonomous car).)
Regarding claim 7
The combination of Tuukkanen, Amelunxen and Zhou teaches the method according to claim 3. Tuukkanen further teaches wherein the selected time instant is earlier in the timeline than the initial time instant. (see para 41- In one instance, a passenger can use the interface element to control and manipulate the visualization of the simulated scene over time (i.e., “playing with the time”) by presenting the future in a variety of increments (e.g., 5 second, 10 second, 20 second). See para 80- As shown, the map user interface 701 of FIGS. 7A-7D (e.g., a UE 105 such as a mobile device) depicts a presentation of a future event over time t1 (FIG. 7A), t2 (FIG. 7B), t3 (FIG. 7C), and t4 (FIG. 7D), by the visualization platform 107 to one or more passengers of the vehicle 703 (e.g., an autonomous car). See para 89- While FIGS. 8A-8D depicts a forward representation of time, the passenger could also use the interface element 805 to move backwards in the simulated of the future event)
Regarding claim 16
Claim 16 is rejected for the same reasons as Claim 2, as they share the same elements.
Regarding claim 17
Claim 17 is rejected for the same reasons as Claim 3, as they share the same elements.
Regarding claim 18
Claim 18 is rejected for the same reasons as Claim 4, as they share the same elements.
Regarding claim 19
Claim 19 is rejected for the same reasons as Claim 5, as they share the same elements.
Regarding claim 20
Claim 20 is rejected for the same reasons as Claim 8, as they share the same elements.
Conclusion
7. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Chu et al. US11921504B1
Discussing the method for generating simulations for evaluating an update to a controller. The controller may be configured to control one or more functionalities of an autonomous and/or a semi-autonomous vehicle.
8. All claims 1-20 are rejected.
9. Any inquiry concerning this communication or earlier communications from the examiner should be directed to PURSOTTAM GIRI whose telephone number is (469)295-9101. The examiner can normally be reached 7:30-5:30 PM, Monday to Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, RENEE CHAVEZ can be reached at 5712701104. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PURSOTTAM GIRI/
Examiner, Art Unit 2186
