DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 9 , 16, and 18-19 are objected to because of the following informalities:
In both claims 9 and 19, “as part of the transition” should instead read “as part of a transition”
In claim 16, “state of the autonomous vehicle the video” should instead read “state of the autonomous vehicle of the video” or similar
In claim 18, “comprising a plurality of instructions… that… cause a VAV demonstration system to…” should instead read “comprising a plurality of instructions… that… causes a VAV demonstration system to…” or “comprising instructions… that… cause a VAV demonstration system to…”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 3 and 18-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 3 recites the limitation "the steering wheel angle data" in line 1. There is insufficient antecedent basis for this limitation in the claim.
Claim 18 recites the limitation "the steering wheel" in line 9. There is insufficient antecedent basis for this limitation in the claim.
Claims 19-20 are rejected for depending upon claim 18.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-8, 15-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over US 2026/0029787 (hereinafter “Qi”) in view of US 2019/0050520 (hereinafter “Alvarez”) and WO 2024/209665 (hereinafter “Kato”).
Regarding Claims 1, 15, and 18, Qi discloses a display device configured to display a VAV demonstration application (fig. 1; par. 0013: “driver-in-loop simulator 10 further include one or more driver view displays 16 and one or more human-machine interface (HMI) displays 18. The driver view display 16 is configured to show a virtual environment, simulating a driving scenario”);
a steering wheel (par. 0013: “input device 14 may be a steering wheel”); and
a control device coupled with the steering wheel and the display device (fig. 1; par. 0013: “simulator 10 configured to simulate an autonomous vehicle… input device 14 may be a steering wheel… one or more driver view displays 16 and one or more human-machine interface (HMI) displays 18;” par. 0004: “a controller in communication with the driver view display, the HMI display, and the input device”), the control device comprising at least one processor in communication with at least one memory device (par. 0013: “The controller includes a processor and a non-transitory computer readable medium in communication with the processor”), the at least one processor programmed to:
control, via a video control scheme, playback of the video data (par. 0004: “a controller in communication with the driver view display, the HMI display, and the input device;” par. 0013: “driver view display 16 is configured to show a virtual environment, simulating a driving scenario;” fig. 1: videos presented on displays 16, 18).
Qi implies but does not explicitly disclose that data is collected from real-world driving sessions of autonomous vehicles. However, Alvarez discloses the VAV demonstration application includes video data and steering wheel data obtained from a real-world driving session of an autonomous vehicle (par. 0020: “Data may be collected from autonomous vehicles, such as when an autonomous vehicle is in an accident or encounters a hazard, and then a simulator may recreate the incident with the performance characteristics of the autonomous vehicle; par. 0023: “When the data collection device 110 interfaces with the OBD, it may extract a vehicle performance profile based on correlation functions across engine speed, wheel speed, vehicle speed, and pedal and steering wheel position messages;” par. 0030: “receive data describing an environment and roadway to construct a simulated environment. This data… may be taken from a real world environment for recreation in the simulator;” par. 0035: “FIGS. 3A and 3B illustrate an example simulated roadway course 300 and a run-time view of a simulation 350… based on a real world environment”).
Alvarez further implies a processor (fig. 7) programmed to: control, via a steering wheel control scheme, movement of the steering wheel based on the steering wheel data and in correspondence with motion of the autonomous vehicle in the video data (par. 0031: “control the simulated vehicle, such as with a steering wheel;” par. 0048: “operation 608 may perform the simulated driving experience with the simulated vehicle using an autonomous vehicle driving system of directives as the driver;” figs. 3A-3B, 5A-5C, 6: the vehicle and steering wheel are controlled based on collected performance information from the real autonomous vehicle, such as the steering wheel data, and on the provided video simulation).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the real-world data and/or steering wheel control of Alvarez with the VAV demonstration/training application of Qi in order to more accurately create a demonstration based on real data (Alvarez, pars. 0020, 0022-0024) and/or to allow both human users and autonomous systems an opportunity to train and/or understand autonomous driving systems (Alvarez, fig. 6; pars. 0018-0020, 0051).
Kato explicitly discloses a processor (par. 0036) programmed to: control, via a steering wheel control scheme, movement of the steering wheel (par. 0039: “the HMD drive unit 29b is controlled to visually present the steering… operation amounts of the exemplary driving to the trainee H, and the trainee H moves their hands… in accordance with the presented… movements, thereby allowing the trainee H to experience the steering… operations of the exemplary driving”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further combine the physical movement of the steering wheel of Kato with the autonomous control of the steering of the vehicle of Qi modified by Alvarez in order to visually demonstrate the steering and/or to show the user the optimal steering angle (Kato, figs. 1, 6-7; par. 0039).
Further regarding Claim 15, Qi modified by Alvarez discloses a method for virtual autonomous vehicle (VAV) demonstration (Qi, fig. 1; par. 0017: “a method 100 is used for simulating an autonomous vehicle”) comprising the above steps (see claim 1).
Further regarding Claim 18, Qi modified by Alvarez discloses one or more non-transitory computer-readable storage media comprising a plurality of instructions stored thereon that, in response to being executed (Qi, par. 0016: “non-transitory computer readable storage device or media 46 includes machine-readable instructions… that when executed by the one or more processors, cause the processors 44 to execute the method”), cause a VAV demonstration system to perform the above steps (see claim 1).
Regarding Claims 2 and 17, Qi modified by Alvarez further discloses the steering wheel data comprises steering wheel angle data obtained from motion of a steering wheel of the autonomous vehicle during the driving session (Alvarez, par. 0020: “extract a vehicle performance profile… steering wheel position messages… the maximum steering radius”).
The combination of the VAV demonstration application of Qi with the extracted steering wheel data of Alvarez described above for claims 1 and 15 would have included the steering wheel angle data.
Regarding Claim 3, Qi modified by Alvarez discloses the use of data structures (Alvarez, par. 0061) but does not specify map data structures. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify Qi so that the steering wheel angle data is mapped and stored in a map data structure of the VAV demonstration application. The KSR Court recognized that “[w]hen there is a design need or market pressure to solve a problem and there are a finite number of identified, predictable solutions, a person of ordinary skill has good reason to pursue the known options within his or her technical grasp.” KSR International Co. v. Teleflex Inc., 550 U.S. at 421, 82 USPQ2d at 1937. In this particular case, Qi modified by Alvarez discloses the use of some kind of data structure to store the collected autonomous vehicle data, which explicitly includes steering wheel angle data. Furthermore, there is a finite set of options in regards to what kind of data structure could be used (e.g., key-value structures, linear structures, hierarchical structures). Any of these options would therefore be obvious for a person having ordinary skill in the art to pursue. Furthermore, Examiner notes a map data structure is very commonly used in the field for storing this kind of data (e.g., a key of a timestamp mapped to a value of what occurs at that timestamp).
Regarding Claim 4, Qi modified by Kato further discloses the at least one processor is further programmed to: determine a video timestamp of a progress in the video data (par. 0069: “Within the comparison display 83, the line TL indicating the time displayed in video 81;” par. 0060: “the driving skill evaluation unit 63 compares time-series data (waveform W1) of the steering angle when trainee H simulates driving a certain training course with time-series data (waveform W2) of the steering angle of exemplary driving actions”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the timestamp determination of Kato with the video presentation of modified Qi in order to effectively determine what point of the video the user is at and in order to later compare data of the user and data of the provided example video based on the time the compared events occurred (Kato, pars. 0060, 0069-0070).
Regarding Claim 5, Qi modified by Kato further discloses the at least one processor is further programmed to: determine a target wheel angle value in the steering wheel data as an angle value at a wheel angle timestamp corresponding to the video timestamp (par. 0070: “at the time indicated by line TL in comparison display 83, the steering angle by trainee H is greater than that of the model driver, resulting in the notification of ‘You are turning the steering wheel a little too much;’” par. 0060: “the driving skill evaluation unit 63 compares time-series data (waveform W1) of the steering angle when trainee H simulates driving a certain training course with time-series data (waveform W2) of the steering angle of exemplary driving actions”).
The combination of the video presentation and VAV demonstration/training application of Qi with the timestamp determination of Kato described above for Claim 4 would have included the determination of a target wheel angle value based on the wheel angle value at that timestamp.
Regarding Claim 6, Qi modified by Kato further discloses the at least one processor is further programmed to: calculate an amount of force to be applied to the steering wheel based on the target wheel angle value and a current wheel angle value of the steering wheel (par. 0070: “at the time indicated by line TL in comparison display 83, the steering angle by trainee H is greater than that of the model driver, resulting in the notification of ‘You are turning the steering wheel a little too much;’” par. 0045: “the driving skill evaluation unit 63 compares the steering angle of the steering wheel 17… calculates the degree of deviation”).
The combination of the video presentation and VAV demonstration/training application of Qi with the determination of a target wheel angle value of Kato described above for Claim 5 would have included this calculation/determination.
Regarding Claim 7, Qi modified by Kato further discloses the at least one processor is further programmed to: calculate the force by: executing a braking routine when a difference between the target wheel angle value and the current angle value is at or below a threshold (par. 0060: “The driving skill evaluation unit 63 calculates the degree of deviation X between the measured data of the trainee H's driving actions and the data of the exemplary driving actions of the exemplary driver… the steering angle… brake operation amount;” par. 0061: “the driving skill evaluation unit 63 may calculate the deviation X for each evaluation item as the percentage of time during which the amount of operation of the steering wheel 17… brake pedal 23 by the trainee H exceeds a predetermined operating error set in advance;” par. 0087: “during the simulated driving experience, the driving training processing unit 61 may visually present the amount of steering input… and brake input to the trainee H based on data of the exemplary driving actions of the model driver;” Examiner notes this calculation is performed whenever the difference threshold is at least above 0).
The combination of the video presentation and VAV demonstration/training application of Qi with the determination of a target wheel angle value of Kato described above for Claim 5 would have included this calculation/determination.
Regarding Claim 8, Qi modified by Kato further discloses the at least one processor is further programmed to: calculate the force by: executing a rotation calculation routine when a difference between the target wheel angle value and current wheel angle value is above a threshold (par. 0045: “the driving skill evaluation unit 63 compares the steering angle of the steering wheel 17… calculates the degree of deviation;” par. 0061: “the driving skill evaluation unit 63 may calculate the deviation X for each evaluation item as the percentage of time during which the amount of operation of the steering wheel 17… by the trainee H exceeds a predetermined operating error set in advance;” Examiner notes this calculation is performed whenever the difference threshold is at least above 0).
The combination of the video presentation and VAV demonstration/training application of Qi with the determination of a target wheel angle value of Kato described above for Claim 5 would have included this calculation/determination.
Regarding Claim 16, Qi further discloses controlling a status light associated with the display device and the steering wheel to synchronize an illumination of the status light with a driving state of the autonomous vehicle the video (par. 0024: “the HMI display 18 may show a green steering wheel and text indicative that the autonomous driving is engaged;” par. 0013: “running a high-quality autonomous vehicle (AV) virtual simulation environment with dynamic scenarios. The driver view display 16 may be in the form of multiple curved screens. The HMI display 18 is configured to simulate the HMI display of an autonomous vehicle. The HMI display 18 may be in communication with the AV simulation environment… may includ[e]… steering wheel/turning lights to be used in the TOC logic”).
Regarding Claim 20, Qi modified by Kato further discloses the plurality of instructions further cause the VAV demonstration system to: determine a video timestamp of a progress in the video data (par. 0069: “Within the comparison display 83, the line TL indicating the time displayed in video 81”);
determine a target wheel angle value in the steering wheel data as an angle value at a wheel angle timestamp corresponding to the video timestamp (par. 0060: “the driving skill evaluation unit 63 compares time-series data (waveform W1) of the steering angle when trainee H simulates driving a certain training course with time-series data (waveform W2) of the steering angle of exemplary driving actions”); and
calculate an amount of force to be applied to the steering wheel based on the target wheel angle value and a current wheel angle value of the steering wheel (par. 0070: “at the time indicated by line TL in comparison display 83, the steering angle by trainee H is greater than that of the model driver, resulting in the notification of ‘You are turning the steering wheel a little too much’”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the timestamp determination of Kato with the video presentation of modified Qi in order to effectively determine what point of the video the user is at and in order to later compare data of the user and data of the provided example video based on the time the compared events occurred (Kato, pars. 0060, 0069-0070).
Claims 9-13 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Qi in view of Alvarez and Kato as applied to claims 1 and 18 above, and further in view of US 2020/0020244 (hereinafter “Best”).
Regarding Claims 9 and 19, Qi discloses one or more selectable scenario videos (par. 0013: “a high-quality autonomous vehicle (AV) virtual simulation environment with dynamic scenarios;” par. 0017: “predefined scenarios include traffic vehicle behavior and targeting routes. The scenario data also includes TOC trigger conditions”), and a scenario video of the one or more selectable scenario videos is selected by a user via a control input of the steering wheel (par. 0005: “driver-in-loop simulator where the input device is a steering wheel… programmed to receive scenario data. The controller is programmed to command the driver view display to show a scenario that is based on the scenario data”).
Qi does not explicitly disclose an idle video. However, Best discloses the video data includes an idle video and one or more selectable scenario videos (par. 0027: “the information console 16 presents menus… accepts inputs such as scenario selection;” Examiner notes the video screen featuring menus and displaying scenarios for selection is considered an “idle video;” fig. 2: video on display 12 when there is no user present is another idle video of the video data).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the idle video of Best with the VAV demonstration/training application of Qi in order to display important information to the user before the scenario video begins (Best, par. 0027) and/or to display an interesting default screen until a user interacts with the system and selects a scenario (Best, fig. 2).
Modified Qi also does not explicitly disclose any transition effects; however, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify Qi so that the at least one processor is further programmed to: display a transition effect as part of the transition between the idle video and the selected scenario video. The KSR Court recognized that “[w]hen there is a design need or market pressure to solve a problem and there are a finite number of identified, predictable solutions, a person of ordinary skill has good reason to pursue the known options within his or her technical grasp.” KSR International Co. v. Teleflex Inc., 550 U.S. at 421, 82 USPQ2d at 1937. In this particular case, when a user selects a video to play which will replace the current idle video, there is a finite number of ways to handle this, e.g., a hard cut or some kind of transition effect. Either of these options would therefore be obvious for a person having ordinary skill in the art to pursue. Furthermore, the use of a transition effect between the two videos represents a mere design choice and offers no specific utility, and therefore, this limitation does not patentably distinguish the claimed invention from the prior art. In re Seid, 161 F.2d 229, 73 USPQ 431 (CCPA 1947) (Claim was directed to an advertising display device comprising a bottle and a hollow member in the shape of a human figure from the waist up which was adapted to fit over and cover the neck of the bottle, wherein the hollow member and the bottle together give the impression of a human body. Appellant argued that certain limitations in the upper part of the body, including the arrangement of the arms, were not taught by the prior art. The court found that matters relating to ornamentation only which have no mechanical function cannot be relied upon to patentably distinguish the claimed invention from the prior art.). See MPEP § 2144.04-I.
Regarding Claim 10, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify Qi so that the at least one processor is further programmed to: display the transition effect in a first direction and then in a second direction opposite the first direction. The KSR Court recognized that “[w]hen there is a design need or market pressure to solve a problem and there are a finite number of identified, predictable solutions, a person of ordinary skill has good reason to pursue the known options within his or her technical grasp.” KSR International Co. v. Teleflex Inc., 550 U.S. at 421, 82 USPQ2d at 1937. In this particular case, there is a finite number of directions in which the transition effect can occur. Any of these options would therefore be obvious for a person having ordinary skill in the art to pursue. Furthermore, the direction of the transition effect represents a mere design choice and offers no specific utility, and therefore, this limitation does not patentably distinguish the claimed invention from the prior art. See MPEP § 2144.04-I.
Regarding Claim 11, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify Qi so that the least one processor is further programmed to: overlay the transition effect over the idle video while the selected scenario video is being loaded. The KSR Court recognized that “[w]hen there is a design need or market pressure to solve a problem and there are a finite number of identified, predictable solutions, a person of ordinary skill has good reason to pursue the known options within his or her technical grasp.” KSR International Co. v. Teleflex Inc., 550 U.S. at 421, 82 USPQ2d at 1937. In this particular case, there is a finite number of ways to implement the transition, e.g., overlaying it over the initial idle video while the selected video loads or implementing it after the video has finished loading. Any of these options would therefore be obvious for a person having ordinary skill in the art to pursue. Furthermore, the placement of the transition effect represents a mere design choice and offers no specific utility, and therefore, this limitation does not patentably distinguish the claimed invention from the prior art. See MPEP § 2144.04-I.
Regarding Claim 12, Qi further discloses a status light, wherein the at least one processor is further programmed to: synchronize an illumination of the status light with a driving state of the autonomous vehicle in at least one of the one or more selectable scenario videos (par. 0024: “the HMI display 18 may show a green steering wheel and text indicative that the autonomous driving is engaged;” par. 0013: “running a high-quality autonomous vehicle (AV) virtual simulation environment with dynamic scenarios. The driver view display 16 may be in the form of multiple curved screens. The HMI display 18 is configured to simulate the HMI display of an autonomous vehicle. The HMI display 18 may be in communication with the AV simulation environment… may includ[e]… steering wheel/turning lights to be used in the TOC logic”).
Regarding Claim 13, Qi further discloses the driving state of the autonomous vehicle includes an autonomous driving state and a manual driving state (par. 0005: “simulate driving by an autonomous vehicle until the controller receives the input that is indicative of the user took control of the simulation in response to detecting the toc event”), and the at least one processor is further programmed to: control the illumination of the status light in a first color corresponding to the autonomous driving state and a second color corresponding to the manual driving state, the second color being different than the first color (par. 0024: “the TOC notification may be a yellow steering wheel with text indicating that the user should take over manually… the HMI display 18 may show a green steering wheel and text indicative that the autonomous driving is engaged”).
Claim 14 is under 35 U.S.C. 103 as being unpatentable over Qi in view of Alvarez and Kato as applied to claim 1 above, and further in view of US 2017/003987 (hereinafter “Harkness”).
Regarding Claim 14, modified Qi does not disclose a user survey. However, Harkness discloses the at least one processor is further programmed to: present, as part of the video control scheme, one or more user surveys on the display device (par. 0015: “computerized means for presenting a user with questions regarding events in the footage… A database records responses to the questions”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the user survey of Harkness with the VAV demonstration/training application of Qi in order to ascertain the effectiveness of the system and/or to generally test user knowledge (Harkness, pars. 0089, 0093, 0015, 0135).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 2024/0177535 (Düser) teaches a system for simulating and testing autonomous vehicle operation based on generated scenarios. The simulation comprises a monitor displaying scenario videos and a steering wheel that the user may use.
US 10,625,676 (Tsimhoni) teaches an autonomous vehicle demonstration/interaction system in real video footage is gathered from sensors on real autonomous vehicles.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JULIE DOSHER whose telephone number is (571) 272-4842. The examiner can normally be reached Monday - Friday, 10 a.m. - 6 p.m. ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Dmitry Suhol can be reached at (571) 272-4430. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.G.D./Examiner, Art Unit 3715
/DMITRY SUHOL/Supervisory Patent Examiner, Art Unit 3715
Prosecution Insights