CTNF 18/952,074 CTNF 100403 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Objections 07-29-01 AIA Claim 8 objected to because of the following informalities: Claim 8, line 2 recites “comprising or more computer readable storage media”. It should read as “comprising one or more computer readable storage media” . Appropriate correction is required. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim (s) 1-3, 7-10, 14-16 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hummelshoj et al. (US 20190228118 A1), hereinafter as Hummelshoj . Regarding claim 1, Hummelshoj teaches A computer-implemented method for generating vehicular-based visualizations (Hummelshoj paragraph [0016] “FIG. 3 illustrates one embodiment of a method associated with varying an amount and type of information that is provided to a vehicle operator to ascertain how the vehicle operator perceives a surrounding environment.”), the method comprising: receiving, by a computing device, a plurality of vehicular parameters associated with a vehicle (Hummelshoj teaches sensor data as a plurality of vehicular parameters, paragraph [0033-0034] “the scene module 220 selects a set of previously gathered sensor data (e.g., video, LIDAR data, etc.) that the scene module 220 transforms into the simulation…… the sensor data can include, for example, scan data that embodies observations of one or more objects in a surrounding environment proximate to the vehicle 100 and/or other aspects about the surroundings.”, paragraph [0073] “The example sensors may be part of the one or more environment sensors 122 and/or the one or more vehicle sensors 121”); analyzing, by the computing device, the plurality of vehicular parameters based on a user profile associated with at least one occupant of the vehicle (Hummelshoj teaches log 260 as the user profile related to the electronic inputs from the vehicle operator, Figure 3, paragraph [0052-0054] “At 350, the interface module 230 monitors for electronic inputs of the vehicle 100 that are generated by an operator …… as the interface module 230 detects the inputs, the interface module 230 logs the inputs into the log 260 as correlating with a particular portion of the driving scene and a particular one of the visualization algorithms 250……. the interface module 230 scores the inputs by comparing the inputs with recorded inputs from when the sensor data was collected. That is, the interface module 230 can compare the inputs from the operator with original inputs to determine how closely the present inputs are in relation to the original inputs.”); and generating, by the computing device, a virtual environment visualization associated with the vehicle based on the analysis (Hummelshoj Figure 3, paragraph [0055] “At 370, the interface module 230 determines whether the control inputs are adequate. In one embodiment, the interface module 230 indicates when the control inputs are out of bounds (e.g., insufficient) to the scene module 220 so that the scene module 220 can adapt which of the algorithms 250 are used to modify the driving scene. In this way, the perception system 170 can customize the visualization such that the operator is provided with various displays of information for the driving scene and a best-fit can be determined.”). Hummelshoj and the current application are in the same field of endeavor, namely generating vehicle related virtual experience. In various embodiments, Hummelshoj teaches a method to modify visualization based on vehicle sensor data and user input to improve efficiency (paragraph [0025] “the perception system can test which information is useful to the operator under various conditions and, thus, identify machine vision approaches that correlate with the visualization algorithm to improve the efficiency of processing sensor data such that a particular processing or reduced set of information can be applied instead of a brute force approach.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of various embodiments of Hummelshoj to improve efficiency. Regarding claim 2, Hummelshoj teaches The computer-implemented method of claim 1, and further teaches wherein analyzing the plurality of vehicular parameters further comprises: analyzing, by the computing device, a plurality of contextual information associated with the at least one occupant (Hummelshoj paragraph [0021] “By modifying how the driving scene is displayed, the perception system can log characteristics about how the operator perceives the surrounding environment to provide controls to the vehicle.”); analyzing, by the computing device, a plurality of feedback information associated with the virtual environment visualization and the at least one occupant (Hummelshoj teaches grading based on user input as the feedback information, paragraph [0007] “in one embodiment, modifies the visualization to provide a reduced/modified view of the driving scenario to the operator. Accordingly, as the operator provides driving inputs while perceiving the modified view, the perception system can grade or otherwise assess whether the operator is controlling the vehicle in a safe or generally acceptable manner for the present driving conditions.”, paragraph [0009] “the perception system dynamically varies which visualization algorithm modifies the visualization of the driving scene as a function of changing driving conditions, feedback about driving inputs, and/or according to other criteria.”); and updating, by the computing device, the user profile based on the plurality of feedback information (Hummelshoj teaches a loop from checking Control Inputs Sufficient 370 to Select Visualization Algorithm 320 in Figure 3, paragraph [0050-0055] “the scene module 220 uses the visualization algorithm to apply one or more techniques to modify which aspects of the driving scene are displayed to the operator. As a general approach, the algorithms 250 focus on select aspects of the simulated environment in order to improve processing efficiency through processing a subset of available sensor data…… as the interface module 230 detects the inputs, the interface module 230 logs the inputs into the log 260 as correlating with a particular portion of the driving scene and a particular one of the visualization algorithms 250…… At 370, the interface module 230 determines whether the control inputs are adequate. In one embodiment, the interface module 230 indicates when the control inputs are out of bounds (e.g., insufficient) to the scene module 220 so that the scene module 220 can adapt which of the algorithms 250 are used to modify the driving scene.”). Regarding claim 3, Hummelshoj teaches The computer-implemented method of claim 1, and further teaches wherein the virtual environment visualization is depicted to the at least one occupant via a computer-mediated reality device (CMR) communicatively coupled to at least one display of the vehicle (Hummelshoj paragraph [0022] “ the simulator is implemented within a functional vehicle by integrating display screens within the vehicle, providing for virtual reality interfaces, providing for augmented reality interfaces, and so on.”, paragraph [0026] “ the vehicle 100 is generally a vehicle simulator that includes a display positioned in a manner so as to simulate views of a surrounding environment of the vehicle 100 to the operator. Of course, in further embodiments, the display may instead be a virtual reality head-mounted device (HMD) or augmented reality HMD…… The simulation system 180 can be integrated with a standard functional vehicle or may be a stand-alone system that is independent of an actual vehicle.”). Regarding claim 7, Hummelshoj teaches The computer-implemented method of claim 1, and further teaches wherein generating the virtual environment visualization comprises: integrating, by the computing device, the vehicle and a subset of the plurality of vehicle parameters into the virtual environment visualization (Hummelshoj paragraph [0050] “the algorithms 250 focus on select aspects of the simulated environment in order to improve processing efficiency through processing a subset of available sensor data.”, paragraph [0073] “ The example sensors may be part of the one or more environment sensors 122 and/or the one or more vehicle sensors 121); and updating, by the computing device, the virtual environment visualization based on the user profile (Hummelshoj Figure 3, paragraph [0055] “At 370, the interface module 230 determines whether the control inputs are adequate. In one embodiment, the interface module 230 indicates when the control inputs are out of bounds (e.g., insufficient) to the scene module 220 so that the scene module 220 can adapt which of the algorithms 250 are used to modify the driving scene. In this way, the perception system 170 can customize the visualization such that the operator is provided with various displays of information for the driving scene and a best-fit can be determined.”). Regarding claim 8, it recites similar limitations of claim 1 but in a computer program product form. The rationale of claim 1 rejection is applied to reject claim 8. In addition, Hummelshoj teaches A computer program product for generating vehicular-based visualizations, the computer program product comprising or more computer readable storage media and program instructions collectively stored on the one or more computer readable storage media, the stored program instructions comprising (Hummelshoj paragraph [0090] “arrangements described herein may take the form of a computer program product embodied in one or more computer-readable media having computer-readable program code embodied, e.g., stored, thereon.”). Hummelshoj and the current application are in the same field of endeavor, namely generating vehicle related virtual experience. In various embodiments, Hummelshoj teaches a method to modify visualization based on vehicle sensor data and user input to improve efficiency (paragraph [0025] “the perception system can test which information is useful to the operator under various conditions and, thus, identify machine vision approaches that correlate with the visualization algorithm to improve the efficiency of processing sensor data such that a particular processing or reduced set of information can be applied instead of a brute force approach.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of various embodiments of Hummelshoj to improve efficiency. Regarding claim 9, claim 9 has similar limitations as claim 2, therefore it is rejected under the same rationale as claim 2. Regarding claim 10, claim 10 has similar limitations as claim 3, therefore it is rejected under the same rationale as claim 3. Regarding claim 14, claim 14 has similar limitations as claim 7, therefore it is rejected under the same rationale as claim 7. Regarding claim 15, it recites similar limitations of claim 1 but in a computer system form. The rationale of claim 1 rejection is applied to reject claim 15. In addition, Hummelshoj teaches A computer system for generating vehicular-based visualizations, the computer system comprising: one or more processors; one or more computer-readable memories; program instructions stored on at least one of the one or more computer-readable memories for execution by at least one of the one or more processors, the program instructions comprising (Hummelshoj paragraph [0082] “The vehicle 100 can include one or more modules, at least some of which are described herein. The modules can be implemented as computer-readable program code that, when executed by a processor 110, implement one or more of the various processes described herein. One or more of the modules can be a component of the processor(s) 110, or one or more of the modules can be executed on and/or distributed among other processing systems to which the processor(s) 110 is operatively connected. The modules can include instructions (e.g., program logic) executable by one or more processor(s) 110. Alternatively, or in addition, one or more data store 115 may contain such instructions.”). Hummelshoj and the current application are in the same field of endeavor, namely generating vehicle related virtual experience. In various embodiments, Hummelshoj teaches a method to modify visualization based on vehicle sensor data and user input to improve efficiency (paragraph [0025] “the perception system can test which information is useful to the operator under various conditions and, thus, identify machine vision approaches that correlate with the visualization algorithm to improve the efficiency of processing sensor data such that a particular processing or reduced set of information can be applied instead of a brute force approach.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of various embodiments of Hummelshoj to improve efficiency. Regarding claim 16, claim 16 has similar limitations as claim 2, therefore it is rejected under the same rationale as claim 2. Regarding claim 20, claim 20 has similar limitations as claim 7, therefore it is rejected under the same rationale as claim 7 . 07-21-aia AIA Claim (s) 4-5, 11-12 and 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hummelshoj et al. (US 20190228118 A1), hereinafter as Hummelshoj, in view of Toyoda et al. (US 20180322783 A1), hereinafter as Toyoda . Regarding claim 4, Hummelshoj teaches The computer-implemented method of claim 1, wherein generating the virtual environment visualization comprises: but is not relied on for the below claim language utilizing, by the computing device, one or more machine learning models to generate one or more predictions associated with a vehicular experience based on the plurality of vehicular parameters; and integrating, by the computing device, a plurality of virtual elements derived from the one or more predictions into the virtual environment visualization. Toyoda teaches utilizing, by the computing device, one or more machine learning models to generate one or more predictions associated with a vehicular experience based on the plurality of vehicular parameters (Toyoda paragraph [0045] “the monitoring module 220 identifies the potential hazards by analyzing the sensor data according to, for example, the hazard model 250. That is, in one embodiment, the monitoring module 220 characterizes aspects of the surrounding environment as provided in the sensor data to determine whether the combination of objects, and other factors constitute one or more potential hazards to the vehicle 100. Additionally, the monitoring module 220 may accept the sensor data as electronic inputs and process the electronic inputs according to a machine learning algorithm and learned aspects embodied in the hazard model 250 to identify the potential hazards.”); and integrating, by the computing device, a plurality of virtual elements derived from the one or more predictions into the virtual environment visualization (Toyoda paragraph [0048-0050] “the engagement module 230 determines which potential hazards identified at 320 will be rendered in the AR system 180 as display scenarios…… when a potential hazard satisfies the hazard threshold (e.g., 30% likelihood of risk or greater), then the engagement module 230 renders graphical elements for the potential hazard…… the engagement module 230 renders graphical elements that correlate with the potential hazards within one or more AR displays of the AR system 180. In one embodiment, the engagement module 230 renders graphical elements within the AR system 180 by rendering overlays on objects/regions within the surrounding environment (e.g., animations, avatars, etc.),”). Hummelshoj and Toyoda are in the same field of endeavor, namely generating vehicle related virtual experience. Toyoda teaches displaying potential hazards using graphical elements to improve driver engagement (paragraph [0020] “the disclosed systems and methods improve driver engagement on driving tasks and awareness of surroundings through motivating the self-engagement of the driver.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Toyoda with the method of Hummelshoj to improve user interaction. Regarding claim 5, Hummelshoj in view of Toyoda teach The computer-implemented method of claim 4, and further teach further comprising: correlating, by the computing device, the plurality of vehicular parameters with the plurality of virtual elements based on the user profile (Toyoda teaches using driver state information collected by monitoring module as part of the user profile, paragraph [0056] “ the engagement module 230 analyzes the driver state information collected by the monitoring module 220 to determine whether the driver is responding to the display scenario by engaging the driving tasks and the surrounding environment in an expected manner. For example, the engagement module 230, in one embodiment, can analyze the driver state information to determine whether the driver is visually scanning a region of the surrounding environment for the potential hazard, whether the driver is adjusting a speed and/or path of the vehicle 100 to account for the potential hazard, whether the driver is obeying laws”); and updating, by the computing device, the virtual environment visualization based on the correlation (Toyoda paragraph [0057] “The engagement module 230, in one embodiment, analyzes the driver state information according to the hazard model 250 and/or a separate driver model that characterizes aspects of the driver behavior in order to determine engagement by the driver. Accordingly, the engagement module 230 can adjust the rendering of the display scenario, at 350, that depicts the potential hazard by further embellishing the graphical elements (e.g., decreasing opacity, increasing brightness, adding additional graphics, etc.)”). Hummelshoj and Toyoda are in the same field of endeavor, namely generating vehicle related virtual experience. Toyoda teaches displaying potential hazards using graphical elements to improve driver engagement (paragraph [0020] “the disclosed systems and methods improve driver engagement on driving tasks and awareness of surroundings through motivating the self-engagement of the driver.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Toyoda with the method of Hummelshoj to improve user interaction. Regarding claim 11, claim 11 has similar limitations as claim 4, therefore it is rejected under the same rationale as claim 4. Regarding claim 12, claim 12 has similar limitations as claim 5, therefore it is rejected under the same rationale as claim 5. Regarding claim 17, claim 17 has similar limitations as claim 4, therefore it is rejected under the same rationale as claim 4. Regarding claim 18, claim 18 has similar limitations as claim 5, therefore it is rejected under the same rationale as claim 5 . 07-21-aia AIA Claim (s) 6, 13 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hummelshoj et al. (US 20190228118 A1), hereinafter as Hummelshoj, in view of Sokolov et al. (US 20220242450 A1), hereinafter as Sokolov . Regarding claim 6, Hummelshoj teaches The computer-implemented method of claim 1, but is not relied on for the below claim language wherein the virtual environment visualization comprises one or more interactive virtual objects configured to receive at least one input from the user. Sokolov teaches wherein the virtual environment visualization comprises one or more interactive virtual objects configured to receive at least one input from the user (Sokolov paragraph [0031] “the fusion of all physical and virtual spatial data into the Metaverse World Model is the foundation for interaction between physical and virtual objects…… virtual world objects such as trucks, buses, cars, vans, motorbikes as well as vulnerable road users such as pedestrians, cyclists and animals can be injected into the metaverse world model requiring physical cars to avoid them”). Hummelshoj and Sokolov are in the same field of endeavor, namely generating vehicle related virtual experience. Sokolov teaches the interaction of virtual objects and real objects to improve accuracy (paragraph [0147] “The Metaverse implementation provides a set of methods designed for various sensor data formats and their data processing systems allowing seamless and accurate infusion of artificial virtual objects and conditions into normal sensor data, reflecting real physical objects and conditions.”, and paragraph [0168] “Software algorithm upgrades and changes can be rapidly provided and tested out in these hybrid real-world and virtual world scenarios, greatly increasing the rapidity of algorithm improvement and testing against a far broader range of scenarios than would be possible with testing limited to real-world only testing.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Sokolov with the method of Hummelshoj to improve interaction and accuracy. Regarding claim 13, claim 13 has similar limitations as claim 6, therefore it is rejected under the same rationale as claim 6. Regarding claim 19, claim 19 has similar limitations as claim 6, therefore it is rejected under the same rationale as claim 6 . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Yamala et al. (US 20230154323 A1) teaches a method to generate presentation of vehicle driver profile. Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIAOMING WEI whose telephone number is (571)272-3831. The examiner can normally be reached M-F 8:00-5:00. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571)272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /XIAOMING WEI/Examiner, Art Unit 2611 /KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611 Application/Control Number: 18/952,074 Page 2 Art Unit: 2611 Application/Control Number: 18/952,074 Page 3 Art Unit: 2611 Application/Control Number: 18/952,074 Page 4 Art Unit: 2611 Application/Control Number: 18/952,074 Page 5 Art Unit: 2611 Application/Control Number: 18/952,074 Page 6 Art Unit: 2611 Application/Control Number: 18/952,074 Page 7 Art Unit: 2611 Application/Control Number: 18/952,074 Page 8 Art Unit: 2611 Application/Control Number: 18/952,074 Page 9 Art Unit: 2611 Application/Control Number: 18/952,074 Page 10 Art Unit: 2611 Application/Control Number: 18/952,074 Page 11 Art Unit: 2611 Application/Control Number: 18/952,074 Page 12 Art Unit: 2611 Application/Control Number: 18/952,074 Page 13 Art Unit: 2611 Application/Control Number: 18/952,074 Page 14 Art Unit: 2611 Application/Control Number: 18/952,074 Page 15 Art Unit: 2611 Application/Control Number: 18/952,074 Page 16 Art Unit: 2611 Application/Control Number: 18/952,074 Page 17 Art Unit: 2611
Prosecution Insights