DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-8 and 10 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Powderly et al. (hereinafter “Powderly”), US Pub. No. 2022/0044021.
Regarding claim 1, Powderly teaches a method ([0097]) comprising: receiving a set of tracking data from one or more sensors in a virtual or augmented reality environment (track gestures made by the user (e.g., hand or finger gestures), detect objects in the world 470 in front of the user, and so forth; hand gesture tracking or recognition may also provide input information; eye tracking is another input (e.g., tracking where the user is looking to control the display technology); the AR display system may be a mixed reality display system; the environmental sensors may include sensors; Figs. 1A, 1B, 2-10, 11A-11G, 12A-12D, 13A-13C, 14A, 14B & [0070], [0104], [0105], [0110]), wherein the tracking data identifies one or more physical properties of a participant in a simulation executing within the virtual or augmented reality environment (the user can simulate walking into the virtual classrooms and engage in a class in an AR/MR/VR environment; [0162]); mapping the set of tracking data to coordinates within the virtual or augmented reality environment (once the wearable system recognizes that an environment is a living room in a user's home, the wearable system can automatically associate the location of the environment with a GPS coordinate or with a location in a world map; as a result, when a user enters the same location in the future, the wearable system can present/block virtual content based on the living room environment; [0095]); generating, using an artificial intelligence model, a set of one or more inferences based at least in part on the set of tracking data and the coordinates within the virtual or augmented reality environment (video tracking, object recognition (location and mapping (vSLAM) techniques using various machine learning algorithms; when a user enters the same location in the future, the wearable system can present/block virtual content based on the living room environment; the wearable system may use a location sensor (e.g., a GPS sensor) to determine the user's location and thereby infer the nature of the user's environment; [0092], [0095], [0146]); translating the inferences into a set of instructions for causing at least one virtual avatar or object to react in the virtual or augmented reality environment (while the user is interacting with visual or audible virtual content, the user of the wearable may encounter a triggering event such as, for example, an emergency condition or an unsafe condition, detecting one or more triggering objects in an environment, or determining characteristics of the user's environment; a wearable device that can present a VR, AR, or MR environment, alone or in combination, for user interaction; add virtual elements to the existing physical objects viewed through the transparent optics of the head mounted display, thereby permitting user interaction with the physical objects; enabling interactions with entertainment related virtual content; Abstract & [0034], [0040], [0160]); and executing the set of instructions that cause the at least one virtual avatar or object to react in the virtual or augmented reality environment (while the user is interacting with visual or audible virtual content, the user of the wearable may encounter a triggering event such as, for example, an emergency condition or an unsafe condition, detecting one or more triggering objects in an environment, or determining characteristics of the user's environment; a wearable device that can present a VR, AR, or MR environment, alone or in combination, for user interaction; add virtual elements to the existing physical objects viewed through the transparent optics of the head mounted display, thereby permitting user interaction with the physical objects; enabling interactions with entertainment related virtual content; Abstract & [0034], [0040], [0160]).
Regarding claim 2, Powderly teaches receiving audio input from the participant (voice recognition technology to determine who is speaking (e.g., whether the speech is from the wearer of the wearable device or another person or voice (e.g., a recorded voice transmitted by a loudspeaker in the environment)) as well as speech recognition technology to determine what is being said; [0106]); wherein the set of one or more inferences is further generated based at least in part on the audio input (speech tracking can be another input can be used alone or in combination with other inputs; speech tracking may include speech recognition, voice recognition, alone or in combination; identify content of the speech by applying various speech recognition algorithms; [0106]).
Regarding claim 3, Powderly teaches eye tracking data from a sensor that tracks position information of at least one pupil of the participant (process image information acquired by the inward-facing imaging system to determine, e.g., the eye pose (position) of the user; [0045]); and wherein the set of one or more inferences is further generated based at least in part on the eye tracking data (process image information acquired by the inward-facing imaging system to determine, e.g., the pupil diameters or orientations of the eyes, eye movements or eye pose of the user; [0045]).
Regarding claim 4, Powderly teaches wherein the tracking data include hand and/or finger tracking data that tracks position information of at least one hand and/or finger of the participant (determining pose data (e.g., hand gestures) that can be used in displaying and understanding the orientation and position of various objects in the user's surroundings; monitor the position, movement, and/or other features or properties of the hands of a user; [0098], [0111]); wherein the set of one or more inferences is further generated based at least in part on the hand and/or finger tracking data (determining pose data (e.g., hand gestures) that can be used in displaying and understanding the orientation and position of various objects in the user's surroundings; hand gesture tracking or recognition may also provide input information; monitor the position, movement, and/or other features or properties of the hands of a user; [0098], [0104], [0111]).
Regarding claim 5, Powderly teaches wherein the tracking data includes biomarker data from one or more biomarker sensors that track one or more biomarkers indicative of an emotional state of the participant (these sensors can measure or estimate the user's physiological parameters such as heart rate, respiratory rate, galvanic skin response, blood pressure, encephalographic state and so on; the wearable system can monitor the user's physiological data while a user is playing a game; if the physiological data indicates that the user is experiencing an agitated emotional state such as being extremely angry due to a loss in a game or extremely scared); [0051], [0139]); and wherein the set of one or more inferences is further generated based at least in part on the one or more biomarkers indicative of the emotional state of the participant (these sensors can measure or estimate the user's physiological parameters such as heart rate, respiratory rate, galvanic skin response, blood pressure, encephalographic state and so on; the wearable system can monitor the user's physiological data while a user is playing a game; if the physiological data indicates that the user is experiencing an agitated emotional state such as being extremely angry due to a loss in a game or extremely scared); [0051], [0139]).
Regarding claim 6, Powderly teaches training, by at least one machine learning algorithm, the artificial intelligence model using a set of training data the includes sensor measurements within an augmented or virtual reality environment (augmented reality (AR) system; the environmental sensors may include inertial measurement units (IMUs); a wearable system configured to display virtual content in a mixed reality or virtual reality environment; [0034], [0051], [0255]).
Regarding claim 7, Powderly teaches wherein translating the inferences into a set of instructions for causing at least one virtual avatar or object to react in the virtual or augmented reality environment (while the user is interacting with visual or audible virtual content, the user of the wearable may encounter a triggering event such as, for example, an emergency condition or an unsafe condition, detecting one or more triggering objects in an environment, or determining characteristics of the user's environment; a wearable device that can present a VR, AR, or MR environment, alone or in combination, for user interaction; add virtual elements to the existing physical objects viewed through the transparent optics of the head mounted display, thereby permitting user interaction with the physical objects; enabling interactions with entertainment related virtual content; Abstract & [0034], [0040], [0160]) comprises: generating, based on the set of one or more inferences using a generative language model, an audio encoding that causes a vocalization from a virtual avatar (the system can incorporate voice recognition technology to determine who is speaking (e.g., a recorded voice transmitted by a loudspeaker in the environment)) as well as speech recognition technology to determine what is being said; [0106]).
Regarding claim 8, Powderly teaches wherein translating the inferences into a set of instructions for causing at least one virtual avatar or object to react in the virtual or augmented reality environment (while the user is interacting with visual or audible virtual content, the user of the wearable may encounter a triggering event such as, for example, an emergency condition or an unsafe condition, detecting one or more triggering objects in an environment, or determining characteristics of the user's environment; a wearable device that can present a VR, AR, or MR environment, alone or in combination, for user interaction; add virtual elements to the existing physical objects viewed through the transparent optics of the head mounted display, thereby permitting user interaction with the physical objects; enabling interactions with entertainment related virtual content; Abstract & [0034], [0040], [0160) comprises: causing the virtual avatar to perform an action within the simulation (the user can simulate walking into the virtual classrooms and engage in a class in an AR/MR/VR environment; the user can interact with objects in the user's FOV; for example, the avatar may represent a virtual image of the user's friend; [0162]).
Regarding claim 10, Powderly teaches translating the inferences into a set of instructions for controlling haptic feedback on a device worn or held by the participant (the wearable system can include a user input device by which the user can input commands to the controller to interact with the wearable system; the user input device can include a haptic device; [0073]); and executing the set of instructions to change a current state of the haptic feedback on the device worn or held by the participant (the wearable system can include a user input device by which the user can input commands to the controller to interact with the wearable system; the user input device can include a haptic device; [0073]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Powderly (see above), in view of Fan, US Pub. No. 2020/0330868.
Regarding claim 9, Powderly fails to explicitly teach updating an objective in a training simulation, wherein the reaction of the virtual avatar or virtual object is further based on the updated objective.
However, in the same field of endeavor, Fan teaches updating an objective in a training simulation (an interface display update method is performed by an electronic device, including: displaying a virtual item currently controlled by a virtual object in a virtual scene; updating the virtual scene, based on the target rotation angle, the first rotation speed of the perspective of the virtual object, and the second rotation speed of the virtual item; simulated training scenario; the virtual scene involved in this application may be used for simulating a three-dimensional virtual space or a two-dimensional virtual space; a user may control a virtual object to move in the virtual scene; Abstract & [0010], [0040], [0041]); wherein the reaction of the virtual avatar or virtual object is further based on the updated objective (an interface display update method is performed by an electronic device, including: displaying a virtual item currently controlled by a virtual object in a virtual scene; updating the virtual scene, based on the target rotation angle, the first rotation speed of the perspective of the virtual object, and the second rotation speed of the virtual item; simulated training scenario; the virtual scene involved in this application may be used for simulating a three-dimensional virtual space or a two-dimensional virtual space; a user may control a virtual object to move in the virtual scene; Abstract & [0010], [0040], [0041]).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Powderly to include the feature of Fan. As such, a person having ordinary skill in the art would appreciate the motivation for doing so would have been to simulate actions of a person and adjust a perspective to display different areas in a virtual scene.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Mealer, US Pub. No. 2019/0180637 teaches a virtual and augmented reality simulations for healthcare clinicians.
Shapira et al. (US Pub. No. 2016/0350973) teaches a multi-person immersive virtual reality environment.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KENNETH B LEE JR whose telephone number is (571)270-3147. The examiner can normally be reached Mon - Fri 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Boddie can be reached at 571-272-0666. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KENNETH B LEE JR/Primary Examiner, Art Unit 2625
Prosecution Insights