DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on June 11, 2025 was filed after the filing date of the application on February 12, 2025. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claim 11 is objected to because of the following informalities:
Claim 11 recites, “…based on the user’s physical interaction with the one or more hand-held objects…” but should recite, “…based on the user’s physical interaction with the one or more hand-held components…”
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-11, 13, 16, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nocon (US 10,444,840).
As to claim 1, Nocon discloses a system for providing an extended reality (XR) simulation (e.g. Figure 1, system 100 implemented as system 100 of Figure 3, e.g. providing an augmented reality (AR) or virtual reality (VR) interactive experience), the system comprising a simulation platform (e.g. one or more computing platforms 102 comprising processor(s) 104 further comprising visual control component 108, haptic control component 110, latency component 112, and synchronization component 114) configured to communicate and exchange data with one or more hand-held components (e.g. one or more input devices 122, further illustrated as input device 302, further comprising one or more haptic feedback devices 124, where column 5, lines 11-51 notes one or more input devices 122 configured to receive user input and may include one or more of a handheld input device, an audio input device, an image capturing device, and/or other devices, where column 5, lines 52-67 notes individual haptic feedback devices of one or more haptic feedback devices 124 may be configured to provide haptic output, where the individual haptic feedback devices may be included in individual input devices) and a wearable display (e.g. one or more presentation platforms/devices 120, where column 4, lines 24 thru column 5, lines 10 notes one or more presentation platforms/devices 122 may include one or more of a display, one or more sensors, and/or other components, where the display may include one or more of screen, a set of screens, a touchscreen, a monitor, a headset (e.g. a head-mounted display, glasses, or goggles (e.g. an AR headset and/or a VR headset)), and/or other displays) for respectively providing an associated user (e.g. providing user 301) with dynamically synchronized haptic feedback and visual feedback based on user interaction with a simulated environment (e.g. with dynamically synchronized visual effects and haptic feedback via one or more haptic feedback devices 124)(Figure 2, method 200, column 12, lines 53 thru column 13, lines 10 notes method 200 to synchronize visual effects and haptic feedback for interactive experiences, where column 2, lines 40-54 notes interactive experiences, e.g. in an interactive space, e.g. one or both of an augmented reality (AR) environment, virtual reality (VR) environment, and/or other interactive spaces, where step 202, column 13, lines 11-24 notes obtaining visual circuit control signals to cause the presentation platforms/devices to present images at a first point in time (performed via visual control component 108), step 204, column 13, lines 25-34 notes obtaining haptic circuit control signals to cause the haptic feedback device to provide haptic output at a second point in time (performed via haptic control component 110), step 206, column 13, lines 35-44 notes determining timing information conveying one or both of latency in the execution of the visual circuit control signals by the presentation platforms/devices and/or latency in the execution of the haptic circuit control signals by the haptic feedback device (performed via latency component 112), and step 208, column 13, lines 45-55 notes based on the timing information, transmitting the visual circuit control signals to the presentation platforms/devices and/or haptic circuit control signals to the haptic feedback device such that the first point in time and the second point in time may occur within a threshold time span (performed via synchronization component 114)).
As noted above, Nocon discloses its one or more input devices 122 may include a handheld device, and further discloses individual one or more haptic feedback devices 124 may be included in individual input devices. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention to modify the one or more input devices to incorporate one or more haptic feedback devices, thus considered the claimed “one or more hand-held components,” yielding predictable results, without changing the scope of the invention.
As to claim 2, Nocon discloses the simulation platform is configured to: communicate and exchange data with the one or more hand-held components via a wired connection (e.g. one or more computing platforms 102 may communicate with one or more input devices 122 via wired connections); and communicate and exchange data with the wearable display via a wireless connection (e.g. one or more computing platforms 102 may communicate with one or more presentation platforms/devices 120 via wired connections)(column 3, lines 18-29 and lines 61-67 notes one or more computing platforms 102, one or more presentation platforms/devices 120, one or more input devices 122, one or more haptic feedback devices 124, and/or other components of system 100 may be operatively linked via one or more electronic communication links, e.g. one or more networks 118, which may include wired and/or wireless connections, where column 4, lines 8-18 notes example where one or more computing platforms 102 and presentation platforms/devices 120 may be in direct wired communication and the one or more input devices 122 and/or one or more haptic feedback devices 124 may communicate wirelessly, but notes this as a non-limiting example, thus denoting communications may be vice versa, e.g. presentation platforms/devices 120 may communicate wirelessly and one or more input devices 122 and/or one or more haptic feedback devices 124 may be in direct wired communication).
As to claim 3, Nocon discloses the simulated environment comprises one or more simulated objects (column 2, lines 40-54 notes interactive space may include one or both of an augmented reality (AR) environment, virtual reality (VR) environment, and/or other interactive spaces, where a virtual reality environment may include views of images forming virtual content within a virtual environment, the virtual content may include one or more of virtual objects, virtual worlds, and/or other virtual content).
As to claim 4, Nocon discloses the simulation platform (e.g. one or more computing platforms 102) is configured to monitor actions of the user (e.g. monitor actions of user 301) to identify user interaction with one or more simulated objects within the simulated environment (e.g. to identify user interaction with one or more simulated objects, e.g. virtual objects within the virtual environment) based on physical user interaction with the one or more hand-held components (e.g. based on user interaction with the one or more input devices 122)(column 5, lines 20-35 notes one or more input devices 112 as a handheld input device configured to receive physical input and/or gesture-based input, which may include one or more physical input mechanisms, one or more virtual input mechanisms, and/or other elements, and may be in the form of a controller, e.g. a remote or a weapon, where column 6, lines 1-16 notes example of the handheld input device as a weapon, where response to depression of a button (e.g. a trigger) of the weapon to simulate a discharge and/or recoil of the weapon, thus considered to “monitor actions” of the user via the one or more input devices to further identify user interaction within the interactive space such that the interaction may be simulated as a haptic output and/or visual display).
As to claim 5, Nocon discloses the simulation platform (e.g. one or more computing platforms 102) comprises a hardware processor (e.g. processor(s) 104) coupled to non-transitory, computer-readable memory (e.g. electronic storage 116) encoded with a computer program (e.g. software algorithms as machine-readable instructions 106 comprising components 108-114) that causes the processor (e.g. causes processor(s) 104, where column 11, lines 51 thru column 12, lines 25 notes electronic storage 116 may include electronic storage media that electronically stores information, including software algorithms, information determined by processor(s) 122, and/or other information that enables system 100 to function as described, where processor(s) 104 may be configured to execute components 108-114) to determine a type and a degree of haptic feedback (e.g. determine a type and degree of haptic output) to be provided to the user via the one or more hand-held components (e.g. to be provided to the user via one or more input devices 122 including one or more haptic feedback devices 124) based on the user's physical interaction with the one or more hand-held components and the type of one or more simulated objects interacted with (e.g. based on the user’s interaction with the one or more input devices 122 and type of simulated objects, e.g. virtual content within the virtual environment)(e.g. as noted in claim 4, column 5, lines 20-35 notes the handheld input device configured to receive physical input and/or gesture-based input, and may include one or more physical input mechanisms, one or more virtual mechanisms, and/or other elements, e.g. motion sensors configured to generate output signals conveying gesture-based inputs, column 5, lines 52 thru column 6, lines 16 notes individual haptic feedback devices 124 may be configured to provide haptic output, e.g. in response to input, the input causing haptic output may include one or more of particular virtual content present by presentation platforms/devices 120, particular input provided via individual ones of one or more input devices 122, a combination of virtual content presented by presentation platforms/devices 120 and input via one or more input devices 122, and/or other input, for example, if the input device is a weapon, the weapon may include a haptic feedback device, which may be configured to provide haptic output in response to depression of a button (or trigger) of the weapon to simulate a discharge and/or recoil the weapon (e.g. type and degree of haptic feedback), the haptic output may be provided in response to seemingly simulations display of virtual content on presentation platforms/devices 120 (e.g. showing the simulation of the discharge and/or recoil), where the “type” of haptic output may correlate with the type of input device, further based on the interaction with the input device, and the “degree” may further be determined based on the usage of the input device, e.g. discharged and/or recoiled as a weapon).
As to claim 6, Nocon discloses the processor (e.g. processor(s) 104 of one or more computing platforms 102) is configured to transmit one or more signals (e.g. transmit one or more haptic circuit control signals, via haptic control component 110), via the wired connection (e.g. via wired connection, e.g. as noted in claim 1), comprising haptic feedback simulation data to the one or more hand-held components (e.g. haptic output (feedback) to the one or more input devices 122 including the one or more haptic feedback devices 124) to thereby cause the one or more hand-held components to provide the determined haptic feedback to the user (e.g. to cause the one or more input devices 122 to provide the determined haptic output (feedback) via the one or more haptic feedback devices 124)(e.g. as noted in claim 1, step 204, obtaining haptic circuit control signals performed via haptic control component 110 (e.g. executed by processor(s) 104), where column 6, lines 41-57 further notes haptic control component 110 may be configured to obtain haptic circuit control signals and/or other information, the haptic circuit control signals dictate control of one or more haptic feedback devices 124 to provide haptic output, where execution of the haptic circuit control signals by one or more haptic feedback devices 124 may cause one or more haptic feedback devices 124 to provide haptic output).
As to claim 7, Nocon discloses a measured latency of transmission of the haptic feedback simulation data to the one or more hand-held components is 1 millisecond or less (column 6, lines 58-67 notes latency component 112 (e.g. executed by processor(s) 104) may be configured to determine timing information and/or other information, which may convey one or more of latency in the execution of visual circuit control signals by presentation platforms/devices 120, latency in the execution of haptic circuit control signals by one or more haptic feedback devices 124, and/or relative latency between the execution of haptic circuit control signals by one or more haptic feedback devices 124 and the execution of visual circuit control signals by presentation platforms/devices 120, where column 8, lines 33-51 notes the transmission of the visual circuit control signals to presentation platforms/devices 120 and/or the haptic circuit control signals to one or more haptic feedback devices 124 may be timed (e.g. one or both may be delayed) so that points in time when one or more haptic feedback devices 124 provide haptic output and points in time when presentation platforms/devices 120 presents images occur within a threshold time span, e.g. for example, within a range of zero to 100 milliseconds, zero to fifty milliseconds, zero to twenty-five milliseconds, zero to ten milliseconds, which encompasses latency of one millisecond or less).
As to claim 8, Nocon discloses haptic feedback provided to the user via the one or more hand-held components (e.g. transmission of haptic circuit control signals to one or more input devices 122 including one or more haptic feedback devices 124 to provide haptic output (e.g. feedback)) and associated visual feedback provided to the user via the wearable display (e.g. transmission of visual circuit control signals to presentation platforms/devices 120 to provide visual content (e.g. presentation images)) have a latency of 1 millisecond or less (column 8, lines 8-32 notes synchronization component (e.g. executed by one or more processor(s) 104) may be configured to instruct one or both of visual control component 108 and/or haptic control component 110 to effectuate transmission of visual circuit control signals and/or haptic circuit control signals, respectively, where as noted in claim 7, column 8, lines 33-51 notes the transmission of the visual circuit control signals to presentation platforms/devices 120 and/or the haptic circuit control signals to one or more haptic feedback devices 124 may be timed (e.g. one or both may be delayed) so that points in time when one or more haptic feedback devices 124 provide haptic output and points in time when presentation platforms/devices 120 presents images occur within a threshold time span, e.g. ranges noted encompasses latency of one millisecond or less).
As to claim 9, Nocon discloses the simulation platform (e.g. one or more computing platforms 102) comprises a hardware processor (e.g. processor(s) 104) coupled to non-transitory, computer-readable memory (e.g. electronic storage 116) encoded with a computer program (e.g. software algorithms as machine-readable instructions 106 comprising components 108-114) that causes the processor (e.g. causes processor(s) 104, where column 11, lines 51 thru column 12, lines 25 notes electronic storage 116 may include electronic storage media that electronically stores information, including software algorithms, information determined by processor(s) 122, and/or other information that enables system 100 to function as described, where processor(s) 104 may be configured to execute components 108-114) to adjust output of digital content to be visually presented to the user via the wearable display (e.g. adjust the output of visual content (e.g. presentation images) to the user via the presentation platforms/devices 120, e.g. as a headset, e.g. visual circuit control signals) based, at least in part, on the user's physical interaction with the one or more hand-held components (e.g. based on the user’s interaction with the one or more input devices 122, e.g. haptic circuit control signals)(e.g. as noted in claim 1, after obtaining visual circuit control signals at step 202 and haptic circuit control signals at step 204, at step 206 determining timing information conveying one or both of latency in the execution of the visual circuit control signals by the presentation platforms/devices and/or latency in the execution of the haptic circuit control signals by the haptic feedback device, and step 208, based on the timing information, transmission of the virtual circuit control signals to the presentation platforms/devices and/or the haptic circuit control signals to the haptic feedback device may be effectuated, where transmission may be configured so that the first point in time and the second point in time may occur within a threshold time span, e.g. as noted in claims 7 and 8).
As to claim 10, Nocon discloses the processor (e.g. one or more processor(s) 104) is configured to transmit one or more signals (e.g. transmit one or more visual circuit control signals, e.g. via visual control component 108), via the wireless connection (e.g. via wireless connection, as noted in claim 1), comprising visual simulation data to the wearable display to be rendered and visually presented to the user (e.g. visual content (e.g. as presentation images) to the presentation platforms/devices 120 to be presented to the user, e.g. user 301)(e.g. as noted in claim 1, step 202, obtaining visual circuit control signals performed via visual control component 108 (e.g. executed by processor(s) 104), where column 6, lines 23-40 further notes visual control component 108 may be configured to obtain visual circuit control signals and/or other information, the visual circuit control signals dictates control of presentation platforms/devices 120 to present images forming virtual content and/or to present other content, where execution of visual circuit control signals by presentation platforms/devices 120 may cause the presentation platforms/devices 120 to present content (e.g. via a display) for viewing by a user).
As to claim 11, Nocon discloses the rendered visual digital content presented to the user via the wearable display mimics movement of the one or more simulated objects interacted with and based on the user's physical interaction with the one or more hand-held objects (e.g. as noted in claims 1, 4, and 10, synchronizing visual effects and haptic feedback for interactive experiences, e.g. in an interactive space, e.g. an augmented reality or virtual reality environment, e.g. the VR environment may include views of images forming virtual content within a virtual environment, virtual content may include one or more virtual objects, virtual worlds, and/or other virtual content, e.g. column 6, lines 1-16 notes example of handheld input device as a weapon, where the haptic feedback may be configured to provide haptic output in response to depression of a button (or trigger) of the weapon to simulate a discharge and/or recoil the weapon (e.g. type and degree of haptic feedback), the haptic output may be provided in response to seemingly simulations display of virtual content on presentation platforms/devices 120 (e.g. showing the simulation of the discharge and/or recoil), thus mimics movement and display of the weapon being discharged and/or recoiled).
As to claim 13, Nocon discloses the visual simulation data is transmitted to the wearable display via a wireless communication protocol (e.g. as noted in claims 1 and 2, the one or more computing platforms 102 may communicate with the one or more presentation platforms/devices 120 via wired or wireless).
As to claim 16, Nocon discloses the one or more hand-held components comprise kinesthetic haptic hardware (e.g. as noted in claim 1, the one or more input devices 122 may include the one or more haptic feedback devices 124, where column 5, lines 64-67 further notes an individual haptic feedback device 124 may comprise one or more of a vibration motor, a heating element, a fan or blower, and/or other device configured to provide haptic output, where a vibration motor may be considered kinesthetic haptic hardware).
As to claim 17, Nocon discloses the wearable display comprises an XR headset (e.g. as noted in claim 1, the presentation platforms/devices 120 may include a headset, e.g. an AR or VR headset).
Claim(s) 12 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nocon (US 10,444,840) as applied to claims 11 and 13 above, and further in view of Kamat (US 2020/0153602).
As to claim 12, Nocon discloses the simulation platform further comprises a simulation engine configured to model the interactions (e.g. one or more computing platforms 102 further comprising various components including visual control component 108), but does not disclose, but Kamat discloses the simulation platform further comprises a simulation engine configured to model the interactions using calculations associated with one or more algorithms comprising at least one of a position-based solver, a collision detection algorithm, a deformable mesh algorithm, and an adaptive mesh algorithm ([0017] notes during launch of a VR application, virtual physics objects in a virtual human body matching actual haptic actuator positions are created and assigned, where during application execution, haptics data is continuously generated for various physics objects in response to user interactions, treating the haptics actuators as in-application physics objects simplifies collision detection algorithms (denoting use), additionally, treating the actuators as in-application objects improves the ability of the system to generate haptics data in real-time and synchronize the haptics data with the rendered video frame, where [0034] further notes a game engine in the application may track object collisions, and may track the collision location, the collision location information may be stored in a data packet and provided to a video frame renderer, this collision information may be passed to the haptics multiplexer and formatter, which process haptics data for different actuators, packetize, and format the haptics data, and output the haptics frames in audio buffers that are synchronized with the video frame data).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify Nocon’s computing platform, e.g. simulation platform, further comprising a visual control component, e.g. simulation engine, to further incorporate Kamat’s game engine for modeling interactions using calculations associated with collision detection to track object collisions and collision locations so that this information may be further used in generating haptics data for different actuators to synchronize haptics data with rendered video frames ([0017], [0034] of Kamat).
As to claim 15, Nocon discloses the visual simulation data comprises required data for visual simulation, wherein the required data for visual simulation is transmitted to the wearable display (e.g. as noted in claim 10, the visual circuit control signals dictates control of presentation platforms/devices 120 to present images forming virtual content and/or to present other content, where execution of visual circuit control signals by presentation platforms/devices 120 may cause the presentation platforms/devices 120 to present content (e.g. via a display, e.g. as a headset) for viewing by a user), but does not disclose, but Kamat discloses wherein the required data for visual simulation is transmitted to the wearable display in each frame of the data transmission ([0027] notes the video data and audio data are combined, e.g. each haptic control event may be synchronized with and packetized within an audio frame, which provides the synchronization of each haptic even with the video data and audio data using underlying protocols, then the receiving device, e.g. an HMD, extracts the haptics data from the audio frame and uses it to drive haptic actuators, [0036] notes HMD 400 includes a receiver chip 402 that separates video data and audio data, the video data is used to drive one or more displays 404 on the HMD 400 (e.g. left and right display panels), the audio data is processed to determine which audio data frames are haptics data frames and which are audio data frames, then audio data frames are used to drive one or more resident speakers 406 on the HMD 400 or one or more external speakers, [0037] notes haptics data frames are passed through the appropriate bus as identified by the computing device, e.g. to drive different haptic actuators, where [0017], [0034] notes video data as rendered video frames).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify Nocon’s visual simulation transmitted to the wearable display to be in each frame of the data transmission as described in Kamat as processing/rendering visual content frame-by-frame is well known in the art to maximize control, accuracy, and quality.
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nocon (US 10,444,840) as applied to claim 13 above, and further in view of Agrawal et al. (US 11,954,258).
As to claim 14, Nocon discloses the wireless communication (e.g. as noted in claim 13), but does not disclose, but Agrawal et al. disclose the wireless communication protocol is a User Datagram Protocol (UDP) connection (Figure 1, column 4, lines 22-34 notes network environment 100 comprising system 102, VR device 104, a haptic device 106, and a server 108, the system 102 may communicate with the VR device 104, haptic device 106, and server 108 via communication network 110, where column 6, lines 12-36 further notes communication network 110 may include a wireless communication network, e.g. User Datagram Protocol (UDP)).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify Nocon’s wireless communication to include User Datagram Protocol (UDP) as described in Agrawal et al. to provide an additional, alternative means of wireless communications, thus expanding the capability and/or functionality of the system (column 6, lines 12-36 of Agrawal et al.).
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nocon (US 10,444,840) as applied to claim 1 above, and further in view of Chowdhary et al. (US 2017/0201808).
As to claim 18, Nocon discloses wearable displays (e.g. as noted in claim 1), but does not disclose, but Chowdhary et al. disclose the system is operable to run on wearable displays having different operating systems ([0073], [0080] notes devices, e.g. Head Mounted Displays (HMDs), Google-Glass, wearable devices, etc., that have a display and are portable running any number of different operating systems, e.g. MS Windows, Apple iOS, Linux, Ubuntu, etc.).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify Nocon’s wearable displays to have different operating systems as described by Chowdhary et al. to further expand the capability and/or functionality of the system to execute on different platforms and/or on different applications ([0073], [0080] of Chowdhury et al.).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JACINTA M CRAWFORD whose telephone number is (571)270-1539. The examiner can normally be reached 8:30a.m. to 4:30p.m.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Y. Poon can be reached at (571)272-7440. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACINTA M CRAWFORD/Primary Examiner, Art Unit 2617
Prosecution Insights