DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 3-6, 13-16 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 3 recites “synthetic hand-held controller input” in lines 1-2. The limitation was already recited in line 2 of claim 2. It is unclear to the Examiner whether the limitation in lines 1-2 of claim 3 is the same or different from the limitation in line 2 of claim 2.
Claim 4 depends on at least claim 3. Therefore, claim 4 is rejected for at least the same reason as claim 3.
Claim 5 recites “synthetic head-mounted display input” in lines 1-2. The limitation was already recited in line 3 of claim 2. It is unclear to the Examiner whether the limitation in lines 1-2 of claim 5 is the same or different from the limitation in line 3 of claim 2.
Claim 6 recites “synthetic input configured for a virtual object” in lines 1-2. The limitation was already recited in line 3 of claim 2. It is unclear to the Examiner whether the limitation in lines 1-2 of claim 5 is the same or different from the limitation in line 3 of claim 2.
Claim 13 recites similar limitations as claim 3. Therefore, the same correction as claim 3 is required.
Claim 14 depends on at least claim 13. Therefore, claim 14 is rejected for at least the same reason as claim 13.
Claim 15 recites similar limitations as claim 5. Therefore, the same correction as claim 5 is required.
Claim 16 recites similar limitations as claim 6. Therefore, the same correction as claim 6 is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. While claim 20 recites a “computer-readable storage medium”, the Applicant's disclosure in paragraph 0029 defines a memory as non-transitory, however, it fails to define the claimed computer-readable storage medium. The United States Patent and Trademark Office (USPTO) is obliged to give the claims their broadest reasonable interpretation consistent with the specifications during proceedings before the USPTO (see In re Zletz, 893 F.2d 319 Fed. Cir. 1989). The broadest reasonable interpretation of a claim drawn to a computer readable medium typically covers forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media, particularly when the specification is silent (see MPEP 2111.01). Thus, the definition of Applicants computer readable medium in the disclosure fails to limit the claim to only non-transitory tangible media, and therefore is non-statutory (see 1351 Off. Gaz. Pat. Office 212 (February 23, 2010)). Applicant is suggested to add “non-transitory” in front of “computer-readable storage medium” in order to overcome the 35 U.S.C. 101 rejection.
To expedite a complete examination of the instant application, the claims rejected under 35 U.S.C. 101 as non-statutory subject matter are further rejected as set forth below in anticipation of applicant amending the claims to place them within the four categories of invention.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-5, 7-9, 11-15, 17-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Urbach (U.S. Patent Application 20170054793) in view of Microsoft (Mixed Reality Documentation).
In regards to claim 1, Urbach teaches a method for native system execution of an application using synthetic input from an external device [e.g. The virtual application execution engine allows for virtual execution of applications by transparently launching and capturing the rendered output of natively executed applications, and delivering the output to other processes on the same platform or to remote devices in a platform agnostic manner (such as through an HTTP stream). An input device (e.g., a joystick, keyboard or other I/O device) expected by the process (executed on the hosting node) that does not physically exist on the remote client can be nevertheless simulated, 0015, 0032], the method comprising:
receiving, at the system over a real-time communication channel established between the system and the external device, synthetic input for an application executing at the system [e.g. User interaction (such as Javascript events, image map events, and changes to text fields) at the remote client are transmitted back to the virtual application execution engine on the hosting node (in one implementation, through HTTP streams), and translated into local events and written to the event loop(s) of the shared process(es), 0032];
executing, at the system, the application in response to the synthetic input [e.g. An input device (e.g., a joystick, keyboard or other I/O device) expected by the process (executed on the hosting node) that does not physically exist on the remote client can be nevertheless simulated. For example, the virtual application execution engine can convert image map inputs from the remote client in a response HTTPstream to equivalent joystick or keyboard input and write or transmit the converted inputs to the process executed on the hosting node, 0032], wherein the executing includes:
streaming, from the system to the external device, the generated visual information, wherein the external device displays the streamed visual information [e.g. delivering the output to other processes on the same platform or to remote devices in a platform agnostic manner (such as through an HTTP stream), 0015].
Urbach does not explicitly teach
an artificial reality (XR) system (emphasis added);
an XR application (emphasis added);
overriding native XR system input for the XR application using the synthetic input; and
generating visual information for the XR system based on the synthetic input; and
streaming, from the XR system to the external device, the generated visual information, wherein the external device displays the streamed visual information.
However, Microsoft teaches
an artificial reality (XR) system [e.g. Mixed Reality device, see Microsoft’s “Perception Simulation” webpage];
an XR application [e.g. application running on the Mixed Reality device, see Microsoft’s “Perception Simulation” webpage];
overriding native XR system input for the XR application using the synthetic input [e.g. bypasses the live sensors on a Mixed Reality device and sends simulated input to applications running on the device, see Microsoft’s “Perception Simulation” webpage]; and
generating visual information for the XR system based on the synthetic input [e.g. The position and orientation of the holographic camera is implicitly controlled by the user's movement. The user's movement is relayed to the application on a frame-by-frame basis via a view transform. The system calculates the view and the projection transforms to use for that frame. Your application must use these transforms to produce correct results. The user’s movement measured by the live sensors are replaced by the simulated inputs, see “Microsoft’s Holographic rendering” and “Perception Simulation” webpages].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified Urbach’s method with the features of
an artificial reality (XR) system;
an XR application;
overriding native XR system input for the XR application using the synthetic input; and
generating visual information for the XR system based on the synthetic input; and
streaming, from the XR system to the external device, the generated visual information, wherein the external device displays the streamed visual information
in the same conventional manner as taught by Microsoft because Microsoft provides a method for simulating tests of an application for a mixed reality device without having to test it using physical inputs [see “Perception Simulation” webpage].
In regards to claim 2, Urbach teaches the method of claim 1, wherein the synthetic input comprises one or more of: synthetic hand-held controller input that simulates system hand-held controller input [e.g. simulated joystick or keyboard, 0032].
Urbach does not explicitly teach XR system hand-held controller input (emphasis added), synthetic head-mounted display input that simulates XR system head-mounted display input, synthetic user gestures and/or user tracking input, or any combination thereof.
However, Microsoft teaches XR system hand-held controller input [e.g. enable and manipulate motion controllers of the mixed reality device, see Perception Simulation webpage], synthetic head-mounted display input that simulates XR system head-mounted display input, synthetic user gestures and/or user tracking input, or any combination thereof [e.g. control properties of a simulated "human", including head position, hand position, and gestures, see “Perception Simulation” webpage].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified Urbach’s method with the features of XR system hand-held controller input (emphasis added), synthetic head-mounted display input that simulates XR system head-mounted display input, synthetic user gestures and/or user tracking input, or any combination thereof in the same conventional manner as taught by Microsoft because Microsoft provides a method for simulating tests of an application for a mixed reality device without having to test it using physical inputs [see “Perception Simulation” webpage].
In regards to claim 3, Urbach does not explicitly teach the method of claim 2, wherein the synthetic input comprises synthetic hand-held controller input, and wherein the synthetic hand-held controller input comprises simulated button press input or simulated movement input.
However, Microsoft teaches the method of claim 2, wherein the synthetic input comprises synthetic hand-held controller input [see rejection of claim 2 above], and wherein the synthetic hand-held controller input comprises simulated button press input or simulated movement input [e.g. Simulates the action of pressing the forefinger to the thumb or pulling the action button on a controller, see “Advanced HoloLens Emulator and Mixed Reality Simulator input” webpage].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified Urbach’s method with the features of the synthetic input comprises synthetic hand-held controller input, and wherein the synthetic hand-held controller input comprises simulated button press input or simulated movement input in the same conventional manner as taught by Microsoft because button press inputs or movement inputs are well known and commonly used in the art of mixed reality systems.
In regards to claim 4, Urbach does not explicitly teach the method of claim 3 wherein the simulated button press input simulates one or more button presses from a hand-held controller of the XR system, and the simulated movement input simulates movement of the hand-held controller of the XR system, the simulated movement comprising 3DoF movement or 6DoF movement.
However, Microsoft teaches the method of claim 3 wherein the simulated button press input simulates one or more button presses from a hand-held controller of the XR system [e.g. simulate up to two hand-held tracked motion controllers, where each simulated controller has home button, menu button, grip button, see “Using the Windows Mixed Reality simulator” webpage], and the simulated movement input simulates movement of the hand-held controller of the XR system [e.g. Each simulated controller has position and orientation in space. The Mixed Reality simulator can simulate the controller movement, see “Using the Windows Mixed Reality simulator” webpage], the simulated movement comprising 6DoF movement [e.g. simulated 6-DOF controller, see “Using the Windows Mixed Reality simulator” webpage].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified Urbach’s method with the features of wherein the simulated button press input simulates one or more button presses from a hand-held controller of the XR system, and the simulated movement input simulates movement of the hand-held controller of the XR system, the simulated movement comprising 6DoF movement in the same conventional manner as taught by Microsoft because button press inputs or movement inputs are well known and commonly used in the art of mixed reality systems.
In regards to claim 5, Urbach does not explicitly teach the method of claim 2, wherein the synthetic input comprises synthetic head-mounted display input, and wherein the synthetic head-mounted display input comprises simulated movement of a head-mounted display of the XR system, the simulated movement comprising 3DoF movement or 6DoF movement.
However, Microsoft teaches the method of claim 2, wherein the synthetic input comprises synthetic head-mounted display input [see rejection of claim 2 above], and wherein the synthetic head-mounted display input comprises simulated movement of a head-mounted display of the XR system [e.g. you can simulate the input of a human looking to a specific, repeatable position, see “Perception Simulation” webpage], the simulated movement comprising 3DoF movement [e.g. motion is controlled with both rotation and translation (movement) along three axes, see “Advanced HoloLens Emulator and Mixed Reality Simulator input” webpage].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified Urbach’s method with the features of wherein the synthetic input comprises synthetic head-mounted display input, and wherein the synthetic head-mounted display input comprises simulated movement of a head-mounted display of the XR system, the simulated movement comprising 3DoF movement in the same conventional manner as taught by Microsoft because movement inputs in a head-mounted display are well known and commonly used in the art of mixed reality systems.
In regards, claim 7, Urbach does not explicitly teach the method of claim 1, wherein the receiving of the synthetic input at the XR system, the executing the XR application, and the streaming of the generated visual information from the XR system to the external device occurs in real-time.
However, Microsoft teaches the method of claim 1, wherein the receiving of the synthetic input at the XR system [e.g. To adjust the speed that the simulated human or input devices will move or rotate in response to keyboard, mouse or gamepad input, select the gear icon next to Input settings, and adjust the sliders, see “Using the HoloLens Emulator” webpage], the executing the XR application [e.g. Your actions move the simulated user and cause interactions with apps that respond as they would on an immersive headset, see “Using the Windows Mixed Reality simulator” webpage], and the streaming of the generated visual information from the XR system to the external device occurs in real-time [e.g. The Perception Simulation Control user interface can display a live view of HoloLens 2 device or Emulator and allow direct control using keyboard, mouse, or gamepad, see “Perception Simulation” webpage].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified Urbach’s method with the features of wherein the receiving of the synthetic input at the XR system, the executing the XR application, and the streaming of the generated visual information from the XR system to the external device occurs in real-time in the same conventional manner as taught by Microsoft because receiving, executing, and streaming on the XR system in real-time is well known and commonly used in the art of mixed reality systems.
In regards to claim 8, Urbach teaches the method of claim 1, wherein a user via interactions with a user interface displayed by the external device and/or a display associated with the external device [e.g. The user on the remote client may interact with the output displayed at the remote client. These interactions, which also can be mouse clicks, keyboard strokes, and the like, are transmitted in messages, 0035].
Urbach does not explicitly teach wherein the synthetic input is received at the external device.
However, Microsoft teaches wherein the synthetic input is received at the external device [e.g. The Simulation control panel lets you view the current position and orientation of the simulated human and input devices. It also allows you to configure both simulated input and devices used for controlling simulated input, such as your PC's keyboard, mouse and gamepad, see “Using the HoloLens Emulator” webpage].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified Urbach’s method with the features of wherein the synthetic input is received at the external device in the same conventional manner as taught by Microsoft because Microsoft provides a method for simulating tests of an application for a mixed reality device without having to test it using physical inputs [see “Perception Simulation” webpage].
In regards to claim 9, Urbach does not explicitly teach the method of claim 1, wherein the XR system is at a location remote from the external device and the user of the external device.
However, Microsoft teaches the method of claim 1, wherein the XR system is at a location remote [e.g. For a remote device, such as HoloLens, enter the IP address of the HoloLens device or emulator (emphasis added), see “Perception simulation (Preview)” webpage] from the external device and the user of the external device [e.g. you can control the emulator remotely using the Perception Simulation Control tool included in the emulator installation or with the Perception Simulation APIs by connecting to the host PC's IP address and Device Portal external port, see “Using the HoloLens Emulator” webpage].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified Urbach’s method with the features of wherein the XR system is at a location remote from the external device and the user of the external device in the same conventional manner as taught by Microsoft because Microsoft provides a method for simulating tests of an application for a mixed reality device without having to test it using physical inputs [see “Perception Simulation” webpage].
In regards to claim 11, the claim recites similar limitations as claim 11, but in the form of an artificial reality (XR) system comprising: one or more processors; and one or more memories storing instructions that, when executed by the one or more processors, cause the XR system to perform the method of claim 1. Furthermore, Urbach teaches a system [e.g. system, 0005] comprising: one or more processors [e.g. processor, 0041]; and one or more memories [e.g. storage device, 0041] storing instructions [e.g. instructions, 0041] that, when executed by the one or more processors, cause the system to perform the method of claim 1. Urbach does not explicitly teach an artificial reality (XR) system (emphasis added). However, Microsoft teaches an artificial reality (XR) system [see rejection of claim 1]. Therefore, the same rationale as claim 11 is applied.
In regards to claim 12, the claim recites similar limitations as claim 2. Therefore, the same rationale as claim 2 is applied.
In regards to claim 13, the claim recites similar limitations as claim 3. Therefore, the same rationale as claim 3 is applied.
In regards to claim 14, the claim recites similar limitations as claim 4. Therefore, the same rationale as claim 4 is applied.
In regards to claim 15, the claim recites similar limitations as claim 5. Therefore, the same rationale as claim 5 is applied.
In regards to claim 17, the claim recites similar limitations as claim 7. Therefore, the same rationale as claim 7 is applied.
In regards to claim 18, the claim recites similar limitations as claim 8. Therefore, the same rationale as claim 8 is applied.
In regards to claim 19, the claim recites similar limitations as claim 9. Therefore, the same rationale as claim 9 is applied.
In regards to claim 20, the claim recites similar limitations as claim 11, but in the form of a computer-readable storage medium storing the method of claim 1. Furthermore, Urbach teaches a computer-readable storage medium [e.g. storage device, 0041] storing the method of claim 1. Therefore, the same rationale as claim 1 is applied.
Allowable Subject Matter
Claims 6, 16 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Claim 10 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
In regards to claim 6, the prior art of record fails to teach or suggest the method of claim 2, wherein the synthetic input comprises synthetic input configured for a virtual object, and wherein the synthetic input configured for a virtual object comprises location coordinates with respect to the virtual object.
In regards to claim 10, the prior art of record fails to teach or suggest the method of claim 1, wherein,
the XR application executes in a sandboxed environment at the XR system,
the external device is connected, via the real-time communication channel, to the sandboxed environment of the XR system, and
the sandboxed environment at the XR system comprises resource restrictions and/or access restrictions.
In regards to claim 16, the claim recites similar limitations as claim 6. Therefore, claim 16 is allowable for at least the same reason as claim 6 if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW SHIN whose telephone number is (571)270-5764. The examiner can normally be reached Monday - Friday from 11:00AM to 7:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at 571-272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW SHIN/Examiner, Art Unit 2612
/Said Broome/Supervisory Patent Examiner, Art Unit 2612
Prosecution Insights