Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-6, 9-17, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Kai Riege, T. Holtkamper, G. Wesche and B. Frohlich, ("The Bent Pick Ray: An Extended Pointing Technique for Multi-User Interaction," 3D User Interfaces (3DUI'06), Alexandria, VA, USA, 2006, pp. 62-65, doi: 10.1109/VR.2006.127.) in view of Dean Ashford (Youtube video, UE4 - Tutorial - VR Widget Interactions!, https://www.youtube.com/watch?v=jC91FFkxgh4 time 10:30 to 14:16, Dec 5, 2018).
As to claim 1, Kai discloses a system comprising:
a hardware computer processor; and a non-transitory computer readable medium having software instructions stored thereon, the software instructions executable by the hardware computer processor to cause the system to perform operations comprising (Page 1, “The 3D images are generated on a two-headed PC with a video refresh rate of 108 Hz and with 1280x1024 resolution and are synchronously output by the DLP projectors.”):
accessing mixed reality environment data, including information regarding a virtual environment and real-world environments of two or more of a plurality of users (Figure 1: Two users collaboratively dragging an object.),
wherein the mixed reality environment data is at least partially visible to each of the plurality of users through a field of view of wearable headsets, the virtual environment including a plurality of virtual objects that are user selectable (Figure 1, Page 1, “Circular polarization filters are used to separate the users’ views; stereo shutter glasses are used for eye separation. To combine both techniques we use BARCO’s modified StereoGraphics ChrystalEyesTM shutter glasses, which have circular polarization filters laminated on the top of the LCD shutter.” Figure 1: Two users collaboratively dragging an object.);
determining that the two or more of the plurality of users are providing inputs in the virtual environment associated with respective rays indicating targeting of the users (Figure 1: Two users collaboratively dragging an object. One ray is colored green and one ray is colored red. Page 2, “in our case a colored straight line is used. As soon as the second user is grabbing an object already picked by the first user, the pick rays of the two users are deformed so that they still emanate from the input device, tangentially to the pointing direction, and touch the selected object at the point of intersection; in other words they have to be bent.”); and
assigning different visual effects to rays of each of the two or more of the plurality of users so that the rays are distinguishable by users viewing the virtual environment; and wherein the system is further caused to perform operations including, for a first user of the plurality of users (Figure 1: Two users collaboratively dragging an object. One ray is colored green and one ray is colored red.):
Kai does not disclose determining a headpose of the first user;
implementing a first input mode with respect to a first virtual object of the plurality of virtual objects upon determining that the headpose of the first user is targeting the first virtual object and contemporaneously upon determining that a first virtual ray of the respective rays from the first user is targeting the first virtual object;
implementing a second input mode, different from the first input mode, with respect to a first virtual object of the plurality of virtual objects upon determining that the headpose of the first user is targeting the first virtual object and contemporaneously upon determining that a first virtual ray of the respective rays from the first user is not targeting the first virtual object; and
implementing a third input mode, different from the first input mode and the second input mode, with respect to the first virtual object of the plurality of virtual objects upon determining that the headpose of the first user is not targeting the first virtual object and contemporaneously upon determining that a first virtual ray of the respective rays from the first user is targeting the first virtual object.
Ashford teaches determining a headpose of the first user (Ashford, 05:14,
PNG
media_image1.png
1312
1779
media_image1.png
Greyscale
Camera is based on what the face is.);
implementing a first input mode with respect to a first virtual object of the plurality of virtual objects upon determining that the headpose of the first user is targeting the first virtual object and contemporaneously upon determining that a first virtual ray of the respective rays from the first user is targeting the first virtual object (Ashford, 06:12-06:27
PNG
media_image2.png
1330
1818
media_image2.png
Greyscale
;
implementing a second input mode, different from the first input mode, with respect to a first virtual object of the plurality of virtual objects upon determining that the headpose of the first user is targeting the first virtual object and contemporaneously upon determining that a first virtual ray of the respective rays from the first user is not targeting the first virtual object (Ashford, 06:13
PNG
media_image3.png
1312
1817
media_image3.png
Greyscale
06:31, the camera faces the target, but the hand laser beams to sky.
PNG
media_image4.png
1304
1828
media_image4.png
Greyscale
; and
implementing a third input mode, different from the first input mode and the second input mode, with respect to the first virtual object of the plurality of virtual objects upon determining that the headpose of the first user is not targeting the first virtual object and contemporaneously upon determining that a first virtual ray of the respective rays from the first user is targeting the first virtual object (10:43, head pose face the ground, laser beams to the object when a trigger is pulled.
PNG
media_image5.png
1313
1826
media_image5.png
Greyscale
).
Riege and Ashford are considered to be analogous art because all pertain to virtual reality. It would have been obvious before the effective filing date of the claimed invention to have modified Riege with the features of “determining a headpose of the first user; implementing a first input mode with respect to a first virtual object of the plurality of virtual objects upon determining that the headpose of the first user is targeting the first virtual object and contemporaneously upon determining that a first virtual ray of the respective rays from the first user is targeting the first virtual object; implementing a second input mode, different from the first input mode, with respect to a first virtual object of the plurality of virtual objects upon determining that the headpose of the first user is targeting the first virtual object and contemporaneously upon determining that a first virtual ray of the respective rays from the first user is not targeting the first virtual object; and implementing a third input mode, different from the first input mode and the second input mode, with respect to the first virtual object of the plurality of virtual objects upon determining that the headpose of the first user is not targeting the first virtual object and contemporaneously upon determining that a first virtual ray of the respective rays from the first user is targeting the first virtual object.” as taught by Ashford. All the claimed elements were known in the prior art and one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art at the time of the invention.
As to claim 2, claim 1 is incorporated and the combination of Riege and Ashford discloses the system is further caused to perform operations including assigning different visual effects to the first virtual object in the first input mode relative to the second input mode (Ashford, 06:12-06:27, first input mode
PNG
media_image2.png
1330
1818
media_image2.png
Greyscale
Ashford, 06:13, second input mode, the target object has different color.
PNG
media_image3.png
1312
1817
media_image3.png
Greyscale
)
As to claim 3, claim 2 is incorporated and the combination of Riege and Ashford discloses the system is further caused to perform operations including assigning different visual effects to the first virtual object in the second input mode relative to the third input mode (Ashford, 06:13, second input mode,
PNG
media_image3.png
1312
1817
media_image3.png
Greyscale
(10:43, head pose face the ground, laser beams to the object when a trigger is pulled, third input mode, the target object is not shown.
PNG
media_image5.png
1313
1826
media_image5.png
Greyscale
).
As to claim 4, claim 3 is incorporated and the combination of Riege and Ashford discloses the system is further caused to perform operations including assigning different visual effects to the first virtual object in the first input mode relative to the third input mode (Ashford, 10:43, head pose face the ground, laser beams to the object when a trigger is pulled, third input mode, the target object is not shown.)
As to claim 5, claim 1 is incorporated and the combination of Riege and Ashford discloses the system is further caused to perform operations including implementing a fourth input mode, different from the first input mode, the second input mode, and the third input mode, with respect to the first virtual object of the plurality of virtual objects upon determining that the headpose of the first user is not targeting the first virtual object and contemporaneously upon determining that a first virtual ray of the respective rays from the first user is not targeting the first virtual object (10:35, the head pose faces sky and the laser beams to sky.
PNG
media_image6.png
1310
1792
media_image6.png
Greyscale
).
As to claim 6, claim 5 is incorporated and the combination of Riege and Ashford discloses the system is further caused to perform operations including implementing different visual effects for first virtual object based on being in the first input mode, the second input mode, the third input mode, or the fourth input mode (Ashford, 06:12-06:27, first input mode. Ashford, 06:13, second input mode, the target object has different color. Ashford, 10:43, head pose face the ground, laser beams to the object when a trigger is pulled, third input mode, the target object is not shown.)
As to claim 9, claim 6 is incorporated and the combination of Riege and Ashford discloses the ray of the first user is different in the fourth input mode relative to the first, second, or third input mode (Ashford, (10:35, the head pose faces sky and the laser beams to sky, fourth input mode, the laser has no ending circle at the end.)
As to claim 10, claim 9 is incorporated and the combination of Riege and Ashford discloses the ray of the first user is shortened in the fourth input mode relative to the first, second, or third input mode (10:35, the head pose faces sky and the laser beams to sky. It seems shorter.
PNG
media_image6.png
1310
1792
media_image6.png
Greyscale
).
.
As to claim 11, the combination of Riege and Ashford discloses a computerized method, performed by a computing system having one or more hardware computer processors and one or more non-transitory computer readable storage device storing software instructions executable by the computing system to perform the computerized method comprising: accessing mixed reality environment data, including information regarding a virtual environment and real-world environments of a plurality of users, wherein the mixed reality environment data is at least partially visible to each of the plurality of users through a field of view of wearable headsets, the virtual environment including a plurality of virtual objects that are user selectable; determining a headpose of a first user of the plurality of users; implementing a first input mode with respect to a first virtual object of the plurality of virtual objects upon determining that the headpose of the first user is targeting the first virtual object and contemporaneously upon determining that a first virtual ray of the respective rays from the first user is targeting the first virtual object; implementing a second input mode, different from the first input mode, with respect to a first virtual object of the plurality of virtual objects upon determining that the headpose of the first user is targeting the first virtual object and contemporaneously upon determining that a first virtual ray of the respective rays from the first user is not targeting the first virtual object; and implementing a third input mode, different from the first input mode and the second input mode, with respect to the first virtual object of the plurality of virtual objects upon determining that the headpose of the first user is not targeting the first virtual object and contemporaneously upon determining that a first virtual ray of the respective rays from the first user is targeting the first virtual object (See claim 1 for detailed analysis.).
As to claim 12, claim 11 is incorporated and the combination of Riege and Ashford discloses determining that two or more of the plurality of users are providing inputs in the virtual environment associated with respective rays indicating targeting of the users; and assigning different visual effects to rays of each of the two or more of the plurality of users so that the rays are distinguishable by users viewing the virtual environment (Riege, Fig. 6, each ray has different color.).
As to claim 13, claim 11 is incorporated and the combination of Riege and Ashford discloses assigning different visual effects to the first virtual object in the first input mode relative to the second input mode (See claim 2 for detailed analysis.).
As to claim 14, claim 11 is incorporated and the combination of Riege and Ashford discloses assigning different visual effects to the first virtual object in the second input mode relative to the third input mode (See claim 3 for detailed analysis.).
As to claim 15, claim 11 is incorporated and the combination of Riege and Ashford discloses assigning different visual effects to the first virtual object in the first input mode relative to the third input mode (See claim 4 for detailed analysis.).
As to claim 16, claim 11 is incorporated and the combination of Riege and Ashford discloses implementing a fourth input mode, different from the first input mode, the second input mode, and the third input mode, with respect to the first virtual object of the plurality of virtual objects upon determining that the headpose of the first user is not targeting the first virtual object and contemporaneously upon determining that a first virtual ray of the respective rays from the first user is not targeting the first virtual object (See claim 5 for detailed analysis.).
As to claim 17, claim 16 is incorporated and the combination of Riege and Ashford discloses implementing different visual effects for first virtual object based on being in the first input mode, the second input mode, the third input mode, or the fourth input mode (See claim 6 for detailed analysis.).
As to claim 20, claim 16 is incorporated and the combination of Riege and Ashford discloses the ray of the first user is different in the fourth input mode relative to the first, second, or third input mode (See claim 9 for detailed analysis.).
Claims 7-8, 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Kai Riege, T. Holtkamper, G. Wesche and B. Frohlich, ("The Bent Pick Ray: An Extended Pointing Technique for Multi-User Interaction," 3D User Interfaces (3DUI'06), Alexandria, VA, USA, 2006, pp. 62-65, doi: 10.1109/VR.2006.127.) in view of Dean Ashford (Youtube video, UE4 - Tutorial - VR Widget Interactions!, https://www.youtube.com/watch?v=jC91FFkxgh4 time 10:30 to 14:16, Dec 5, 2018), further in view of Powerly et al. (US Pub 2017/0287225 A1).
As to claim 7, claim 6 is incorporated and the combination of Riege and Ashford does not disclose the system is further caused to perform operations including implementing a glowing visual effect for the first virtual object in the second input mode relative to the fourth input mode.
Powerly teaches implementing a glowing visual effect for the first virtual object in the second input mode relative to the fourth input mode (Powerly, ¶0125, “The wearable system may assign a focus indicator to one or more objects indicating the direction and/or location of the cone. For example, the wearable system may assign a focus indicator to an object which is in front of the user and intersects with the user's direction of gaze. The focus indicator can comprise a halo, a color, a perceived size or depth change (e.g., causing the target object to appear closer and/or larger when selected), a change in the shape of the cursor sprite graphic (e.g. the cursor is changed from a circle to an arrow), or other audible, tactile, or visual effects which draw the user's attention.”).
Riege, Ashford and Powerly are considered to be analogous art because all pertain to virtual reality. It would have been obvious before the effective filing date of the claimed invention to have modified Riege with the features of “implementing a glowing visual effect for the first virtual object in the second input mode relative to the fourth input mode” as taught by Powerly. The suggestion/motivation would have been in order to assign a focus indicator to an object which is in front of the user and intersects with the user's direction of gaze (Powerly, ¶0125.)
As to claim 8, claim 6 is incorporated and the combination of Riege, Ashford and Powerly discloses the system is further caused to perform operations including implementing a labeled visual effect for the first virtual object in the second input mode relative to the fourth input mode (Powerly, ¶0125, “The wearable system may assign a focus indicator to one or more objects indicating the direction and/or location of the cone. For example, the wearable system may assign a focus indicator to an object which is in front of the user and intersects with the user's direction of gaze. The focus indicator can comprise a halo, a color, a perceived size or depth change (e.g., causing the target object to appear closer and/or larger when selected), a change in the shape of the cursor sprite graphic (e.g. the cursor is changed from a circle to an arrow), or other audible, tactile, or visual effects which draw the user's attention.”).
As to claim 18, claim 16 is incorporated and the combination of Riege, Ashford and Powerly discloses implementing a glowing visual effect for the first virtual object in the second input mode relative to the fourth input mode (See claim 7 for detailed analysis.).
As to claim 19, claim 16 is incorporated and the combination of Riege, Ashford and Powerly discloses implementing a labeled visual effect for the first virtual object in the second input mode relative to the fourth input mode (See claim 8 for detailed analysis.).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Sooch et al. (US Pub 2020/0281058) teaches based on the length of time the button is held, the brightness or color temperature will increase or decrease more or less.
Rogers (US Pub 2019/0340816 A1) teaches positional data of the pointer of the virtual reality input device that is projected on a surface (e.g., panel displaying an application) within a virtual reality environment.
Galor (US Patent 9,122,311 B2) teaches highlight a given interactive item 38 by presenting a visual effect.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YU CHEN whose telephone number is (571)270-7951. The examiner can normally be reached on M-F 8-5 PST Mid-day flex.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached on 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YU CHEN/Primary Examiner, Art Unit 2613
Prosecution Insights