Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 41-50, 52--60 and 161 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Mathey-Owens et al. (US 2018004283 A1).
Regarding claim 41, Mathey-Owens et al. discloses a non-transitory computer-readable medium containing instructions that when executed by at least one processor cause the at least one processor to perform incremental convergence operations in an extended reality environment (the scene may comprise a virtual reality scene or a mixed reality scene, para. 0001), the operations comprising:
displaying a plurality of dispersed virtual objects across a plurality of virtual regions (Examiner articulates that objects in different areas of the scene are separately selectable, corresponding to plural virtual regions), the plurality of virtual regions including at least a first virtual region and a second virtual region that differs from the first virtual region (one or more objects (e.g., physical objects and/or virtual objects) may be present in a scene (i.e., virtual reality or mixed reality scene) with which a user wishes to interact, para. 0003);
receiving an initial kinesics input tending toward the first virtual region (…obtain movement data by tracking the movement of an input object (e.g., hand, finger, stylus, remote control, etc.) controlled by the user through the environment, para. 0021);
highlighting a group of adjacent virtual objects in the first virtual region based on the initial kinesics input (objects that fall within or partially within the 3D selection space may be highlighted or otherwise indicated as being included in the 3D selection space as the movement continues and the 3D selection space expands. Thus, the user may visualize or preview what objects currently lie within or move into the 3D selection space as the movement is detected, para. 0024; select the objects and/or present visual indicators (e.g., dotted line, highlight, color, etc.) to indicate that the objects have been selected, para. 0166);
receiving a refined kinesics input tending toward a particular virtual object from among the highlighted group of adjacent virtual objects (multiple gestures or movements of the input object may be used to identify objects that are included or at least partially included in the 3D selection space. For example, an initial group of objects may be selected as they fall within or at least partially within a 3D selection space, para. 0028; objects 218 may be real, physical objects, or virtual objects projected into the scene by the computing device 208. The object(s) 218 which fall within, or at least partially within (i.e., more than a threshold amount), the 3D selection space, such as objects 218A and 218B, may be selected, while objects that fall outside, or at least partially outside (i.e., more than a threshold amount), the 3D selection space, such as object 220, remain unselected, para. 0046); and
triggering a functionality associated with the particular virtual object based on the refined kinesics input (selecting multiple objects in an environment or scene, the techniques described herein may apply to any type of action on one or more objects, such as resizing, moving, coloring, etc., para. 0034; selection module 116 may perform one or more operations on the objects included in the 3D selection space, para. 0166).
Regarding claim 42, Mathey-Owens et al. discloses the non-transitory computer readable medium of claim 41, wherein the virtual objects of the plurality of dispersed virtual objects include at least one of virtual widgets or options in a virtual menu (objects may be directly interacted with (e.g., moving, coloring/painting, etc., para. 0057 which is typical of icons/ widgets or menu items in a 3d graphical user interface).
Regarding claim 43, Mathey-Owens et al. discloses non-transitory computer readable medium of claim 41, wherein the initial kinesics input includes a combination of a gaze detection and a gesture detection (selection of one or more objects in a 3D scene using gaze tracking and movement tracking. In some examples, multiple objects (i.e., physical objects and/or virtual objects) may be present and/or projected in a scene. A user may wish to select or interact with the multiple objects as a group. A computing device may use movement sensors (e.g., environment camera, accelerometer, depth sensing cameras, magnetic field sensors, etc.) to obtain movement data by tracking the movement of an input object (e.g., hand, finger, stylus, remote control, etc.) controlled by the user through the environment, para. 0021; location of the eyes of the user may be tracked and vectors (i.e., gaze vectors) originating at or near the user's eyes which intersect with points along the shape may be calculated. The computing system may use these vectors to extend and/or expand a virtual representation of the shape defined by the movement of the input object in a direction substantially parallel to a vector that starts at or near the eyes of the user and extends to a position on the shape, such as a center of the shape, to create a virtual 3D selection space or volume in the scene, para. 0023).
Regarding claim 44, Mathey-Owens et al. discloses the non-transitory computer readable medium of claim 43, wherein when the detected gaze tends towards the first virtual region and the detected gesture tends towards the second virtual region, the operations further include highlighting the group of adjacent virtual objects in the first virtual region over a group of virtual objects in the second virtual region (objects that fall within or partially within the 3D selection space may be highlighted or otherwise indicated as being included in the 3D selection space as the movement continues and the 3D selection space expands. Thus, the user may visualize or preview what objects currently lie within or move into the 3D selection space as the movement is detected, para. 0024).
Regarding claim 45, Mathey-Owens et al. discloses the non-transitory computer readable medium of claim 43, wherein when the detected gaze tends toward the first virtual region and the detected gesture tends toward an area without virtual objects, the operations further include highlighting the group of adjacent virtual objects in the first virtual region (the method of the applied prior art uses gaze to define vectors and a 3D selection volume even where the gesture path itself does not intersect all objects; objects intersected by the volume extended along a gaze vector are highlighted).
Regarding claim 46, Mathey-Owens et al. discloses the non-transitory computer readable medium of claim 41, wherein the initial kinesics input is associated with a first type of kinesics input and the refined kinesics input is associated with a second type of kinesics input that differs from the first type of kinesics input (second movement of the input object may then be detected subsequent to selecting the initial group of objects which selects a subgroup of the objects located within or at least partially within the 3D selection space, para. 0028).
Regarding claim 47, Mathey-Owens et al. discloses the non-transitory computer readable medium of claim 41, wherein highlighting the group of adjacent virtual objects in the first virtual region based on the initial kinesics input includes changing an appearance of each virtual object in the group of adjacent virtual objects in the first virtual region (objects within or at least partially within the 3D selection space may be highlighted, outlined, colored, or have a visible indicator (e.g., number, arrow, etc.) positioned near them, or any other indicator usable to indicate that the objects have been selected, para. 0027).
Regarding claim 48, Mathey-Owens et al. discloses the non-transitory computer readable medium of claim 41, wherein highlighting the group of adjacent virtual objects in the first virtual region based on the initial kinesics input includes displaying controls that enable interaction with at least one virtual object in the group of adjacent virtual objects of the first virtual region (objects within or at least partially within the 3D selection space may be highlighted, outlined, colored, or have a visible indicator (e.g., number, arrow, etc.) positioned near them, or any other indicator usable to indicate that the objects have been selected, para. 0027; user 202 may interact with the object(s) 218 included in the selection space. For example, the user 202 may perform various actions on the object (e.g., moving, resizing, coloring, etc.), and/or may deselect one or more of the objects included in the selection space, para. 0050).
Regarding claim 49, Mathey-Owens et al. discloses the non-transitory computer readable medium of claim 41, wherein upon receiving the refined kinesics input, the operations further include cancelling the highlighting of at least some of the virtual objects in the group of adjacent virtual objects (once the 3D selection space 316 is created (and similarly for an infinite 3D selection space), the user 302 may interact with the objects 318 included in the selection space. For example, the user 302 may perform various actions on the object (e.g., moving, resizing, coloring, etc.), and/or may deselect one or more of the objects included in the selection space, para. 0056).
Regarding claim 50, Mathey-Owens et al. discloses the non-transitory computer readable medium of claim 41, wherein:
the particular virtual object is an icon (indicators 322 may be highlighting the selected objects 318, a color change in the objects 318, an icon (e.g., arrow, number, etc.) placed nearby the selected objects 318, para. 0055); and wherein the triggered functionality includes causing an action associated with the particular virtual object including activating a script associated with the icon (responsive to the end of the movement of the input object 306, or responsive to detecting an ending action, the shapes included in the 3D selection space may automatically have an action performed on them without being selected first. In some examples, the user may have previously entered a mode, such as a paint mode, where the action is predetermined and taken on the objects included or at least partially included in the 3D selection space, para. 0057).
Regarding claim 52, Mathey-Owens et al. discloses the non-transitory computer readable medium of claim 41, wherein the operations further include:
prior to receipt of the initial kinesics input, receiving a preliminary kinesics input tending toward the second virtual region (multiple gestures or movements of the input object may be used to identify objects that are included or at least partially included in the 3D selection space. For example, an initial group of objects may be selected as they fall within or at least partially within a 3D selection space formed by a first movement of the input object, para. 0028);
highlighting a group of adjacent virtual objects in the second virtual region based on the preliminary kinesics input (second movement of the input object may then be detected subsequent to selecting the initial group of objects which selects a subgroup of the objects located within or at least partially within the 3D selection space. In this way, multiple gestures may be used to select groups and subgroups within those groups which are interacted with, para. 0028); and
upon receipt of the initial kinesics input, cancelling the highlighting of the group of adjacent virtual objects in the second virtual region (deselect one or more of the objects included in the selection space, para. 0050) and highlighting the group of adjacent virtual objects in the first virtual region (user 202 may perform various actions on the object (e.g., moving, resizing, coloring, etc, para. 0050).
Regarding claim 53, Mathey-Owens et al. discloses the non-transitory computer readable medium of claim 41, wherein the operations further include:
determining a certainty level associated with the initial kinesics input; and selecting a visual property of the highlighting based on the certainty level.
Regarding claim 54, Mathey-Owens et al. discloses the non-transitory computer readable medium of claim 41, wherein the operations further include:
determining a certainty level associated with the refined kinesics input (scene 300 may include multiple objects 318 that are included or at least partially included in the 3D selection space 316, and objects 320 that are not included, or less than a threshold portion is included, in the 3D selection space 316, para. 0055); and
forgoing triggering the functionality associated with the particular virtual object when the certainty level is lower than a threshold (if the virtual object is below a threshold, said object is not selected and an action is not performed).
Regarding claim 55, Mathey-Owens et al. discloses the non-transitory computer readable medium of claim 41, wherein the operations further include:
determining a first certainty level associated with the initial kinesics input (scene 300 may include multiple objects 318 that are included or at least partially included in the 3D selection space 316, and objects 320 that are not included, or less than a threshold portion is included, in the 3D selection space 316, para. 0055);
forgoing highlighting the group of adjacent virtual objects in the first virtual region when the first certainty level is lower than a first threshold (if the virtual object is below a threshold, said object is not selected and an action is not performed);
determining a second certainty level associated with the refined kinesics input; (scene 300 may include multiple objects 318 that are included or at least partially included in the 3D selection space 316, and objects 320 that are not included, or less than a threshold portion is included, in the 3D selection space 316, para. 0055) and
forgoing triggering the functionality associated with the particular virtual object when the second certainty level is lower than a second threshold (if the virtual object is below a threshold, said object is not selected and an action is not performed).
Regarding claim 56, Mathey-Owens et al. discloses the non-transitory computer readable medium of claim 41, wherein the first virtual region is selected based on the initial kinesics input (e.g., gesture(s) combined with gaze vectors to define the target spatial region).
Regarding claim 57, Mathey-Owens et al. discloses the non-transitory computer readable medium of claim 41, wherein:
the plurality of virtual regions are a plurality of virtual regions of an extended reality environment, displaying the plurality of dispersed virtual objects includes causing a wearable extended reality appliance (an example environment 100 in which multiple object selection techniques can be implemented to include a head-mounted computing device 102A and implemented using any other computing device including or having access to a camera or other sensor capable of tracking a location of eyes and movement of a user, para. 0037 and see examples (Oculus Rift® by Oculus VR, Google Glass by Google Inc., or HoloLens) of head-mounted computer devices in para. 0032) to display the plurality of dispersed virtual objects, highlighting the group of adjacent virtual objects includes causing the wearable extended reality appliance to highlight the group of adjacent virtual objects, and the functionality associated with the particular virtual object includes causing the wearable extended reality appliance to start displaying a particular virtual content (Examiner articulates that objects in different areas of the scene are separately selectable, corresponding to plural virtual regions), the plurality of virtual regions including at least a first virtual region and a second virtual region that differs from the first virtual region (one or more objects (e.g., physical objects and/or virtual objects) may be present in a scene (i.e., virtual reality or mixed reality scene) with which a user wishes to interact, para. 0003. See also para. 0046).
Regarding claim 58, Mathey-Owens et al. discloses the non-transitory computer readable medium of claim 41, wherein the highlighted group of adjacent virtual objects in the first virtual region includes at least two virtual objects of the plurality of dispersed virtual objects (user 104 is attempting to virtually select multiple objects in the scene 106. The environment 100 may comprise a physical, or real world, environment. As noted above, the scene 106 may be a computer generated scene, a real-world scene, or a mixed reality scene. For example, the scene may include the real world environment and one or more virtual objects, paras. 0038, para. 0046).
Claim 59, a method claim, is rejected for the same reason as claim 41 and see para. 0179.
Claim 60, a system claim, is rejected for the same reason as claim 41 and see para. 0039.
Regarding claim 161, Mathey-Owens et al. discloses the non-transitory computer readable medium of claim 41, wherein the refined kinesics input is associated with greater precision than the initial kinesics input (e.g., the gyroscope 1136 can be used for accurate recognition of movement within a 3D scene of a video game application or some other application while in some configurations, an application program utilizes greater precision output from the gyroscope 1136 combined with the accelerometer 1134 to enhance control of some functionality of the application program.
Allowable Subject Matter
Claim 51 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THOMAS J LETT whose telephone number is (571)272-7464. The examiner can normally be reached Mon-Fri 9-6 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571) 272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THOMAS J LETT/Primary Examiner, Art Unit 2611
Prosecution Insights