Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on June 1, 2026 has been considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-7, 10-14 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Lucas et al (US 10,515,484) in view of Bradski et al (US 2016/0026253).
As per claim 1 Lucas et al discloses: A method for providing virtual object 502 accessibility in an artificial reality (XR) environment, the method comprising:
detecting, by an XR system and in a real-world environment, that a {figure 2: 202} is within a field-of-view of a user of the XR system {figure 2: 204};
in response to the detecting of the virtual object 502 locked to a representation of the {figure 5 & [column 5, line 64 through column 6, line 5 ] FIG. 5 illustrates a view of an interactive space showing a set of virtual objects, a user object 502, and a virtual cursor 504 positionally linked with user object 502. The set of virtual objects may include one or more of a first virtual object 506, a second virtual object 508, a third virtual object 510, a fourth virtual object 512 and/or other virtual objects. The view in FIG. 5 generally shows user object 502 prior to, or approaching, an interaction with one or more objects in the set of virtual objects.};
detecting, based on determined positions of both i) a hand of the user and ii) the virtual object 502, locked to the representation of the {figure 2: 208}; and
based on the detecting of the intent to interact with the first virtual object 502 by the hand of the user, displaying a second virtual object 502, associated with the first virtual object 502, in the XR environment {[ column 8, lines 50-55] In some implementations, a user (or multiple users) may utilize multiple hands in the interactive space. In some implementations, individual hands may be associated with individual threshold distances. Accordingly, in some implementations, one or more virtual objects may be correlated with the location of one or more hands.}.
As per claim 11 Lucas et al discloses: The method of claim 1, wherein a location of the displayed first virtual object 502 is locked to a location based on a location of the representation of the {figure 2: 204 & [column 5, line 64 through column 6, line 5 ] FIG. 5 illustrates a view of an interactive space showing a set of virtual objects, a user object 502, and a virtual cursor 504 positionally linked with user object 502. The set of virtual objects may include one or more of a first virtual object 506, a second virtual object 508, a third virtual object 510, a fourth virtual object 512 and/or other virtual objects. The view in FIG. 5 generally shows user object 502 prior to, or approaching, an interaction with one or more objects in the set of virtual objects.}.
As per claim 15 Lucas et al discloses: A computer-readable storage medium storing instructions, for providing virtual object 502 accessibility in an artificial reality (XR) environment, the instructions, when executed by a computing system, cause the computing system to:
detect, by an XR system and in a real-world environment, a {figure 2: 202}, within a threshold distance of a field-of-view of the XR system {figure 2: 204};
in response to the detecting of the virtual object 502 locked to a representation of the figure 5 & [column 5, line 64 through column 6, line 5 ] FIG. 5 illustrates a view of an interactive space showing a set of virtual objects, a user object 502, and a virtual cursor 504 positionally linked with user object 502. The set of virtual objects may include one or more of a first virtual object 506, a second virtual object 508, a third virtual object 510, a fourth virtual object 512 and/or other virtual objects. The view in FIG. 5 generally shows user object 502 prior to, or approaching, an interaction with one or more objects in the set of virtual objects.};
detect an intent to interact with the virtual object 502 by a hand of the user {figure 2: 208}; and
based on the detecting of the intent to interact with the virtual object 502 by the hand, perform an action, associated with the virtual object 502, in the XR environment {[ column 8, lines 50-55] In some implementations, a user (or multiple users) may utilize multiple hands in the interactive space. In some implementations, individual hands may be associated with individual threshold distances. Accordingly, in some implementations, one or more virtual objects may be correlated with the location of one or more hands.}.
As per claim 19 Lucas et al discloses: A computing system for providing wrist- or forearm-locked virtual object 502 accessibility in an artificial reality (XR) environment, the computing system comprising:
one or more processors 104; and one or more memories 115 storing instructions that, when executed by the one or more processors 104, cause the computing system to:
detect, by an XR system and in a real-world environment, that a {figure 2: 202}, is within a field-of-view of a user the XR system {figure 2: 204};
in response to the detecting of the virtual object 502 locked to a representation of the wrist or forearm of the user in the XR environment {figure 5 & [column 5, line 64 through column 6, line 5 ] FIG. 5 illustrates a view of an interactive space showing a set of virtual objects, a user object 502, and a virtual cursor 504 positionally linked with user object 502. The set of virtual objects may include one or more of a first virtual object 506, a second virtual object 508, a third virtual object 510, a fourth virtual object 512 and/or other virtual objects. The view in FIG. 5 generally shows user object 502 prior to, or approaching, an interaction with one or more objects in the set of virtual objects.};
detect, based on determined position of both i) a hand of the user and ii)the virtual object 502, locked to the representation of the {figure 2: 208}; and
based on the detecting of the intent to interact with the virtual object 502 by the hand of the user,, perform an action, associated with the virtual object 502, in the XR environment {[ column 8, lines 50-55] In some implementations, a user (or multiple users) may utilize multiple hands in the interactive space. In some implementations, individual hands may be associated with individual threshold distances. Accordingly, in some implementations, one or more virtual objects may be correlated with the location of one or more hands.}.
Regarding claims 1, 11 and 19 Lucas et al is silent as to: a wrist or forearm. With respect to claim 1, 11 and 19 Bradski et al discloses: [1288] In one example, the user may select a particular icon 12510 (FIG. 125F); the system may have some indicator to denote that it has now been selected (e.g., denoted by a different color, etc.). As shown in FIG. 125G, the user may drag the selected icon 12510 to his wrist. This action may be recognized by the system, indicating to the user that this application may be opened. Here, the user has selected a virtual object icon (e.g., a diamond shaped icon, as shown in the FIGS. 125G). Based on the icon selection, the other virtual icons may fade away and a virtual fading pattern may be projected on the user's wrist, as shown in FIGS. 125H and 125I respectively.
It would have been obvious to a person having ordinary skill in the art at the time the invention was effectively filed to provide the method of Lucas et al with a user’s wrist or forearm as taught by Bradski et al. The rationale is as follows: one of ordinary skill in the art at the time the invention was effectively filed would have been motivated to provide a method with a user’s wrist or forearm to provide a more area and a user-friendly way to interact in artificial reality.
Regarding claim 2 Lucas et al is silent as to: The method of claim 1, wherein the second virtual object 502 is a virtual menu providing access to one or more functions of the XR system or one or more functions of an XR application executing in the XR environment. With respect to claim 2 Bradski et al discloses: [1240] The user selectable virtual icons may represent applications (e.g., social media application, Web browser, email, etc.), functions, menus, virtual rooms or virtual spaces, etc. . . As described above, the user interface coordinates may be tied to the determined location of the user's center such that it is tied to the user's body.
Regarding claim 3 Lucas et al is silent as to: The method of claim 1, wherein the hand is on a different arm of a body, of the user, from the wrist. With respect to claim 3 Bradski et al discloses: the hand is on a different arm of a body, of the user, from the {figure 125A-125D}
Regarding claim 4 Lucas et al is silent as to: The method of claim 3, wherein the detected intent to interact is detecting a tap by the hand on the first virtual object 502. With respect to claim 4 Bradski et al discloses: The method of claim 3, wherein the detected intent to interact is a tap by the hand on the first virtual object {[1040] In some implementations, fingers gestures may be used as input for the AR system. Finger gestures can take a variety of forms and may, for example, be based on inter-finger interaction, pointing, tapping, rubbing, etc.}
It would have been obvious to a person having ordinary skill in the art at the time the invention was effectively filed to provide the method of Lucas et al with a virtual menu tappable by a hand on a different arm of a body, of the user, from the wrist as taught by Bradski et al. The rationale is as follows: one of ordinary skill in the art at the time the invention was effectively filed would have been motivated to provide a method with a virtual menu tappable by a hand on a different arm of a body, of the user, from the wrist to provide a user a simplified means to access different selectable items.
As per claim 5 Lucas et al discloses: The method of claim 1, wherein the detected intent is based on detecting a gesture made by the hand {[column 8, line 56 through column 9, line 4] In some implementations, interaction component 112 may be configured to determine, based on a correlation of a location of the user's hand with a virtual object, and the pose of the user's hand, the user's intent to perform an interaction of the with the virtual object. In some implementations, the pose of the user's hand may be indicative of gesture-based interactions being performed by the user. A gesture may include one or more of clicking, reaching, grabbing, releasing, swiping, pinching, pulling, throwing, pointing, and/or other gestures and/or actions. By way of non-limiting illustration, interaction component 112 may utilize one or more gesture recognition techniques to identify one or more gestures and/or actions being performed by a human hand based on the pose of the hand, the correlation of a virtual object with a location of the user's hand, and/or other information.}.
As per claim 6 Lucas et al discloses: The method of claim 5, wherein the gesture is a pinch gesture by the hand {[column 8, lines 63-65] A gesture may include one or more of clicking, reaching, grabbing, releasing, swiping, pinching, pulling, throwing, pointing, and/or other gestures and/or actions. }.
As per claim 7 Lucas et al discloses: The method of claim 1, wherein detecting the intent to interact with the first virtual object 502 includes determining a duration of the intent to interact with the first virtual object 502 {[column 8, lines 11-20] The hand component 110 may be configured to determine pose of objects in the real world. The determination of pose may be based on output signals from one or more hand tracking devices 12 and/or other information. By way of non-limiting illustration, hand component 110 may be configured to determine pose of the hand over time. The pose may refer to one or more of heading or orientation (e.g., which way the palm is facing), whether the hand is closed (e.g., in a gripped position) or open (e.g., one or more digits extended), and/or other information.}, and wherein the method further comprises: selecting the second virtual object 502 based on the duration of the interaction with the first virtual object 502 {[ column 8, lines 50-55] In some implementations, a user (or multiple users) may utilize multiple hands in the interactive space. In some implementations, individual hands may be associated with individual threshold distances. Accordingly, in some implementations, one or more virtual objects may be correlated with the location of one or more hands.}.
As per claim 10 Lucas et al discloses: The method of claim 1, wherein the intent to interact with the first virtual object 502 is a first intent to interact, and wherein the method further comprises: detecting a second intent to interact with the first virtual object 502 by the hand of the user; and based on the detecting the second intent to interact with the first virtual object 502 {[column 11, lines 21-26] FIG. 7 illustrates the view of the interactive space of FIGS. 5 and 6, further illustrating a highlight 505 of virtual cursor 504 reflecting a determined pose, or a change in pose, of user object 502. By way of non-limiting illustration, user object 502 may have changed from an open hand pose (e.g., the pose shown in FIGS. 5 and 6) to a gripped hand pose.},
Regarding claim 10 Lucas et al is silent as to: ceasing display of the second virtual object 502 in the XR environment. Regarding claim 14 Lucas et al is silent as to: The method of claim 1 wherein the XR environment includes interactive content, and wherein the method further comprises: ceasing rendering of a portion of the interactive content, within a threshold distance of the first virtual object 502, while the first virtual object 502 is displayed.
It would have been obvious to a person having ordinary skill in the art at the time the invention was effectively filed to specify that the method of Lucas et al ceased display of the second virtual object in the XR environment. The rationale is as follows: one of ordinary skill in the art at the time the invention was effectively filed would have been motivated to specify that a method ceased display of the second virtual object in the XR environment to clear the display or either by turning off the display.
As per claim 12 Lucas et al discloses: The method of claim 1, wherein the first virtual object 502 is selected based on a determination of which wrist or forearm is detected {figure 2: 202}.
As per claim 13 Lucas et al discloses: The method of claim 1, wherein the first virtual object 502 is displayed further based on detection of greater than a threshold amount of a palm of the user facing the XR system {[column 8, lines 11-20] The hand component 110 may be configured to determine pose of objects in the real world. The determination of pose may be based on output signals from one or more hand tracking devices 12 and/or other information. By way of non-limiting illustration, hand component 110 may be configured to determine pose of the hand over time. The pose may refer to one or more of heading or orientation (e.g., which way the palm is facing), whether the hand is closed (e.g., in a gripped position) or open (e.g., one or more digits extended), and/or other information. Note: a threshold amount lacks specificity to distinguish over an amount disclosed in the method of the applied prior art.}.
As per claim 14 Lucas et al discloses: The method of claim 1, wherein the XR environment includes interactive content, and wherein the method further comprises: ceasing rendering of a portion of the interactive content, within a threshold distance of the first virtual object, while the first virtual object is displayed { [1096] In response to the orientations, changes in position (e.g., movements) and/or interactions, the AR system may change one or more aspects of the rendering the virtual user interface, causing corresponding inputs to be provided to a computer or some other device . . . Likewise, the AR system may respond by stopping the rendering of virtual interface elements which would otherwise appear on the faces now hidden from the view of the user.}.
As per claim 20 Lucas et al discloses: wherein the virtual object 502 is a first virtual object 502, and wherein the action includes displaying a second virtual object 502, associated with the first virtual object 502, in the XR environment {[ column 8, lines 50-55] In some implementations, a user (or multiple users) may utilize multiple hands in the interactive space. In some implementations, individual hands may be associated with individual threshold distances. Accordingly, in some implementations, one or more virtual objects may be correlated with the location of one or more hands.}.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Lucas et al (US 10,515,484) in view of Bradski et al (US 2016/0026253) as applied to claim 1 above, and further in view of Emami et al (US 2024/0281071).
Regarding claim 9 Lucas et al is silent as to: The method of claim 1, wherein the first virtual object 502 is mapped to a physical button on a controller of the XR system. With respect to claim 9 Emami et al discloses: The method of claim 1, wherein the first virtual object is mapped to a physical button on a controller of the XR system. {[0049] FIGS. 5A and 5B illustrate two hand gestures in connection with a controller. FIG. 5A illustrates an example 525 where a controller 500 is intended to be gripped by a user's hand. In general, the user's index finger will be contacting button 502 when controller 500 is fully gripped. A capacitance sensor, pressure sensor, IR sensor, computer vision analysis of an image of the user's hand in relation to the button 502, or other means can determine that the user's index finger is contacting button 502. The system detecting that the user's finger is touching button 502 is indicated by the shaded “cap sense” state 501, where the button 502 can have states not being touched (e.g., no cap sense and not pressed, capacitance sensed 501, or button pressed 503). In examples, the avatar representation on the UI, when controller 500 is fully gripped and the user's index finger is contacting button 502, is shown at 510 as having the representation of the user's hand (e.g., a hand of the user's avatar) curled around touching a representation of button 502 on a representation of controller 500.}
It would have been obvious to a person having ordinary skill in the art at the time the invention was effectively filed to provide the method of Lucas et al as modified by Bradski et al with the first virtual object is mapped to a physical button on a controller of the XR system as taught by Emami et al. The rationale is as follows: one of ordinary skill in the art at the time the invention was effectively filed would have been motivated to provide a method with the first virtual object is mapped to a physical button on a controller of the XR system so that the user can see the controller, the buttons and the movement thereof in a augmented reality environment thereby eliminating the need for to remove a head mounted display.
Allowable Subject Matter
Claim 8 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claims 15, 17 and 18 are allowed.
Response to Arguments
Applicant's arguments filed June 1, 2026 have been fully considered but they are not persuasive. Applicant asserts in the second full paragraph on page 3 the following:
Neither the cited passages of Lucas or Braski, nor their combination, teach determining any relationship between a wrist or forearm and a field-of-view of a user of an XR system, thus they do not teach "detecting that a wrist or forearm is within a field-of-view of a user of the XR system," as claim 1 now recites. Further, as the cited passages of Lucas and Braski do not teach any such detection of the wrist or forearm in relation a user's field of view, they certainly cannot teach taking an action (namely "displaying a first virtual object locked to a representation of the wrist or forearm"), "in response to the detecting of the wrist or forearm is within the field-of-view of the user of the XR system," as claim 1 now further recites.
Contrary to applicant’s assertion, as stated and mapped supra, Bradski depicts in figure 5 and discloses in [column 5, line 64 through column 6, line 5 ] the following:
FIG. 5 illustrates a view of an interactive space showing a set of virtual objects, a user object 502, and a virtual cursor 504 positionally linked with user object 502. The set of virtual objects may include one or more of a first virtual object 506, a second virtual object 508, a third virtual object 510, a fourth virtual object 512 and/or other virtual objects. The view in FIG. 5 generally shows user object 502 prior to, or approaching, an interaction with one or more objects in the set of virtual objects.
Additionally, for interaction and an interactive space, the wrist or forearm must be in field-of-view of a user to operate as intended. Therefore, Lucas as modified by Bradski teaches the claimed invention.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID D DAVIS whose telephone number is (571)272-7572. The examiner can normally be reached Monday - Friday, 8 a.m. - 4 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ke Xiao can be reached at 571-272-7776. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID D DAVIS/Primary Examiner, Art Unit 2627
DDD
Prosecution Insights