DETAILED ACTION
Claims 1-3, 5-12, and 14-20 filed May 4th 2026 are pending in the current action.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed May 4th 2026 have been fully considered but they are not persuasive.
Applicant’s arguments with respect to the teachings of Martin have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
With regards towards the amended limitation, the scope of “the physical tracking system” is undefined. The specification provides no mention of “the physical tracking system” or its contents.
Applicant asserts that Wu teaches “estimating” which is in contrast to the “directly tracking” required by the claim. However, the Applicant’s specification ties the term “directly tracking” with a camera detection (See Applicant ¶62 where “In various implementations, process 900 can accomplish this e.g., by directly tracking the user's hand (e.g., using computer vision to recognize the user's hand and its relational movement in the environment)”) The Examiner would argue that the tracking performed by the camera is an estimate; tracking of a hand using a camera requires estimation of the regions occluded from the camera. (See Applicant ¶66 The technology can also use cameras/vision to determine hand gesture when gripping controller in place of or in addition to the touch sensing. In some implementations, this can be done by receiving input from a camera, that can be located on the controller or on the head mounted device, and that captures images of the user's hand and determines, based on the images, a posture such as a 3D model of the hand, a skeletal representation of the hand, relative coordinates or distances for specified points on a hand (e.g., various joints and fingertips), etc.) Thus, the estimation of joint position performed by Wu is similar in operation to the hand tracking being performed by the Applicant.
The applicant asserts that Wu does not describe “controlling a representation of a finger.” The Examiner would like to clarify. Wu states in ¶4: “The present disclosure provides a finger-gesture detection device, which can detect a finger state of a user.” Wu ¶19 further states: “the virtual finger gesture of the virtual hand gesture to make the fingers straightened and closed with respect to the palm.” Wu ¶3 states: “a conventional control handle can only provide a simple input function such as object clicking or object selection without detecting a bending degree of a finger, so the control handle cannot be applied to a virtual environment to which a finger gesture needs to be applied, such as piano playing or keyboard tapping.” Thus, Wu provides suggestion that their detection of the bending degree of a user’s finger provides an improvement over the conventional prior art by detecting detailed finger gestures to apply towards piano playing and keyboard tapping.
In the interview on April 29th 2026, the agreement reached was pertaining to the cap sense and the virtual representation as shown in Figs. 8A-D in the applicant’s specification. The Examiner has modified the 103 rejection with Palmaro (US2017/0323483) to help remedy the combination.
Claim Rejections - 35 USC § 101
The amendment filed May 4th 2026 resolves the prior 101 rejection and the rejection is withdrawn.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
Claim 1-3, 5-12, and 14-20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Claim 1 recites “directly track, using one or more physical tracking systems of the controller device, a portion of the user’s hand that is not in contact with the one or more physical tracking systems of the controller device.” The specification does not explicitly state the term “physical tracking system” and the scope of the limitation is unclear. Claims 2 and 3 provide contradictory interpretations of the limitation “using one or more physical tracking systems of the controller device.” Claim 2 is directed to the infrared and/or capacitive sensors embedded within the controller, whereas claim 3 is directed to a camera. As best as the Examiner can tell, the camera is external to the controller as outlined in Applicant’s ¶36 where “one or more cameras included in the HMD 200 or 250, or from external cameras, can monitor the positions and poses of the user's hands to determine gestures and other hand and body motions.” Thus, the camera would not be “of the controller.” Claims 10 and 19 recite similar limitations and are rejected for similar reasons.
Claims 6 and 15 recite “a measured angle” as opposed to “a detected angle.” The difference is not explicitly disclosed in the specification. The specification does not recite “measured.”
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5-8, 10-12, 14-17, 19 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US2021/0089127) in view of Palmaro (US2017/0323483)
Consider claim 1, where Wu teaches a method for providing direct touch interaction with virtual objects while a user holds a controller device of an artificial reality system, (See Wu Fig. 1C, 1D and ¶43 where a user holds a controller handle 50 with worn sensors 120 around the knuckles of the user’s fingers where the virtual fingers interact with virtual objects so that an actuator (not shown) may be placed in each of the second wearing portions 120 to provide individual tactile feedback feelings of different fingers of the user (i.e., only a finger interacting with a virtual object will feel the corresponding tactile feedback)) the method comprising: providing, by the artificial reality system and while a user's hand holds the controller device, a representation of the user's hand in an artificial reality environment; (See Wu Figs 2, 3, and ¶48-51 where the sensors 120 are worn around the k3 knuckles and with the bending degree of the third knuckle K3 of each finger, the bending degrees of the first knuckle K1 and the second knuckle K2 are estimated, and finally the gesture of each finger is obtained) directly tracking, using one or more physical tracking systems of the controller device, a portion of the user's hand that is not in contact with the one or more physical tracking systems of the controller device; (See Wu Fig. 1B, 4 and ¶40 where As shown in FIG. 1B, in the present embodiment, the control handle 50 further includes a first sensing component 54. The first sensing component 54 is, for example, a capacitive sensor or a pressure sensor, and is disposed on the grip portion 52. the first sensing component 54 disposed on the control handle 50, when any of the fingers of the user touches the position corresponding to the control handle 50 (i.e., the first sensing component 54), the IMU of the first sensor 130 on the finger is automatically corrected. Thus, a capacitive sensor that detects a touch with the grip portion of the controller) and controlling, in the artificial reality environment while the user's hand continues to hold the controller device and based on the tracked portion of the user's hand not being in contact with the one ore more physical tracking systems of the controller device, a representation of the portion of the user's hand that is not in contact with the controller device performing a direct touch interaction with a virtual object. (See Wu Fig. 1C, 1D and ¶43 where a user holds a controller handle 50 with worn sensors 120 around the knuckles of the user’s fingers where the virtual fingers interact with virtual objects so that an actuator (not shown) may be placed in each of the second wearing portions 120 to provide individual tactile feedback feelings of different fingers of the user (i.e., only a finger (specifically, the K1, K2 portions of the finger) interacting with a virtual object will feel the corresponding tactile feedback))
Wu teaches controlling; however, Wu does not explicitly teach illustrating. However, in an analogous field of endeavor Palmaro teaches illustrating. (See Palmaro Fig 6 and ¶53-55, 74 The finger gesture 508 may be similar to the rewind gesture 402 or the forward gesture 408, but may be detected based on a two-finger gesture whereby two fingers (e.g., one from each hand 500, 502) are revolved such that each finger revolves around the other in a circular motion. ) Thus, it would have been obvious for one of ordinary skill in the art that the virtual hand of Wu (See Wu ¶18)used for controlling objects could be rendered into a virtual scene as taught by Palmaro. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of recognizing that the virtual system taught in Wu is capable of rendering and illustrating pertinent finger gestures.
Consider claim 2, where Wu in view of Palmaro teaches the method of claim 1, wherein the one or more physical tracking systems of the controller device include a capacitance or an infrared (IR) sensor, (See Wu Fig. 1B, 4 and ¶40 where As shown in FIG. 1B, in the present embodiment, the control handle 50 further includes a first sensing component 54. The first sensing component 54 is, for example, a capacitive sensor or a pressure sensor, and is disposed on the grip portion 52. the first sensing component 54 disposed on the control handle 50, when any of the fingers of the user touches the position corresponding to the control handle 50 (i.e., the first sensing component 54), the IMU of the first sensor 130 on the finger is automatically corrected. Thus, a capacitive sensor that detects a touch with the grip portion of the controller) and wherein the directly tracking the that a portion of the user's hand, that is not in contact with the one or more physical tracking systems of the controller device is based on detecting a change capacitance or infrared (IR) sensor change in a button of the controller device. (See Wu ¶53 where after entering a game, according to an interaction between the user and a virtual object, the computer will continuously record a real-time detection result (e.g., a magnetic value, a resistance value, a capacitance value, or other data) detected by each finger.)
Consider claim 3, where Wu in view of Palmaro teaches the method of claim 1, wherein the one or more physical tracking systems of the controller device include a camera, and wherein the directly tracking the portion of the user's hand, that is not in contact with the one or more physical tracking systems of the controller device is performed by applying computer vision, to one or more captured images depicting at least part of the hand of the user. (See Palmaro Fig. 6 and ¶23, 30, 36 where the VR interaction system provides temporal control actions to the user (e.g., a VR game developer) through a series of user-performed gestures performed by the user using hand tracking devices (e.g., one or more input devices tracking motion of the user, such as handheld tracking devices or optical tracking devices). In still other embodiments, hand motion of the user 100 may be tracked without, or in addition to, the handhelds or wearable hand tracking devices via a hand position sensor (not shown, e.g., using optical methods to track the position and orientation of the user's hands) such as, for example, those made commercially available by Leap Motion, Inc. (a California corporation). For example, the user 100 may be using the VR engine 212 to create a new virtual environment, and the actions refer to the user 100 creating and manipulating objects within the environment. The user actions may include, for example, position, orientation, and motion (POM) information of the user 100 (e.g., from the various VR input devices 206), or POM and state information of virtual objects within the virtual environment and with which the user 100 interacts during a recorded session. The POM information may include POM of the head of the user (e.g., as provided by the HMD 102), POM of the hands of the user (e.g., as provided by the handhelds), POM of the affected virtual objects altered by the user 100, and the voice commands of the user (e.g., as provided by a microphone, not separately depicted). For example, the POM of the head of the user 100 is determined by sensors in the HMD 102, or by an external head tracking device (e.g., external optical motion tracking device). The POM of the hands of the user is determined by sensors in handheld devices (e.g., the handhelds), or by an external hand tracking device.) Thus, it would have been obvious for one of ordinary skill in the art that the virtual hand of Wu (See Wu ¶18) used for controlling objects could be rendered into a virtual avatar as taught by Palmaro. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of using commercially available solutions such as those provided by Leap Motion, Inc. (See Palmaro ¶30)
Consider claim 5, where Wu in view of Palmaro teaches the method of claim 1, wherein the illustrating the representation of the portion of the user's hand, that is not in contact with the one or more physical tracking systems of the controller device, performing the direct touch interaction includes illustrating one or more fingers of the portion of the user's hand making a pointing gesture. (See Wu Figs 2, 3, and ¶48-51 where the sensors 120 are worn around the k3 knuckles and with the bending degree of the third knuckle K3 of each finger, the bending degrees of the first knuckle K1 and the second knuckle K2 are estimated, and finally the gesture of each finger is obtained.)
Consider claim 6, where Wu in view of Palmaro teaches the method of claim 1, wherein the directly tracking the portion of the user's hand is not in contact with the one or more physical tracking systems of the controller device includes measuring an angle of a finger of the user in relation to the user's hand; and wherein the illustrating the representation of the portion of the user's hand, that is not in contact with the one or more physical tracking systems of the controller device, performing the direct touch interaction includes showing the representation of the user's hand with a finger at the measured angle. (See Wu ¶57-59 where based on the foregoing, when the finger-gesture detection device of the present disclosure is worn on a hand of a user, the position of the first sensor corresponds to a third knuckle of a finger of the user. Therefore, the first sensor may detect the gesture of the third knuckle of each finger, and derive the gesture of each finger by estimating a finger joint motion. By integrating the finger-gesture detection device of the present disclosure and the control handle, the bending degree and gesture of each finger can be detected. As a result, the user can perform more complicated input functions in VR, which in turn brings out diversified VR interactions and allows the user to play games in a more natural way. The correction method can establish different detection ranges for different hand types or different grip modes of different users, thereby more accurately mapping a finger gesture in VR. Thus, mapping the bending degree to the virtual finger more accurately)
Consider claim 7, where Wu in view of Palmaro teaches the method of claim 1, wherein the directly tracking that a portion of the user's hand is not in contact with the controller device includes detecting that a user is not touching at least a particular button of the controller device; (See Wu Fig 5a and ¶50-51 where a second sensing component 140a that measures the distance between the finger from the second sensing component to calculate the degree of bending, thus being able to sense when the finger is in contact (distance of zero) or not in contact (distance greater than zero)) and wherein the illustrating the representation, of the portion of the user's hand that is not in contact with the controller device, performing the direct touch interaction includes showing the representation of the user's hand with a finger at a predetermined angle to the representation of the user's hand. (See Wu ¶57-59 where based on the foregoing, when the finger-gesture detection device of the present disclosure is worn on a hand of a user, the position of the first sensor corresponds to a third knuckle of a finger of the user. Therefore, the first sensor may detect the gesture of the third knuckle of each finger, and derive the gesture of each finger by estimating a finger joint motion. By integrating the finger-gesture detection device of the present disclosure and the control handle, the bending degree and gesture of each finger can be detected. As a result, the user can perform more complicated input functions in VR, which in turn brings out diversified VR interactions and allows the user to play games in a more natural way. The correction method can establish different detection ranges for different hand types or different grip modes of different users, thereby more accurately mapping a finger gesture in VR. Thus, mapping the bending degree to the virtual finger more accurately)
Consider claim 8, where Wu in view of Palmaro teaches the method of claim 1, wherein the illustrating the representation of the portion of the user's hand that is not in contact with the controller device includes showing a representation of the controller device as being held by a representation of the user's hand. (See Palmaro Fig 6 and ¶53-55, 74 The finger gesture 508 may be similar to the rewind gesture 402 or the forward gesture 408, but may be detected based on a two-finger gesture whereby two fingers (e.g., one from each hand 500, 502) are revolved such that each finger revolves around the other in a circular motion. ) Thus, it would have been obvious for one of ordinary skill in the art that the virtual hand of Wu (See Wu ¶18)used for controlling objects could be rendered into a virtual scene as taught by Palmaro. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of recognizing that the virtual system taught in Wu is capable of rendering and illustrating pertinent finger gestures.
Consider claim 10, where Wu teaches a non-transitory computer-readable storage medium storing instructions, for providing direct touch interaction with virtual objects while a user holds a controller device of an artificial reality system, the instructions, when executed by a computing system, (See Wu Fig. 1C, 1D and ¶43 where a user holds a controller handle 50 with worn sensors 120 around the knuckles of the user’s fingers where the virtual fingers interact with virtual objects so that an actuator (not shown) may be placed in each of the second wearing portions 120 to provide individual tactile feedback feelings of different fingers of the user (i.e., only a finger interacting with a virtual object will feel the corresponding tactile feedback)) cause the computing system to: provide, by the artificial reality system and while a user's hand holds the controller device, a representation of the user's hand in an artificial reality environment; track that a portion of the user's hand is not in contact with the one or more physical tracking systems of the controller device; (See Wu Figs 2, 3, and ¶48-51 where the sensors 120 are worn around the k3 knuckles and with the bending degree of the third knuckle K3 of each finger, the bending degrees of the first knuckle K1 and the second knuckle K2 are estimated, and finally the gesture of each finger is obtained) and control, in the artificial reality environment while the user's hand continues to hold the controller device and based on the tracked portion of the user's hand not being in contact with the one or more physical tracking systems of the controller device, a representation of the portion of the user's hand that is not in contact with the one or more physical tracking systems of the controller device performing a direct touch interaction with a virtual object. (See Wu Fig. 1C, 1D and ¶43 where a user holds a controller handle 50 with worn sensors 120 around the knuckles of the user’s fingers where the virtual fingers interact with virtual objects so that an actuator (not shown) may be placed in each of the second wearing portions 120 to provide individual tactile feedback feelings of different fingers of the user (i.e., only a finger (specifically, the K1, K2 portions of the finger) interacting with a virtual object will feel the corresponding tactile feedback))
Wu teaches controlling; however, Wu does not explicitly teach illustrating. However, in an analogous field of endeavor Palmaro teaches illustrating. (See Palmaro Fig 6 and ¶53-55, 74 The finger gesture 508 may be similar to the rewind gesture 402 or the forward gesture 408, but may be detected based on a two-finger gesture whereby two fingers (e.g., one from each hand 500, 502) are revolved such that each finger revolves around the other in a circular motion. ) Thus, it would have been obvious for one of ordinary skill in the art that the virtual hand of Wu (See Wu ¶18)used for controlling objects could be rendered into a virtual scene as taught by Palmaro. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of recognizing that the virtual system taught in Wu is capable of rendering and illustrating pertinent finger gestures.
Consider claim 11, where Wu in view of Palmaro teaches the non-transitory computer-readable storage medium of claim 10, wherein the one or more physical tracking systems of the controller device include a capacitance or an infrared (IR) sensor, (See Wu Fig. 1B, 4 and ¶40 where As shown in FIG. 1B, in the present embodiment, the control handle 50 further includes a first sensing component 54. The first sensing component 54 is, for example, a capacitive sensor or a pressure sensor, and is disposed on the grip portion 52. the first sensing component 54 disposed on the control handle 50, when any of the fingers of the user touches the position corresponding to the control handle 50 (i.e., the first sensing component 54), the IMU of the first sensor 130 on the finger is automatically corrected. Thus, a capacitive sensor that detects a touch with the grip portion of the controller) and wherein the directly tracking the that a portion of the user's hand, that is not in contact with the one or more physical tracking systems of the controller device is based on detecting a change capacitance or infrared (IR) sensor change in a button of the controller device. (See Wu ¶53 where after entering a game, according to an interaction between the user and a virtual object, the computer will continuously record a real-time detection result (e.g., a magnetic value, a resistance value, a capacitance value, or other data) detected by each finger.)
Consider claim 12, where Wu in view of Palmaro teaches the non-transitory computer-readable storage medium of claim 10, wherein the one or more physical tracking systems of the controller device include a camera, and wherein the directly tracking the portion of the user's hand, that is not in contact with the one or more physical tracking systems of the controller device is performed by applying computer vision, to one or more captured images depicting at least part of the hand of the user. (See Palmaro Fig. 6 and ¶23, 30, 36 where the VR interaction system provides temporal control actions to the user (e.g., a VR game developer) through a series of user-performed gestures performed by the user using hand tracking devices (e.g., one or more input devices tracking motion of the user, such as handheld tracking devices or optical tracking devices). In still other embodiments, hand motion of the user 100 may be tracked without, or in addition to, the handhelds or wearable hand tracking devices via a hand position sensor (not shown, e.g., using optical methods to track the position and orientation of the user's hands) such as, for example, those made commercially available by Leap Motion, Inc. (a California corporation). For example, the user 100 may be using the VR engine 212 to create a new virtual environment, and the actions refer to the user 100 creating and manipulating objects within the environment. The user actions may include, for example, position, orientation, and motion (POM) information of the user 100 (e.g., from the various VR input devices 206), or POM and state information of virtual objects within the virtual environment and with which the user 100 interacts during a recorded session. The POM information may include POM of the head of the user (e.g., as provided by the HMD 102), POM of the hands of the user (e.g., as provided by the handhelds), POM of the affected virtual objects altered by the user 100, and the voice commands of the user (e.g., as provided by a microphone, not separately depicted). For example, the POM of the head of the user 100 is determined by sensors in the HMD 102, or by an external head tracking device (e.g., external optical motion tracking device). The POM of the hands of the user is determined by sensors in handheld devices (e.g., the handhelds), or by an external hand tracking device.) Thus, it would have been obvious for one of ordinary skill in the art that the virtual hand of Wu (See Wu ¶18) used for controlling objects could be rendered into a virtual avatar as taught by Palmaro. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of using commercially available solutions such as those provided by Leap Motion, Inc. (See Palmaro ¶30)
Consider claim 14, where Wu in view of Palmaro teaches the non-transitory computer-readable storage medium of claim 10, wherein the illustrating the representation of the portion of the user's hand, that is not in contact with the one or more physical tracking systems of the controller device, performing the direct touch interaction includes illustrating one or more fingers of the portion of the user's hand making a pointing gesture. (See Wu Figs 2, 3, and ¶48-51 where the sensors 120 are worn around the k3 knuckles and with the bending degree of the third knuckle K3 of each finger, the bending degrees of the first knuckle K1 and the second knuckle K2 are estimated, and finally the gesture of each finger is obtained.)
Consider claim 15, where Wu in view of Palmaro teaches the non-transitory computer-readable storage medium of claim 10, wherein the directly tracking that a portion of the user's hand is not in contact with the one or more physical tracking systems of the controller device includes measuring an angle of a finger of the user in relation to the user's hand; and wherein the illustrating the representation of the portion of the user's hand, that is not in contact with the one or more physical tracking systems of the controller device, performing the direct touch interaction includes showing the representation of the user's hand with a finger at the measured angle. (See Wu ¶57-59 where based on the foregoing, when the finger-gesture detection device of the present disclosure is worn on a hand of a user, the position of the first sensor corresponds to a third knuckle of a finger of the user. Therefore, the first sensor may detect the gesture of the third knuckle of each finger, and derive the gesture of each finger by estimating a finger joint motion. By integrating the finger-gesture detection device of the present disclosure and the control handle, the bending degree and gesture of each finger can be detected. As a result, the user can perform more complicated input functions in VR, which in turn brings out diversified VR interactions and allows the user to play games in a more natural way. The correction method can establish different detection ranges for different hand types or different grip modes of different users, thereby more accurately mapping a finger gesture in VR. Thus, mapping the bending degree to the virtual finger more accurately)
Consider claim 16, where Wu in view of Palmaro teaches the non-transitory computer-readable storage medium of claim 10, wherein the directly tracking that a portion of the user's hand is not in contact with the one or more physical tracking systems of the controller device includes detecting that a user is not touching at least a particular button of the controller device; (See Wu Fig 5a and ¶50-51 where a second sensing component 140a that measures the distance between the finger from the second sensing component to calculate the degree of bending, thus being able to sense when the finger is in contact (distance of zero) or not in contact (distance greater than zero)) and wherein the illustrating the representation, of the portion of the user's hand that is not in contact with the one or more physical tracking systems of the controller device, performing the direct touch interaction includes showing the representation of the user's hand with a finger at a predetermined angle to the representation of the user's hand. (See Wu ¶57-59 where based on the foregoing, when the finger-gesture detection device of the present disclosure is worn on a hand of a user, the position of the first sensor corresponds to a third knuckle of a finger of the user. Therefore, the first sensor may detect the gesture of the third knuckle of each finger, and derive the gesture of each finger by estimating a finger joint motion. By integrating the finger-gesture detection device of the present disclosure and the control handle, the bending degree and gesture of each finger can be detected. As a result, the user can perform more complicated input functions in VR, which in turn brings out diversified VR interactions and allows the user to play games in a more natural way. The correction method can establish different detection ranges for different hand types or different grip modes of different users, thereby more accurately mapping a finger gesture in VR. Thus, mapping the bending degree to the virtual finger more accurately)
Consider claim 17, where Wu in view of Palmaro teaches the non-transitory computer-readable storage medium of claim 10, wherein the providing the representation of the user's hand includes showing a representation of the controller device as being held by the representation of the user's hand. (See Palmaro Fig 6 and ¶53-55, 74 The finger gesture 508 may be similar to the rewind gesture 402 or the forward gesture 408, but may be detected based on a two-finger gesture whereby two fingers (e.g., one from each hand 500, 502) are revolved such that each finger revolves around the other in a circular motion. ) Thus, it would have been obvious for one of ordinary skill in the art that the virtual hand of Wu (See Wu ¶18)used for controlling objects could be rendered into a virtual scene as taught by Palmaro. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of recognizing that the virtual system taught in Wu is capable of rendering and illustrating pertinent finger gestures.
Consider claim 19, where Wu teaches a computing system, for providing direct touch interaction with virtual objects while a user holds a controller device of an artificial reality system, the computing system comprising: one or more processors; and one or more memories storing instructions that, when executed by the one or more processors, (See Wu Fig. 1C, 1D and ¶43 where a user holds a controller handle 50 with worn sensors 120 around the knuckles of the user’s fingers where the virtual fingers interact with virtual objects so that an actuator (not shown) may be placed in each of the second wearing portions 120 to provide individual tactile feedback feelings of different fingers of the user (i.e., only a finger interacting with a virtual object will feel the corresponding tactile feedback)) cause the computing system to: provide, by the artificial reality system and while a user's hand holds the controller device, a representation of the user's hand in an artificial reality environment; track that a portion of the user's hand is not in contact with the one or more physical tracking systems of the controller device; (See Wu Figs 2, 3, and ¶48-51 where the sensors 120 are worn around the k3 knuckles and with the bending degree of the third knuckle K3 of each finger, the bending degrees of the first knuckle K1 and the second knuckle K2 are estimated, and finally the gesture of each finger is obtained) and control, in the artificial reality environment while the user's hand continues to hold the controller device and based on the tracked portion of the user's hand not being in contact with the one or more physical tracking systems of the controller device, a representation of the portion of the user's hand that is not in contact with the one or more physical tracking systems of the controller device performing a direct touch interaction with a virtual object. (See Wu Fig. 1C, 1D and ¶43 where a user holds a controller handle 50 with worn sensors 120 around the knuckles of the user’s fingers where the virtual fingers interact with virtual objects so that an actuator (not shown) may be placed in each of the second wearing portions 120 to provide individual tactile feedback feelings of different fingers of the user (i.e., only a finger (specifically, the K1, K2 portions of the finger) interacting with a virtual object will feel the corresponding tactile feedback))
Wu teaches controlling; however, Wu does not explicitly teach illustrating. However, in an analogous field of endeavor Palmaro teaches illustrating. (See Palmaro Fig 6 and ¶53-55, 74 The finger gesture 508 may be similar to the rewind gesture 402 or the forward gesture 408, but may be detected based on a two-finger gesture whereby two fingers (e.g., one from each hand 500, 502) are revolved such that each finger revolves around the other in a circular motion. ) Thus, it would have been obvious for one of ordinary skill in the art that the virtual hand of Wu (See Wu ¶18)used for controlling objects could be rendered into a virtual scene as taught by Palmaro. One of ordinary skill in the art would have been motivated to perform the modification for the advantage of/ benefit of recognizing that the virtual system taught in Wu is capable of rendering and illustrating pertinent finger gestures.
Consider claim 20, where Wu in view of Palmaro teaches the computing system of claim 19, wherein the one or more physical tracking systems of the controller device include a capacitance or an infrared (IR) sensor, (See Wu Fig. 1B, 4 and ¶40 where As shown in FIG. 1B, in the present embodiment, the control handle 50 further includes a first sensing component 54. The first sensing component 54 is, for example, a capacitive sensor or a pressure sensor, and is disposed on the grip portion 52. the first sensing component 54 disposed on the control handle 50, when any of the fingers of the user touches the position corresponding to the control handle 50 (i.e., the first sensing component 54), the IMU of the first sensor 130 on the finger is automatically corrected. Thus, a capacitive sensor that detects a touch with the grip portion of the controller) and wherein the directly tracking the that a portion of the user's hand, that is not in contact with the one or more physical tracking systems of the controller device is based on detecting a change capacitance or infrared (IR) sensor change in a button of the controller device. (See Wu ¶53 where after entering a game, according to an interaction between the user and a virtual object, the computer will continuously record a real-time detection result (e.g., a magnetic value, a resistance value, a capacitance value, or other data) detected by each finger.)
Allowable Subject Matter
Claims 9, 18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: Claim 9 recites the limitation "in response to detecting that the tracked movement moved the user's hand within a threshold distance of an object configured for direct touch interaction, hiding the representation of the controller." While this limitation can be found in Kim et al. (US2020/0159337) where Fig. 10-13 and ¶132-143 teaches the unification of the user's hand's virtual representation and the controller's virtual representation. However, it would have been non-obvious to implement the integration of the hand and controller specifically when within threshold distance of touching a virtual object as claimed.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM LU whose telephone number is (571)270-1809. The examiner can normally be reached 10am-6:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Eason can be reached at 571-270-7230. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
WILLIAM LU
Primary Examiner
Art Unit 2624
/WILLIAM LU/Primary Examiner, Art Unit 2624
Prosecution Insights