Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-7, 9-17 and 19-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Guo (WO 2023016302).
Claims 1 and 11, Guo disclose an input apparatus and method, comprising:
a camera (280, fig. 2), configured to capture a plurality of hand images of a user; and
a processor (210, fig. 2), communicatively connected to the camera, configured to execute the following operations:
determining a first gesture of the user based on a plurality of first hand images of the hand images; in response to the first gesture matching an activating gesture, generating a virtual keyboard on a virtual plane at a first time point, wherein the virtual plane is generated based on a palm position corresponding to the first gesture (Fig. 5. When the head-mounted device 31 displays the virtual keyboard in the virtual space, the virtual keyboard can be displayed at any position in the virtual space. In some embodiments, the virtual keyboard can be displayed directly below the hand 32 or at a position where the hand is about to fall. In this way, the user does not need to change the position of the virtual keyboard through operations such as dragging, which improves user experience. It can be understood that the head-mounted device 31 can establish a hand model of the hand 32 in real time, and display the hand model in the virtual space. The virtual keyboard is displayed directly below the hand model corresponding to the hand 32);
determining a second gesture of the user based on a plurality of second hand images corresponding to a second time point of the hand images, wherein the first time point is earlier than the second time point; and in response to the second gesture matching a typing gesture, generating an input command corresponding to the typing gesture based on a movement between the second gesture and the virtual keyboard (For example, in scenarios such as virtual reality (Virtual Reality, VR), augmented reality (Augmented Reality, AR), and mixed reality technology (Mixed Reality, MR), users input text information through a virtual keyboard displayed on an electronic device. At this time, the virtual input elements are keys in the virtual keyboard);
wherein the operation of generating the input command corresponding to the typing gesture further comprising:
calculating a first moving path of each of a plurality of fingertips based on the second hand images;
determining whether each of the moving paths of the fingertips is perpendicular to the virtual plane (abstract, Fig. 7, Fig. 14 and Fig. 15, once the keyboard is activated the user “simulates” finger tip presses in order to activate the virtual keys, the camera for the virtual keyboard images the hands and determines depth changes that are inherent to key presses for each individual finger, said depth measure is perpendicular to the virtual plane of the virtual keyboard); and
and in response to the first moving path of one of the fingertips perpendicular to the virtual plane, generating the input command of a key corresponding to the one of the fingertips (abstract).
Claims 2 and 12, Guo discloses the operation of generating the virtual keyboard further comprising: generating the virtual plane below the palm position based on the palm position corresponding to the first gesture; and generating the virtual keyboard on the virtual plane (When the head-mounted device 31 displays the virtual keyboard in the virtual space, the virtual keyboard can be displayed at any position in the virtual space. In some embodiments, the virtual keyboard can be displayed directly below the hand 32 or at a position where the hand is about to fall. In this way, the user does not need to change the position of the virtual keyboard through operations such as dragging, which improves user experience. It can be understood that the head-mounted device 31 can establish a hand model of the hand 32 in real time, and display the hand model in the virtual space. The virtual keyboard is displayed directly below the hand model corresponding to the hand 32).
Claims 4 and 14, Guo discloses the virtual keyboard inherently comprises a key feature for executing an editing function corresponding to an editing gesture (for example, pressing a delete or insert key).
Claims 5 and 15, Guo discloses calculating a plurality of hand joint points in the hand images; and determining the first gesture and the second gesture based on the hand joint points (fig. 9).
Claims 6 and 16, Guo discloses calculating a plurality of fingertip positions in the second hand images; and calculating a key corresponding to each of the fingertip positions on the virtual keyboard (abstract).
Claims 7 and 17, Guo disclose in response to the second gesture matching a closing gesture, terminating the virtual keyboard (That is to say, when the head-mounted device 31 detects that the distance between the hand 32 and the physical surface 33 is greater than the second distance threshold, it automatically turns off the gesture input function and removes the displayed virtual keyboard. Of course, the preset closing conditions are not limited to the examples mentioned above) and (In other embodiments, the user can also manually turn off the gesture input function. At this point, the user can turn off the gesture input function through a menu or a button).
Claims 9 and 19, Guo discloses directional keys (fig. 6) which have a capability of selecting a cursor position based on a fingertip position; and generating an input content based on the cursor position and an input command.
Claims 10 and 11, Guo discloses calculating a second moving path of one of a plurality of fingertips in the second hand images; and inherently selecting a plurality of texts based on the second moving path (for example, the copy feature of the virtual keyboard).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 8 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guo (WO 2023016302) in view of Antoniac et al. (US 20160224123).
Guo does not but Antoniac discloses a hand open pose to a hand closed pose for generating a close/terminate command. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to use Antoniac’s a hand open pose to a hand closed pose feature in the invention of Geo to remove the virtual keyboard because Guo teaches any hand gestures can be used to remove the virtual keyboard (That is to say, when the head-mounted device 31 detects that the distance between the hand 32 and the physical surface 33 is greater than the second distance threshold, it automatically turns off the gesture input function and removes the displayed virtual keyboard. Of course, the preset closing conditions are not limited to the examples mentioned above).
Response to Arguments
Applicant's arguments filed 1/29/2026 have been fully considered but they are not persuasive.
Applicant argues that prior art fails to teach newly added limitations to independent claims, the examiner respectfully disagrees. Specifically, Guo explicitly teaches that once the keyboard is activated the user “simulates” fingertip presses in order to activate the virtual keys, the camera for the virtual keyboard images the hands and determines depth changes that are inherent to key presses for each individual finger, said depth measure is perpendicular to the virtual plane of the virtual keyboard (Fig. 7, Fig. 14 and Fig. 15);
This is equivalent to the added limitation determining whether each of the moving paths of the fingertips is perpendicular to the virtual plane. To explain further in order for the virtual keyboard system of Guo to operate properly the camera for tracking the individual fingers and hands of the user must be operation to track all 3D movement for the different points of the hand shown in Fig. 9. The system will then abstract the hand into different parts. The finger tips then align with the virtual keyboard based on XY coordinate system in parallel with the plane of the virtual keyboard on the physical surface as shown in Fig. 7 or Fig. 14. Then in order to provide an input of a key, the vertical or depth coordinate must then be tracked by the camera. The movement of the fingertip itself doesn’t not necessarily have to be exclusively perpendicular, since the keypress must also be have an associated XY coordinate in order to determine the location on the virtual keyboard. However, the depth direction or the Z axis being tracked must have a certain value beyond which the simulate “press” is recognized by the system. Thus said Z coordinate change is necessarily perpendicular to the virtual plane of the virtual keyboard.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ke Xiao whose telephone number is 571-272-7776. The examiner can normally be reached Mon – Fri from 6AM-2PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KE XIAO/SPE, Art Unit 2627
Prosecution Insights