DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 6/11/2026 has been entered. Currently, claims 1-20 are pending.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Forlines et al. in US 2019/0294258 (hereinafter Forlines).
Regarding claim 1, Forlines disclose a method comprising: obtaining image data of a first hand (Forlines’s par. 232: hand postures measured by camera or ToF-3D sensors) in accordance with an indication of peripheral input (Forlines’s par. 179-182, 199: upon hands/fingers in a typing or resting-on-keys state of the touch sensitive keyboard, then further gestures are detected) from a user interaction (Forlines’s par. 179: hover) with a peripheral event (Forlines’s par. 179: typing) at a peripheral device (Forlines’s par. 179: keyboard); determining, based on the image data (Forlines’s par. 232: hand postures measured by camera or ToF-3D sensors), a hand pose of the first hand (Forlines’s par. 232: hand posture measured by camera, e.g. par. 186: palm on trackpad); determining a hand pose (Forlines’s par. 228: palm up or down, par. 168: curling [such as for closing of par. 183, 228]) corresponding to a predetermined input gesture (Forlines’s par. 228: closing/opening hand, holding palm up/down, par. 183: “grab”, hand in shape of holding mouse) associated with a user input action (Forlines’s par. 228: orchestra control, par. 183: drag of a window, interaction with a stylus, movement of mouse) for interacting with a user interface (Forlines’s par. 227-228: instrument/orchestra, par. 183: touch sensitive device); and in accordance with a determination that characteristics of the first hand (Forlines’s par. 186: fingers resting on keyboard) in the image data (Forlines’s par. 232: hand postures measured by camera) satisfy a peripheral use criterion (Forlines’s par. 179, 186: typing state): processing the peripheral input from the peripheral device (Forlines’s par. 179, 186: typing), and rejecting track input (Forlines’s par. 186: ignoring track input).
Forlines fails to explicitly disclose determining that the hand pose (based on image data) corresponds to a predetermined input gesture associated with a user input action for interacting with a user interface, or explicitly rejecting the user input action associated with the predetermined input gesture.
However, Forlines does disclose ignoring input by the palm resting below the keyboard when the user’s fingers are resting on the keyboard for typing (Forlines’s par. 186), and input gestures that correspond to the palm (Forlines’s par. 228: palm down for orchestra control, par. 168: curling as detected by separation between fingertip and palm such as hand closing/opening for grab of windows, interaction with stylus or movement of mouse per par. 183).
Therefore, it would have been obvious to one of ordinary skill in the art, that Forlines palm input gestures (Forlines’s par. 168, 183, 228) are based on the hand’s posture image data (Forlines’s par. 232) in order to obtain the benefit of a hand skeleton that enables reliable gesture recognition (Forlines’s par. 232). Furthermore, it would also have been obvious to one of ordinary skill in the art, that these palm input gestures (Forlines’s par. 168, 183, 228) are ignored when they are detected to correspond to the palm below the keyboard (Forlines’s par. 186), in order to obtain the predictable result of mitigating accidental input (Forlines’s par. 186), in applications that include both the trackpad and the keyboard (Forlines’s Figs. 11-12, 15).
By doing such combination Forlines disclose a method comprising:
obtaining image data of a first hand (Forlines’s par. 232: hand postures measured by camera or ToF-3D sensors) in accordance with an indication of peripheral input (Forlines’s par. 179-182, 199: upon hands/fingers in a typing or resting-on-keys state of the touch sensitive keyboard, then further gestures are detected) from a user interaction (Forlines’s par. 179: hover) with a peripheral event (Forlines’s par. 179: typing) at a peripheral device (Forlines’s par. 179: keyboard);
determining, based on the image data (Forlines’s par. 232: hand postures measured by camera or ToF-3D sensors), a hand pose of the first hand (Forlines’s par. 232: hand posture measured by camera, e.g. par. 186: palm on trackpad);
determining that the hand pose (Forlines’s par. 186: palm resting below keyboard, such as with a palm up or down per par. 228, or hand curling per par. 168) corresponds to a predetermined input gesture (Forlines’s par. 186 palm resting below keyboard upon combination includes palm input gestures such as closing/opening hand, holding palm up/down or the hand in shape of holding a mouse per par. 183, 228) associated with a user input action (Forlines’s par. 228: orchestra control, par. 183: drag of a window, interaction with a stylus, movement of mouse) for interacting with a user interface (Forlines’s par. 227-228: instrument/orchestra, par. 183: touch sensitive device); and
in accordance with a determination that characteristics of the first hand (Forlines’s par. 186: fingers resting on keyboard) in the image data (Forlines’s par. 232: hand postures measured by camera) satisfy a peripheral use criterion (Forlines’s par. 179, 186: typing state):
processing the peripheral input from the peripheral device (Forlines’s par. 179, 186: typing), and
rejecting the user input action (Forlines’s par. 186: ignoring track input when palm is detected below keyboard, where the track input upon combination includes the orchestra control of par. 228, the drag of a window, interaction with a stylus or movement of a mouse by the hand shaped as a mouse of par. 183) associated with the predetermined input gesture (Forlines’s par. 186 palm resting below keyboard upon combination includes palm input gestures such as closing/opening hand, holding palm up/down or the hand in shape of holding a mouse per par. 183, 228).
Regarding claim 8, Forlines disclose a non-transitory computer readable medium (Forlines’s par. 61: [memory] in MCU, FPGA, CPU, GPU) comprising computer readable code executable by one or more processors (Forlines’s par. 61, 262) to:
perform the steps as explained for claim 1.
Regarding claim 15, Forlines disclose a system (Forlines’s par. 61, 262) comprising:
one or more processors (Forlines’s par. 61, 262); and
one or more computer readable media (Forlines’s par. 61: [memory] in MCU, FPGA, CPU, GPU) comprising computer readable code executable by the one or more processors (Forlines’s par. 61, 262) to: perform the steps as explained for claim 1.
Regarding claims 2, 9 and 16, Forlines disclose wherein the predetermined input gesture (Forlines’s par. 186 palm resting below keyboard upon combination includes palm input gestures such as closing/opening hand, holding palm up/down or the hand in shape of holding a mouse per par. 183, 228) is detected based on hand tracking data from a current frame and one or more prior frames (Forlines’s par. 103-106: object tracking [finger] frame-to-frame).
But Forlines fails to explicitly disclose the hand tracking data from a current frame of the image data and one or more prior frames. However, because Forlines does disclose using cameras to measure hand postures to compute a hand skeleton mode of the user hand for gesture recognition (Forlines’s par. 232), it would also have been obvious to one of ordinary skill in the art, that the hand tracking data (Forlines’s par. 103-106) is also from a current frame of the image data and one or more prior frames (Forlines’s par. 232: based on camera for gesture recognition, and thus using frames of image to perform the hand tracking of par. 103-106), in order to obtain the predictable result of reliable gesture recognition which would enable the gestures described with the keyboard (Forlines’s par. 232).
Regarding claims 3, 10 and 17, Forlines fails to explicitly disclose determining that a relationship between the predetermined input gesture and the peripheral event satisfy a cancellation criterion, wherein the peripheral event is processed further in response to a determination that the cancellation criterion fails to be satisfied.
However, Forlines does disclose a typing state cancellation criterion (Forlines’s par. 186: the condition “hand leaving to the right of the keyboard” which changes the state from typing to mouse, thus cancelling typing state) that is based on a horizontal movement of the right hand leaving the keyboard to the right (Forlines’s par. 186), and discretized states that are triggered based on hand conditions (Forlines’s par. 179).
Therefore, it would also have been obvious to one of ordinary skill in the art that a relationship between the input gesture of an accidental input to the trackpad (Forlines’s par. 186) and the peripheral event of typing when the fingers are resting on the keyboard (Forlines’s par. 186) is the right hand leaving the keyboard to transition to mouse state (Forlines’s par. 186), and that the typing would continue to be processed in response to the transition to a mouse state not being triggered (Forlines’s par. 179, 186), in order to obtain the predictable result of transition of states only when triggered by the hand movement (Forlines’s par. 179, 186).
By doing such combination, Forlines further disclose further comprising:
determining that a relationship (Forlines’s par. 186 hand movement) between the predetermined input gesture (Forlines’s par. 186 palm resting below keyboard upon combination includes palm input gestures such as closing/opening hand, holding palm up/down or the hand in shape of holding a mouse per par. 183, 228) and the peripheral event (Forlines’s par. 179: typing state of the touch sensitive keyboard by fingers on keyboard) satisfy a cancellation criterion (Forlines’s par. 186: the condition “hand leaving to the right of the keyboard” which changes the state from typing to mouse, thus cancelling typing state), wherein
the peripheral event (Forlines’s par. 179: typing state of the touch sensitive keyboard) is processed further (Forlines’s par. 179, 186: typing) in response to a determination that the cancellation criterion (Forlines’s par. 186: hand leaving keyboard to indicate a mouse state) fails to be satisfied (Forlines’s par. 186: when the hand does leave the keyboard, and thus the transition between states is not triggered per par. 179).
Regarding claims 4, 11 and 18, Forlines fails to explicitly disclose wherein the predetermined input gesture is further processed to provide a graphical indication that the predetermined input gesture is detected.
However, Forlines does disclose graphical indications when input gestures are detected (Forlines’s par. 153: holographic visual feedback).
Therefore, it would have been obvious to one of ordinary skill in the art, that Forlines’s predetermined input gesture (Forlines’s par. 228: closing/opening hand, holding palm up/down, par. 183: “grab”, hand in shape of holding mouse) is further processed to provide a graphical indication that the input gesture is detected (Forlines’s par. 153: holographic visual feedback);
In order to obtain the predictable result of visual feedback (Forlines’s par. 153).
Regarding claims 5, 12 and 19, Forlines disclose further comprising:
detecting a hand movement of the first hand (Forlines’s par. 186: horizontal movement of the hand to the right);
determining whether the hand movement satisfies a cancellation threshold (Forlines’s par. 186: the condition “hand leaving to the right of the keyboard” which changes the state from typing to mouse, thus cancelling typing state); and
in accordance with a determination that the hand movement satisfies the cancellation threshold (Forlines’s par. 186: the condition “hand leaving to the right of the keyboard” which changes the state from typing to mouse, thus cancelling typing state), determining that the first hand is in a gesture input mode (Forlines’s par. 179: hover or mouse state).
Regarding claims 6, 13 and 20, Forlines disclose further comprising:
determining a second hand pose for the first hand (Forlines’s Figs. 19 and par. 195-196: finger resting in the center of the K key);
determining, based on the second hand pose (Forlines’s Figs. 19 and par. 195-196: finger resting in the center of the K key), that the first hand is in a peripheral use mode (Forlines’s Figs. 18-19 and par. 194, 196: key is resting on keyboard, thus on typing state);
detecting a second input gesture from the first hand determined to be in the peripheral use mode (Forlines’s Figs. 19 and par. 195-196: rocking of finger over the K key); and
in accordance with a determination that the second input gesture is allowable during the peripheral use mode (Forlines’s Figs. 19 and par. 194-196: rocking of finger over the K key is allowed in typing state which is entered when fingers rest on keyboard), processing a second user input action associated with the second input gesture (Forlines’s par. 196: control direction of cursor based on rocking of finger over the K key).
Regarding claims 7 and 14, Forlines disclose wherein a second hand is determined to be in a gesture input mode (Forlines’s Figs. 16-17 and par. 192-193: e.g. the left hand is in typing because two fingers rest on the CTRL and ALT keys, and the right hand generates touch input events, e.g. swiping right input action).
Response to Arguments
Applicant's arguments filed 6/11/2026 have been fully considered but they are not persuasive.
On the Remarks pg. 8, Applicant argues that Forlines fails to show how the input action associated with, for example, the open/closed hand is rejected based on the peripheral user criterion and that Forlines’s par. 186 only describes that accidental input is mitigated by turning off or ignoring capacitive contact from a palm resting on a laptop trackpad during typing which is distinct from ignoring a detected hand pose.
The office must respectfully disagree, the claims are rejected under 103 over Forlines. As explained in the rejection above, Forlines does not explicitly disclose the hand pose [detected by camera] corresponds to the predetermined input gestures of par. 183, 228, or rejecting the user input action associated with the predetermined input gesture. These two limitations are made obvious by Forlines disclosing that input by the palm resting below the keyboard is ignored when typing (Forlines’s par. 186), and by gestures corresponding to the touch sensitive device enabled to be more reliable by use of a camera or TOF3D sensor (Forlines’s par. 232). Upon combination, Forlines’s ignoring trackpad input (Forlines’s par. 186) includes ignoring input by the hand posing in predetermined input gestures over the trackpad, these input gestures are detected by ToF3D sensors and cameras (Forlines’s par. 232). This combination is reasonable when a touch sensitive device includes both the keyboard and the trackpad (e.g. Figs. 11-12, 15), where both the keyboard and the trackpad are sensitive to proximity (Forlines’s par. 187).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Liliana Cerullo whose telephone number is (571)270-5882. The examiner can normally be reached 8AM to 3PM MT. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amr Awad can be reached at 571-272-7764. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LILIANA CERULLO/ Primary Examiner, Art Unit 2621
Prosecution Insights