DETAILED ACTION
This is a first office action in response to application No. 19/340,547 filed on 09/25/2025, in which claims 1 - 20 are presented for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
1. Claim 1 is objected to because of the following informalities: The claim recites “determining a touch status based on second sensor data,” based on par. [0019] of the specification the limitation read “determining a touch status of the hand based on the second sensor data,”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
2. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
3. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
5. Claims 1 - 20 are rejected under 35 U.S.C. 103 as being unpatentable over GUO “US 2025/0130712” in view of Jamin “WO 2022/200686”.
Re-claim 1, Guo teaches a method (par. [0005] a virtual input element display method) comprising:
determining, based on first sensor data of a first type, a hand location of a hand; (fig. 3 and pars. [0167] the head mounted device 31 collects an environmental image of a surrounding environment by using the binocular camera, obtains depth information of the surrounding environment based on an image photographed by each camera in the binocular camera, [0171] The head mounted device 31 may detect a distance between the hand 32 and the physical surface 33 in real time, and compare the distance and the first distance threshold. Pars. [0139] and [0143])
initiating a monitoring process of a surface of the hand, (par. [0172] when the distance between the hand 32 and the physical surface 33 is less than the first distance threshold, the head mounted device 31 automatically enables the gesture input function)
determining a touch status based on second sensor data, (par. [0081] In a process in which the user enters the information by using the virtual keyboard, the head mounted device continuously photographs a plurality of frames of hand images of the user by using a built-in camera from a top-down angle. Based on the plurality of consecutive frames of hand images, the head mounted device recognizes a finger location by using a gesture recognition algorithm, to recognize a key “pressed in simulation” by the user, and further recognizes an input gesture to obtain input information of the user. And pars. [0220] and [0255] The electronic device detects, by using a gesture recognition algorithm, whether the finger of the user is in contact with a target physical surface, to determine whether gesture input is successful.) and
Guo does not explicitly teach in response to determining that the hand location satisfies a proximity threshold to a physical surface:
initiating a monitoring process of a surface of the hand, wherein the monitoring process comprises capturing second sensor data of a second type different than the first sensor data,
triggering a user input action based on the touch status.
However, Jamin teaches in response to determining that the hand location satisfies a proximity threshold to a physical surface: (page 8; line 33 through page 9; line 9, an apparatus is provided which may detect how an object is reacting haptically (through vibrations and sound, for example), to what hands of a user are doing to it. The apparatus may be able to tell the exact moment the user has touched the object, and/or the moment a trigger on the object has been clicked, and/or whether or not the user's fingers are sliding across a surface of the object or just hovering. Advantageously, this may enable using everyday objects as controllers. The object may not need to comprise electronics or be powered on to produce any feedback, but the haptic feedback may be received from the object inherently in response to contacting the hand of the user.)
initiating a monitoring process of a surface of the hand, wherein the monitoring process comprises capturing second sensor data of a second type different than the first sensor data, (page 20; line 31 through page 21; line 18, however, what may not be possible is for the optical system to detect the user tapping the back of the plate in order to shift gears or to detect the exact moment of the tapping when the fingertip is not visible. The wristband 100 may enable detection of such non-visible taps based on haptic feedback from the plate detected by the at least one sensor of the wristband 100 in response to the taps made by the user's hand wearing the wristband 100. Moreover, the taps may be registered substantially in real-time, i.e. as soon as the fingers touch the plate and the resulting wave/vibration movement travels along the hand to sensors at the wristband. Hence, the wristband 100 provides a simple arrangement to also detect manipulation of objects which may not be easy to detect even with complicated or costly devices. In an embodiment, the wristband 100 may be used in combination with a camera-based hand tracking system to complement the hand tracking and/or make hand inter action detection more accurate. The camera-based hand tracking system may be mounted, for example, to a heads-up display. The heads-up display may refer to any trans parent display that presents data without requiring users to look away from their usual viewpoints.)
triggering a user input action based on the touch status. (page 22; lines 9 - 16, the wristband 100 may be configured to detect the exact moment the user touches and then presses a key down to confirm the key press together with the optical tracking system, such as a camera on the wristband 100. Without the wristband 100, the press events may not be detectable with the optical hand tracking on its own since the hand may occlude the cameras view to the fingers and their actions itself.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to further modify the invention of Guo with the teaching of Jamina plurality of different types of sensors for improved accuracy of the detection.
Re-claim 2, Guo teaches wherein the first sensor data comprises camera data. (par. [0268] The another electronic device may be a controller, may be a portable electronic device such as a mobile phone or a tablet computer, or may be a wearable electronic device such as a smart band or a smartwatch. These electronic devices include vibration motors.)
Re-claim 3, Guo teaches the method of claim 1, further comprising: determining, from the second sensor data, a surface type and a contact state. (pars. [0141] FIG. 3 is a schematic diagram of a VR scenario according to an embodiment of this application. As shown in FIG. 3, in physical space, the scenario may include a head mounted device 31, a hand 32 of a user, and a physical surface 33 (for example, a desktop, where the physical surface may be a physical plane, and the plane is more convenient for typing). [0165] For example, the head mounted device 31 obtains data of an inertia measurement unit (IMU) of the VR controller, or six degrees of freedom (dof) locations of the VR controller. When a variation of the IMU data or the dof data is small, it may be determined that the VR controller is in the static state. Otherwise, it is determined that the VR controller is in the non-static state. and [0166] - [0168])
Re-claim 4, Guo in view of Jamin teaches all the limitations of claim 1, Jamin teaches wherein determining the hand location comprises:
obtaining one or more joint locations for the hand based on hand tracking data; generating a hand geometry based on the one or more joint locations; and determining the hand location based on a depth value for the hand geometry. (page 8; line 20 through page 9; line 1 It is an objective to provide an apparatus comprising a wristband which is able to detect the way a user of the wristband interacts with other objects. This may comprise, for example, detection of touch events of foreign objects, detection of what type of object the user is in contact with (e.g. based on a weight, surface texture, acoustics properties, and the like) and/or detecting how the foreign object reacts to hand manipulation. The wristband may comprise at least one sensor configured to be in contact with skin of a wrist of the user. In an embodiment, the wristband may comprise a plurality of different types of sensors for improved accuracy of the detection. According to an embodiment, an apparatus is provided which may detect how an object is reacting haptically (through vibrations and sound, for example), to what hands of a user are doing to it. The apparatus may be able to tell the exact moment the user has touched the object, and page 18; lines 8 - 13 FIG. 5 illustrates an example of a first type of interaction a wristband 100 is configured to detect wherein an object 502 and a hand 500 are in contact. The hand interaction event may comprise, for example, a touch of an object. In an embodiment, the hand interaction event may further comprise releasing the object. The hand interaction event may comprise location of the touch, e.g. to which part of the object 502 a finger of the user has touched, or which finger touched the object 502.)
Re-claim 5, Guo teaches wherein the monitoring process comprises:
obtaining, by a camera, image data of the hand; (pars. [0165] - [0167])
identifying a target location on a back surface of the hand; (pars. [0168] - [0169])
and
probing and collect the second sensor data from the target location from a non-
camera sensor. (pars. [0170] - [0172])
Re-claim 6, Guo teaches wherein the monitoring process comprises:
obtaining, by a camera, image data of the hand; (par. [0138] When the VR system is specifically the system shown in FIG. 2A, a sensor is integrated into the head mounted display device 22, and is configured to collect hand data of the user. For example, the sensor is an image sensor, and the hand data is hand image data. The host 21 receives the hand data from the head mounted display device 22, processes the hand data to obtain a hand value,)
identifying a first target location on a first finger of the hand; (par. [0141] FIG. 3 is a schematic diagram of a VR scenario according to an embodiment of this application. As shown in FIG. 3, in physical space, the scenario may include a head mounted device 31, a hand 32 of a user, and a physical surface 33 (for example, a desktop, where the physical surface may be a physical plane, and the plane is more convenient for typing)
identifying a second target location on a second finger of the hand; (par. [0269] For example, refer to a schematic diagram of vibration feedback in a VR scenario shown in FIG. 14. Based on the scenario shown in FIG. 3, after the head mounted device 31 displays, based on the size of the hand 32 of the user, the virtual keyboard that adapts to the hand 32, the user touches the target physical surface 33 by using a finger, to simulate to tap the key in the virtual keyboard by using a finger, so as to enter corresponding information.) and
probing and collect the second sensor data from the first target location and the second target location. (par. [0165] For example, the head mounted device 31 obtains data of an inertia measurement unit (IMU) of the VR controller, or six degrees of freedom (dof) locations of the VR controller. When a variation of the IMU data or the dof data is small, it may be determined that the VR controller is in the static state. Otherwise, it is determined that the VR controller is in the non-static state. And pars. [0173] - [0174])
Re-claim 7, Guo teaches the method of claim 6, further comprising:
detecting a multifinger gesture based on the second sensor data; and activating a user input action based on the multifinger gesture. (par. [0269] For example, refer to a schematic diagram of vibration feedback in a VR scenario shown in FIG. 14. Based on the scenario shown in FIG 3, after the head mounted device 31 displays, based on the size of the hand 32 of the user, the virtual keyboard that adapts to the hand 32, the user touches the target physical surface 33 by using a finger, to simulate to tap the key in the virtual keyboard by using a finger, so as to enter corresponding information.)
Re-claim 8, the rejected of claim 1 is incorporated herein and only further limitations are addressed below.
Guo teaches a non-transitory computer readable medium comprising computer readable code executable by the one or more processors to: (pars. [0128] and [0356])
Re-claims 9 - 14, are rejected as applied to claims 2 - 7 above because the scope and contents of the recited limitations are substantially the same.
Re-claim 15, the rejected of claim 1 is incorporated herein and only further limitations are addressed below.
Guo teaches a system comprising: one or more processors; and one or more computer readable media comprising computer readable code executable by the one or more processors to: (pars. [0128] and [0356])
Re-claims 16 - 20, are rejected as applied to claims 3 - 7 above because the scope and contents of the recited limitations are substantially the same.
Contact Information
6. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Sosina Abebe whose telephone number is (571) 270-7929. The examiner can normally be reached on Mon-Friday from 9:00-5:30 If attempts to reach the examiner by telephone are unsuccessful, the examiner's Supervisor, Temesghen Ghebretinsae can be reached on (571) 272-3017. The fax phone number for the organization where this application or proceeding is assigned is 703-872-9306. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/S.A/Examiner, Art Unit 2626
/TEMESGHEN GHEBRETINSAE/ Supervisory Patent Examiner, Art Unit 2626 6/25/26B
Prosecution Insights