DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed February 9, 2026 have been fully considered but they are not persuasive.
A portion of independent claims 1, 8 and 15 duplicated below has been amended as follows:
“recognizing a set of handshape gesture components from a set of defined handshape gesture components including handshape gesture components that are if an exclusive handshape component is recognized, then no other handshape gesture component in the set of defined handshape gesture components should be recognized, and redundant handshape gesture components that intersect in intention or are strictly more specific than another redundant handshape gesture component”
Regarding the limitation “recognizing a set of handshape gesture components from a set of defined handshape gesture components including handshape gesture components that are exclusive such if an exclusive handshape component is recognized, then no other handshape gesture component in the set of defined handshape gesture components should be recognized”, Applicant argues that Bedikian does not disclose this feature. Para. [0052] of the present disclosure is the only portion of the present disclosure that mentions the exclusion and it does not elaborate on this feature. Based on this portion of the present specification, the broadest reasonable interpretation (BRI) for this limitation is that if a handshape gesture among a plurality of predefined handshape gestures is recognized and accepted as being the intended handshape gesture, no other recognized handshape gestures are accepted as the intended gesture, i.e., the accepted intended handshape gesture is accepted to the exclusion of the other recognized, unaccepted, handshape gestures.
First of all, the use of the word “should” in this limitation renders portions of the limitation optional. Nevertheless, Bedikian discloses this feature. In Bedikian, a captured handshape gesture is compared to a set of predefined handshape gestures contained in a library of predefined gesture templates (para. [0014]) to determine which template most closely matches the captured gesture. The most closely matching template is then accepted as the intended gesture to the exclusion of any other recognized gestures that did not match the template as closely. See para. [0054], for example: “[i]n one embodiment, one or more components of trajectory information about a sensed gesture-and potentially other ges-ture primitives-are mathematically compared against the stored trajectories to find potential matches from which a best match (or best matches) may be selected, and the gesture is recognized as corresponding to the located data- base entry based upon qualitative, statistical confidence factors or other quantitative criteria indicating a degree of match. For example, a confidence factor that exceeds a threshold can indicate a potential match.”
Applicant argues that the recognition and acceptance of the dominant gesture in Bedikian over other recognized gestures does not meet this limitation, but Applicant provides no reasoning as to why it does not meet the limitation. The examiner disagrees for the reasons described above. Furthermore, Bedikian also teaches that some gestures are prioritized and accepted as the intended gesture to the exclusion of the other recognized gestures. In Bedikian, a plurality of gestures are recognized and a determination is made as to which of the gestures is the dominant gesture. Based on that determination, the action is performed on the virtual object and the other gestures are ignored, i.e., the dominant gesture is accepted as the intended gesture to the exclusion of the other recognized gestures. (para. [0013] and [0019]). Therefore, for this additional reason Bedikian teaches this limitation.
Applicant further argues that Bedikian does not disclose the newly-added limitation of recognizing “redundant handshape gesture components that intersect in intention or are strictly more specific than another redundant handshape gesture component”. The examiner agrees.
The BRI for “redundant handshape gesture”, based on para. [0052] of the present specification, is a handshape gesture that can have the same meaning as, or a meaning that intersects with the meaning of, a different handshape gesture. Canberk discloses this limitation. Para. [0125] of Canberk discusses several different hand gestures that all correspond to a selecting gesture, and thus they overlap in meaning: “[t]he predefined selecting gesture 810 in some example implementations includes tapping the thumb against a side of an extended finger (or another finger), tapping the thumb near the tip end of the extended finger (or another finger), flexing the thumb, contracting all or part of the hand in a grasping motion, lingering the thumb at a generally stationary location relative to the display….”
Claim Interpretation
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation (BRI) of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification.
The following terms in the claims have been given the following broadest reasonable interpretations (BRIs) in light of the specification:
1. “recognizing a set of handshape gesture components from a set of defined handshape gesture components including handshape gesture components that are exclusive such if an exclusive handshape component is recognized, then no other handshape gesture component in the set of defined handshape gesture components should be recognized”, based on para. [0052] of the present specification, is that when a handshape gesture among a plurality of predefined handshape gestures is recognized and accepted as being the intended handshape gesture, no other recognized handshape gestures are accepted as the intended gesture, i.e., the accepted intended handshape gesture is accepted to the exclusion of the other recognized, unaccepted, handshape gestures.
2. “redundant handshape gesture”, based on para. [0052] of the present specification, is a handshape gesture that can have the same meaning as, or a meaning that intersects with the meaning of, a different handshape gesture.
3. virtual object: in a user interface that is displayed in a two-dimensional (2D) or three-dimensional (3D) rendering, interactive elements that are generated for display to appear as a part of, and/or overlaid upon, the surrounding environment that is also being displayed. The BRI is based on paras. [0002] and [0017] of the present disclosure.
4. augmented reality (AR): augmented reality, virtual reality and any hybrids of these technologies. The BRI is based on para. [0002] of the present disclosure.
5. overlap between a hand of the user and the virtual object: in a three-dimensional (3D) coordinate system, an intersection of the hand of the user and the virtual object in two, but not necessarily three, dimensions of the 3D coordinate system. The BRI is based on paras. [0098]-[0099] of the present disclosure.
6. gesture: a movement made by a user of the AR system moving and positioning portions of the user's body while those portions of the user's body are detectable by the AR system. The BRI is based on para. [0018] of the present disclosure.
7. redisplaying the virtual object based on the operation performed on the virtual object: displaying the virtual object as it would appear as a result of the user’s interaction with the virtual object. The BRI is based on paras. [0108] of the present disclosure.
8. skeletal model of the hand: a model based on landmark features of a hand. The BRI is based on paras. [0046] of the present disclosure.
Should applicant wish different definitions, Applicant should point to the portions of the specification that clearly show a different definition.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 7-11, 14-17 and 20-26 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publ. Appl. No. 2022/0206588 A1 to Canberk, et al. (hereinafter referred to as “Canberk”) in view of U.S. Publ. Appl. No. 2024/0061511 A1 to Bedikian et al. (hereinafter referred to as “Bedikian”) and further in view of U.S. Publ. Appl. No. 2014/0184496 A1 to Gribetz et al. (hereinafter referred to as “Gribetz”).
Regarding claim 1, Canberk discloses a computer-implemented method (para. [0137], Fig. 4) comprising:
displaying (Fig. 2A, displays 180A, 180B, para. [0036]) a virtual object (paras. [0102]-[0104], virtual element is defined as “including any graphical element presented on a display…including a virtual object or graphical icon”), in an Augmented Reality (AR) overlay provided to a user by an AR system (Fig. 13, block 1302, paras. [0102]-[0104]; see also para. [0098]: “The processor 432 of the eyewear device 100 may position a virtual object 608 (such as the key shown in FIG. 6) within the environment 600 for viewing during an augmented reality experience”; see also para. [0105]: “[t]he virtual element 700 is presented as an overlay relative the physical environment 600. The effect, as shown, allows the viewer to see and interact with the virtual element 700 while the surrounding environment 600 also remains visible through the display 180B.”);
capturing, by one or more cameras of the AR system, video frame data of a hand of the user (para. [0023]: “[t]he method includes capturing frames of video data with the camera system);
determining using the video frame data an overlap between a hand of the user and the virtual object (camera system 114 captures an interaction of the user’s hand 621 with the virtual object 712 and one or more processors 432 determine a spatial relationship between the user’s hand and the virtual object, Fig. 7, para. [0108]; see also para. [0128]: “The process of controlling the movement in this example includes determining whether the current thumb position 621 is near the game piece 755 (e.g., whether the thumb is detected within a predefined area on the display relative to the game piece 755)”; para. [0081] discusses that various hand shapes and hand gestures can be used, and therefore is not limited to using the thumb position 621; Canberk does not explicitly disclose determining an overlap between the user’s hand and the virtual object, but only that the thumb position 621 needs to be near the virtual object 755 (para. [0128]));
determining a gesture being made by the user (Fig. 9, hand shapes 801-803; Fig. 13, block 1308 and paras. [0113]-[0115] disclose detecting a series of handshapes in video data 900 and determining whether they match a “predefined series of hand gestures 485 stored in the hand gesture library 480….”) by performing operations comprising:
generating skeletal model data using the video frame data (paras. [0118]-[0123] discuss using machine learning to generate the hand feature model; paras. [0081] and [0122]-[0123] discuss a hand gesture library 480 that was created by the hand feature model; the library 480 contains hand gestures and poses and may include “other information about orientation, along with three-dimensional coordinates for the wrist, the fifteen interphalangeal joints, the five fingertips and other skeletal or soft-tissue landmarks”; because the model of Canberk includes skeletal landmarks of all of these portions of the hand, it constitutes a skeletal model);
recognizing a set of handshape gesture components from a set of defined handshape gesture components including handshape gesture components that are exclusive such if an exclusive handshape component is recognized, then no other handshape gesture component in the set of defined handshape gesture components should be recognized, and redundant handshape gesture components that intersect in intention or are strictly more specific than another redundant handshape gesture component (see the BRIs for this limitation above; paras. [0081] and [0113]-[0115] disclose recognizing handshape gestures by comparing pixels in the captured video frames to handshape gestures stored in the library 480 to determine whether a particular handshape gesture or a particular series of handshape gestures have been recognized; Canberk does not explicitly disclose recognizing a handshape gesture component that is exclusive such that if it is recognized no other handshape gesture component should be recognized; Canberk discloses recognizing redundant handshape gestures in para. [0125]: “[t]he predefined selecting gesture 810 in some example implementations includes tapping the thumb against a side of an extended finger (or another finger), tapping the thumb near the tip end of the extended finger (or another finger), flexing the thumb, contracting all or part of the hand in a grasping motion, lingering the thumb at a generally stationary location relative to the display….”); and
recognizing the gesture using the set of hand shape gesture components (paras. [0081] and [0113]-[0115]);
in response to determining the gesture is a zoom in gesture, increasing a size of the virtual object within the AR overlay (this zoom-in limitation is not explicitly disclosed in Canberk);
in response to determining the gesture is a zoom out gesture, reducing the size of the virtual object within the AR overlay (this zoom-out limitation is not explicitly disclosed in Canberk); and
redisplaying the virtual object (the virtual object is displayed based on the operation performed, as described in para. [0127]: “the virtual element 700 (e.g., the slider 712) moves left and right, in accordance with the associated left-right action”; compare the positions of the slider 712 on the display 180b in Figs. 7 and 8);
As indicated above, Canberk does not explicitly disclose that the user’s hand position must be overlapping the position of the virtual object to interact with the virtual object 755, but only that the thumb position needs to be in a particular predefined spatial relationship relative to the virtual object 755, i.e., it has to be “near” the virtual object: “The process of controlling the movement in this example includes determining whether the current thumb position 621 is near the game piece 755 (e.g., whether the thumb is detected within a predefined area on the display relative to the game piece 755)” (para. [0128]).
Bedikian, in the same field of endeavor, discloses determining, by the one or more processors (Fig. 9B, processor 920, para. [0135]), using one or more cameras (Fig. 1A, motion capture hardware 108, para. [0052]) of the AR system, an overlap between a hand of the user and the virtual object (paras. [0123]-[0126] and Figs. 8A-8B discuss determining whether the control object intersects with the virtual object; see also para. [0009]: “In an embodiment, determining whether motion information defines an engagement gesture can include finding an intersection (also referred to as a contact, pierce, or a "virtual touch") of motion of a control object with a virtual control surface”; the control object in Bedikian can be the user’s hand, as discussed in para. [0050]: “Control objects include, e.g., hands, fingers, feet, or other anatomical parts”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the AR system and algorithms of Canberk based on the teachings of Bedikian such that the determination that is made in Canberk of whether the hand or some portion of the hand is “near” the virtual object would require that the hand or hand portion actually intersect, i.e., overlap, the virtual object as taught by Bedikian. Doing so would better ensure that the AR system of Canberk does not perform an operation on the virtual object that the user did not intend. The modification could have been made by one of ordinary skill in the art before the effective filing data of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results.
Regarding the limitation “recognizing a set of handshape gesture components from a set of defined handshape gesture components including handshape gesture components that are exclusive such if an exclusive handshape component is recognized, then no other handshape gesture component in the set of defined handshape gesture components should be recognized”, as indicated above, Canberk discloses recognizing handshape gestures by comparing pixels in the captured video frames to handshape gestures stored in the library 480 to determine whether a particular handshape gesture or a particular series of handshape gestures have been recognized, but does not explicitly disclose recognizing handshape gesture components that are exclusive such that if an exclusive handshape component is recognized, then no other handshape gesture component should be recognized.
Bedikian discloses recognizing a set of handshape gesture components forming a dominant gesture that is exclusive in that it is accepted as being the intended handshape gesture component over other non-dominant recognized handshape gesture components (para. [0013]: “[s]ome embodiments discern, in real time, a dominant gesture from unrelated movements that may each qualify as a gesture, and may output a signal indicative of the dominant gesture. In various embodiments, the gesture-recognition system identifies a user's dominant gesture when more than one gesture (e.g., an arm-waving gesture and a finger-flexing gesture) is detected. For example, the gesture-recognition system may computationally represent the waving gesture as a waving trajectory and the finger-flexing gestures as five separate (and smaller) trajectories. Each trajectory may be converted into a vector along, for example, six Euler degrees of freedom in Euler space. The vector with the largest magnitude represents the dominant component of the motion (e.g., waving in this case) and the rest of vectors may be ignored.”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the AR system and algorithms of Canberk based on the teachings of Bedikian such that a dominant hand gesture component is recognized and accepted as the intended gesture to the exclusion of other recognized handshape gesture components as taught by Bedikian. A person of ordinary skill in the art would have been motivated to make the modification to ensure that some gestures, such as those with higher priority or importance, are accepted and acted upon over other recognized gestures, such as those of lower priority or importance. The modification could have been made by one of ordinary skill in the art before the effective filing data of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (modifying software to prioritize certain handshape gestures over others).
Regarding the limitation “in response to determining the gesture is a zoom in gesture, increasing a size of the virtual object within the AR overlay”, Canberk does not explicitly disclose a zoom in gesture. Bedikian discloses that the gesture is a zoom in gesture and the operation performed on the virtual object is an increase in a size of the virtual object (para. [0147], Figs. 11A and 11B: “FIGS. 11A and 11B illustrate a zooming action performed by two fingers (thumb and index finger) accord-ing to various embodiments.”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the AR system and algorithms of Canberk based on the teachings of Bedikian such that the processor 432 of Canberk determines when a hand gesture should be recognized as a zoom-in operation as taught by Bedikian, and in response, the one or more processors 432 increase the size of the virtual object within the AR overlay. Canberk teaches a variety of hand and finger gestures that are recognized by the AR system. A person of ordinary skill in the art would have been motivated to make the modification to provide the AR system of Canberk with the capability of performing an additional useful operation for users. The modification could have been made by one of ordinary skill in the art before the effective filing data of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (i.e., training the AR system of Canberk to recognize a zoom-in gesture and including it in the gesture library 480 of Canberk).
Regarding the limitation “in response to determining the gesture is a zoom out gesture, reducing the size of the virtual object within the AR overlay”, neither Canberk nor Bedikian explicitly discloses this limitation. Gribetz, in the same field of endeavor, discloses an AR system (Fig. 1A, sensing and display apparatus 1300) that includes a zoom-out gesture that results in an operation being performed that reduces the size of the virtual object (para. [0988]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the AR system and algorithms of Canberk based on the teachings of Gribetz such that the processor 432 of Canberk determines when a hand gesture should be recognized as a zoom-out operation as taught by Gribetz, and in response, reduces the size of the virtual object within the AR overlay. A person of ordinary skill in the art would have been motivated to make the modification to to provide the AR system of Canberk with the capability for users of performing an additional useful operation. The modification could have been made by one of ordinary skill in the art before the effective filing data of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (i.e., training the AR system of Canberk, as modified by Bedikian, to recognize a zoom-out gesture as taught by Gribetz and including it in the gesture library 480 of Canberk).
Regarding claim 2, Canberk discloses that the virtual object is not interacted with by the user unless the user’s hand or hand portion is detected as being “near” the virtual object (Canberk, para. [0128]). This indicates that the processor 432 makes a determination of whether the hand or hand portion of the user is sufficiently near the virtual object and ends interaction by the user with the virtual object when the hand or hand portion is no longer sufficiently near the virtual object. However, Canberk does not explicitly state that interaction is ended when the hand of the user is no longer near the virtual object
Bedikian discloses determining, using video frame data (para. [0074]), that the hand of the user no longer overlaps the virtual object (para. [0126] discusses determining a “dis-intersection or a non-intersec-tion of the control object with the virtual control construct”; see also Fig. 8B-1, block 826) and based on determining that the hand of the user no longer overlaps the virtual object, ending, by the one or more processors, interaction of the user with the virtual object (Bedikian, para. [0126], the processor only performs an operation on the virtual object when it determines that there is engagement, i.e., intersection, of the hand of the user with the virtual object and ends interaction of the user with the virtual object when there is no intersection between the user’s hand and the virtual object; see also para. [0107]: “When the control object 504 (e.g., as shown, the user's index finger) ‘touches’ or ‘pierces’ the virtual plane (i.e., when its spatial location coincides with, inter-sects, or moves beyond the virtual plane's computationally defined spatial location), the cursor 506 and/or machine interface operates in the engaged mode (FIG. 5B); other-wise, the cursor and/or machine interface operates in the disengaged mode (FIG. 5A).”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the AR system and algorithms of Canberk based on the teachings of Bedikian such that the processor 432 of Canberk ends interaction by the user when it determines that the user’s hand no longer intersects the virtual object, as taught by Bedikian A person of ordinary skill in the art would have been motivated to make the modification to better ensure that the AR system of Canberk does not perform an operation on the virtual object that the user did not intend. The modification could have been made by one of ordinary skill in the art before the effective filing data of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (modifying software to cause the system to operate on the virtual object only when intersection/overlap has been detected).
Regarding the claim 3 limitation of generating a skeletal model of the hand of the user based on the tracking video frame data, Canberk et al discloses generating a hand feature model of the user based on the tracking video frame data (paras. [0118]-[0123] discuss using machine learning to generate the hand feature model; in paras. [0081] and [0122]-[0123], Canberk discusses a hand gesture library 480 that was created by the hand feature model. The library 480 contains hand gestures and poses and may include “other information about orientation, along with three-dimensional coordinates for the wrist, the fifteen interphalangeal joints, the five fingertips and other skeletal or soft-tissue landmarks.”). Because the model of Canberk includes skeletal landmarks of all of these portions of the hand, it constitutes a skeletal model.
Regarding the claim 3 limitation of determining the overlap between the hand of the user and the virtual object based on the skeletal model and 3D coordinate data of the virtual object, as indicated above, Canberk does not explicitly disclose that the user’s hand position has to be overlapping the position of the virtual object to interact with the virtual object 755, but only that the hand position needs to be in a particular predefined spatial relationship (i.e., near) relative to the virtual object 755 to interact with the virtual model. Canberk discloses that the skeletal model is based on 3D coordinate data (paras. [0081], [0115] and [0123]) and that the user’s hand is detected in 3D coordinates (para. [0115]).
Bedikian discloses determining the overlap between the hand of the user and the virtual object based on the 3D coordinate data of the virtual object. Para. [0116] and Fig. 6 show and describe the hand of the user and the virtual object being in the same 3D coordinate system. Para. [0070] discusses the gesture recognition module 116 converting the recognized gestures into at least a 3D coordinate system (x, y, z, roll, pitch and yaw). Para. [0099] describes an intersection occurring between the control object (e.g., the index finger of the user’s hand) and the virtual object when they have at least one point in 3D space in common.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the AR system and algorithms of Canberk based on the teachings of Bedikian such that the processor 432 of Canberk determines the overlap between the hand of the user and the virtual object based on the skeletal model of Canberk and based on 3D coordinate data of the virtual object as taught by Bedikian A person of ordinary skill in the art would have been motivated to make the modification to ensure that the AR system of Canberk does not perform an operation on the virtual object unless there is an intersection between the user’s hand and the virtual object at a common point in 3D space. This, in turn, would help ensure that operations by the user on the virtual object would not occur unless they are intended by the user, such as when the user accidentally places a hand near the virtual object without intending to interact with it. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (modifying software to cause the system to operate on the virtual object only when intersection/overlap has been detected).
Regarding claim 4, the rejection of claim 3 above applies mutatis mutandis to the corresponding limitations of claim 4. Regarding the claim 4 limitation of determining the overlap between the hand of the user and the virtual object based on video frame data using one or more computer vision methodologies, Canberk discloses using machine vision for tracking video frame data of the user’s hand and the virtual object (paras. [0030]-[0031], [0118]).
Regarding claim 7, Canberk discloses that the AR system comprises a head-worn device (Figs. 1A-2B; para. [0034]).
Regarding claim 8, the rejection of claim 1 above applies mutatis mutandis to the corresponding limitations of claim 8. The only elements that claim 8 recites that are not recited in claim 1 are the computing apparatus, one or more processors of the computing apparatus and a memory of the computing apparatus. Regarding the limitation of the computing apparatus, Canberk discloses a computing apparatus (Fig. 4, 100) comprising one or more processors (Fig. 4, 432) and a memory (Fig. 4, 434) that stores instructions for execution by the processor 432 (para. [0069]).
Regarding claim 9, the rejection of claim 2 above applies mutatis mutandis to the corresponding limitations of claim 9.
Regarding claim 10, the rejection of claim 3 above applies mutatis mutandis to the corresponding limitations of claim 10.
Regarding claim 11, the rejection of claim 4 above applies mutatis mutandis to the corresponding limitations of claim 11.
Regarding claim 14, the rejection of claim 7 above applies mutatis mutandis to the corresponding limitations of claim 14.
Regarding claim 15, the rejection of claim 1 above applies mutatis mutandis to the corresponding limitations of claim 15. The only elements that claim 15 recites that are not recited in claim 1 are the non-transitory computer-readable storage medium including instruc-tions that when executed by a computer, cause the computer to perform the steps recited in claim 15, which are also recited in claim 1. Canberk discloses a non-transitory computer-readable storage medium (Fig. 4, memory 434) including instruc-tions for execution by a computer (processor 432; para. [0069]).
Regarding claim 16, the rejection of claim 2 above applies mutatis mutandis to the corresponding limitations of claim 16.
Regarding claim 17, the rejection of claim 3 above applies mutatis mutandis to the corresponding limitations of claim 17.
Regarding claim 20, the rejection of claim 7 above applies mutatis mutandis to the corresponding limitations of claim 20.
Regarding claim 21, as indicated above in the rejection of claim 3, Canberk discloses generating a hand feature model of the user based on the tracking video frame data (paras. [0118]-[0123] discuss using machine learning to generate the hand feature model; in paras. [0081] and [0122]-[0123], Canberk discusses a hand gesture library 480 that was created by the hand feature model. The library 480 contains hand gestures and poses and may include “other information about orientation, along with three-dimensional coordinates for the wrist, the fifteen interphalangeal joints, the five fingertips and other skeletal or soft-tissue landmarks.”) Because the model includes skeletal landmarks of all of these portions of the hand, it constitutes a skeletal model.
Regarding the limitation “determining the gesture comprises recognizing derived continuous gesture components composed of derived continuous features extracted from skeletal model data at multiple timestamps to form a continuous stream of data”, Canberk discloses that the gestures comprise continuous features extracted from the model (paras. [0121]-[0122]) at multiple timestamps to form a continuous stream of data (paras. [0002], [0030] and [0031]: “real-time tracking of hand gestures and micro-scale movements” and “[t]he term ‘gesture’ refers to the active movement of an object, such as a hand, through a series of poses). Regarding the limitation “at multiple timestamps”, the BRI for the term timestamp, based on the plain meaning as understood by those of ordinary skill in the art before the effective filing date, is that it means a digital record of when an event occurred. Because Canberk discloses that the hand gestures and the corresponding micro-scale movements are tracked in real time, this means that the timing of these movements is recorded, which means that the movements are timestamped.
Regarding claim 22, Canberk does not explicitly discuss that recognizing the derived continuous gesture components includes applying a specified level of smoothing to the derived continuous features. Bedikian discloses applying a smoothing filter to derived continuous gesture components (Col. 11, line 28-Col. 13, line 21).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the AR system and algorithms of Canberk based on the teachings of Bedikian such that the processor 432 of Canberk applies a specific level of smoothing to the derived continuous feature during the gesture recognition process as taught by Bedikian. A person of ordinary skill in the art would have been motivated to make the modification to filter out anomalies in the data and shakiness of the user’s fingers as taught by Bedikian. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (implementing known smoothing filter with the gesture recognition algorithm as taught by Bedikian to acquired data).
Regarding claim 23, the rejection of claim 21 applies mutatis mutandis to claim 23.
Regarding claim 24, the rejection of claim 22 applies mutatis mutandis to claim 24.
Regarding claim 25, the rejection of claim 21 applies mutatis mutandis to claim 25.
Regarding claim 26, the rejection of claim 22 applies mutatis mutandis to claim 26.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL J SANTOS whose telephone number is (571)272-2867. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Bella can be reached on (571)272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL J. SANTOS/ Examiner, Art Unit 2667
/MATTHEW C BELLA/ Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights