Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to amendment filed 03/30/2026 in which the claims 1-2, 4-20 are pending.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 11/26/2025, 03/16/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Double Patenting
Terminal disclaimer approved 05/04/2025 overcomes the non-statutory double patenting rejection of U.S. Patent No. US 11,546,505 B2.
Response to Arguments
Applicant's arguments filed 03/30/2026 have been fully considered but they are not persuasive.
Overview of the Claim Amendments
Independent claim 1 has been amended to delete the "defining" step because this
limitation is recited in the preceding claim phrase. Claim 1 has also been amended to recite:
"generating an imaginary plane relative to the shutter gesture location, wherein
the imaginary plane is oriented according to the shutter gesture;
"estimating an eyewear device location relative to one or more of the imaginary plane or the shutter gesture location; ...."
Similar amendments have been made to independent claims 10 and 15.
Dependent claims 4, 12, and 17 have been amended to conform to the amendments made to the independent claims.
Support for these amendments can be found in the claims, in the drawings (especially Figures 7 and 8), and in the written description (especially at paragraphs [0124] through [0127] and in the other paragraphs describing Figures 7 and 8).
No new matter has been added to the claims.
With respect to all amendments, the Applicant has not dedicated or abandoned any unclaimed subject matter. Moreover, the Applicant has not acquiesced to any characterizations of the invention, any objections, or any rejections of the claims made by the Examiner.
ARGUMENT
A primafacie case of obviousness requires that the cited prior art references teach or suggest all the features of the claimed invention. MPEP § 2142; In re Vaeck, 947 F.2d 488, 20 USPQ2d 1438 (Fed. Cir. 1991). "The key to supporting any rejection under 35 U.S.C. § 103 is the clear articulation of the reason(s) why the claimed invention would have been obvious." MPEP § 2141; see also, KSR Int'l Co. v. Teleflex Inc., 550 U.S. 398, 418 (2007), quoting In re Kahn, 441 F.3d 977, 988, 78 USPQ2d 1329, 1336 (Fed. Cir. 2006) ("there must be some articulated reasoning with some rational underpinning to support the legal conclusion of obviousness").
I. The proposed combinations of Prada Gomez with Ravasz and Alvarez(or with Bradski) do not teach or suggest "generating an imaginaryplane ..." as recited in the claims, as amended.
The cited references do not teach or suggest "generating an imaginary plane relative to the shutter gesture location, wherein the imaginary plane is oriented according to the shutter gesture; ...."
Prada Gomez at column 17 describes identifying "a triggering object" but does not
describe "detecting ... a shutter gesture location ...." The rejection at page 12 relies on Ravasz for 'defining the shutter gesture location.' However, the interaction location described in Ravasz is fundamentally tied to an anatomical anchor (e.g., a "representation of a wrist" at column 16). In Ravasz, the user interface is "overlaid on the wrist" which means the interaction location is limited to an anatomical boundary.
In contrast, claim 1, as amended, recites "generating an imaginary plane ..." which is a virtual geometric construct - not an anatomical boundary (like the wrist in Ravasz). A person of ordinary skill would recognize that overlaying a user interface onto a virtual wrist is technically distinct from generating an imaginary plane that is generated independently of any specific location, such as an anatomical anchor.
Examiner respectfully disagrees and clarifies that in response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, Ravasz clearly discloses the amended claim language “ generating an imaginary plane relative to the shutter gesture location, wherein the imaginary plane is oriented according to the shutter gesture; estimating an eyewear device location relative to one or more of the imaginary plane or the shutter gesture location” as described in detail below.
Ravasz discloses the artificial reality system may further trigger generation and rendering of the graphical user interface elements in response to detection of particular gestures in combination with other conditions, such as the position and orientation of the particular gestures in a physical environment relative to a current field of view of the user, which may be determined by real-time gaze tracking of the user, or relative to a pose of an HMD worn by the user. Col 6 lines 27-41 teaches Artificial reality system 10 may trigger generation and rendering of graphical user interface elements 124, 126 in response to other conditions, such as a current state of one or more applications being executed by the system, or the position and orientation of the particular detected gestures in a physical environment in relation to a current field of view 130 of user 110, as may be determined by real-time gaze tracking of the user, or other conditions. More specifically, as further described herein, image capture devices 138 of HMD 112 capture image data representative of objects in the real world, physical environment that are within a field of view 130 of image capture devices 138. Field of view 130 typically corresponds with the viewing perspective of HMD 112.
It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of involves determining that a particular hand is associated with a wearable head-mounted display (HMD) via analysis of video data including hand gesture an image is generated from video data of a sub-FOV within the closed area of Prada Gomez with the artificial reality system comprises an image capture device, which is provided to capture image data representative of a physical environment. A head-mounted display (HMD) is provided to output artificial reality content of Ravasz with the method of receive an image including a gesture made by a user from the capture device and analyze the image to identify the gesture made by the user in the image. The processor determines action to be performed on the image based on the identified gesture and determines a selection area for the gesture. The processor identifies an area-of-interest in the image based on the selection area of the gesture and performs action on the area-of-interest. Alvarez in order to provide a system in which system performs operations on a smaller region, thus efficiently and potentially reducing power consumption and increasing speed of the operations.
Further with respect to applicants argument that “The rejection at page 12 relies on Ravasz for 'defining the shutter gesture location.' However, the interaction location described in Ravasz is fundamentally tied to an anatomical anchor (e.g., a "representation of a wrist" at column 16). In Ravasz, the user interface is "overlaid on the wrist" which means the interaction location is limited to an anatomical boundary. Examiner clarifies that Ravasz discloses in col 4 lines 45-51 that in some examples, artificial reality system 10 may detect a predefined gesture based on additional conditions being satisfied, such as the position and orientation of portions of arm 134 (e.g., a wrist) and/or hand 132 (or digits thereof) in a physical environment in relation to a current field of view 130 of user 110, as may be determined by real-time gaze tracking of the user, or other conditions.
Col 6 lines 42-57 further discloses the artificial reality application may present a real-world image of hand 132 and/or arm 134 of user 110 within artificial reality content 122 comprising mixed reality, augmented reality, and/or any other combination of information directly reproducing a physical environment with computer-mediated content. In either example, user 110 is able to view the portions of his/her hand 132 and/or arm 134 that are within field of view 130 as objects within the virtual environment represented by artificial reality content 122. Col 6 line 58- col 7 line 5 During operation, artificial reality system 10 performs object recognition within image data captured by image capture devices 138 of HMD 112 (and/or by external cameras 102) to identify hand 132, including optionally identifying individual fingers or the thumb, and/or all or portions of arm 134 of user 110. Further, artificial reality system 10 tracks the position, orientation, and configuration of hand 132 (optionally including particular digits of the hand) and/or portions of arm 134 over a sliding window of time. The artificial reality application analyzes any tracked motions, configurations, positions, and/or orientations of hand 132 and/or portions of arm 134 to identify one or more gestures performed by particular objects, e.g., hand 132 (including but not limited to one or more particular digits of hand 132) and/or portions of arm 134 (or specific portions thereof, such as a wrist) of user 110.
Col 7 lines 7-29 teaches to detect the gesture(s), the artificial reality application may compare the motions, configurations, positions and/or orientations of hand 132 and/or portions of arm 134 to gesture definitions stored in a gesture library of artificial reality system 10, where each gesture in the gesture library may be each mapped to one or more actions. In some examples, detecting movement may include tracking positions of one or more of the digits (individual fingers and thumb) of hand 132, including whether any of a defined combination of the digits (such as an index finger and thumb) are brought together to touch or approximately touch in the physical environment, or to bookend or encircle a user interface element (e.g., an assistant element or a display element) presented as part of artificial reality content 122. In other examples, detecting movement may include tracking an orientation of hand 132 (e.g., fingers pointing toward HMD 112 or away from HMD 112) and/or an orientation of arm 134 (i.e., the normal of the arm facing toward HMD 112) relative to the current pose of HMD 112. The position and orientation of the respective portion or entirety of hand 132 or arm 134 thereof may alternatively be referred to as the pose of hand 132 or arm 134, or a configuration of hand 132 or arm 134.
Thus Ravasz interaction location described in Ravasz is not fundamentally tied to an anatomical anchor (e.g., a "representation of a wrist" at column 16) and nor the user interface is just "overlaid on the wrist" which means the interaction location is limited to an anatomical boundary, as argued by applicant, instead Ravasz also has describes that detecting movement may include tracking positions of one or more of the digits (individual fingers and thumb) of hand 132, including whether any of a defined combination of the digits (such as an index finger and thumb) are brought together to touch or approximately touch in the physical environment, or to bookend or encircle a user interface element (e.g., an assistant element or a display element) presented as part of artificial reality content 122) and To detect the gesture(s), the artificial reality application may compare the motions, configurations, positions and/or orientations of hand 132 and/or portions of arm 134 to gesture definitions stored in a gesture library of artificial reality system 10, where each gesture in the gesture library may be each mapped to one or more actions. In some examples, detecting movement may include tracking positions of one or more of the digits (individual fingers and thumb) of hand 132, including whether any of a defined combination of the digits (such as an index finger and thumb) are brought together to touch or approximately touch in the physical environment, or to bookend or encircle a user interface element (e.g., an assistant element or a display element) presented as part of artificial reality content 122.
Applicant further argues that Moreover, the cited references lack the 3D spatial orientation logic that would be required to generate an imaginary plane. Prada Gomez describes identifying "triggering objects" in a 2D video frame and matching the object to a library of shapes. Ravasz describes identifying12 2D hand shapes. Neither reference describes generating an imaginary plane relative to a 3D position in physical space (i.e., the shutter gesture location).
The rejection fails to explain why a person of ordinary skill, starting with the HMD and triggering-object-recognition system of Prada Gomez would look to the wrist-anchored "pulling" interface of Ravasz, and then further seek out the 2D "area of interest" logic in Alvarez. Without the benefit of the applicant's disclosure, there is no reason to conclude that a person of ordinary skill would find these references compatible. "Knowledge of the applicant's disclosure must be put aside .... Impermissible hindsight must be avoided and the legal conclusion must be reached on the basis of the facts gleaned from the prior art." MPEP 2142 (italics added).
The proposed combination of Prada Gomez with Ravasz and Alvarez, even if attempted, would not include "generating an imaginary plane ..." as recited in the claims, as amended. The resulting combination, at best, would include the processes of detecting triggering objects (Prada Gomez), presenting a wrist-anchored user interface (Ravasz), and identifying a smaller area of interest that may not include interference (Alvarez). Such a system would still lack the functionality of "generating an imaginary plane relative to the shutter gesture location, wherein the imaginary plane is oriented according to the shutter gesture; ...."
In addition, the objective in Ravasz of presenting a stable, wrist-anchored user interface is functionally incompatible with a system that generates a free-floating imaginary plane relative to the detected location of a shutter gesture. A person of ordinary skill would be discouraged from making the proposed combination because the wrist-anchored user interface in Ravasz would frustrate the process of generating and using an imaginary plane. The proposed combination cannot technically arrive at the elements recited in the claims without significant and unguided technical changes that are not suggested in any of the references.
For at least these reasons, the proposed combination of Prada Gomez with Ravasz and Alvarez does not teach or suggest all the limitations recited in independent claims 1, 10, and 15, as amended. The dependent claims 2, 4-9, 11-14, and 16-20 are not obvious for the same reasons the independent claims are not obvious, and because the dependent claims recited additional and non-obvious features. Even if a person of ordinary skill were motivated to attempt a combination that included Bradski, the general descriptions of object tracking in Bradski would not cure the deficiencies noted above.
Moreover, the piecemeal addition of Bradski to Prada Gomez, Ravasz, and Alvarez suggests that the proposed combination was arrived at using impermissible hindsight. "Combining prior art references without evidence of such a suggestion, teaching, or motivation simply takes the inventor's disclosure as a blueprint for piecing together the prior art to defeat patentability - the essence of hindsight." In re Dembiczak, 175 F.3d 994, 999 (Fed. Cir. 1999).
Examiner respectfully diagress and clarifies that In response to applicant's argument that the examiner's conclusion of obviousness is based upon improper hindsight reasoning, it must be recognized that any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971).
Examiner further clarifies that Prada Gomez discloses provide an augmented reality or heads-up display where the projected image or graphic can be superimposed over or provided in conjunction with a real-world view as perceived by the user through the lens elements. col 8 lines 54-60 teaches video camera 120 may be forward facing to capture at least a portion of a real-world view perceived by the user. This forward facing image captured by the video camera 120 may then be used to generate an augmented reality where computer generated images appear to interact with the real-world view perceived by the user.
Ravasz discloses artificial reality system 10 uses information captured from a real-world, three-dimensional (3D) physical environment to render artificial reality content 122 for display to user 110. Using HMD 112 as a frame of reference, and based on a current field of view 130 as determined by a current estimated pose of HMD 112, the artificial reality application renders 3D artificial reality content which, in some examples, may be overlaid, at least in part, upon the real-world, 3D physical environment of user 110. During this process, the artificial reality application uses sensed data received from HMD 112, such as movement information and user commands, and, in some examples, data from any external sensors 90, such as external cameras 102A and/or 102B, to capture 3D information within the real world, physical environment, such as motion by user 110 and/or feature tracking information with respect to user 110. Based on the sensed data, the artificial reality application determines a current pose for the frame of reference of HMD 112 and, in accordance with the current pose, renders the artificial reality content 122.
col 12 lines 50-56 teaches Artificial reality system 20 uses data received from cameras 102, HMDs 112, and controllers 114 to capture 3D information within the real-world environment, such as motion by users 110 and/or tracking information with respect to users 110 and objects 108, for use in computing updated pose information for a corresponding frame of reference of HMDs 112col 17 lines 63-66 teaches responsive to control by application engine 320, rendering engine 322 generates 3D artificial reality content for display to the user by application engine 340 of HMD 112.Col 18 lines 1-21 teaches Based on the current viewing perspective, rendering engine 322 constructs the 3D, artificial reality content which may in some cases be overlaid, at least in part, upon the real-world 3D environment of user 110. During this process, pose tracker 326 operates on sensed data received from HMD 112, such as movement information and user commands, and, in some examples, data from any external sensors 90 (FIGS. 1A, 1B), such as external cameras, to capture 3D information within the real-world environment, such as motion by user 110 and/or feature tracking information with respect to user 110. Based on the sensed data, pose tracker 326 determines a current pose for the frame of reference of HMD 112 and, in accordance with the current pose, constructs the artificial reality content for communication, via the one or more I/O interfaces 315, to HMD 112 for display to user 110.
Thus Ravasz discloses generating an imaginary plane relative to a 3D position in physical space (i.e., the shutter gesture location).
Alvarez further describes in Para[0032] teaches The processor may detect gestures using conventional image analysis techniques, motion data, and/or depth of field measurements. [0034] Because a user gesture may likely found in the foreground of an image, the processor may utilize depth of field measurements to detect the presence of a foreground gesture. For example, in some embodiments, the processor may receive stereoscopic data based on images from a plurality of cameras. The processor may then utilize the stereoscopic images to calculate near-field objects in the image or video. In some embodiments, the mere presence of a near-field object may be used to determine the presence of a gesture. In another example, the processor may receive depth data from depth sensors, such as infrared or ultrasound sensors. The sensor(s) may generate a map indicating the distance from the camera to various objects in the image. When there are points in a particular plane of depth that form a hand-shape, the processor may determine that a gesture is present in the image. Para[0055] teaches The processor may also use stereoscopic images to calculate the depth of image coordinates. For motion gestures, the processor may utilize the “real world” coordinates to stitch together images. For example, the processor may use traditional map projection techniques, such as rectilinear projection. FIGS. 2A-2C illustrate example gestures that define corresponding selection areas.
Thus the cited references does not lack the 3D spatial orientation logic that would be required to generate an imaginary plane. For at least these reasons, the proposed combination of Prada Gomez with Ravasz and Alvarez teaches all the limitations recited in independent claims 1, 10, and 15, as amended.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
9. Claims 1-2, 4, 6-8, 10, 12, 15, 17, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Prada Gomez et al. (US 9,024,842 B1) in view of Ravasz et al. (US 11,003,307 B1) and Alvarez et al. (US 2016/0203360 A1)
Regarding claim 1, Prada Gomez discloses a method of capturing a still image using an eyewear device, the eyewear device (FIG. 1a, the eyeglasses 102 ) comprising a camera system (a video camera 120), an image processing system, and a display, (the display 220 could correspond to an image processing and display system for making images viewable to a user (wearer) of an HMD) the method comprising: capturing frames of video data of a physical environment with the camera system (col 17 lines 25-35 teaches HMD could capture video data from the FOV of the video camera, and then identify a triggering object in the captured data); detecting a shutter gesture (col 15 lines 56-col 16 line 22 teaches hand gestures can be used to provide user input to a wearable HMD col 16 lines 25-65 teaches a. Detection of Hand Gestures & fig. 5 HMD can recognize that a hand gesture is being or has been made);
Prada Gomez does not explicitly disclose detecting a shutter gesture and a shutter gesture location in a first frame of the captured frames of video data with the image processing system, wherein the shutter gesture location is defined relative to the physical environment; generating an imaginary plane relative to the shutter gesture location, wherein the imaginary plane is oriented according to the shutter gesture; estimating an eyewear device location relative to one or more of the imaginary plane or the shutter gesture location; and presenting an indicator using the eyewear device, wherein the indicator comprises a brief alert selected from the group consisting of a simulated shutter effect presented on the display, a flash presented on the display, a rectangular shape presented on the display, and a sound emitted from a loud speaker; identifying a captured still image among a subset of the captured frames of video data excluding the first frame; presenting on the display the captured still image at a display location, wherein the display location is correlated with the shutter gesture location and the eyewear device location.
However Ravasz discloses detecting a shutter gesture and a shutter gesture location in a first frame of the captured frames of video data with the image processing system, wherein the shutter gesture location is defined relative to the physical environment ( Abstract & Col 2 lines 23-40 teaches image capture device is configured to capture image data representative of a physical environment, gesture detector configured to identify, from the image data, a gesture. col 7 lines 7- 20 teaches to detect the gesture(s), the artificial reality application may compare the motions, configurations, positions and/or orientations of hand 132 and/or portions of arm 134 to gesture definitions stored in a gesture library of artificial reality system 10, where each gesture in the gesture library may be each mapped to one or more actions, including whether any of a defined combination of the digits (such as an index finger and thumb) are brought together to touch or approximately touch in the physical environment); generating an imaginary plane relative to the shutter gesture location, wherein the imaginary plane is oriented according to the shutter gesture (Col 6 lines 27-41 teaches Artificial reality system 10 may trigger generation and rendering of graphical user interface elements 124, 126 in response to other conditions, such as a current state of one or more applications being executed by the system, or the position and orientation of the particular detected gestures in a physical environment in relation to a current field of view 130 of user 110, as may be determined by real-time gaze tracking of the user, or other conditions); estimating an eyewear device location relative to one or more of the imaginary plane or the shutter gesture location (col 6 lines 27-41 teaches the artificial reality system may further trigger generation and rendering of the graphical user interface elements in response to detection of particular gestures in combination with other conditions, such as the position and orientation of the particular gestures in a physical environment relative to a current field of view of the user, which may be determined by real-time gaze tracking of the user, or relative to a pose of an HMD worn by the user, col 4 lines 45-50 teaches artificial reality system 10 may detect a predefined gesture based on additional conditions being satisfied, such as the position and orientation of portions of arm 134 (e.g., a wrist) and/or hand 132 (or digits thereof) in a physical environment in relation to a current field of view 130 of user 110, as may be determined by real-time gaze tracking of the user, or other conditions Col 16 lines 34-38 teaches control unit 210 may detect certain predefined gestures performed at locations generally corresponding to the location of the display element to gate UI elements within artificial reality content 122 displayed via HMD 112); and presenting an indicator using the eyewear device, wherein the indicator comprises a brief alert selected from the group consisting of a simulated shutter effect presented on the display, a flash presented on the display, a rectangular shape presented on the display, and a sound emitted from a loud speaker (Col 6 lines 13-26 & Fig. 1A teaches user interface elements may, for example, include, be, or be part of a graphical user interface, such as a menu or sub-menu with which user 110 interacts to operate the artificial reality system, or individual user interface elements selectable and manipulatable by user 110, such as toggle elements, drop-down elements, menu selection elements, two-dimensional or three-dimensional shapes, graphical input keys or keyboards, content display windows and the like. While depicted as a two-dimensional element, for example, UI element 126 may be a two-dimensional or three-dimensional shape that is manipulatable by user 110 by performing gestures to translate, scale, and/or rotate the shape within the virtual environment represented by artificial reality content 122. col 6 lines 27-40 teaches artificial reality system 10 may trigger generation and rendering of graphical user interface elements 124, 126 in response to other conditions, such as a current state of one or more applications being executed by the system, or the position and orientation of the particular detected gestures in a physical environment in relation to a current field of view 130 of user 110, as may be determined by real-time gaze tracking of the user, or other conditions. More specifically, image capture devices 138 of HMD 112 capture image data representative of objects in the real world, physical environment that are within a field of view 130 of image capture devices 138. Field of view 130 typically corresponds with the viewing perspective of HMD 112); presenting on the display the captured still image at a display location, wherein the display location is correlated with the shutter gesture location and the eyewear device location (Col 5 lines 19-22 teaches artificial reality system 10 uses information captured from a real-world, three-dimensional (3D) physical environment to render artificial reality content 122 for display to user 110. Col 5 lines 36-55 teaches during operation, the artificial reality application constructs artificial reality content 122 for display to user 110 by tracking and computing pose information for a frame of reference, typically a viewing perspective of HMD 112. Using HMD 112 as a frame of reference, and based on a current field of view 130 as determined by a current estimated pose of HMD 112, the artificial reality application renders 3D artificial reality content which, in some examples, may be overlaid, at least in part, upon the real-world, 3D physical environment of user 110. During this process, the artificial reality application uses sensed data received from HMD 112, such as movement information and user commands, and, in some examples, data from any external sensors 90, such as external cameras 102A and/or 102B, to capture 3D information within the real world, physical environment, such as motion by user 110 and/or feature tracking information with respect to user 110. Based on the sensed data, the artificial reality application determines a current pose for the frame of reference of HMD 112 and, in accordance with the current pose, renders the artificial reality content 122). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of involves determining that a particular hand is associated with a wearable head-mounted display (HMD) via analysis of video data including hand gesture an image is generated from video data of a sub-FOV within the closed area of Prada Gomez with the method which involves detection of particular position and orientation of the particular detected gestures in a physical environment in relation to a current field of view of user which may be determined by real-time gaze tracking of the user, or relative to a pose of an HMD worn by the user of Ravasz in order to provide an artificial reality system leverages the ease of performing these gestures during the course of regular artificial reality operation.
Prada Gomez in view of Ravasz does not explicitly disclose identifying a captured still image among a subset of the captured frames of video data excluding the first frame; However Alvarez discloses identifying a captured still image among a subset of the captured frames of video data excluding the first frame (para[0061] teaches multiple images may be taken before, after, and/or during the appearance of the gesture. The set of images are used to identify the area of interest in the scene depicted with the gesture itself not present. For example, the camera may capture a set of images using a burst capture mode. If the camera is not subject to significant motion, the processor can apply the selection area of the gesture to an image in the burst set taken just after the gesture leaves the frame of the camera. This may allow the area of interest to not include any interference from the gesture being in the image). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of involves determining that a particular hand is associated with a wearable head-mounted display (HMD) via analysis of video data including hand gesture an image is generated from video data of a sub-FOV within the closed area of Prada Gomez with Ravasz with the method of identifies an area-of-interest in the image based on the selection area of the gesture and performs action on the area-of-interest of Alvarez in order to provide a system in which system performs operations on a smaller region, thus efficiently and potentially reducing power consumption and increasing speed of the operations.
Regarding claim 2, Alvarez discloses the method of claim 1, wherein presenting the captured image further comprises: selecting the subset of captured frames, wherein the subset comprises one or more frames preceding or following the first frame (para[0061] teaches multiple images may be taken before, after, and/or during the appearance of the gesture. The set of images are used to identify the area of interest in the scene depicted with the gesture itself not present. For example, the camera may capture a set of images using a burst capture mode. If the camera is not subject to significant motion, the processor can apply the selection area of the gesture to an image in the burst set taken just after the gesture leaves the frame of the camera. This may allow the area of interest to not include any interference from the gesture being in the image). Motivation to combine as indicated in claim 1.
Regarding 4, Prada Gomez in view of Ravasz and Alvarez discloses the method of claim 1, Prada Gomez further discloses wherein detecting the shutter gesture further comprises: detecting a hand shape in the first frame; generating the imaginary plane oriented according to the hand shape (Fig. 7); determining whether the detected hand shape matches one or more hand gestures stored in a hand gesture library (col 16 lines 58-67 teaches HMD can recognize that a hand gesture is being or has been made, and may then identify or match the recognized hand gesture against a library (or other list or collection) of known hand gesture in order to determine one or more actions to take or invoke in response to the gesture) wherein the one or more stored hand gestures comprises a gesture selected from the group consisting of an extended finger appearing to tap the imaginary plane (col 16 lines 38-44 teaches cursor-like pointer gesture as viewed and interpreted by the HMD 502. For example, a cursor-like pointer gesture could be used to identify an object in the FOV of the HMD 502 that a wearer of the HMD wishes to signify as important or noteworthy, or to highlight or underline an object or text that appears in the FOV. However, this possible function (cursor-like pointer gesture)) .
Prada Gomez in view of Alvarez does not explicitly disclose further discloses and a finger of a single hand in momentary contact with a thumb of the single hand. However Ravasz further discloses and a finger of a single hand in momentary contact with a thumb of the single hand (Col 7 lines 7-20 teaches to detect the gesture(s), the artificial reality application may compare the motions, configurations, positions and/or orientations of hand 132 and/or portions of arm 134 to gesture definitions stored in a gesture library of artificial reality system 10, including whether any of a defined combination of the digits (such as an index finger and thumb) are brought together to touch or approximately touch in the physical environment & Fig. 7B & Col 28 lines 35-40 teaches configuration 606, in which the index finger and thumb of hand 132 form a circle or approximately a circle). Motivation to combine as indicated in claim 1.
Regarding claim 6, Prada Gomez discloses the method of claim 1, wherein detecting the shutter gesture further comprises: detecting a border gesture in the first frame, wherein the border gesture defines a field of view (FIGS. 9a and 9b & Col 22 lines 56-67 illustrate how the apparent size of a symbolic right angle the FOV of a HMD can be used to define or demark a portion of the FOV for image selection); and presenting the captured still image in accordance with the defined field of view (col 23 lines 1-10 teaches Upon determining that a right-angle-shaped hand gesture has been made, the HMD generates an image of the FOV within the bounding box 904-1).
Regarding claim 7, Prada Gomez discloses the method of claim 6, wherein detecting the shutter gesture further comprises: presenting a border indicator on the display, the indicator comprising a rectangular border sized according to the defined field of view (FIG. 9a, 9b, 10a, 10b teaches depicts the closed area as the dotted rectangle 1004 & Col 28 lines 40-50 teaches at step 1306, the wearable HMD determines a closed area in the FOV bounded by a rectangle having a first corner at a vertex of the symbolic right angle at the first location and a diagonally-opposite second corner at a vertex of the symbolic right angle at the second location. The bounded area may be considered as being form by a motion of dragging the vertex from the first corner to the second corner. Again, this is illustrated in FIG. 10, which depicts the closed area as the dotted rectangle 1004).
Regarding claim 8, Prada Gomez discloses the method of claim 6, wherein detecting the shutter gesture further comprises: detecting a border hand shape in the first frame; determining whether the detected border hand shape matches one or more border hand gestures stored in a hand gesture library (col 16 lines 58-67 teaches HMD can recognize that a hand gesture is being or has been made, and may then identify or match the recognized hand gesture against a library (or other list or collection) of known hand gesture in order to determine one or more actions to take or invoke in response to the gesture), wherein the one or more stored border hand gestures comprises a gesture selected from the group consisting of (a) an L shape comprising an index finger of a first hand spaced apart from a thumb of the first hand, wherein the L shape defines an approximate corner of the field of view (col 22 lines 29- 65 & FIGS. 8a and 8b and FIGS. 9a and 9b illustrate one mode use of a right-angle-shaped gesture in a FOV of a HMD. FIGS. 10a and 10b illustrate another mode of use of a right-angle-shaped gesture that involves dragging a symbolic right angle across the FOV of a HMD), (b) the L shape together with a similar L shape made by a second hand, wherein the two L shapes define two or more approximate corners of the field of view (Col 23 lines 65 -col 24 lines 15 FIG. 10b is an illustrative representation of using a two-handed, right-angle-shaped hand gesture in a field of view to select an image area of a display) and (c) an extended finger appearing to tap an imaginary plane, wherein the field of view comprises a default size and shape relative to the extended finger tap (col 16 lines 40-45 teaches for example, to some type of cursor-like pointer gesture as viewed and interpreted by the HMD 502. For example, a cursor-like pointer gesture could be used to identify an object in the FOV of the HMD 502 that a wearer of the HMD wishes to signify as important or noteworthy, or to highlight or underline an object or text that appears in the FOV) .
Regarding claim 10, Prada Gomez discloses an image capture system, comprising: an eyewear device (FIG. 1a, the eyeglasses 102) comprising a processor, a memory (col 8 lines 25-40 The on-board computing system 118 may include, for example, a one or more processors and one or more forms of memory), a camera system (a video camera 120), an image processing system, and a display (the display 220 could correspond to an image processing and display system for making images viewable to a user (wearer) of an HMD); programming in the memory, wherein execution of said programming by the processor configures the eyewear device to perform functions, including functions to: capture frames of video data of a physical environment with the camera system (col 17 lines 25-35 teaches HMD could capture video data from the FOV of the video camera, and then identify a triggering object in the captured data); detect a shutter gesture (col 15 lines 56-col 16 line 22 teaches hand gestures can be used to provide user input to a wearable HMD col 16 lines 25-65 teaches a. Detection of Hand Gestures & fig. 5 HMD can recognize that a hand gesture is being or has been made);
Prada Gomez does not explicitly disclose detect a shutter gesture and a shutter gesture location in a first frame of the captured frames of video data with the image processing system, wherein the shutter gesture location is defined relative to the physical environment; generate an imaginary plane relative to the shutter gesture location, wherein the imaginary plane is oriented according to the shutter gesture; estimate an eyewear device location relative to one or more of the imaginary plane or the shutter gesture location; and present an indicator using the eyewear device, wherein the indicator comprises a brief alert selected from the group consisting of a simulated shutter effect presented on the display, a flash presented on the display, a rectangular shape presented on the display, and a sound emitted from a loudspeaker; identify a captured still image among a subset of the captured frames of video data excluding the first frame; and present on the display the captured still image at a display location, wherein the display location is correlated with the shutter gesture location and the eyewear device location. However Ravasz discloses detect a shutter gesture and a shutter gesture location in a first frame of the captured frames of video data with the image processing system, wherein the shutter gesture location is defined relative to the physical environment (Abstract & Col 2 lines 23-40 teaches the image capture device is configured to capture image data representative of a physical environment, gesture detector configured to identify, from the image data, a gesture, col 7 lines 7- 20 teaches to detect the gesture(s), the artificial reality application may compare the motions, configurations, positions and/or orientations of hand 132 and/or portions of arm 134 to gesture definitions stored in a gesture library of artificial reality system 10, where each gesture in the gesture library may be each mapped to one or more actions, including whether any of a defined combination of the digits (such as an index finger and thumb) are brought together to touch or approximately touch in the physical environment); generate an imaginary plane relative to the shutter gesture location, wherein the imaginary plane is oriented according to the shutter gesture (Col 6 lines 27-41 teaches Artificial reality system 10 may trigger generation and rendering of graphical user interface elements 124, 126 in response to other conditions, such as a current state of one or more applications being executed by the system, or the position and orientation of the particular detected gestures in a physical environment in relation to a current field of view 130 of user 110, as may be determined by real-time gaze tracking of the user, or other conditions); estimate an eyewear device location relative to one or more of the imaginary plane or the shutter gesture location (col 6 lines 27-41 teaches the artificial reality system may further trigger generation and rendering of the graphical user interface elements in response to detection of particular gestures in combination with other conditions, such as the position and orientation of the particular gestures in a physical environment relative to a current field of view of the user, which may be determined by real-time gaze tracking of the user, or relative to a pose of an HMD worn by the user, col 4 lines 45-50 teaches artificial reality system 10 may detect a predefined gesture based on additional conditions being satisfied, such as the position and orientation of portions of arm 134 (e.g., a wrist) and/or hand 132 (or digits thereof) in a physical environment in relation to a current field of view 130 of user 110, as may be determined by real-time gaze tracking of the user, or other conditions Col 16 lines 34-38 teaches control unit 210 may detect certain predefined gestures performed at locations generally corresponding to the location of the display element to gate UI elements within artificial reality content 122 displayed via HMD 112; Col 16 lines 34-38 teaches control unit 210 may detect certain predefined gestures performed at locations generally corresponding to the location of the display element to gate UI elements within artificial reality content 122 displayed via HMD 112); and present an indicator using the eyewear device, wherein the indicator comprises a brief alert selected from the group consisting of a simulated shutter effect presented on the display, a flash presented on the display, a rectangular shape presented on the display, and a sound emitted from a loudspeaker (col 6 lines 27-40 teaches artificial reality system 10 may trigger generation and rendering of graphical user interface elements 124, 126 in response to other conditions, such as a current state of one or more applications being executed by the system, or the position and orientation of the particular detected gestures in a physical environment in relation to a current field of view 130 of user 110, as may be determined by real-time gaze tracking of the user, or other conditions. More specifically, image capture devices 138 of HMD 112 capture image data representative of objects in the real world, physical environment that are within a field of view 130 of image capture devices 138. Field of view 130 typically corresponds with the viewing perspective of HMD 112); and present on the display the captured still image at a display location, wherein the display location is correlated with the shutter gesture location and the eyewear device location (Col 5 lines 19-22 teaches artificial reality system 10 uses information captured from a real-world, three-dimensional (3D) physical environment to render artificial reality content 122 for display to user 110. Col 5 lines 36-55 teaches during operation, the artificial reality application constructs artificial reality content 122 for display to user 110 by tracking and computing pose information for a frame of reference, typically a viewing perspective of HMD 112. Using HMD 112 as a frame of reference, and based on a current field of view 130 as determined by a current estimated pose of HMD 112, the artificial reality application renders 3D artificial reality content which, in some examples, may be overlaid, at least in part, upon the real-world, 3D physical environment of user 110. During this process, the artificial reality application uses sensed data received from HMD 112, such as movement information and user commands, and, in some examples, data from any external sensors 90, such as external cameras 102A and/or 102B, to capture 3D information within the real world, physical environment, such as motion by user 110 and/or feature tracking information with respect to user 110. Based on the sensed data, the artificial reality application determines a current pose for the frame of reference of HMD 112 and, in accordance with the current pose, renders the artificial reality content 122). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of involves determining that a particular hand is associated with a wearable head-mounted display (HMD) via analysis of video data including hand gesture an image is generated from video data of a sub-FOV within the closed area of Prada Gomez with the method which involves detection of particular position and orientation of the particular detected gestures in a physical environment in relation to a current field of view of user which may be determined by real-time gaze tracking of the user, or relative to a pose of an HMD worn by the user of Ravasz in order to provide an artificial reality system leverages the ease of performing these gestures during the course of regular artificial reality operation.
Prada Gomez in view of Ravasz does not explicitly disclose identify a captured still image among a subset of the captured frames of video data excluding the first frame. However Alvarez discloses identify a captured still image among a subset of the captured frames of video data excluding the first frame (para[0061] teaches because the gesture may overlap the area of interest, the processor may utilize a second image to identify an area of interest. Multiple images may be taken before, after, and/or during the appearance of the gesture. The set of images are used to identify the area of interest in the scene depicted with the gesture itself not present. For example, the camera may capture a set of images using a burst capture mode. If the camera is not subject to significant motion, the processor can apply the selection area of the gesture to an image in the burst set taken just after the gesture leaves the frame of the camera. This may allow the area of interest to not include any interference from the gesture being in the image). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of involves determining that a particular hand is associated with a wearable head-mounted display (HMD) via analysis of video data including hand gesture An image is generated from video data of a sub-FOV within the closed area of Prada Gomez with Ravasz with the method of identifies an area-of-interest in the image based on the selection area of the gesture and performs action on the area-of-interest of Alvarez in order to provide a system in which system performs operations on a smaller region, thus efficiently and potentially reducing power consumption and increasing speed of the operations.
Regarding claim 12, Prada Gomez in view of Ravasz and Alvarez discloses the image capture system of claim 10, Prada Gomez further discloses wherein the function to detect the shutter gesture further comprises functions to: detect a hand shape in the first frame; generate an imaginary plane oriented according to the hand shape (Fig. 7); and determine whether the detected hand shape matches one or more hand gestures stored in a hand gesture library (col 16 lines 58- 67 teaches HMD can recognize that a hand gesture is being or has been made, and may then identify or match the recognized hand gesture against a library (or other list or collection) of known hand gestures in order to determine one or more actions to take or invoke in response to the gesture), wherein the one or more stored hand gestures comprises a gesture selected from the group consisting of an extended finger appearing to tap an imaginary plane (col 16 lines 40-45 teaches for example, to some type of cursor-like pointer gesture as viewed and interpreted by the HMD 502. For example, a cursor-like pointer gesture could be used to identify an object in the FOV of the HMD 502 that a wearer of the HMD wishes to signify as important or noteworthy, or to highlight or underline an object or text that appears in the FOV.)
Prada Gomez in view of Alvarez does not explicitly disclose and a finger of a single hand in momentary contact with a thumb of the single hand. However Ravasz and a finger of a single hand in momentary contact with a thumb of the single hand (Col 7 lines 7-20 teaches to detect the gesture(s), the artificial reality application may compare the motions, configurations, positions and/or orientations of hand 132 and/or portions of arm 134 to gesture definitions stored in a gesture library of artificial reality system 10, including whether any of a defined combination of the digits (such as an index finger and thumb) are brought together to touch or approximately touch in the physical environment & Fig. 7B & Col 28 lines 35-40 teaches configuration 606, in which the index finger and thumb of hand 132 form a circle or approximately a circle). Motivation to combine as indicated in claim 10.
Regarding claim 15, Prada Gomez discloses a non-transitory computer-readable medium storing program code which, when executed (col 5 lines 30-35 teaches non-transitory tangible computer readable executable instructions could also be stored on some form of non-transitory tangible computer readable storage medium), is operative to cause an electronic processor to perform the steps of: capturing frames of video data of a physical environment with a camera system coupled to an eyewear device (FIG. 1a, the eyeglasses 102, col 17 lines 25-35 teaches HMD could capture video data from the FOV of the video camera, and then identify a triggering object in the captured data) comprising a processor, a memory (col 8 lines 25-40 The on-board computing system 118 may include, for example, a one or more processors and one or more forms of memory), an image processing system, and a display the display 220 could correspond to an image processing and display system for making images viewable to a user (wearer) of an HMD); detecting a shutter gesture (col 15 lines 56-col 16 line 22 teaches hand gestures can be used to provide user input to a wearable HMD col 16 lines 25-65 teaches a. Detection of Hand Gestures & fig. 5 HMD can recognize that a hand gesture is being or has been made);
Prada Gomez does not explicitly disclose detecting a shutter gesture and a shutter gesture location in a first frame of the captured frames of video data with the image processing system, wherein the shutter gesture location is defined relative to the physical environment; generating an imaginary plane relative to the shutter gesture location, wherein the imaginary plane is oriented according to the shutter gesture; estimating an eyewear device location relative to one or more of the imaginary plane and the shutter gesture location; and presenting an indicator using the eyewear device, wherein the indicator comprises a brief alert selected from the group consisting of a simulated shutter effect presented on the display, a flash presented on the display, a rectangular shape presented on the display, and a sound emitted from a loudspeaker; identifying a captured still image among a subset of the captured frames of video data excluding the first frame; presenting on the display the captured still image at a display location, wherein the display location is correlated with the shutter gesture location and the eyewear device location. However Ravasz discloses detecting a shutter gesture and a shutter gesture location in a first frame of the captured frames of video data with the image processing system, wherein the shutter gesture location is defined relative to the physical environment (Abstract & Col 2 lines 23-40 teaches image capture device is configured to capture image data representative of a physical environment, gesture detector configured to identify, from the image data, a gesture, col 7 lines 7- 20 teaches to detect the gesture(s), the artificial reality application may compare the motions, configurations, positions and/or orientations of hand 132 and/or portions of arm 134 to gesture definitions stored in a gesture library of artificial reality system 10, where each gesture in the gesture library may be each mapped to one or more actions, including whether any of a defined combination of the digits (such as an index finger and thumb) are brought together to touch or approximately touch in the physical environment); (Col 6 lines 27-41 teaches Artificial reality system 10 may trigger generation and rendering of graphical user interface elements 124, 126 in response to other conditions, such as a current state of one or more applications being executed by the system, or the position and orientation of the particular detected gestures in a physical environment in relation to a current field of view 130 of user 110, as may be determined by real-time gaze tracking of the user, or other conditions); estimate an eyewear device location relative to one or more of the imaginary plane or the shutter gesture location (col 6 lines 27-41 teaches the artificial reality system may further trigger generation and rendering of the graphical user interface elements in response to detection of particular gestures in combination with other conditions, such as the position and orientation of the particular gestures in a physical environment relative to a current field of view of the user, which may be determined by real-time gaze tracking of the user, or relative to a pose of an HMD worn by the user, col 4 lines 45-50 teaches artificial reality system 10 may detect a predefined gesture based on additional conditions being satisfied, such as the position and orientation of portions of arm 134 (e.g., a wrist) and/or hand 132 (or digits thereof) in a physical environment in relation to a current field of view 130 of user 110, as may be determined by real-time gaze tracking of the user, or other conditions Col 16 lines 34-38 teaches control unit 210 may detect certain predefined gestures performed at locations generally corresponding to the location of the display element to gate UI elements within artificial reality content 122 displayed via HMD 112; Col 16 lines 34-38 teaches control unit 210 may detect certain predefined gestures performed at locations generally corresponding to the location of the display element to gate UI elements within artificial reality content 122 displayed via HMD 112); and presenting an indicator using the eyewear device, wherein the indicator comprises a brief alert selected from the group consisting of a simulated shutter effect presented on the display, a flash presented on the display, a rectangular shape presented on the display, and a sound emitted from a loud speaker (col 6 lines 27-40 teaches artificial reality system 10 may trigger generation and rendering of graphical user interface elements 124, 126 in response to other conditions, such as a current state of one or more applications being executed by the system, or the position and orientation of the particular detected gestures in a physical environment in relation to a current field of view 130 of user 110, as may be determined by real-time gaze tracking of the user, or other conditions. More specifically, image capture devices 138 of HMD 112 capture image data representative of objects in the real world, physical environment that are within a field of view 130 of image capture devices 138. Field of view 130 typically corresponds with the viewing perspective of HMD 112); presenting on the display the captured still image at a display location, wherein the display location is correlated with the shutter gesture location and the eyewear device location (Col 5 lines 19-22 teaches artificial reality system 10 uses information captured from a real-world, three-dimensional (3D) physical environment to render artificial reality content 122 for display to user 110. Col 5 lines 36-55 teaches during operation, the artificial reality application constructs artificial reality content 122 for display to user 110 by tracking and computing pose information for a frame of reference, typically a viewing perspective of HMD 112. Using HMD 112 as a frame of reference, and based on a current field of view 130 as determined by a current estimated pose of HMD 112, the artificial reality application renders 3D artificial reality content which, in some examples, may be overlaid, at least in part, upon the real-world, 3D physical environment of user 110. During this process, the artificial reality application uses sensed data received from HMD 112, such as movement information and user commands, and, in some examples, data from any external sensors 90, such as external cameras 102A and/or 102B, to capture 3D information within the real world, physical environment, such as motion by user 110 and/or feature tracking information with respect to user 110. Based on the sensed data, the artificial reality application determines a current pose for the frame of reference of HMD 112 and, in accordance with the current pose, renders the artificial reality content 122,). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of involves determining that a particular hand is associated with a wearable head-mounted display (HMD) via analysis of video data including hand gesture an image is generated from video data of a sub-FOV within the closed area of Prada Gomez with the method which involves detection of particular position and orientation of the particular detected gestures in a physical environment in relation to a current field of view of user which may be determined by real-time gaze tracking of the user, or relative to a pose of an HMD worn by the user of Ravasz in order to provide an artificial reality system leverages the ease of performing these gestures during the course of regular artificial reality operation.
Prada Gomez in view of Ravasz does not explicitly disclose identifying a captured still image among a subset of the captured frames of video data excluding the first frame; However Alvarez discloses identifying a captured still image among a subset of the captured frames of video data excluding the first frame (para[0061] teaches multiple images may be taken before, after, and/or during the appearance of the gesture. The set of images are used to identify the area of interest in the scene depicted with the gesture itself not present. For example, the camera may capture a set of images using a burst capture mode. If the camera is not subject to significant motion, the processor can apply the selection area of the gesture to an image in the burst set taken just after the gesture leaves the frame of the camera. This may allow the area of interest to not include any interference from the gesture being in the image). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of involves determining that a particular hand is associated with a wearable head-mounted display (HMD) via analysis of video data including hand gesture An image is generated from video data of a sub-FOV within the closed area of Prada Gomez with Ravasz with the method of identifies an area-of-interest in the image based on the selection area of the gesture and performs action on the area-of-interest of Alvarez in order to provide a system in which system performs operations on a smaller region, thus efficiently and potentially reducing power consumption and increasing speed of the operations.
Regarding claim 17, Prada Gomez in view of Ravasz and Alvarez discloses the non-transitory computer-readable medium storing program code of claim 15, Prada Gomez further discloses wherein the program code, when executed, is operative to cause the electronic processor to perform the further steps of: detecting a hand shape in the first frame; generating an the imaginary plane oriented according to the hand shape (Fig. 7); and determining whether the detected hand shape matches one or more hand gestures stored in a hand gesture library (col 16 lines 58- 67 teaches HMD can recognize that a hand gesture is being or has been made, and may then identify or match the recognized hand gesture against a library (or other list or collection) of known hand gestures in order to determine one or more actions to take or invoke in response to the gesture), wherein the one or more stored hand gestures comprises a gesture selected from the group consisting of an extended finger appearing to tap an imaginary plane (col 16 lines 40-45 teaches for example, to some type of cursor-like pointer gesture as viewed and interpreted by the HMD 502. For example, a cursor-like pointer gesture could be used to identify an object in the FOV of the HMD 502 that a wearer of the HMD wishes to signify as important or noteworthy, or to highlight or underline an object or text that appears in the FOV.)
Prada Gomez in view of Alvarez does not explicitly disclose and a finger of a single hand in momentary contact with a thumb of the single hand. However Ravasz and a finger of a single hand in momentary contact with a thumb of the single hand (Col 7 lines 7-20 teaches to detect the gesture(s), the artificial reality application may compare the motions, configurations, positions and/or orientations of hand 132 and/or portions of arm 134 to gesture definitions stored in a gesture library of artificial reality system 10, including whether any of a defined combination of the digits (such as an index finger and thumb) are brought together to touch or approximately touch in the physical environment & Fig. 7B & Col 28 lines 35-40 teaches configuration 606, in which the index finger and thumb of hand 132 form a circle or approximately a circle). Motivation to combine as indicated in claim 15.
Regarding claim 19, Prada Gomez discloses the non-transitory computer-readable medium storing program code of claim 15, wherein the program code, when executed, is operative to cause the electronic processor to perform the further steps of: detecting a border hand shape in the first frame, wherein the border hand shape defines a field of view; determining whether the detected border hand shape matches one or more border hand gestures stored in a hand gesture library (col 16 lines 58-67 teaches HMD can recognize that a hand gesture is being or has been made, and may then identify or match the recognized hand gesture against a library (or other list or collection) of known hand gesture in order to determine one or more actions to take or invoke in response to the gesture), wherein the one or more stored border hand gestures comprises a gesture selected from the group consisting of (a) an L shape comprising an index finger of a first hand spaced apart from a thumb of the first hand, wherein the L shape defines an approximate corner of the field of view (col 22 lines 29- 65 & FIGS. 8a and 8b and FIGS. 9a and 9b illustrate one mode use of a right-angle-shaped gesture in a FOV of a HMD. FIGS. 10a and 10b illustrate another mode of use of a right-angle-shaped gesture that involves dragging a symbolic right angle across the FOV of a HMD), (b) the L shape together with a similar L shape made by a second hand, wherein the two L shapes define two or more approximate corners of the field of view Col 23 lines 65 -col 24 lines 15 FIG. 10b is an illustrative representation of using a two-handed, right-angle-shaped hand gesture in a field of view to select an image area of a display), and (c) an extended finger appearing to tap an imaginary plane, wherein the field of view comprises a default size and shape relative to the extended finger tap (col 16 lines 40-45 teaches for example, to some type of cursor-like pointer gesture as viewed and interpreted by the HMD 502. For example, a cursor-like pointer gesture could be used to identify an object in the FOV of the HMD 502 that a wearer of the HMD wishes to signify as important or noteworthy, or to highlight or underline an object or text that appears in the FOV); and presenting the captured still image in accordance with the defined field of view.( FIGS. 9a and 9b, col 23 lines 1-10 teaches Upon determining that a right-angle-shaped hand gesture has been made, the HMD generates an image of the FOV within the bounding box 904-1).
10. Claims 5, 9, 11, 13, 14, 16, 18, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Prada Gomez et al. (US 9,024,842 B1) in view of Ravasz et al. (US 11,003,307 B1) and Alvarez et al. (US 2016/0203360 A1) in further view of Bradski et al. (EP 3699736 A1).
Regarding claim 5, Prada Gomez in view of Ravasz and Alvarez discloses the method of claim 1, Prada Gomez in view of Ravasz and Alvarez does not explicitly disclose wherein the eyewear device comprises a localization system, the method further comprising: determining the eyewear device location relative to the physical environment using the localization system; and presenting the captured still image at the display location in accordance with the eyewear device location, such that the captured still image appears persistently anchored relative to the physical environment. However Bradski discloses does not explicitly disclose wherein the eyewear device comprises a localization system, the method further comprising: determining the eyewear device location relative to the physical environment using the localization system (Para[1253] teaches display a user interface of the user's choosing, the AR system must determine a location of the user in the world (e.g., the world coordinate frame). For example, the user's location may be determined through any of the localization techniques (e.g., GPS, Bluetooth, topological map, map points related to the user's AR system, etc.). Once the user's location in the world coordinate frame has been determined, a relationship between the user's hands/finger etc. in relation to the user's AR system may be determined, Para[1297] Thus, the AR system detects and responds to gestures (e.g., throwing gestures, pointing gestures) which allow freeform location-specification denoting which virtual content should be rendered or moved. For example, where a user desires a virtual display, monitor or screen, the user may specify a location in the physical environment in the user's field of view in which to cause the virtual display, monitor or screen to appear, Para[0610] & Fig. 29 teaches the user’s individual AR system may detect a location and orientation of the user within the world); and presenting the captured still image at the display location in accordance with the eyewear device location, such that the captured still image appears persistently anchored relative to the physical environment. (para[01740] & Fig. 145 teaches at F14502 a virtual image is displayed to a user. At 14506, the AR system may calculate a location of the user's eye pupil. This may be calculated through the various techniques outlined above. At 14508, the AR system, may user the calculated location of the user's eye pupil to determine a location at which a pixel of the virtual image is displayed to the user. para[0897] teaches thus, as the user interfaces with the AR system in the blended virtual interface mode, various physical objects may be displayed to the user as rendered physical objects. This may be especially useful for allowing the user to interface with the AR system, while still being able to safely navigate the local, physical environment. & Fig. 58, 86A). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of identifies an area-of-interest in the image based on the selection area of the gesture and performs action on the area-of-interest of Prada Gomez in view of Ravasz and Alvarez with the determining a location of the gesture in relation to the user's augmented reality display system, wherein the virtual user interface is displayed at the determined location of Bradski in order to provide a system in which appreciated that localization information may be utilized to facilitate accurate rendering in the user's view of the pertinent world (e.g., such information would facilitate the glasses to know where they are with respect to the real world).
Regarding claim 9, Prada Gomez in view of Ravasz and Alvarez discloses the method of claim 1, Prada Gomez in view of Ravasz and Alvarez does not explicitly disclose, wherein presenting the captured image further comprises: broadcasting to other eyewear devices one or more of the captured still image and the shutter gesture location. However Bradski discloses wherein presenting the captured image further comprises: broadcasting to other eyewear devices one or more of the captured still image and the shutter gesture location. (para[0641] teaches user's individual AR system contains information about the user's head pose and orientation in a space, information about hand movement etc. of the user, information about the user's eyes and eye gaze, information about any totems that are being used by the user. Thus, the user's individual AR system already holds a lot of information about the user's interaction within a particular space that is transmitted to the passable world model. his information may then be reliably used to create avatars for the user and help the avatar communicate with other avatars or users of that space. It should be appreciated that in one or more embodiments, third party cameras may not be needed to animate the avatar. Rather, the avatar may be animated based on the user's individual AR system, and then transmitted to the cloud to be viewed/interacted with by other users of the AR system, para[0646]. para[0817] teaches once the objects are recognized, the information may be transmitted to one or more user wearable systems 4920. For example, the AR system 4900 may transmit data pertaining to a scene in a first location (e.g., San Francisco) to one or more users having wearable systems in New York City. Utilizing the data in the map database 4910 (e.g., data received from multiple cameras and other inputs, the object recognizers and other software components map the points collected through the various images, recognize objects etc.), para[0821], [0822]). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of identifies an area-of-interest in the image based on the selection area of the gesture and performs action on the area-of-interest of Prada Gomez in view of Ravasz and Alvarez with the method in which user's individual AR system already holds a lot of information about the user's interaction within a particular space that is transmitted to the passable world model of Bradski in order to provide a system to be viewed/interacted with by other users of the AR system.
Regarding claim 11, Prada Gomez in view of Ravasz and Alvarez discloses the image capture system of claim 10, Alvarez further discloses, wherein the function to present the captured image further comprises functions to: select the subset of captured frames, wherein the subset comprises one or more frames preceding or following the first frame (para[0061] teaches multiple images may be taken before, after, and/or during the appearance of the gesture);
Prada Gomez in view of Ravasz and Alvarez does not explicitly disclose and broadcast to other eyewear devices one or more of the captured still image and the shutter gesture location. However Bradski discloses and broadcast to other eyewear devices one or more of the captured still image and the shutter gesture location. (para[0641] teaches user's individual AR system contains information about the user's head pose and orientation in a space, information about hand movement etc. of the user, information about the user's eyes and eye gaze, information about any totems that are being used by the user. Thus, the user's individual AR system already holds a lot of information about the user's interaction within a particular space that is transmitted to the passable world model. his information may then be reliably used to create avatars for the user and help the avatar communicate with other avatars or users of that space. It should be appreciated that in one or more embodiments, third party cameras may not be needed to animate the avatar. Rather, the avatar may be animated based on the user's individual AR system, and then transmitted to the cloud to be viewed/interacted with by other users of the AR system, para[0646]. para[0817] teaches once the objects are recognized, the information may be transmitted to one or more user wearable systems 4920. For example, the AR system 4900 may transmit data pertaining to a scene in a first location (e.g., San Francisco) to one or more users having wearable systems in New York City. Utilizing the data in the map database 4910 (e.g., data received from multiple cameras and other inputs, the object recognizers and other software components map the points collected through the various images, recognize objects etc.), para[0821], [0822]). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of identifies an area-of-interest in the image based on the selection area of the gesture and performs action on the area-of-interest of Prada Gomez in view of Ravasz and Alvarez with the method in which user's individual AR system already holds a lot of information about the user's interaction within a particular space that is transmitted to the passable world model of Bradski in order to provide a system to be viewed/interacted with by other users of the AR system.
Regarding claim 13, Prada Gomez in view of Ravasz and Alvarez discloses the image capture system of claim 10, Prada Gomez in view of Ravasz and Alvarez does not explicitly disclose wherein the eyewear device comprises a localization system, and wherein the execution of the programming further configures the eyewear device to: determining the eyewear device location relative to the physical environment using the localization system; and present the captured still image at the display location in accordance with the eyewear device location, such that the captured still image appears persistently anchored relative to the physical environment. However Bradski discloses wherein the eyewear device comprises a localization system, and wherein the execution of the programming further configures the eyewear device to: determining the eyewear device location relative to the physical environment using the localization system; (Para[1253] teaches display a user interface of the user's choosing, the AR system must determine a location of the user in the world (e.g., the world coordinate frame). For example, the user's location may be determined through any of the localization techniques discussed above (e.g., GPS, Bluetooth, topological map, map points related to the user's AR system, etc.). Once the user's location in the world coordinate frame has been determined, a relationship between the user's hands/finger etc. in relation to the user's AR system may be determined, Para[1297] Thus, the AR system detects and responds to gestures (e.g., throwing gestures, pointing gestures) which allow freeform location-specification denoting which virtual content should be rendered or moved. For example, where a user desires a virtual display, monitor or screen, the user may specify a location in the physical environment in the user's field of view in which to cause the virtual display, monitor or screen to appear, Para[0610] & Fig. 29 teaches the user’s individual AR system may detect a location and orientation of the user within the world); and present the captured still image at the display location in accordance with the eyewear device location, such that the captured still image appears persistently anchored relative to the physical environment. (para[01740] & Fig. 145 teaches at F14502 a virtual image is displayed to a user. At 14506, the AR system may calculate a location of the user's eye pupil. This may be calculated through the various techniques outlined above. At 14508, the AR system, may user the calculated location of the user's eye pupil to determine a location at which a pixel of the virtual image is displayed to the user. para[0897] teaches thus, as the user interfaces with the AR system in the blended virtual interface mode, various physical objects may be displayed to the user as rendered physical objects. This may be especially useful for allowing the user to interface with the AR system, while still being able to safely navigate the local, physical environment. & Fig. 58, 86A). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of identifies an area-of-interest in the image based on the selection area of the gesture and performs action on the area-of-interest of Prada Gomez in view of Ravasz and Alvarez with the determining a location of the gesture in relation to the user's augmented reality display system, wherein the virtual user interface is displayed at the determined location of Bradski in order to provide a system in which appreciated that localization information may be utilized to facilitate accurate rendering in the user's view of the pertinent world (e.g., such information would facilitate the glasses to know where they are with respect to the real world).
Regarding claim 14, Prada Gomez in view of Ravasz and Alvarez discloses the image capture system of claim 10, Prada Gomez further discloses wherein the function to detect the shutter gesture further comprises functions to: detect a border hand shape in the first frame, wherein the border hand shape defines a field of view; determine whether the detected border hand shape matches one or more border hand gestures stored in a hand gesture library (col 16 lines 58-67 teaches HMD can recognize that a hand gesture is being or has been made, and may then identify or match the recognized hand gesture against a library (or other list or collection) of known hand gesture in order to determine one or more actions to take or invoke in response to the gesture), wherein the one or more stored border hand gestures comprises a gesture selected from the group consisting of (a) an L shape comprising an index finger of a first hand spaced apart from a thumb of the first hand, wherein the L shape defines an approximate corner of the field of view (col 22 lines 29- 65 & FIGS. 8a and 8b and FIGS. 9a and 9b illustrate one mode use of a right-angle-shaped gesture in a FOV of a HMD. FIGS. 10a and 10b illustrate another mode of use of a right-angle-shaped gesture that involves dragging a symbolic right angle across the FOV of a HMD), (b) the L shape together with a similar L shape made by a second hand, wherein the two L shapes define two or more approximate corners of the field of view (Col 23 lines 65 -col 24 lines 15 FIG. 10b is an illustrative representation of using a two-handed, right-angle-shaped hand gesture in a field of view to select an image area of a display), and (c) an extended finger appearing to tap an imaginary plane, wherein the field of view comprises a default size and shape relative to the extended finger tap (col 16 lines 40-45 teaches for example, to some type of cursor-like pointer gesture as viewed and interpreted by the HMD 502. For example, a cursor-like pointer gesture could be used to identify an object in the FOV of the HMD 502 that a wearer of the HMD wishes to signify as important or noteworthy, or to highlight or underline an object or text that appears in the FOV); present a border indicator on the display, the indicator comprising a rectangular border sized according to the defined field of view (FIGS. 9a and 9b & Col 22 lines 56-67 illustrate how the apparent size of a symbolic right angle the FOV of a HMD can be used to define or demark a portion of the FOV for image selection); present the captured still image in accordance with the defined field of view (col 23 lines 1-10 teaches Upon determining that a right-angle-shaped hand gesture has been made, the HMD generates an image of the FOV within the bounding box 904-1).
Prada Gomez in view of Ravasz and Alvarez does not explicitly disclose and broadcast to other eyewear devices one or more of the captured still image, the shutter gesture location, the border indicator, and the defined field of view. However Bradski discloses and broadcast to other eyewear devices one or more of the captured still image, the shutter gesture location, the border indicator, and the defined field of view (para[0641] teaches user's individual AR system contains information about the user's head pose and orientation in a space, information about hand movement etc. of the user, information about the user's eyes and eye gaze, information about any totems that are being used by the user. Thus, the user's individual AR system already holds a lot of information about the user's interaction within a particular space that is transmitted to the passable world model. his information may then be reliably used to create avatars for the user and help the avatar communicate with other avatars or users of that space. It should be appreciated that in one or more embodiments, third party cameras may not be needed to animate the avatar. Rather, the avatar may be animated based on the user's individual AR system, and then transmitted to the cloud to be viewed/interacted with by other users of the AR system, para[0646]. para[0817] teaches once the objects are recognized, the information may be transmitted to one or more user wearable systems 4920. For example, the AR system 4900 may transmit data pertaining to a scene in a first location (e.g., San Francisco) to one or more users having wearable systems in New York City. Utilizing the data in the map database 4910 (e.g., data received from multiple cameras and other inputs, the object recognizers and other software components map the points collected through the various images, recognize objects etc.), para[0821], [0822]). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of identifies an area-of-interest in the image based on the selection area of the gesture and performs action on the area-of-interest of Prada Gomez in view of Ravasz and Alvarez with the method in which user's individual AR system already holds a lot of information about the user's interaction within a particular space that is transmitted to the passable world model of Bradski in order to provide a system to be viewed/interacted with by other users of the AR system.
Regarding claim 16, Prada Gomez in view of Ravasz and Alvarez discloses the non-transitory computer-readable medium storing program code of claim 15, Alvarez further discloses wherein the program code, when executed, is operative to cause the electronic processor to perform the further steps of: selecting the subset of captured frames, wherein the subset comprises one or more frames preceding or following the first frame (para[0061] teaches multiple images may be taken before, after, and/or during the appearance of the gesture);
Prada Gomez in view of Ravasz and Alvarez does not explicitly disclose and broadcasting to other eyewear devices one or more of the captured still image and the shutter gesture location. However Benko disclose broadcast to other eyewear devices one or more of the captured still image and the shutter gesture location. However Bradski discloses and broadcasting to other eyewear devices one or more of the captured still image and the shutter gesture location (para[0641] teaches user's individual AR system contains information about the user's head pose and orientation in a space, information about hand movement etc. of the user, information about the user's eyes and eye gaze, information about any totems that are being used by the user. Thus, the user's individual AR system already holds a lot of information about the user's interaction within a particular space that is transmitted to the passable world model. his information may then be reliably used to create avatars for the user and help the avatar communicate with other avatars or users of that space. It should be appreciated that in one or more embodiments, third party cameras may not be needed to animate the avatar. Rather, the avatar may be animated based on the user's individual AR system, and then transmitted to the cloud to be viewed/interacted with by other users of the AR system, para[0646]. para[0817] teaches once the objects are recognized, the information may be transmitted to one or more user wearable systems 4920. For example, the AR system 4900 may transmit data pertaining to a scene in a first location (e.g., San Francisco) to one or more users having wearable systems in New York City. Utilizing the data in the map database 4910 (e.g., data received from multiple cameras and other inputs, the object recognizers and other software components map the points collected through the various images, recognize objects etc.), para[0821], [0822]). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of identifies an area-of-interest in the image based on the selection area of the gesture and performs action on the area-of-interest of Prada Gomez in view of Ravasz and Alvarez with the method in which user's individual AR system already holds a lot of information about the user's interaction within a particular space that is transmitted to the passable world model of Bradski in order to provide a system to be viewed/interacted with by other users of the AR system.
Regarding claim 18, Prada Gomez in view of Ravasz and Alvarez discloses the non-transitory computer-readable medium storing program code of claim 15, Prada Gomez in view of Ravasz and Alvarez does not explicitly disclose wherein the eyewear device comprises a localization system, and wherein the program code, when executed, is operative to cause the electronic processor to perform the further steps of: determining the eyewear device location relative to the physical environment using the localization system; and presenting the captured still image at the display location in accordance with the eyewear device location, such that the captured still image appears persistently anchored relative to the physical environment; and presenting the captured still image at the display location in accordance with the eyewear device location, such that the captured still image appears persistently anchored relative to the physical environment. However Bradski discloses wherein the eyewear device comprises a localization system, and wherein the program code, when executed, is operative to cause the electronic processor to perform the further steps of: determining the eyewear device location relative to the physical environment using the localization system (Para[1253] teaches display a user interface of the user's choosing, the AR system must determine a location of the user in the world (e.g., the world coordinate frame). For example, the user's location may be determined through any of the localization techniques discussed above (e.g., GPS, Bluetooth, topological map, map points related to the user's AR system, etc.). Once the user's location in the world coordinate frame has been determined, a relationship between the user's hands/finger etc. in relation to the user's AR system may be determined, Para[1297] Thus, the AR system detects and responds to gestures (e.g., throwing gestures, pointing gestures) which allow freeform location-specification denoting which virtual content should be rendered or moved. For example, where a user desires a virtual display, monitor or screen, the user may specify a location in the physical environment in the user's field of view in which to cause the virtual display, monitor or screen to appear, Para[0610] & Fig. 29 teaches the user’s individual AR system may detect a location and orientation of the user within the world); presenting the captured still image at the display location in accordance with the eyewear device location, such that the captured still image appears persistently anchored relative to the physical environment. (para[01740] & Fig. 145 teaches at F14502 a virtual image is displayed to a user. At 14506, the AR system may calculate a location of the user's eye pupil. This may be calculated through the various techniques outlined above. At 14508, the AR system, may user the calculated location of the user's eye pupil to determine a location at which a pixel of the virtual image is displayed to the user. para[0897] teaches thus, as the user interfaces with the AR system in the blended virtual interface mode, various physical objects may be displayed to the user as rendered physical objects. This may be especially useful for allowing the user to interface with the AR system, while still being able to safely navigate the local, physical environment. & Fig. 58, 86A). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of identifies an area-of-interest in the image based on the selection area of the gesture and performs action on the area-of-interest of Prada Gomez in view of Ravasz and Alvarez with the determining a location of the gesture in relation to the user's augmented reality display system, wherein the virtual user interface is displayed at the determined location of Bradski in order to provide a system in which appreciated that localization information may be utilized to facilitate accurate rendering in the user's view of the pertinent world (e.g., such information would facilitate the glasses to know where they are with respect to the real world).
Regarding claim 20, Prada Gomez in view of Ravasz and Alvarez discloses the non-transitory computer-readable medium storing program code of claim 19, Prada Gomez further discloses wherein the program code, when executed, is operative to cause the electronic processor to perform the further steps of: presenting a border indicator on the display, the indicator comprising a rectangular border sized according to the defined field of view (FIG. 9a, 9b, 10a, 10b teaches depicts the closed area as the dotted rectangle 1004 & Col 28 lines 40-50 teaches at step 1306, the wearable HMD determines a closed area in the FOV bounded by a rectangle having a first corner at a vertex of the symbolic right angle at the first location and a diagonally-opposite second corner at a vertex of the symbolic right angle at the second location. The bounded area may be considered as being form by a motion of dragging the vertex from the first corner to the second corner. Again, this is illustrated in FIG. 10, which depicts the closed area as the dotted rectangle 1004. );
Prada Gomez in view of Ravasz and Alvarez does not explicitly disclose and broadcasting to other eyewear devices one or more of the captured still image, the shutter gesture location, the border indicator, and the defined field of view. However Bradski discloses and broadcasting to other eyewear devices one or more of the captured still image, the shutter gesture location, the border indicator, and the defined field of view. para[0641] teaches user's individual AR system contains information about the user's head pose and orientation in a space, information about hand movement etc. of the user, information about the user's eyes and eye gaze, information about any totems that are being used by the user. Thus, the user's individual AR system already holds a lot of information about the user's interaction within a particular space that is transmitted to the passable world model. his information may then be reliably used to create avatars for the user and help the avatar communicate with other avatars or users of that space. It should be appreciated that in one or more embodiments, third party cameras may not be needed to animate the avatar. Rather, the avatar may be animated based on the user's individual AR system, and then transmitted to the cloud to be viewed/interacted with by other users of the AR system, para[0646]. para[0817] teaches once the objects are recognized, the information may be transmitted to one or more user wearable systems 4920. For example, the AR system 4900 may transmit data pertaining to a scene in a first location (e.g., San Francisco) to one or more users having wearable systems in New York City. Utilizing the data in the map database 4910 (e.g., data received from multiple cameras and other inputs, the object recognizers and other software components map the points collected through the various images, recognize objects etc.), para[0821], [0822]). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of identifies an area-of-interest in the image based on the selection area of the gesture and performs action on the area-of-interest of Prada Gomez in view of Ravasz and Alvarez with the method in which user's individual AR system already holds a lot of information about the user's interaction within a particular space that is transmitted to the passable world mode of Bradski in order to provide a system to be viewed/interacted with by other users of the AR system.
Conclusion
11. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROWINA J CATTUNGAL whose telephone number is (571)270-5922. The examiner can normally be reached Monday-Thursday 7:30am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Brian Pendleton can be reached on (571) 272-7527. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROWINA J CATTUNGAL/Primary Examiner, Art Unit 2425
Prosecution Insights