DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in responsive to communication(s):
Application filed on 5/6/2024 with effective filing date of 5/6/2024.
The status of the claims is summarized as below:
Claims 1-20 are pending.
Claims 1, 8, and 15 are independent claims.
Specification
The disclosure is objected to because of the following informalities:
“network interface 610” is cited in ¶0053 in the filed specification, where Fig. 6 labels network interface as 614.
“GPU 608” is cited in ¶0054 in the filed specification, where Fig. 6 labels GPU as 616.
“graphics subsystem 614” is cited in ¶0055 in the filed specification, where Fig. 6 labels graphics subsystem as 620.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-2, 4, 6, 8-9, 11, 13-16, 18, 20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Amariutei et al. (US Pub 20150309629, hereinafter Amariutei).
Per claim 8, Amariutei teaches:
A method for providing a user interface through a head-mounted display worn by a user, comprising: (abstract [0028, 0030]: method for utilizing real world objects to interact with a user interface, where the user device 100 can be a head-mounted display (HMD) or other wearable computing device in AR (augmented reality), such as Google Glass);
identifying and tracking a hand of the user, including identifying and tracking a thumb and fingertips of the hand of the user; ([0033, 0038-0039, 0065-0069] Fig. 5A-5B: a camera as part of the user device 100 along with tracking module 157 are used to track objects; specifically regions of the interactive surface area, such as phalanx of the user’s hand which include fingertip of the hand, are tracked; also the interacting object is also tracked; [0052] user’s thumb may be used as interacting object, so a single hand can be used to make selections);
providing a real-time view of the user's hand through the head-mounted display; ([0030, 0088] the camera of the HMD provides a live image of the user’s hand overlaid with commands through HMD’s display; i.e. Fig. 5A-5B);
rendering a plurality of graphical user interface (UI) elements in respective association with the fingertips of the user's hand in the real-time view of the hand through the head-mounted display; ([0030, 0124-0125] Fig. 5A-B, Fig. 7: Fig. 7 shows a user interface of a calculator application overlaid to regions of a hand 700 from real-time view of the camera and displayed on the display of the HMD, where the displayed view includes visual depicture of commands overlaid to regions of the hand; Fig. 7 shows “1”, “4”, “7”, “+” user interface elements are associated with fingertips of the hand);
wherein detection of touching of the thumb to a given fingertip activates selection of the graphical UI element associated with the given fingertip to trigger an action associated with the graphical UI element. ([0050, 0052, 0056-0057] The user’s thumb may be used as interacting object to select commands overlaid on the phalanges regions of the user’s hand; when the interacting object, i.e. thumb, is in proximity with a particular region of the interactive surface area, i.e. pointer finger on the left hand, the associated command can be initiated; for example, Fig. 6A shows three commands on an index finger, a selection of command “3” can be made when thumb is used as the interacting object to touch the fingertip of the index finger, and associated action with command “3” is initiated).
Per claim 9, Amariutei teaches all the limitations of claim 8, and further teaches:
wherein the plurality of graphical UI elements are rendered as overlays on the respective fingertips of the user's hand. ([0030, 0088, 0124-0125] Fig. 5A-B, Fig. 7: Fig. 7 shows a user interface of a calculator application mapped to regions of a hand 700 from the view of the camera of the HMD, where the display may include the hand image with commands overlaid on respective regions of the hand including plurality of fingertips).
Per claim 11, Amariutei teaches all the limitations of claim 8, and further teaches:
wherein rendering the graphical UI elements is in response to detecting viewing by the user of the hand. ([0047] interactive surface area may be predefined by the user or indicated by the user; when a user wiggle their fingers when the hand is in view of the camera/display, the device is prompted to select the user’s hand as interactive surface area, and subsequently overlay/render the UI elements on the hand).
Per claim 13, Amariutei teaches all the limitations of claim 8, and further teaches:
wherein identifying and tracking the user's hand includes capturing images of the user's hand by a camera integrated with the head-mounted display. ([0065, 0067] user’s hand’s may be captured by camera and tracked; [0028, 0034] camera 140 may be integrated into the user’s device 100 which may be a head-mounted display).
Per claim 14, Amariutei teaches all the limitations of claim 8, and further teaches:
wherein detection of swiping of the thumb across fingers of the user's hand triggers scrolling of the graphical UI elements. ([0084, 0052] a slider control can be rendered on an extended finger, where thumb can be the interactive object controlling the slider).
Per claim 1, claim 1 is a method claim with broader but substantially similar limitations as claim 8, and is likewise rejected.
Per claim 2, 4, 6, claims 2, 4 and 6 include limitations that are substantially the same as claims 9, 11 and 13 respectively, and are likewise rejected.
Per claim 15, claim 15 is a non-transitory medium ([0035] Fig. 1, memory 154) claim that include limitations that are substantially the same as claim 1, which is also broader than claim 8, and is likewise rejected.
Per claim 16, 18, 20, claims 16, 18 and 20 include limitations that are substantially the same as claims 9, 11 and 13 respectively, and are likewise rejected.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being as being unpatentable over Amariutei, in view of Norieda et al. (US Pub 20180260033, hereinafter Norieda).
Per claim 7, Amariutei teaches all the limitations of claim 1, and further teaches:
wherein ([0084, 0052] a slider control can be rendered on an extended finger, where thumb can be the interactive object controlling the slider).
But Amariutei does not explicitly teach activating selection of the element triggers rendering of another element: “wherein activating selection of the graphical UI element triggers rendering of a graphical slider element …”.
However, Norieda teaches:
wherein activating selection of the graphical UI element triggers rendering of a graphical ([0185] Fig. 21 shows the input method selection menu 310 in response to user selection of input button 302 from Fig. 20).
Norieda and Amariutei are analogous art because Norieda also teaches method of overlaying menu on user’s body in an AR/HMD environment. Therefore, it would have been obvious to one of ordinary skills in art before the effective filing date, having the teachings of Amariutei and Norieda before him/her, to modify the teachings of Amariutei to include the teachings of Norieda so that upon selection of an GUI element, new menu can be rendered in place of existing one. One would be motivated to make the combination, with a reasonable expectation of success, because it would enable update on the rendered menu displayed on user’s hand in response to a selection to a displayed menu item, providing an improved interactive user interface on a real world object such as user’s hands, enhancing user experiences.
Claim(s) 3, 10, 17 is/are rejected under 35 U.S.C. 103 as being as being unpatentable over Amariutei, in view of Dash (US Pub 20170090747, hereinafter Dash).
Per claim 10, Amariutei teaches all the limitations of claim 8, but does not explicitly teach “wherein the rendering of the graphical UI elements is configured to track movements of the fingertips to maintain positioning of the graphical UI elements in respective association with the fingertips”.
However, Dash teaches:
wherein the rendering of the graphical UI elements is configured to track movements of the fingertips to maintain positioning of the graphical UI elements in respective association with the fingertips. ([0003, 0028-0030] system 200 from Fig. 2 comprises a display/transparent reflector 204 for reflecting rendered image such as the character map image 228/238 shown in Fig. 2A-2B, where the character map is rendered based on the tracked position and orientation of user’s hands; [0033-0035] Fig. 3A-3C show when the fingers of the hand moves, the positions of the fingers including the fingertips are tracked to render the projected character map based on the movement of the fingers; i.e. 300n from Fig. 3A vs. 304n from Fig. 3C).
Dash and Amariutei are analogous art because Dash also teaches method of overlaying menu on user’s hand in an AR/HMD environment. Therefore, it would have been obvious to one of ordinary skills in art before the effective filing date, having the teachings of Amariutei and Dash before him/her, to modify the teachings of Amariutei to include the teachings of Dash so that position of user’s finger can be tracked to prepare character/menu overlay based on the position and orientations of user’s hand/fingers. One would be motivated to make the combination, with a reasonable expectation of success, because it would generate a rendered character/menu map that is warped around the current position/orientation of user’s hand/fingers, providing a more realistic view of the character/menu overlay, and improving user experiences.
Per claim 3, 17, claims 3 and 17 include limitations that are substantially the same as claim 10, and are likewise rejected as claim 10.
Claim(s) 5, 12, 19 is/are rejected under 35 U.S.C. 103 as being as being unpatentable over Amariutei, in view of Abovitz et al. (US Pub 20150247975, hereinafter Abovitz).
Per claim 12, Amariutei teaches all the limitations of claim 8, but Amariutei does not explicitly teach “wherein rendering the graphical UI elements is in response to detecting the user's hand in an open-hand pose with a palm side of the hand facing the user”.
Abovitz, however, teaches:
wherein rendering the graphical UI element is in response to detecting the user's hand in an open-hand pose with a palm side of the hand facing the user. ([0656] Fig. 52A: in response to detecting a first defined gesture of user opening or displaying open palm of hand, the AR system renders a primary menu in a field of view of the user so as to appear to be overlaid on the user’s hand).
Abovitz and Amariutei are analogous art because Abovitz also teaches method of overlaying menu on user’s hand in an AR/HMD environment. Therefore, it would have been obvious to one of ordinary skills in art before the effective filing date, having the teachings of Amariutei and Abovitz before him/her, to modify the teachings of Amariutei to include the teachings of Abovitz so that a predefined gesture such as open hand with palm up can trigger the display of menu overlay. One would be motivated to make the combination, with a reasonable expectation of success, because it would provide predefined gestures to trigger display of the menu overlaid on top of the selected interactive surface area, providing more customization for users to indicate selection of interactive surface area, and improving user experiences.
Per claim 5, 19, claims 5 and 19 include limitations that are substantially the same as claim 12, and are likewise rejected as claim 12.
Conclusion
The examiner requests, in response to this Office action, support by shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line no(s) in the specification and/or drawing figure(s). This will assist the examiner in prosecuting the application.
When responding to this office action, Applicant is advised to clearly point out the patentable novelty which he or she thinks the claims present, in view of the state of the art disclosed by the references cited or the objections made. He or she must also show how the amendments avoid such references or objections, See 37 CFR 1.111(c).
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHOEBE X PAN whose telephone number is (571)270-7794. The examiner can normally be reached M-F 9am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Fred Ehichioya can be reached at (571) 272-4034. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHOEBE X PAN/Examiner, Art Unit 2179
/IRETE F EHICHIOYA/Supervisory Patent Examiner, Art Unit 2179
Prosecution Insights