DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is made non-final.
Claims 1-24 are pending in the case. Claims 1, 9, and 17 are independent claims.
Priority
The instant application claims domestic benefit of provisional application 63/556,592 filed 02/22/2024 and provisional application 63/586,969 filed 09/29/2023.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 6, 9-12, 14, 17-20, and 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Estrada Diaz (US 2022/0121808 A1), in view of Hill et al. (US 2016/0313902 A1).
Regarding claim 1, Estrada Diaz teaches a method comprising:
at an electronic device in communication with one or more displays, one or more input devices, and one or more cameras (FIGS. 1A and 3, [0034-0036], and [0053-0058]: AR glasses 300, corresponding to wearable electronic device 102, is in communication with display device 304, input devices such as sensing system 310 and/or control system 312, and camera 316; For additional details, see computer system 200 of FIG. 2 and [0041-0052]. As supported in [0041], “the computer system 200 can include some or all components described below with reference to wearable electronic device 102, FIG. 6, and/or other wearable device with processing power.” Note the display device categorized as output devices 204, the input devices 202, and camera categorized as input devices 202):
capturing, via the one or more cameras, one or more images of a physical environment, including a first object (step 504 of FIG. 5 and [0072]; FIG. 4 and [0059-0067]: for example, an image including a first object/form 402);
in accordance with detecting, in the one or more images, a first region of the first object and that one or more first criteria are satisfied, displaying, via the one or more displays, a first user interface element including first suggested text in a computer-generated environment, wherein the first user interface element is displayed at a first location (steps 506-510 of FIG. 5 and [0073-0083]; FIG. 4 and [0059-0067], [0077], and [0079]: continuing the example, a first region, such as a blank field for a driver’s license number, is detected. One or more first criteria are satisfied in response to detection of a hover input over the first region. A first user interface element 416 includes first suggested text and displayed at a first location); and
in accordance with detecting, in the one or more images, a second region, different from the first region, of the first object and that one or more second criteria are satisfied, displaying, via the one or more displays, a second user interface element including second suggested text in the computer-generated environment, wherein the second user interface element is displayed at a second location, (steps 506-510 of FIG. 5 and [0073-0083]; FIG. 4 and [0059-0067], [0079]: continuing the example, note how the user moves onto the next blank field, representing a second region. Repeating the same process done for the first region, one or more second criteria are satisfied in response to detection of a hover input over the second region, such as phone or whatever information is prompted. As supported in [0067] and [0076], information for a second user interface element/visual cue is provided and displayed at a second location).
Estrada Diaz does not explicitly teach displaying the first user interface element at a first location, based on the first region, and displaying the second user interface element at a second location, different from the first location, based on the second region.
Hill teaches displaying a first user interface element at a first location, based on the first region, and a second user interface element is displayed at a second location, different from the first location, based on the second region (FIGS. 3A-D and [0060-0069]: as seen in FIG. 3A, first object corresponds to objects 104. A user may gaze toward first region corresponding to object 104D as seen in FIG. 3B, satisfying a criterion of a first criteria when detecting the first region for prompting a visual cue. A user may similarly gaze toward second region corresponding to object 104E as seen in FIG. 3D, satisfying a criterion of a second criteria when detecting the second region for prompting a visual cue).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the display of the user interface elements in their respective locations of Estrada Diaz by incorporating the teachings of Hill so as to include displaying a first user interface element at a first location, based on the first region, and a second user interface element is displayed at a second location, different from the first location, based on the second region. Doing so would reduce the likelihood for user error, as the user can more clearly associate the information provided by the respective user interface element for the appropriate field. In this way, the user would be less likely to confuse information specific for one region for another region.
Regarding claim 2, Estrada Diaz in view of Hill teaches the method of claim 1. Estrada Diaz further teaches wherein:
the first region corresponds to a first fillable field; and
the second region corresponds to a second fillable field, different from the first fillable field (FIG. 4 and [0059-0067], steps 506-510 of FIG. 5 and [0073-0083]: for example, the first region corresponds to a first fillable field, such as driver’s license number, and the second region corresponds to a second fillable field, such as phone or whatever else information is prompted).
Regarding claim 3, Estrada Diaz in view of Hill teaches the method of claim 1. Although Estrada Diaz teaches a gaze tracking component that “determine[s] where the user is currently looking (e.g., with reference to a document/form)” ([0043] and [0045]) as well as a criterion that is satisfied when the user “focuses” on some region ([0027]: “The AR glasses may detect input fields in such a form, retrieve information pertaining to the input fields, and logically provide the retrieved information as assistive data, according to which input field a user may be hovered upon and/or otherwise focused upon.”), Estrada Diaz does not explicitly teach the one or more first criteria include a criterion that is satisfied when gaze of a user of the electronic device is directed toward the first region when detecting, in the one or more images, the first region of the first object; and the one or more second criteria include a criterion that is satisfied when the gaze of the user is directed toward the second region when detecting, in the one or more images, the second region of the first object.
Hill teaches the one or more first criteria include a criterion that is satisfied when gaze of a user of the electronic device is directed toward the first region when detecting, in the one or more images, the first region of the first object; and the one or more second criteria include a criterion that is satisfied when the gaze of the user is directed toward the second region when detecting, in the one or more images, the second region of the first object (FIGS. 3A-D and [0060-0069]: as seen in FIG. 3A, first object corresponds to objects 104. A user may gaze toward first region corresponding to object 104D as seen in FIG. 3B, satisfying a criterion of a first criteria when detecting the first region for prompting a visual cue. A user may similarly gaze toward second region corresponding to object 104E as seen in FIG. 3D, satisfying a criterion of a second criteria when detecting the second region for prompting a visual cue).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the gaze tracking component and “focus” aspects of Estrada Diaz by incorporating the teachings of Hill so as to include the one or more first criteria include a criterion that is satisfied when gaze of a user of the electronic device is directed toward the first region when detecting, in the one or more images, the first region of the first object; and the one or more second criteria include a criterion that is satisfied when the gaze of the user is directed toward the second region when detecting, in the one or more images, the second region of the first object. Doing so would remove reliance on an additional, peripheral device or object, such as Estrada Diaz’s writing utensil 110, to satisfy the criteria. Instead, the user may more conveniently place visual focus, which naturally coincides with a point of interest, to obtain relevant information.
Regarding claim 4, Estrada Diaz in view of Hill teaches the method of claim 1. Estrada Diaz further teaches wherein:
the one or more first criteria include a criterion that is satisfied when a respective object associated with a user of the electronic device is detected within a threshold distance of the first region when detecting, in the one or more images, the first region of the first object; and
the one or more second criteria include a criterion that is satisfied when the respective object associated with the user is detected within the threshold distance of the second region when detecting, in the one or more images, the second region of the first object ([0028], step 506 of FIG. 5 and [0073-0074], FIG. 4 and [0059-0067], [0077], and [0079]: a respective object/writing utensil 110 must be within a threshold distance for the camera 316 and/or sensing system 310 to detect the hover input. For example, a distance in which writing utensil 110 is outside of the field of view of the AR glasses 300 would fail the criterion, as opposed to a writing utensil hovering closer to the form and within a threshold distance that is inside the field of view of the AR glasses. This criterion is applied similarly for the one or more first criteria and the one or more second criteria; See also example of FIG. 1B and [0037-0040] in which detection of a hovering input is implemented as a criterion for some region of a form).
Regarding claim 6, Estrada Diaz in view of Hill teaches the method of claim 1. Estrada Diaz further teaches wherein:
the first suggested text is displayed based on one or more features of the first region, including first text that is included in the first region; and
the second suggested text is displayed based on one or more features of the second region, including second text that is included in the second region (FIG. 4 and [0059-0067], steps 506-510 of FIG. 5 and [0073-0083]: for example, the first region corresponds to a field for a driver’s license number, as indicated by the first text, and the second region corresponds to a field for a phone number, as indicated by the second text. The respective suggested text is displayed based on the respective text included in the respective region).
Regarding claims 9-12 and 14, the claims recite an electronic device comprising: one or more processors; memory; and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing (FIGS. 1A and 3, [0034-0036], and [0053-0058]; FIG. 2 and [0041-0052]) a method comprising steps corresponding to the method of claims 1-4 and 6, respectively, and are therefore rejected on the same premises.
Regarding claims 17-20 and 22, the claims recite a non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device (FIGS. 1A and 3, [0034-0036], and [0053-0058]; FIG. 2 and [0041-0052]), cause the electronic device to perform a method comprising steps corresponding to the method of claims 1-4 and 6, respectively, and are therefore rejected on the same premises.
Claim(s) 5, 13, and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Estrada Diaz (US 2022/0121808 A1), in view of Hill et al. (US 2016/0313902 A1), in view of Bladel et al. (US 2019/0296984 A1).
Regarding claim 5, Estrada Diaz in view of Hill teaches the method of claim 1. Estrada Diaz further teaches wherein:
the one or more first criteria include a criterion that is satisfied when a respective object associated with a user of the electronic device is detected as being directed toward the first region
the one or more second criteria include a criterion that is satisfied when the respective object associated with the user is detected as being directed toward the second region ([0028], step 506 of FIG. 5 and [0073-0074], FIG. 4 and [0059-0067]: a respective object/writing utensil 110, via a hover input, is directed toward some region as detected by the camera 316 and/or sensing system 310. This criterion is applied similarly for the one or more first criteria and the one or more second criteria; See also example of FIG. 1B and [0037-0040] in which detection of a hovering input is implemented as a criterion for some region of a form).
Estrada Diaz in view of Hill does not explicitly teach the respective object being directed toward the respective region for at least a threshold amount of time.
Bladel teaches wherein:
the one or more first criteria include a criterion that is satisfied when a respective object associated with a user of the electronic device is detected as being directed toward the first region for at least a threshold amount of time when detecting, in the one or more images, the first region of the first object; and
the one or more second criteria include a criterion that is satisfied when the respective object associated with the user is detected as being directed toward the second region for at least the threshold amount of time when detecting, in the one or more images, the second region of the first object (FIGS. 4 and 6 and [0050-0054], steps 703-704 of FIG. 7 and [0062-0065]: a criterion is satisfied when the user hovers a cursor over some region for at least a threshold amount of time when detecting some region, such as a region corresponding to a registered domain name. This criterion is similarly applied across the respective criteria for the respective region, such as a first region for a first domain name and a second region for a second domain name).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the physical hovering aspect of Estrada Diaz by incorporating the teachings of Bladel so as to include the one or more first criteria include a criterion that is satisfied when a respective object associated with a user of the electronic device is detected as being directed toward the first region for at least a threshold amount of time when detecting, in the one or more images, the first region of the first object; and the one or more second criteria include a criterion that is satisfied when the respective object associated with the user is detected as being directed toward the second region for at least the threshold amount of time when detecting, in the one or more images, the second region of the first object. Doing so would reduce the likelihood of unintentional triggering of display of first and/or second user interface elements since Bladel introduces a threshold amount of time requirement to satisfy the respective criterion. In this way, processing resources is conserved as additional information of the first and/or second user interface elements is less likely to be unnecessarily displayed.
Regarding claim 13, the claim recites an electronic device corresponding to the method of claim 5 and is therefore rejected on the same premise.
Regarding claim 21, the claim recites a non-transitory computer readable storage medium corresponding to the method of claim 5 and is therefore rejected on the same premise.
Claim(s) 7, 8, 15, 16, 23, and 24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Estrada Diaz (US 2022/0121808 A1), in view of Hill et al. (US 2016/0313902 A1), in view of Boyd et al. (US 2022/0319082 A1).
Regarding claim 7, Estrada Diaz in view of Hill teaches the method of claim 1. Estrada Diaz further teaches further comprising:
while displaying the first user interface element at the first location in the computer-generated environment, detecting, in the first region of the first object, first text, wherein the first text was not detected prior to displaying the first user interface element (FIG. 4 and [0059-0067]: while visual cue 416 is displayed, first text is detected in the first region as the user writes “LL113” with their writing utensil 110).
Estrada Diaz in view of Hill does not explicitly teach in response to detecting the first text in the first region of the first object, ceasing display of the first user interface element in the computer-generated environment.
Boyd teaches while displaying the first user interface element at the first location in the computer-generated environment, detecting, in the first region of the first object, first text, wherein the first text was not detected prior to displaying the first user interface element; and
in response to detecting the first text in the first region of the first object, ceasing display of the first user interface element in the computer-generated environment (FIG. 12 and [0173-0174]: as seen in user interface 1200, while first user interface element 1214 is displayed, first text is detected in first region/overlay 1208. As supported in [0173], “The text content of the overlay 1208 may be entered via a keyboard input device 1210 included in the user interface 1200.” Thus, the first text corresponds to the user typing “#MoviePremiere” in overlay 1208. In response to detecting the first text, display of the first user interface element 1214 ceases, as seen in user interface 1220).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Estrada Diaz by incorporating the teachings of Boyd so as to include while displaying the first user interface element at the first location in the computer-generated environment, detecting, in the first region of the first object, first text, wherein the first text was not detected prior to displaying the first user interface element; and in response to detecting the first text in the first region of the first object, ceasing display of the first user interface element in the computer-generated environment. Doing so would reduce clutter on the user interface since the user has completed entering text corresponding to the suggested text.
Regarding claim 8, Estrada Diaz in view of Hill teaches the method of claim 1. Estrada Diaz further teaches further comprising:
after displaying the first user interface element at the first location in the computer-generated environment, detecting, in the first region of the first object, first text, wherein the first text was not detected prior to displaying the first user interface element (FIG. 4 and [0059-0067]: after visual cue 416 is displayed, first text is detected in the first region as the user writes “LL113” with their writing utensil 110).
Estrada Diaz in view of Hill does not explicitly teach in response to detecting the first text: in accordance with a determination that at least a portion of the first text is different from a corresponding portion of the first suggested text, displaying, via the one or more displays, a visual indication in the computer-generated environment.
Boyd teaches after displaying the first user interface element at the first location in the computer-generated environment, detecting, in the first region of the first object, first text, wherein the first text was not detected prior to displaying the first user interface element; and
in response to detecting the first text:
in accordance with a determination that at least a portion of the first text is different from a corresponding portion of the first suggested text, displaying, via the one or more displays, a visual indication in the computer-generated environment (FIG. 12 and [0173-0174]: as seen in user interface 1200, after first user interface element 1214 is displayed, first text is detected in first region/overlay 1208. As supported in [0173], “The text content of the overlay 1208 may be entered via a keyboard input device 1210 included in the user interface 1200.” The currently typed “#MovieP” is missing a corresponding portion of the first suggest text “#MoviePremiere” seen in first user interface element 1214. Accordingly, a visual indication corresponding to user interface element 1212 is displayed).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Estrada Diaz by incorporating the teachings of Boyd so as to include after displaying the first user interface element at the first location in the computer-generated environment, detecting, in the first region of the first object, first text, wherein the first text was not detected prior to displaying the first user interface element; and in response to detecting the first text: in accordance with a determination that at least a portion of the first text is different from a corresponding portion of the first suggested text, displaying, via the one or more displays, a visual indication in the computer-generated environment. Doing so would allow the user to visually confirm whether the user is entering the correct text.
Regarding claims 15 and 16, the claims recite an electronic device corresponding to the method of claims 7 and 8, respectively, and are therefore rejected on the same premises.
Regarding claims 23 and 24, the claims recite a non-transitory computer readable storage medium corresponding to the method of claims 7 and 8, respectively, and are therefore rejected on the same premises.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure, including:
US 2021/0158044 A1: auto-complete suggestions integrated into a field-of-view of a user
US 2022/0058879 A1: assistive information for augmented reality
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KENNY NGUYEN whose telephone number is (571)272-4980. The examiner can normally be reached M-Th 7AM to 5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, KIEU D VU can be reached on (571)272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KENNY NGUYEN/Primary Examiner, Art Unit 2171
Prosecution Insights