Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 3/10/2026 has been entered.
Response to Amendment
The amendment filed on 2/12/2026 has been entered and made of record. Claims 1, 10-12 and 20 are amended. Claims 1-20 are pending.
Response to Arguments
Applicant’s arguments with respect to the rejections of independent claims 1 & 20 have been fully considered but they are moot because the arguments do not apply to the references being used in the current rejection.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1 and 20 are rejected under 35 U.S.C. 102(a)(1) or (a)(2) as being anticipated by Singh et al. (US 2025/0005868 A1).
As to Claim 1, Singh teaches A device for displaying information, the device comprising: at least one memory; and at least one processor coupled to the at least one memory and configured to (Singh, Fig 2):
obtain an image of a scene from a first camera; output the image for display at a display panel (Singh discloses “This may involve the system examining images or video footage captured by an XR device, which represent the scene as observed by the user through an optical-see-through (OST) or visual-see-through (VST) display” in [0054]);
detect an object in the image (Singh discloses “Following the image capture, the system then proceeds to analyze the image to identify the object 10” in [0142], see also [0097]);
determine that a user is gazing at a representation of the object in the image displayed at the display panel, based on an image of the user obtained from a second camera (Singh discloses “In some examples, the user's gaze patterns may be monitored, e.g., to identify gaze locks, referring to when the user's gaze remains focused on an area in the XR environment for longer than a pre-determined period. In some examples, images corresponding to the identified gaze lock may be captured and analyzed to identify objects and their common attributes” in [0012]);
based on determining that the user is gazing at the representation of the object, output information associated with the object for display at the display panel (Singh discloses “In response to this determination, a virtual overlay may be displayed, e.g., providing information relating to the object” in [0008].)
Claim 20 recites similar limitations as claim 1 but in a method form. Therefore, the same rationale used for claim 1 is applied.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4-5, 7-10 and 13-20 are rejected under 35 U.S.C. 103 as being unpatentable over by Vaught et al. (US 2013/0187835 A1) in view of Singh et al. (US 2025/0005868 A1).
As to Claim 1, Vaught teaches A device for displaying information, the device comprising: at least one memory; and at least one processor coupled to the at least one memory and configured to (Vaught, Fig 1, 8-9 & 11):
obtain an image of a scene from a first camera; output the image for display at a display panel; detect an object in the image (Vaught discloses “Accordingly, embodiments are disclosed herein that relate to the use of a see-through display system, such as a head-mounted display (HMD), that utilizes gaze detection and outward-facing image sensor( s) to acquire and utilize information related to particular objects in the field of view of the user that are the subject of a user's current focus, such as objects shown on a display screen external to the see-through display device” in [0016]; see also [0027]. Here, HMD can be a pass-through HMD, for example, Singh discloses “This may involve the system examining images or video footage captured by an XR device, which represent the scene as observed by the user through an optical-see-through (OST) or visual-see-through (VST) display” in [0054]);
determine that a user is gazing at a representation of the object in the image displayed at the display panel, based on an image of the user obtained from a second camera (Vaught discloses “This may comprise, for example, receiving image data from one or more inward-facing image sensors, and analyzing the image data to determine the position of the user's eye based upon location of glints reflected from the eye of the user. This may allow the detection of a direction of gaze of each eye of the user, as indicated at 1006” in [0038], see also [0027-0028]); and
based on determining that the user is gazing at the representation of the object, output information associated with the object for display at the display panel (Vaught discloses “In some embodiments, contextual information related to an object of interest on an external display screen may be obtained from a source other than a device controlling the external display screen. In such embodiments, image recognition techniques may be used to determine an identity of the object on the external display screen at which the user is gazing so that contextually relevant information about the object may be obtained and displayed” in [0017]; “In response, the see-through display system 100 retrieves and displays a list 206 of messages currently in the inbox of an email account linked to the user's mobile device 104. The user may then be able to open a selected message by gazing at the selected message in the inbox displayed on the see-through display system 100” in [0018]; “The see-through display system 800 comprises one or more lenses 802 that form a part of a see-through display subsystem 804, such that images may be projected onto the lenses 802, or produced by image-producing elements (e.g. see-through OLED displays) located within the lenses 802. The see-through display system 800 further comprises one or more outward facing image sensors 806 configured to acquire images of a background scene being viewed by a user” in [0032]; “The see-through display system 800 further comprises a gaze detection subsystem 810 configured to detect a direction of gaze of each eye of a user…Changes in the glints from the user's eyeballs as determined from image data gathered via image sensor(s) 814 may be used to determine a direction of gaze. Further, a location at which gaze lines projected from the user's eyes intersect the external display may be used to determine an object at which the user is gazing (e.g. a virtual object displayed on an external display)” in [0033]. See also [0036] and Fig 3-6. Singh, [0008].)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Vaught with the teaching of Singh so that a head-mounted display can provide both virtual reality, augmented reality or mixed reality environment (Singh, [0047]).
As to Claim 2, Vaught in view of Singh teaches The device of claim 1, wherein the first camera comprises a scene-facing camera (Vaught discloses “The see-through display system 800 further comprises one or more outward facing image sensors 806 configured to acquire images of a background scene being viewed by a user” in [0032]. Singh, [0141].)
As to Claim 4, Vaught in view of Singh teaches The device of claim 2, wherein the at least one processor is further configured to initiate an application that activates the scene-facing camera (Vaught discloses HMD including outward facing image camera in [0016,0035, 0037]. Singh further discloses initiating XR session to capture images of the scene in Fig 11.)
As to Claim 5, Vaught in view of Singh teaches The device of claim 1, wherein the second camera comprises a user-facing camera (Vaught discloses “This may comprise, for example, receiving image data from one or more inward-facing image sensors, and analyzing the image data to determine the position of the user's eye based upon location of glints reflected from the eye of the user. This may allow the detection of a direction of gaze of each eye of the user, as indicated at 1006” in [0038].)
As to Claim 7, Vaught in view of Singh teaches The device of claim 5, wherein the at least one processor is further configured to initiate an application that activates the user-facing camera (Vaught discloses HMD including inward-facing image sensor in [0038]. Singh further discloses initiating XR session to identify user’s gazing focused on a particular area for a longer than time period in Fig 11.)
As to Claim 8, Vaught in view of Singh teaches The device of claim 1, wherein the at least one processor is further configured to initiate an application that activates at least one of the first camera or the second camera (Vaught discloses HMD including outward facing image camera in [0016,0035, 0037] and inward-facing image sensor in [0038]. Singh further discloses initiating XR session to identify user’s gazing focused on a particular area for a longer than time period and capture images of the scene in Fig 11.)
As to Claim 9, Vaught in view of Singh teaches The device of claim 1, wherein the device comprises a handheld device (Vaught discloses “The see-through display system 702 may comprise any suitable type of device, including but not limited to the HMD shown in FIGS. 1-6” in [0025]. Singh discloses a smartphone and tablet in [0129].)
As to Claim 10, Vaught in view of Singh teaches The device of claim 1, wherein the device comprises the first camera, the second camera, and the display panel (Vaught, Fig 1 & 8-9. Singh, Fig 7-8.)
As to Claim 13, Vaught in view of Singh teaches The device of claim 1, wherein the at least one processor is further configured to initiate an action responsive to determining that the object is the object of interest of the user (Vaught discloses “As can be seen in FIG. 2, gaze lines 200, 202 are determined to lead from the eyes of the user to an email user interface tile 204. In response, the see-through display system 100 retrieves and displays a list 206 of messages currently in the inbox of an email account linked to the user's mobile device 104. The user may then be able to open a selected message by gazing at the selected message in the inbox displayed on the see-through display system 100” in [0018]; see also [0029-0031, 0036].)
As to Claim 14, Vaught in view of Singh teaches The device of claim 13, wherein the action is based on the object (Vaught discloses “obtaining an identity of an object displayed on the external display screen at the location determined, and performing an action based upon the identity of the object” in [0002], see also [0018, 0028, 0042-0043].)
As to Claim 15, Vaught in view of Singh teaches The device of claim 14, wherein the action is at least one of: translating text, wherein the object comprises the text; preparing a communication, wherein the object comprises contact information and the communication is based on the contact information; requesting data from a server, wherein the object comprises a barcode or a quick-response (QR) code and the requested data is based on the barcode or the QR code; identifying the object; or logging the object (Vaught, [0018, 0029-0031, 0036].)
As to Claim 16, Vaught in view of Singh teaches The device of claim 13, wherein the action is initiated further responsive to an instruction from the user (Vaught, [0020, 0023-0024, 0036, 0043].)
As to Claim 17, Vaught in view of Singh teaches The device of claim 16, wherein the information displayed at the display comprises a prompt relative to the instruction (Vaught, [0023-0024, 0035].)
As to Claim 18, Vaught in view of Singh teaches The device of claim 17, further comprising interpreting, as the instruction, at least one of: a vocalization of the user; a hand gesture of the user; a head motion of the user; or an eye motion of the user (Vaught discloses “The use of motion data may allow changes in gaze location to be tracked even if image data from the outward-facing image sensor(s) 806 cannot be resolved Likewise, the motion sensors 818, as well as the microphone(s) 808 and the gaze detection subsystem 810, also may be employed as user input devices, such that a user may interact with see-through display system 800 via gestures of the eye, neck and/or head, as well as via verbal commands” in [0035], see also [0024].)
As to Claim 19, Vaught in view of Singh teaches The device of claim 1, wherein the at least one processor is further configured to detect a plurality of objects in the scene based on images of the scene (Vaught, [0023, 0041] and claims 8 & 18.)
Claim 20 recites similar limitations as claim 1 but in a method form. Therefore, the same rationale used for claim 1 is applied.
Claims 3 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over by Vaught in view of Singh and Ha (US 2021/0034870 A1).
As to Claim 3, Vaught in view of Singh teaches The device of claim 2, wherein the scene-facing camera is configured to capture images by default when the device is active (Vaught discloses “The see-through display system 702 may take any suitable form” in [0032]; “for example, receiving two-dimensional and/or three-dimensional still or video image data from one or more outward-facing image sensors on a see-through display device, and locating the external display screen in the image data” in [0037]. It is not critical to the setting of one camera as default. It is rendered obvious as a design choice (see MPEP 2144.04) that would have no impact on the function or results of the claimed invention. For example, Ha discloses “By default, the recognition camera 145 captures a scene in the direction along which the user's eyes are located” in [0228].)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Vaught and Singh with the teaching of Ha so as to set some default values for some device elements during the device calibration process.
As to Claim 6, Vaught in view of Singh teaches The device of claim 5, wherein the user-facing camera is configured to capture images by default when the device is active (Vaught discloses an inward-facing image sensor in [0038]. It is not critical to the setting of one camera as default. It is rendered obvious as a design choice (see MPEP 2144.04) that would have no impact on the function or results of the claimed invention. For example, Ha discloses “By default, the recognition camera 145 captures a scene in the direction along which the user's eyes are located but may be configured to capture the image of the user's eyes reflected from the transparent display 110 and detect the eye motion, looking direction of the eyes, and size change of the eyes” in [0228].)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Vaught and Singh with the teaching of Ha so as to set some default values for some device elements during the device calibration process.
Claims 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over by Vaught in view of Singh and Zimmermann et al. (US 2021/0312684 A1).
As to Claim 11, Vaught in view of Singh teaches The device of claim 1, wherein: the first camera is configured to capture a field of view of the scene; the display panel occupies a portion of a field of view of the user; and the field of view of the scene is different than the portion of the field of view of the user occupied by the display panel (Vaught discloses “Accordingly, embodiments are disclosed herein that relate to the use of a see-through display system, such as a head-mounted display (HMD), that utilizes gaze detection and outward-facing image sensor(s) to acquire and utilize information related to particular objects in the field of view of the user that are the subject of a user's current focus, such as objects shown on a display screen external to the see-through display device” in [0016]. Zimmermann further discloses “The wearable system 400 can include an outward-facing imaging system 464 (e.g., a digital camera) that images a portion of the world 470. This portion of the world 470 may be referred to as the field of view (FOV) of a world camera and the imaging system 464 is sometimes referred to as an FOV camera. The FOV of the world camera may or may not be the same as the FOV of a viewer 210 which encompasses a portion of the world 470 the viewer 210 perceives at a given time.” in [0096], see also [0271].)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Vaught and Singh with the teaching of Zimmermann so as to explain the difference between the field of view of the captured scene and the field of view of user.
As to Claim 12, Vaught in view of Singh teaches The device of claim 1, wherein a line of sight between the user and the object is different than a line of sight between the user and the representation of the object in the image displayed by the display panel (Zimmermann, [0175-0180]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Vaught and Singh with the teaching of Zimmermann so as to generate a scaled object that maintains roughly the same eye height as the viewer (Zimmermann, [0174].)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEIMING HE whose telephone number is (571)270-1221. The examiner can normally be reached on Monday-Friday, 8:30am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached on 571-272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEIMING HE/
Primary Examiner, Art Unit 2611
Prosecution Insights