DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Response to Amendment
The amendment filed on May 1, 2026 has been entered. Claims 1-15 are now pending in the application. Applicant's amendments have addressed all informalities as previously set forth in the non-final action mailed on February 3, 2026.
Response to Arguments
Applicant’s arguments filed May 1, 2026 with respect to previous claims 1-15 have been considered but are moot because the arguments do not apply to any of the combination of references being used in the current rejection.
In regards to independent claim 1, the Lopez reference is now cited as it discloses an augmented reality (AR) device that can access a library of applications or user interfaces (UIs) designed to control a set of devices. The AR device can determine which UI to present based on detection of a device to be controlled near the AR device (see abstract).
Lopez discloses that FIG. 8 shows five example UIs 810-850 for different target devices. The UI 810 is presented to the right of a thermostat and shows the current temperature (in Celsius or Fahrenheit) and mode of the thermostat as well as temperatures in three other rooms which can be interpreted as an example in which display information that the user would be unable to acquire about the device of interest or a “cognitive gap” regarding the user’s knowledge based. Next, Lopez details the UI 830 is presented for a door. The UI 830 includes a security image superimposed over the door and a message indicating that the door is currently locks. The UI 840 is presented for a refrigerator. The UI 840 indicates the current temperature of the refrigerator and also provides information regarding the contents of the refrigerator, this is interpreted as additional examples of providing display information the user would be unable to acquire about the device of interest. In addition, the reference discloses that the example UIs 810-850 may be displayed in response to detecting the user's gaze upon the associated target device. In some example embodiments, the displayed UI may be hidden by detecting that the user's gaze is no longer on the UI and has remained off of the UI for a predetermined period of time (e.g., 3 seconds), by detecting that the user's head is directed in a direction that removes the target device from the user's field of view (see paragraphs [0054], [0056], and [0059]) as further detailed in the rejections of the office action below.
In regards to independent claim 11, this claim recites limitations similar in scope to that of claim 1, and thus remains rejected under the same rationale as provided above and further detailed in the rejections of the office action below.
In regards to dependent claims 7-11 and 15, these claims depend from the rejected base claims 1 and 11, and therefore they remain rejected under the same rationale as provided above and further detailed in the rejections of the office action below.
Allowable Subject Matter
Claims 2-6 and 12-14 are objected to as being dependent upon a rejected base claim, but would be allowable if the claims are rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter:
In regards to dependent claim 2, none of the cited prior art alone or in combination provides motivation to teach “receiving device identification information and notification information from the device of interest; identifying whether the device of interest includes a display screen based on the device identification information; and based on identifying that the device of interest does not include the display screen, determining to display the notification information” as the references only teach techniques for object identification and rendering additional information about the corresponding object via augmented reality as well as identifying devices via AR with functions for identifying the devices facing direction, however the references fail to explicitly disclose the combination of concepts for identifying whether a target device possesses a screen and when the determination finds that it doesn’t, providing notification info to the AR device, in conjunction with the features of claim 1, for the purpose of displaying additional info of the device of interest using an AR image.
In addition, there is no teaching, suggestion, or motivation found in the current references and none that can be inferred from the examiner’s own knowledge with respect to the current limitation.
In regards to dependent claim 12, this claim recites limitations similar in scope to claim 2, and thus is objected to under the same rationale as provided above.
In regards to dependent claims 3-6, 13, and 14, these claims depend from objected to base claims 2 and 12, and thus are objected to under the same rationale as provided above.
As allowable subject matter has been indicated, applicant's reply must either comply with all formal requirements or specifically traverse each requirement not complied with. See 37 CFR 1.111(b) and MPEP § 707.07(a).
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 7-11, and 15 are rejected under 35 U.S.C. 102(a)(1)/(a)(2) as being anticipated by Lopez (US 2016/0274762 A1, hereinafter referenced “Lopez”).
In regards to claim 1 (Currently Amended). Lopez discloses a method, performed by an augmented reality (AR) device (Lopez, Abstract), the method comprising:
-identifying a device of interest (Lopez, para [0062]; Reference discloses in operation 1010, the system detects one or more target devices in the user's field of view and waits for a selection of a device (i.e. identified device of interest));
-determining to display information about the device of interest when a user is unable to acquire the information about the device if interest (Lopez, Fig. 8 and para [0054], [0056], and [0059]; Reference at [0054] discloses FIG. 8 shows five example UIs 810-850 for different target devices. The UI 810 is presented to the right of a thermostat and shows the current temperature (in Celsius or Fahrenheit) and mode of the thermostat as well as temperatures in three other rooms (i.e. display information the user would be unable to acquire about the device of interest). Para [0056] discloses the UI 830 is presented for a door. The UI 830 includes a security image superimposed over the door and a message indicating that the door is currently locked…The UI 840 is presented for a refrigerator. The UI 840 indicates the current temperature of the refrigerator and also provides information regarding the contents of the refrigerator (i.e. display information the user would be unable to acquire about the device of interest). Para [0059] discloses the example UIs 810-850 may be displayed in response to detecting the user's gaze upon the associated target device. In some example embodiments, the displayed UI may be hidden by detecting that the user's gaze is no longer on the UI and has remained off of the UI for a predetermined period of time (e.g., 3 seconds), by detecting that the user's head is directed in a direction that removes the target device from the user's field of view);
-and displaying an AR image corresponding to the information about the device of interest based on the determination (Lopez, Fig. 8 and para [0054], [0056], and [0059]; Reference at para [0056] discloses the UI 830 is presented for a door. The UI 830 includes a security image superimposed over the door and a message indicating that the door is currently locked…The UI 840 is presented for a refrigerator. The UI 840 indicates the current temperature of the refrigerator and also provides information regarding the contents of the refrigerator (i.e. examples of display information the user would be unable to acquire about the device of interest). Para [0059] discloses the example UIs 810-850 may be displayed in response to detecting the user's gaze upon the associated target device. In some example embodiments, the displayed UI may be hidden by detecting that the user's gaze is no longer on the UI and has remained off of the UI for a predetermined period of time (e.g., 3 seconds), by detecting that the user's head is directed in a direction that removes the target device from the user's field of view).
In regards to claim 7 (Currently Amended). Lopez discloses the method of claim 1.
Lopez further discloses
-wherein the device of interest comprises at least one of: a device in which an event occurred among devices connected to the AR device, a device selected by a user from among the devices connected to the AR device, or a device newly connected to the AR device (Lopez, para [0062]; Reference discloses FIG. 10 shows a flowchart for an example method 1000 of providing interaction with target devices using an AR device. In operation 1010, the system detects one or more target devices in the user's field of view and waits for a selection of a device. The system detects the selection of a target device in operation 1020. The selection of the target device may be performed by detecting that the user's gaze is directed to the target device (i.e. a device selected by a user from among the devices connected to the AR device)).
In regards to claim 8 (Currently Amended). Lopez discloses the method of claim 1.
Lopez further discloses
-wherein the identifying the device of interest comprises: identifying, through an eye tracking sensor (Lopez, para [0042]; Reference discloses once an image is captured by the sensor of the camera module 280, the image is sent to an eye tracking layer for further processing (e.g. detection of eye features and calculation of gaze direction or point of regard)… These features may be used by a gaze estimation module 230, which may be in charge of calculating the point of regard of the user), a device at which a user is gazing; identifying whether the device at which the user is gazing is connected to the AR device; and based on identifying that the device at which the user is gazing is connected to the AR device, determining the device at which the user is gazing as the device of interest (Lopez, para [0062]; Reference discloses FIG. 10 shows a flowchart for an example method 1000 of providing interaction with target devices using an AR device. In operation 1010, the system detects one or more target devices in the user's field of view and waits for a selection of a device. The system detects the selection of a target device in operation 1020. The selection of the target device may be performed by detecting that the user's gaze is directed to the target device (i.e. determining the device at which the user is gazing as the device of interest)).
In regards to claim 9 (Original). Lopez discloses the method of claim 8.
Lopez further discloses
-wherein the identifying, through the eye tracking sensor, the device at which the user is gazing comprises: acquiring, through the eye tracking sensor (Lopez, para [0042]; Reference discloses once an image is captured by the sensor of the camera module 280, the image is sent to an eye tracking layer for further processing (e.g. detection of eye features and calculation of gaze direction or point of regard)… These features may be used by a gaze estimation module 230, which may be in charge of calculating the point of regard of the user), a gaze direction of the user; acquiring, through a camera, a real world scene image; identifying at least one device from the real world scene image (Lopez, para [0050]; Reference discloses with respect to FIGS. 4-13, a VR device would perform in largely the same manner as an AR device, except that the user would be viewing a computer-generated image instead of a physical room. In some example embodiments, the VR device includes one or more world-facing cameras that record the scene in front of the user. The VR device may include images of the scene captured in this way within the virtual scene, overlaid on top of the virtual scene, or both. Similarly, the devices seen by the user are virtual, and may correspond to physical devices or virtual ones. For example, a user that is travelling may use VR to tour his own living room and control physical devices therein); and determining the device at which the user is gazing based on a location of the at least one device in the real world scene image and the gaze direction of the user (Lopez, para [0062]; Reference discloses FIG. 10 shows a flowchart for an example method 1000 of providing interaction with target devices using an AR device. In operation 1010, the system detects one or more target devices in the user's field of view and waits for a selection of a device. The system detects the selection of a target device in operation 1020. The selection of the target device may be performed by detecting that the user's gaze is directed to the target device).
In regards to claim 10 (Original). Lopez discloses the method of claim 1.
Lopez further discloses
-wherein the displaying the AR image corresponding to the information about the device of interest comprises: acquiring, through a camera, a real world scene image; identifying the device of interest from the real world scene image; determining an area to display the AR image based on a location of the device of interest; and displaying the AR image on the determined area (Lopez, para [0050]; Reference discloses with respect to FIGS. 4-13, a VR device would perform in largely the same manner as an AR device, except that the user would be viewing a computer-generated image instead of a physical room. In some example embodiments, the VR device includes one or more world-facing cameras that record the scene in front of the user. The VR device may include images of the scene captured in this way within the virtual scene, overlaid on top of the virtual scene, or both. Similarly, the devices seen by the user are virtual, and may correspond to physical devices or virtual ones. For example, a user that is travelling may use VR to tour his own living room and control physical devices therein).
In regards to claim 11 (Currently Amended). Lopez discloses an augmented reality (AR) device configured to display information about a device connected to the AR device (Lopez, Abstract), the AR device comprising:
-a display (Lopez, para [0030]; Reference discloses the AR device 100 includes a display 130);
-a memory storing a program comprising one or more instructions (Lopez, para [0040]; Reference discloses FIG. 2 is a block diagram of an example software architecture 200 for facilitating eye tracking control. Any one or more of the components of the software architecture 200 may run on either a central processing unit (CPU) of the AR device 100 or on a combination of a CPU and a graphics processing unit (GPU) of the AR device 100);
-and at least one processor configured to execute the one or more instructions, wherein the one or more instructions, when executed by the at least one processor (Lopez, para [0040]; Reference discloses FIG. 2 is a block diagram of an example software architecture 200 for facilitating eye tracking control. Any one or more of the components of the software architecture 200 may run on either a central processing unit (CPU) of the AR device 100 or on a combination of a CPU and a graphics processing unit (GPU) of the AR device 100), cause the AR device to:
-identify a device of interest (Lopez, para [0062]; Reference discloses in operation 1010, the system detects one or more target devices in the user's field of view and waits for a selection of a device (i.e. identified device of interest)), para [0080]; Reference discloses specifically, the display unit 151 may display an image in which at least one object associated with the augmented reality information is displayed. For example, when the camera 121 photographs an object associated with the augmented reality information (i.e. object interpreted as identified object of interest)),
-determine to display information about the device of interest when a user is unable to acquire the information about the device if interest (Lopez, Fig. 8 and para [0054], [0056], and [0059]; Reference at [0054] discloses FIG. 8 shows five example UIs 810-850 for different target devices. The UI 810 is presented to the right of a thermostat and shows the current temperature (in Celsius or Fahrenheit) and mode of the thermostat as well as temperatures in three other rooms (i.e. display information the user would be unable to acquire about the device of interest). Para [0056] discloses the UI 830 is presented for a door. The UI 830 includes a security image superimposed over the door and a message indicating that the door is currently locked…The UI 840 is presented for a refrigerator. The UI 840 indicates the current temperature of the refrigerator and also provides information regarding the contents of the refrigerator (i.e. display information the user would be unable to acquire about the device of interest). Para [0059] discloses the example UIs 810-850 may be displayed in response to detecting the user's gaze upon the associated target device. In some example embodiments, the displayed UI may be hidden by detecting that the user's gaze is no longer on the UI and has remained off of the UI for a predetermined period of time (e.g., 3 seconds), by detecting that the user's head is directed in a direction that removes the target device from the user's field of view),
-and display an AR image corresponding to the information about the device of interest through the display based on the determination (Lopez, Fig. 8 and para [0054], [0056], and [0059]; Reference at para [0056] discloses the UI 830 is presented for a door. The UI 830 includes a security image superimposed over the door and a message indicating that the door is currently locked…The UI 840 is presented for a refrigerator. The UI 840 indicates the current temperature of the refrigerator and also provides information regarding the contents of the refrigerator (i.e. examples of display information the user would be unable to acquire about the device of interest). Para [0059] discloses the example UIs 810-850 may be displayed in response to detecting the user's gaze upon the associated target device. In some example embodiments, the displayed UI may be hidden by detecting that the user's gaze is no longer on the UI and has remained off of the UI for a predetermined period of time (e.g., 3 seconds), by detecting that the user's head is directed in a direction that removes the target device from the user's field of view).
In regards to claim 15. Lopez discloses the AR device of claim 11.
Lopez further discloses
-wherein the one or more instructions, when executed by the at least one processor, further cause the AR device to: identify, through an eye tracking sensor (Lopez, para [0042]; Reference discloses once an image is captured by the sensor of the camera module 280, the image is sent to an eye tracking layer for further processing (e.g. detection of eye features and calculation of gaze direction or point of regard)… These features may be used by a gaze estimation module 230, which may be in charge of calculating the point of regard of the user), a device at which a user is gazing, identify whether the device at which the user is gazing is connected to the AR device, and based on identifying that the device at which the user is gazing is connected to the AR device, determine the device at which the user is gazing as the device of interest (Lopez, para [0062]; Reference discloses FIG. 10 shows a flowchart for an example method 1000 of providing interaction with target devices using an AR device. In operation 1010, the system detects one or more target devices in the user's field of view and waits for a selection of a device. The system detects the selection of a target device in operation 1020. The selection of the target device may be performed by detecting that the user's gaze is directed to the target device (i.e. determining the device at which the user is gazing as the device of interest)).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: See the Notice of References Cited (PTO-892)
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TERRELL M ROBINSON whose telephone number is (571)270-3526. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, KENT CHANG can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TERRELL M ROBINSON/Primary Examiner, Art Unit 2614
Prosecution Insights