DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 15 April 2026 has been entered.
Response to Arguments
Applicant’s arguments, see Response to Office Action mailed 16 January 2026, filed 15 April 2026, with respect to the rejection(s) of claim(s) 1-2 and 4-11 under 35 USC §102 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Pinel et al. (US 2020/0026097 A1).
Claim Objections
Claims 1 and 7 are objected to because of the following informalities: The claims recite multiple iterations of “Internet of Things... device” followed by an abbreviation in paragraphs. The multiple recitations and different abbreviations appears to convey they are multiple and distinct devices. However, reading the IOT devices as distinct devices may render the claim unclear and therefor indefinite. For example, if each of the three recitation of IOT/IoT is read as separate and distinct devices, it is unclear how the of displaying one auxiliary image corresponding to a first device is received by a second device or which third device receives the command. If the IOT/IoT devices are the same device, Examiner suggest amending the claim to remove all but the first recitation of “Internet of Things” and replace them with an abbreviation which does not vary. For the purposes of Examination, the claims are read as one IOT device receiving the command, update the display content, and receive the confirmation. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-2 and 4-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Olwal et al. (US 2023/0086766 A1) in view of Pinel et al. (US 2020/0026097 A1).
Regarding claim 1, Olwal discloses a rapid user input determination system comprising: a control device [Olwal: FIG. 2]; a head-mounted device signal-connected to the control device [Olwal: ¶ [0054] The wearable computing device 100 is depicted as AR glasses in this example. In general, the device 100 may include any or all components of systems 100 and/or 200 and or 600. The wearable computing device 100 may also be indicated as smart glasses representing an optical head-mounted display device designed in the shape of a pair of eyeglasses] and comprising at least one display and at least one eye-tracking unit [Olwal: ¶ [0007]: In an example embodiment, the detecting a defocus event associated with a first region of the content may be electronically detecting a user having previously focused on the first region of the content to no longer focus the first region (e.g., determined by a gaze trajectory of a user's eye or of the user's eyes and thus a viewing focus of the user)]; and a motion sensing device [Olwal: ¶ [0047]: In some implementations, the defocus event may include user-related motion such as a head turn, a head lift or lower, a device removal from the user (e.g., removing a wearable computing device), and/or any combination thereof; and ¶ [0065] The sensor system 214 may also include an inertial motion unit (IMU) sensor 220. The IMU sensor 220 may detect motion, movement, and/or acceleration of the wearable computing device 100. The IMU sensor 220 may include various different types of sensors such as, for example, an accelerometer, a gyroscope, a magnetometer, and other such sensors. In some implementations, the sensor system 214 may include screen embedded sensors that may detect particular user actions, motions, etc., directly from the virtual screen]; wherein, the control device is configured to receive a command as a user input and to output a corresponding content based on the command to the head-mounted device and the motion sensing device [Olwal: ¶ [0067]: In some implementations, the camera 224 may be a point tracking camera that can, for example, detect and follow one or more optical markers on an external device, such as, for example, optical markers on an input device or finger on a screen. The input may be detected by an input detector 215, for example]; the at least one display is configured to demonstrate a display content, the display content comprises a virtual reality image or an augmented reality image, the virtual reality image or the augmented reality image includes an auxiliary image and to update the display content based on the corresponding content received from the control device [Olwal: ¶ [0067]], the eye-tracking unit is configured to track a movement and a motion trajectory of at least one of the user's eyes [Olwal: ¶ [0007]] to determine a visual focus of the user on the display content [Olwal: ¶ [0007]], the motion sensing device is configured to detect a motion or a posture of the user's hand and to transmit a hand- gesture information to the control device [Olwal: ¶ [0067]], wherein the motion and the posture are respectively defined as a triggering action and a triggering posture[Olwal: ¶ [0038]: For example, if the user performs an action that triggers an object to move, the user may follow the trajectory of the object. While the user looks away (i.e., defocus on content), the object continues to move in a trajectory. Thus, when the user refocuses on the content, the model 116 may generate and render focus transition markers to guide the user to the current position of the object], wherein the command comprises the control device to confirm the visual focus of the user on the display content and to receive the triggering action or the triggering posture of the user as the command [Olwal: ¶ [0038]: For example, if the user performs an action that triggers an object to move, the user may follow the trajectory of the object. While the user looks away (i.e., defocus on content), the object continues to move in a trajectory. Thus, when the user refocuses on the content, the model 116 may generate and render focus transition markers to guide the user to the current position of the object], and wherein at least a portion of the motion sensing device is embedded in or detachably mounted on the head-mounted device [Olwal: FIG. 1].
Olwal may not explicitly disclose the control device is configured to receive a command as a user input and to output a corresponding content based on the command to a Internet of Things (IOT) device; the auxiliary image corresponding to at least one Internet of Things (IoT) device, the command comprises the control device to confirm the visual focus of the user on the display content and to receive the triggering action or the triggering posture of the user as the command, which is then received by the Internet of Things (IoT) device to perform a corresponding action (emphasis added).
However, Pinel discloses A rapid user input determination system comprising: a control device [Pinel: ¶ [0020] In some examples, the user controls one or more of the electronic devices 105A-105C based on controls inputs received by the smart contact lens 115 while the user views the one or more of the electronic devices 105A-105C being controlled. For example, when the user focuses on one of the electronic devices 105A-105C and performs one of the control gestures, the electronic devices 105A-105C, a control signal is transmitted to the one of the electronic devices 105A-105C focused on by the user. In some examples, the smart contact lens 115 determines and transmits the control signal. In other examples, the processing device 120 determines and transmits the control signal, for example, based on one or more signals received by the processing device 120 from the smart contact lens 115 and including at least some of the control inputs. In various examples, the control signal is transmitted substantially directly from the smart contact lens 115 or the processing device 120 to one or more of the electronic devices 105A-105C, via a plurality of nodes (e.g., such as mesh network nodes or Internet-of-Things (IOT) nodes), or via the Internet and one or more networking devices facilitating Internet-based communication.]; a head-mounted device signal-connected to the control device and comprising at least one display and at least one eye-tracking unit [Pinel: FIG. 2; and ¶ [0003] Aspects of the present disclosure provides for a computer program product comprising a computer readable storage medium having program instructions embodied therewith. The program instructions are executable by a processor of a smart contact lens to cause the processor of the smart contact lens to measure, by the smart contact lens, dimensions of a user's eye to determine a focal length of the user, measure, by the smart contact lens, a magnetic field proximate to the user's eye to determine a direction of focus of the user, receive, by the smart contact lens, a control input from the user, determine, by the smart contact lens, a control signal corresponding to the focal length of the user, the direction of focus of the user, and the received control input, generate, by the smart contact lens, the determined control signal, and control a first electronic device by transmitting, by the smart contact lens, the generated control signal to the first electronic device.]; and a motion sensing device [Pinel: ¶ [0004] Other aspects of the present disclosure provide for a computer-implemented method. The computer-implemented method comprises measuring, by a smart contact lens, dimensions of a user's eye to determine a focal length of the user, measuring, by the smart contact lens, a magnetic field proximate to the user's eye to determine a direction of focus of the user, receiving, by the smart contact lens, a control input from the user, determining, by the smart contact lens, a control signal corresponding to the focal length of the user, the direction of focus of the user, and the received control input, generating, by the smart contact lens, the determined control signal, and controlling a first electronic device by transmitting, by the smart contact lens, the generated control signal to the first electronic device]; wherein, the control device is configured to receive a command as a user input and to output a corresponding content based on the command to a Internet of Things (IOT) device [Pinel: ¶ [0020]], the head- mounted device and the motion sensing device [Pinel: ¶ [0004]]; the at least one display is configured to demonstrate a display content [Pinel: ¶ [0017]: In some examples, the smart contact lens 115 includes circuitry or other electrical components for displaying one or more digital images on a surface of the smart contact lens 115 and/or projecting a digital image into a space outside of the smart contact lens 115 (e.g., appearing in front of the user, to the sides of the user, etc.)], the display content comprises a virtual reality image or an augmented reality image, the virtual reality image or the augmented reality image includes an auxiliary image corresponding to at least one Internet of Things (IoT) device [Pinel: ¶ [0022] In yet other examples, the user controls one or more virtual elements based on controls inputs received by the smart contact lens 115 while the user views the virtual elements. For example, the smart contact lens is suitable for use in a virtual reality environment or an augmented reality environment in which virtual elements are presented to the user (e.g., via the smart contact lens 115 and/or via any other device) and the user interacts with the virtual elements via one or more of the control gestures], and to update the display content based on the corresponding content received from the control device [Pinel: ¶ [0019] In an example of operation of the system 100, the smart contact lens 115 controls one or more of the electronic devices 105A-105C based on control inputs received from the user. In various examples, the control inputs take numerous forms, the scope of which is not limited herein. For example, the control inputs include detections, determinations, and/or measurements of any one or more control gestures such as a prolonged gaze at one of the electronic devices 105A-105C or the uncontrollable object 110 (e.g., a period of intense visual focus on one of the electronic devices 105A-105C or the uncontrollable object 110), movement of the user's eyes, movement of the user's eyelids, movement of the user's head, movement of the user's body, gestures made by the user's hands in view of the user's eyes (and therefore the smart contact lens 115), or any combination of the foregoing], the eye-tracking unit is configured to track a movement and a motion trajectory of at least one of the user's eyes to determine a visual focus of the user on the display content [Pinel: ¶ [0019]: For example, the control inputs include detections, determinations, and/or measurements of any one or more control gestures such as a prolonged gaze at one of the electronic devices 105A-105C or the uncontrollable object 110 (e.g., a period of intense visual focus on one of the electronic devices 105A-105C or the uncontrollable object 110), movement of the user's eyes, movement of the user's eyelids, movement of the user's head, movement of the user's body, gestures made by the user's hands in view of the user's eyes (and therefore the smart contact lens 115), or any combination of the foregoing], the motion sensing device is configured to detect a motion or a posture of the user's hand and to transmit a hand- gesture information the control device [Pinel: ¶ [0019]], wherein the motion and the posture are respectively defined as a triggering action and a triggering posture, wherein the command comprises the control device to confirm the visual focus of the user on the display content and to receive the triggering action or the triggering posture of the user as the command, which is then received by the Internet of Things (IoT) device to perform a corresponding action [Pinel: ¶ [0017]-[0019]], and at least a portion of the motion sensing device is embedded in or detachably mounted on the head-mounted device [Pinel: ¶ [0004]].
It would have been obvious to one having ordinary skill in the art before the effective filing date to combine the control of external devices of Pinel with the control process of Olwal in order to provide improved user experience, and to provide the music control of Olwal: ¶ [0045] on an external device.
Regarding Claim 2, Olwal in view of Pinel discloses all the limitations of Claim 1, and is analyzed as previously discussed with respect to that claim.
Furthermore, Olwal in view of Pinel discloses wherein the head-mounted device further comprises at least one camera [Olwal: ¶ [0062]: In some implementations, the image sensor 216 is a red, green, blue (RGB) camera. In some examples, the image sensor 216 includes a pulsed laser sensor (e.g., a LiDAR sensor) and/or depth camera. For example, the image sensor 216 may be a camera configured to detect and convey information used to make an image. In some implementations, the image sensor 216 is an eye tracking sensor (or camera), such as eye/gaze tracker 218 that captures movements of an eye of a user accessing device 100, for example] and at least one inertial measurement unit [Olwal: ¶ [0065]: The sensor system 214 may also include an inertial motion unit (IMU) sensor 220. The IMU sensor 220 may detect motion, movement, and/or acceleration of the wearable computing device 100], the at least one camera is configured to detect an external environment of the head-mounted device to collect multiple images or multiple video records in real time [Olwal: ¶ [0067]: The sensor system 214 may also include a camera 224 capable of capturing still and/or moving images. In some implementations, the camera 224 may be a depth camera that can collect data related to distances of external objects from the camera 224], and to transmit the images or video records to the control device [Olwal: FIG. 2], the at least one inertial measurement unit is configured to detect a movement of the user's head [Olwal: ¶ [0109]: In some implementations, the tracking may be performed by the IMU 220. In some implementations, tracking may include assessing and tracking signals that include any one or more (or any combination of) gaze signals (e.g., eye gaze), head tracking signals], and a position of the display content demonstrated on the at least one display can be adjusted based on a detecting result of the inertial measurement unit [Olwal: ¶ [0109]: In some implementations, models 248 may be used to assess tracked focus or related signals to determine next steps and/or next focus events. Any combination of the above may be used to ascertain and track the focus of the user].
Regarding Claim 4, Olwal in view of Pinel discloses all the limitations of Claim 1, and is analyzed as previously discussed with respect to that claim.
Furthermore, Olwal in view of Pinel discloses wherein the motion sensing device comprises a depth camera, a radar, or a wristband [Olwal: ¶ [0067]: depth camera].
Regarding Claim 5, Olwal in view of Pinel discloses all the limitations of Claim 2, and is analyzed as previously discussed with respect to that claim.
Furthermore, Olwal in view of Pinel discloses wherein the motion sensing device comprises a depth camera, a radar, or a wristband [Olwal: ¶ [0067]: depth camera].
Regarding Claim 6, Olwal in view of Pinel discloses all the limitations of Claim 3, and is analyzed as previously discussed with respect to that claim.
Furthermore, Olwal in view of Pinel discloses wherein the motion sensing device comprises a depth camera, a radar, or a wristband [Olwal: ¶ [0067]: depth camera].
Regarding Claim 7, Olwal in view of Pinel discloses all the limitations of Claim 1, and is analyzed as previously discussed with respect to that claim.
Furthermore, Olwal in view of Pinel discloses a method of using the rapid user input determination system as claimed in claim 1 and comprising following steps: demonstrating a display content on a head-mounted device [Olwal: ¶ [0088]: he gaze target 310 may change over time as the user interacts with content 310, 312, 314, and/or other content configured for display in screen 302A], the display content comprises a user interface [Olwal: ¶ [0020]: UI], a virtual reality image or an augmented reality image [Olwal: ¶ [0021]], the virtual reality image or the augmented reality image includes an auxiliary image corresponding to at least one Internet of Things (IoT) device [Pinel: ¶ [0020]]; determining a target of the auxiliary image corresponding to an Internet of Things (IOT) device in the display content [Pinel: ¶ [0017][-[0020]] selected by a user with the eye-tracking unit [Olwal: ¶ [0087]: the wearable computing device 100 may determine via eye/gaze tracker 218 (or other device associated with sensor system 214) that a user operating device 100 is paying attention to (e.g., focusing on) the content 306]; and detecting a triggering action or a triggering posture with the motion sensing device to confirm the target selected by the user [Olwal: ¶ [0089]: The interruption may be visual, audial, or tactile in nature. The interruption may trigger the eye gaze of the user to move to another focus or be removed from an original focus (e.g., gaze target). For example, the user may hear another user call out the user's name, as shown by indicator 318. The user may then defocus from content 306 in screen 302B and the focus (e.g., attention, gaze target) may change, as shown by map 316, which depicts the eye gaze change of the user. In this example, the eye/gaze tracker 218, for example, may detect that the user is looking beyond the screen 302B and may begin to trigger changes in presentation of content on device 100], the Internet of Things (IOT) perform the corresponding action [Pinel: ¶ [0017]-[0020]].
Regarding Claim 8, Olwal in view of Pinel discloses all the limitations of Claim 7, and is analyzed as previously discussed with respect to that claim.
Furthermore, Olwal in view of Pinel discloses wherein a part of the display content interacts with an image of an external environment, the part of the display content corresponds to edges or contours of the image, or tracks a moving object displayed in the image [Olwal: ¶ [0104]: For example, if the user performs an action that triggers an object to move, the device 100 (and the user) may follow the trajectory of the object. While the user is defocused (e.g., looking away), the object will keep moving, and when the user refocuses (e.g., looks back), the device 100 may guide the user to the current position of the object using content, shapes, highlights, or other such marker].
Regarding Claim 9, Olwal in view of Pinel discloses all the limitations of Claim 7, and is analyzed as previously discussed with respect to that claim.
Furthermore, Olwal in view of Pinel discloses wherein the user interface includes multiple selection boxes, multiple symbols, selection boxes text strings or at least one button [Olwal: ¶ [0144]: In some implementations, one or more input devices included on, or connect to, the computing device 650 can be used as input to the AR space. The input devices can include, but are not limited to, a touchscreen, a keyboard, one or more buttons, a trackpad, a touchpad, a pointing device, a mouse, a trackball, a joystick, a camera, a microphone, earphones or buds with input functionality, a gaming controller, or other connectable input device].
Regarding Claim 10, Olwal in view of Pinel discloses all the limitations of Claim 9, and is analyzed as previously discussed with respect to that claim.
Furthermore, Olwal in view of Pinel discloses wherein the display content dynamically displays a position of the user's visual focus on the display in real time and a content shown at the position overlays other contents of the display content [Olwal: ¶ [0121]: The device 100 may then obtain a model for the UI. The model for the UI may define a plurality of states of and user interactions associated with the UI. Based on the tracked focus of the user, the model of the attention, and a determined state, from the plurality of states, of the UI, the device 100 may trigger rendering, for a second time period, at least one focus transition marker (e.g., marker 326) overlaid on at least a portion of the second region of screen 302D and the content therein].
Regarding Claim 11, Olwal in view of Pinel discloses all the limitations of Claim 7, and is analyzed as previously discussed with respect to that claim.
Furthermore, Olwal in view of Pinel discloses wherein, when the user's visual focus is located on one of the selection boxes or the auxiliary images, a display state of the selection box or the auxiliary image changes, and a change of the display state involves color changes or blinking [Olwal: ¶ [0146]: In some implementations, one or more output devices included on the computing device 650 can provide output and/or feedback to a user of the AR headset 690 in the AR space. The output and feedback can be visual, tactical, or audio. The output and/or feedback can include, but is not limited to, vibrations, turning on and off or blinking and/or flashing of one or more lights or strobes, sounding an alarm, playing a chime, playing a song, and playing of an audio file].
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN R MESSMORE whose telephone number is (571)272-2773. The examiner can normally be reached Monday-Friday 9-5 EST/EDT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chris Kelley can be reached at 571-272-7331. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONATHAN R MESSMORE/Primary Examiner, Art Unit 2482
Prosecution Insights