DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed on 03/12/2026 has been entered and considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3, 4, and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li (U.S. Pub. No. 2017/0336439) in view of Min (U.S. Pub. No. 2006/0044265), and further in view of Berkovich (U.S. Pub. No. 2021/0133452).
As to claim 1, Li teaches a method (method of Fig. 5) for visual-inertial tracking with an eyewear device (the VIO is accurately estimating the motion of the device and any fault that causes the device not to accurately estimate the motion causes subdetectors to independently detect different failure conditions, [0017], lines 1-7), the method comprising:
monitoring (checking the feature tracks) a plurality of sensors (214 and 216) of a visual inertial odometry system (VIOS) ([0017], lines 1-4 and a component 430 for visual sensor check subdetector and inertial measurement unit sensor 440, [0034], lines 1-5), wherein each of the plurality of sensors (214 and 216) provide input for determining a position of the eyewear device (position in the environment 212; [0024], lines 1-6 and [0025], lines 1-5);
determining a status of the VIOS ([0034], lines 8-13);
Li does not mention adjusting the plurality of sensors,
Min teaches adjusting the plurality of sensors based on the determined status (adjusting the sensing distance of detection signals based on the determined status of the menu key 426, [0051], lines 6-9), and determining the position of the eyewear device using the using the subset of sensors (sensor 222 having multiple sensors 232 and 234, Fig. 2, [0042], lines 4-12 and [0059], lines 6-12 and the position of the user’s hand is determined for the eyewear device within the environment as can be seen in Fig. 7, [0068], lines 1-11).
Therefore it would have been obvious to one of ordinary skilled in the art at the time the invention was filed to have added the adjusting the sensor and determining the position of the eyewear device of Min to the visual inertial tracking of Li because to provide an HMD information terminal combining an HMF with an information terminal so that a user’s motion can be sensed and recognized as key input without any separate input device, [0014], lines 3-7.
Prior art references of Li and Min do not teach selecting a subset of the plurality of sensors and powering off the remaining sensors,
Berkovich teaches wherein the adjusting (adjusting the volume of image data transmitted over physical link, [0105], lines 1-2) comprises selecting a subset of the plurality of sensors and powering off the remaining sensors (powering off is interpreted as disabled or not allowed to transmit sensor data, some of the sensors are not selected and some are selected to transmit sensor data, [0105], lines 8-13).
Therefore, it would have been obvious to one of ordinary skilled in the art at the time the invention was filed to have added the selecting some of the sensors and disabling some of the sensors of Berkovich to the device of Li as modified by Min because the controller 812 can select the subset of sensors to transmit image data at a higher resolution and/or at a higher frame rate, whereas the sensors that are not selected can transmit image data at a lower resolution and/or at a lower frame rate, [0105], lines 14-18.
As to claim 3, Li teaches the plurality of sensors include an inertial measurement unit (IMU) (IMU sensor data subdetectors, [0034], lines 8-9) and a first camera ([0027], lines 9-11).
As to claim 4, Li teaches the first camera (214) is a first visible light camera (the sensor is an imaging sensor, as can be seen in Fig. 2, the sensor works in an environment receiving visible light in order to capture a wide angle imaging, [0023], lines 3-10), and wherein the plurality of sensors further includes one or more of a second visible light camera, a first depth camera, a second depth camera, another IMU, a radar system, or a GPS (the second sensor 216 is a narrow angle imaging sensor in the local environment 212, and therefore receives a second visible light for narrow angle imaging, [0023], lines 10-19 and [0024], lines 1-3 for cameras 214 and 216).
As to claim 16, Li teaches a non-transitory computer-readable medium (1000/1004) storing program code (1040) for visual-inertial tracking when executed by an eyewear device (100) (the VIO is accurately estimating the motion of the device and any fault that causes the device not to accurately estimate the motion causes subdetectors to independently detect different failure conditions, [0017], lines 1-7) having a plurality of sensors (214 and 216), a processor (1004, Fig. 10), and a memory (RAM, [0045], lines 10-15), the program code ([0045], lines 15-19), when executed, is operative to cause an electronic processor to perform the steps of:
monitoring (checking the feature tracks) a plurality of sensors (214 and 216) of a visual inertial odometry system (VIOS) ([0017], lines 1-4 and a component 430 for visual sensor check subdetector and inertial measurement unit sensor 440, [0034], lines 1-5), wherein each of the plurality of sensors (214 and 216) provide input for determining a position of the eyewear device (position in the environment 212; [0024], lines 1-6 and [0025], lines 1-5);
determining a status of the VIOS ([0034], lines 8-13);
Li does not mention adjusting the plurality of sensors,
Min teaches adjusting the plurality of sensors based on the determined status (adjusting the sensing distance of detection signals based on the determined status of the menu key 426, [0051], lines 6-9), and
determining the position of the eyewear device using the adjusted plurality of sensors (the position of the user’s hand is determined for eyewear device within the environment as can be seen in Fig. 7, [0068], lines 1-11).
Therefore it would have been obvious to one of ordinary skilled in the art at the time the invention was filed to have added the adjusting the sensor and determining the position of the eyewear device of Min to the visual inertial tracking of Li because to provide an HMD information terminal combining an HMF with an information terminal so that a user’s motion can be sensed and recognized as key input without any separate input device, [0014], lines 3-7.
wherein the determining the position of the eyewear device comprises determining the position of the eyewear device using the subset of sensors (sensor 222 having multiple sensors 232 and 234, Fig. 2, [0042], lines 4-12 and [0059], lines 6-12 and the position of the user’s hand is determined for the eyewear device within the environment as can be seen in Fig. 7, [0068], lines 1-11);
Prior art references of Li and Min do not teach selecting a subset of the plurality of sensors and powering off the remaining sensors,
Berkovich teaches the adjusting (adjusting the volume of image data transmitted over physical link, [0105], lines 1-2) comprises selecting a subset of the plurality of sensors and placing the remaining sensors in a lower power mode (Lower power mode is interpreted as disabled or not allowed to transmit sensor data, some of the sensors are not selected and some are selected to transmit sensor data, [0105], lines 8-13), wherein the lower power mode includes one or more of reducing frame rate, resolution, quality, or a combination thereof (higher or lower frame rate, [0105], lines 13-18);
Therefore, it would have been obvious to one of ordinary skilled in the art at the time the invention was filed to have added the selecting some of the sensors and disabling some of the sensors of Berkovich to the device of Li as modified by Min because the controller 812 can select the subset of sensors to transmit image data at a higher resolution and/or at a higher frame rate, whereas the sensors that are not selected can transmit image data at a lower resolution and/or at a lower frame rate, [0105], lines 14-18.
Claim(s) 10 and 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li (U.S. Pub. No. 2017/0336439) in view of Min (U.S. Pub. No. 2006/0044265).
As to claim 10, Li teaches an eyewear device (100) with visual-inertial tracking (the VIO is accurately estimating the motion of the device and any fault that causes the device not to accurately estimate the motion causes subdetectors to independently detect different failure conditions, [0017], lines 1-7), the eyewear device (100) comprising:
a visual inertial odometry system (VIOS) ([0017], lines 1-4 and a component 430 for visual sensor check subdetector and inertial measurement unit sensor 440, [0034], lines 1-5) including a plurality of sensors (214 and 216), wherein the plurality of sensors (214 and 216) include an inertial measurement unit (IMU) (IMU sensor data subdetectors, [0034], lines 8-9) and a first camera ([0027], lines 9-11),
wherein each of the plurality of sensors (214 and 216) provide input for determining a position of the eyewear device (position within the environment 212; [0024], lines 1-6 and [0025], lines 1-5);
a processor (1000/400) configured to determine a status of the VIOS ([0034], lines 8-13),
and
a frame supporting the VIOS and the processor (the frame of the HMD 100 protects the processor and other circuits within the frame, Fig. 1A), the frame configured to be worn on the head of a user (the frame has two straps 118 and the HMD 100 is meant to immerse the user in whatever image is being displayed on the device, [0019],
Li further teaches wherein the VIOS ([0017], lines 1-4 and a component 430 for visual sensor check subdetector and inertial measurement unit sensor 440, [0034], lines 1-5) is configured to capture images with the first camera (214/216, [0024], lines 1-6), and the processor is configured to identify a physical environment of the eyewear device ([0044], lines 6-11) and determine the status of the VIOS based on the identified physical environment (the display controller 1006, which is a part of the processor controls the display device to display the modified imagery at the display device, [0044], lines 1-6, wherein as can be seen in Fig. 2, the images captured by the elements 234, 242, and 238 determine an image of the environment 212 on the display, [0027], lines 9-14).
Li does not mention adjusting the plurality of sensors,
Min teaches adjust the plurality of sensors based on the determined status (adjusting the sensing distance of detection signals based on the determined status of the menu key 426, [0051], lines 6-9), and determine the position of the eyewear device using the adjusted plurality of sensors (the position of the user’s hand is determined for eyewear device within the environment as can be seen in Fig. 7, [0068], lines 1-11);
Therefore it would have been obvious to one of ordinary skilled in the art at the time the invention was filed to have added the adjusting the sensor and determining the position of the eyewear device of Min to the visual inertial tracking of Li because to provide an HMD information terminal combining an HMF with an information terminal so that a user’s motion can be sensed and recognized as key input without any separate input device, [0014], lines 3-7.
As to claim 11, Li teaches the first camera (214) is a first visible light camera (the sensor is an imaging sensor, as can be seen in Fig. 2, the sensor works in an environment receiving visible light in order to capture a wise angle imaging, [0023], lines 3-10), and wherein the plurality of sensors further includes one or more of a second visible light camera, a first depth camera, a second depth camera, another IMU, a radar system, or a GPS (the second sensor 216 is a narrow-angle imaging sensor in the local environment 212, and therefore receives a second visible light for narrow angle imaging, [0023], lines 10-19).
Response to Arguments
Applicant's arguments filed 03/12/2026 have been fully considered but they are not persuasive.
Applicant on page 10 of the remarks argues that the Office’s proposed combination is improper. Li is fundamentally directed to fault detection in a motion tracking system, not to dynamically adjusting sensors to conserve power or alter position determination. Li’s subdetectors operate “independently and in parallel to a main system on a device” to detect faults, and the system issues a fault signal when an error conduction is detected, which may result in a reset of the motion tracking filter. Applicant further argues that Li’s entire purpose is to maintain accuracy in 6DOF pose estimation by detecting a breakdowns in motion tracking. Dynamically powering off sensors would directly undermine Li’s objective of accurately estimating the motion of the device, as doing so would deprive the fault detection system of the sensor data it needs to identify faults and maintain accurate pose estimation. A person of ordinary skill would not modify Li in a manner that contradicts its central purpose. Applicant further describes the system of Min and concludes that Min provides no teaching of adjusting sensors of a VIOS based on a determined status of that system.
Examiner respectfully disagrees because Li’s monitoring is a feature tracking, which is defined in paragraph [0023] in addition to the paragraphs mentioned in the office action as “A feature track is generated by identifying one or more image features in a first image frame and then matching those one or more image features with one or more corresponding image features in consecutive frames.” This definition clearly shows monitoring, which is performed by sensors 214 and 216. Therefore, the prior art reference of Li teaches the monitoring using the sensors of VIOS and the two sensors 214 and 216 provide input for determining a position of the eyewear device. This feature of Li along with visual inertial tracking with an eyewear device by estimating the motion of the device and any fault that causes the device not to accurately estimate the motion is combined with the prior art reference of Min. These features are independently monitor and track the user’s eyewear device. The claims do not mention specifically the function and details of determining status for adjusting the plurality of sensors. Li clearly teaches a method to determine tracking of eyewear device and Min teaches the adjusting the sensing distance of detection signals, which is similar to determining position in the environment of Li. Therefore, the two references can be combined and result in a functional system. Examiner suggests that the applicant should describe the “determining of status” and “adjusting the plurality of sensors” limitations in more details in order to overcome the prior art references and the combination thereof.
Applicant further argues that Min is directed to a fundamentally different problem than Li and describes this difference on pages 10 and 11 of the remarks and mentions that there is no articulated rationale or motivation why a person of ordinary skill would combine a fault detection system for visual inertial odometry of Li with a key input recognition system of Min.
Examiner respectfully disagrees as the claims do not describe the method used to determine statues of the VIOS and does not provide a method for adjusting the plurality of sensors. The prior art references of Li and Min independently teach the limitations and both references show a method wherein Li clearly teaches to determine tracking of eyewear device and Min teaches the adjusting the sensing distance of detection signals, which is similar to determining position in the environment of Li.
Allowable Subject Matter
Claims 2, 5-9, 12-15, 17-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims and overcome obvious type double patenting.
Claim 2 is objected to because the prior art references do not teach selecting a sampling rate for one of the plurality of sensors based on the determined status; and sampling the one of the plurality of sensors at the selected sampling rate, wherein the determining the position of the eyewear device comprises determining the position of the eyewear device using the one of the plurality of sensors at the selected sampling rate.
Claim 5 is objected to because the prior art references do not teach identifying a physical environment of the eyewear device; comparing the identified physical environment to a prior physical environment; identifying new information in the identified physical environment; and determining the status of the VIOS based on the new information in the identified physical environment.
Claim 7 is objected to because the prior art references do not teach determining at least one of a motion parameter value or an uncertainty parameter value of the eyewear device; and mapping the determined at least one of the motion parameter value or the uncertainty parameter value to one of a plurality of VIOS status configuration options.
Claim 12 is objected to because the prior art references do not teach compare the identified physical environment to a prior physical environment; identify new information in the identified physical environment; and determine the status of the VIOS based on the new information in the identified physical environment.
Claim 13 is objected to because the prior art references do not teach determining at least one of a motion parameter value or an uncertainty parameter value of the eyewear device; and mapping the determined at least one of the motion parameter value or the uncertainty parameter value to one of a plurality of VIOS status configuration options.
Claim 17 is objected to because the prior art references do not teach selecting a sampling rate for one of the subset of sensors based on the determined status; and sampling the one of the plurality of sensors at the selected sampling rate, wherein the determining the position of the eyewear device comprises determining the position of the eyewear device using the one of the subset of sensors at the selected sampling rate.
Claim 18 is objected to because the prior art references do not teach capturing images with the first camera; identifying a physical environment of the eyewear device; comparing the identified physical environment to a prior physical environment; identifying new information in the identified physical environment; and determining the status of the visual-inertial tracking system based on the new information in the identified physical environment.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Inquiry
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PEGEMAN KARIMI whose telephone number is (571)270-1712. The examiner can normally be reached Monday-Friday; 9:00am-4:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chanh Nguyen can be reached at 5712727772. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PEGEMAN KARIMI/Primary Examiner, Art Unit 2623