DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
2. Claim(s) 1, 3, 7, 11, 13, 16, and 20-26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pillans et al. (US Patent/PGPub. No. 20240089605) in view of Maric et al. (US Patent/PGPub. No. 20210333823).
Regarding Claim 1, (Currently Amended)
Pillans et al. teach
a method ([0006], FIG. 1, i.e. a method of operating a head-mounted device), the method comprising:
obtaining depth information ([0045], FIG. 3-5, i.e. to obtain information relating to … object detection, spatial (depth)) associated with an object ([0005], FIG. 3-5, i.e. external objects) in a physical environment ([0005], FIG. 3-5, i.e. real-world environment) of a head-wearable device ([0005], FIG. 3-5, i.e. head-mounted device);
obtaining first image data from a first camera ([0061], FIG. 3-5, i.e. mages captured by the secondary cameras) of a first type ([0062], FIG. 3-5, i.e. 52, 54, and 56 and/or other tracking or motion sensing cameras) … monochrome image sensors);
obtaining second image data from a second camera ([0033], FIG. 3-5, i.e. camera 50 … capturing images) of a second type ([0062], FIG. 3-5, i.e. the main pass-through camera(s) 50 can be implemented as color image sensors), different from the first type ([0062], FIG. 3-5, i.e. “color image sensors” different from “monochrome image sensors”) [, wherein the first camera and the second camera are located behind a cover window of the head-wearable device (taught by Maric et al. please see below)]; and
tracking movement of the head-wearable device or the object based at least in part upon the depth information ([0045], FIG. 3-5, i.e. captured images from camera 50 … to obtain information relating to … object detection, spatial (depth)), the first image data, and the second image data ([0005], FIG. 3-5, i.e. one or more sensors for capturing images from the real-world environment, for tracking external objects and/or the motion and orientation of the head-mounted device).
However, Pillans et al. do not explicitly teach
wherein the first camera and the second camera are located behind a cover window of the head-wearable device.
In the same field of endeavor, Maric et al. teach
wherein the first camera and the second camera ([0047], FIG. 1-2 & 4-6, i.e. at least two openings 58 (e.g., through-hole openings), each of which is configured to receive an associated camera. Each camera 46) are located behind a cover window ([0052], FIG. 1-2 & 4-6, i.e. Camera windows 62 (please note that “F” denotes region in front of the head-mounted device 10 and also “positive z” denotes wearer side)) of the head-wearable device ([0024], FIG. 1-2 & 4-6, i.e. head-mounted devices such as electronic device 10).
It would have been obvious to a person having ordinary skill in the art at the time the invention’s effective date was filed to combine Pillans et al. teaching method of head-mounted tracking user or object in an environment using plurality of cameras with Maric et al. teaching method of head-mounted comprising plurality of cameras located behind windows to effectively provide protection and other optical characteristics for the cameras by placing them behind the windows (Maric et al.’s [0052]).
Regarding Claim 3, (Currently Amended)
the method of claim 1, wherein
Pillans et al. teach
the first type of camera is a monochrome camera ([0062], FIG. 3-5, i.e. 52, 54, and 56 … monochrome image sensors), and the second type of camera is a color camera ([0062], FIG. 3-5, i.e. 50 can be implemented as color image sensors).
Regarding Claim 7, (Currently Amended)
the method of claim 1, wherein
Maric et al. teach
the cover window of the head-wearable device (i.e. please see above citation(s)) comprises glass ([0051], FIG. 5-6, i.e. display 14F may be covered with … a layer of glass (please note that cover window 62 is part of display/cover 14F/CG)).
Regarding Claim 11, (Currently Amended)
Pillans et al. teach
a system ([0023], FIG. 2, i.e. system 8), comprising:
one or more processors ([0024], FIG. 2, i.e. Processing circuitry in control circuitry 12 may be based on one or more microprocessors); and
one or more memories storing instructions ([0024], FIG. 2, i.e. Control circuitry 12 may include storage … Software code may be stored on storage in circuitry 12 and run on processing circuitry in circuitry 12 to implement control operations) that, when executed by the one or more processors, cause the system (i.e. please see above citation(s)) to:
obtain depth information ([0045], FIG. 3-5, i.e. to obtain information relating to … object detection, spatial (depth)) associated with an object ([0005], FIG. 3-5, i.e. external objects) in a physical environment ([0005], FIG. 3-5, i.e. real-world environment) of a head-wearable device ([0005], FIG. 3-5, i.e. head-mounted device);
obtain first image data from a first camera ([0061], FIG. 3-5, i.e. mages captured by the secondary cameras) of a first type ([0062], FIG. 3-5, i.e. 52, 54, and 56 and/or other tracking or motion sensing cameras) … monochrome image sensors);
obtain second image data from a second camera ([0033], FIG. 3-5, i.e. camera 50 … capturing images) of a second type ([0062], FIG. 3-5, i.e. the main pass-through camera(s) 50 can be implemented as color image sensors), different from the first type ([0062], FIG. 3-5, i.e. “color image sensors” different from “monochrome image sensors”) [, wherein the first camera and the second camera are located behind a cover window of the head-wearable device (taught by Maric et al. please see below)]; and
track movement of the head-wearable device or the object based at least in part upon on the depth information ([0045], FIG. 3-5, i.e. captured images from camera 50 … to obtain information relating to … object detection, spatial (depth)), the first image data, and the second image data ([0005], FIG. 3-5, i.e. one or more sensors for capturing images from the real-world environment, for tracking external objects and/or the motion and orientation of the head-mounted device).
However, Pillans et al. do not explicitly teach
wherein the first camera and the second camera are located behind a cover window of the head-wearable device.
In the same field of endeavor, Maric et al. teach
wherein the first camera and the second camera ([0047], FIG. 1-2 & 4-6, i.e. at least two openings 58 (e.g., through-hole openings), each of which is configured to receive an associated camera. Each camera 46) are located behind a cover window ([0052], FIG. 1-2 & 4-6, i.e. Camera windows 62 (please note that “F” denotes region in front of the head-mounted device 10 and also “positive z” denotes wearer side)) of the head-wearable device ([0024], FIG. 1-2 & 4-6, i.e. head-mounted devices such as electronic device 10).
It would have been obvious to a person having ordinary skill in the art at the time the invention’s effective date was filed to combine Pillans et al. teaching of head-mounted device tracking user or object in an environment using plurality of cameras with Maric et al. teaching of head-mounted device comprising plurality of cameras located behind windows to effectively provide protection and other optical characteristics for the cameras by placing them behind the windows (Maric et al.’s [0052]).
Regarding Claim 13, (Currently Amended)
the system of claim 11, wherein
Pillans et al. teach
the first type of camera is a monochrome camera ([0062], FIG. 3-5, i.e. 52, 54, and 56 … monochrome image sensors), and the second type of camera is a color camera ([0062], FIG. 3-5, i.e. 50 can be implemented as color image sensors).
Regarding Claim 16, (Currently Amended)
the system of claim 11, wherein
Maric et al. teach
the cover window of the head-wearable device (i.e. please see above citation(s)) comprises glass ([0051], FIG. 5-6, i.e. display 14F may be covered with … a layer of glass (please note that cover window 62 is part of display/cover 14F/CG)).
Regarding Claim 20, (Currently Amended)
Pillans et al. teach
a non-transitory computer-readable storage medium having instructions ([0024], FIG. 2, i.e. Control circuitry 12 may include storage … Software code may be stored on storage in circuitry 12 and run on processing circuitry in circuitry 12 to implement control operations) embodied thereon, wherein the instructions, when executed by one or more processors ([0024], FIG. 2, i.e. Processing circuitry in control circuitry 12 may be based on one or more microprocessors), cause a system ([0023], FIG. 2, i.e. system 8) to:
obtain depth information ([0045], FIG. 3-5, i.e. to obtain information relating to … object detection, spatial (depth)) associated with an object ([0005], FIG. 3-5, i.e. external objects) in a physical environment ([0005], FIG. 3-5, i.e. real-world environment) of a head-wearable device ([0005], FIG. 3-5, i.e. head-mounted device);
obtain first image data from a first camera ([0061], FIG. 3-5, i.e. mages captured by the secondary cameras) of a first type ([0062], FIG. 3-5, i.e. 52, 54, and 56 and/or other tracking or motion sensing cameras) … monochrome image sensors);
obtain second image data from a second camera ([0033], FIG. 3-5, i.e. camera 50 … capturing images) of a second type ([0062], FIG. 3-5, i.e. the main pass-through camera(s) 50 can be implemented as color image sensors), different from the first type ([0062], FIG. 3-5, i.e. “color image sensors” different from “monochrome image sensors”) [, wherein the first camera and the second camera are located behind a cover window of the head-wearable device (taught by Maric et al. please see below)]; and
track movement of the head-wearable device or the object based at least in part upon on the depth information ([0045], FIG. 3-5, i.e. captured images from camera 50 … to obtain information relating to … object detection, spatial (depth)), the first image data, and the second image data ([0005], FIG. 3-5, i.e. one or more sensors for capturing images from the real-world environment, for tracking external objects and/or the motion and orientation of the head-mounted device).
However, Pillans et al. do not explicitly teach
wherein the first camera and the second camera are located behind a cover window of the head-wearable device.
In the same field of endeavor, Maric et al. teach
wherein the first camera and the second camera ([0047], FIG. 1-2 & 4-6, i.e. at least two openings 58 (e.g., through-hole openings), each of which is configured to receive an associated camera. Each camera 46) are located behind a cover window ([0052], FIG. 1-2 & 4-6, i.e. Camera windows 62 (please note that “F” denotes region in front of the head-mounted device 10 and also “positive z” denotes wearer side)) of the head-wearable device ([0024], FIG. 1-2 & 4-6, i.e. head-mounted devices such as electronic device 10).
It would have been obvious to a person having ordinary skill in the art at the time the invention’s effective date was filed to combine Pillans et al. teaching of head-mounted device tracking user or object in an environment using plurality of cameras with Maric et al. teaching of head-mounted device comprising plurality of cameras located behind windows to effectively provide protection and other optical characteristics for the cameras by placing them behind the windows (Maric et al.’s [0052]).
Regarding Claim 21, (New)
the method of claim 1, wherein
Pillans et al. teach
the depth information (i.e. please see above citation(s)) is obtained from a depth-tracking component ([0046], FIG. 3-5, i.e. depth, and location (or generally spatial) information from various sensors in device 10 (e.g., using information from cameras 50, 52, 54, 56, flicker information from a flicker sensor 102, and/or other sensing components within sensor block 16 in FIG. 2)) of the head-wearable device (i.e. please see above citation(s)).
Regarding Claim 22, (New)
the method of claim 21, wherein
Pillans et al. teach
the depth-tracking component (i.e. please see above citation(s)) comprises an illumination source configured to illuminate an area ([0029], FIG. 3-5, i.e. a target is illuminated by the beams of light) with a structured light pattern ([0029], FIG. 3-5, i.e. three-dimensional image sensors such as structured light sensors).
Regarding Claim 23, (New)
the method of claim 22, wherein
Pillans et al. teach
the illumination source (i.e. please see above citation(s)) emits infrared light ([0029], FIG. 3-5, i.e. cameras (e.g., infrared).
Regarding Claim 24, (New)
the method of claim 23, wherein
Pillans et al. teach
the depth-tracking component (i.e. please see above citation(s)) comprises an infrared sensor ([0029], FIG. 3-5, i.e. cameras (e.g., infrared).
Regarding Claim 25, (New)
the method of claim 1,
Pillans et al. teach
further comprising:
obtaining third image data ([0046], FIG. 3-5, i.e. captured images output from pass-through camera(s) 50) from a third camera ([0046], FIG. 3-5, i.e. pass-through camera(s) 50 (please note that there are two cameras 50s)), wherein the depth information is obtained via a depth-tracking component ([0046], FIG. 3-5, i.e. depth, and location (or generally spatial) information from various sensors in device 10 (e.g., using information from cameras 50, 52, 54, 56 (please note that there are two of each 50s, 52s, 54s, and 56s cameras)) of the head-wearable device, the first camera and the second camera are included in a first forward-facing component assembly ([0037], FIG. 3-5, i.e. Both pass-through cameras 50 on the left and right sides of device 10 are front-facing cameras) arranged on a first side ([0037], FIG. 3-5, i.e. right-hand side … camera 50) of the depth-tracking component ([0038], FIG. 3-5, i.e. right-hand side … object/orientation tracking camera 52), and the third camera ([0037], FIG. 3-5, i.e. left-hand side … another … camera 50) is included in a second forward-facing component assembly ([0037], FIG. 3-5, i.e. Both pass-through cameras 50 on the left and right sides of device 10 are front-facing cameras) arranged on a second side ([0037], FIG. 3-5, i.e. left-hand side) of the depth-tracking component (i.e. please see above citation(s)), opposite the first side ([0037], FIG. 3-5, i.e. left and right sides).
Regarding Claim 26, (New)
the method of claim 3, wherein
Pillans et al. teach
a pass through view ([0037], FIG. 3-5, i.e. a pass-through (scene)) of the physical environment is provided (i.e. please see above citation(s)), using a display ([0021], FIG. 3-5, i.e. left display 14 for presenting an image … to a user's left eye … and a right display 14 for presenting an image to a user's right eye) of the head-wearable device, based on the second image data (i.e. please see above citation(s)).
3. Claim(s) 4-5, and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pillans et al. (US Patent/PGPub. No. 20240089605) and Maric et al. (US Patent/PGPub. No. 20210333823) in view of Stahl et al. (US Patent/PGPub. No. 10914957).
Regarding Claim 4, (Currently Amended)
Pillans et al. and Maric et al. teach
the method of claim 3.
However, Pillans et al. and Maric et al. do not explicitly teach
wherein the monochrome camera has a higher refresh rate than a refresh rate of the color camera.
In the same field of endeavor, Stahl et al. teach
wherein the monochrome camera has a higher refresh rate (Col. 8, Ln. 45-55, FIG. 2, i.e. head pose sensors 222 … with a frame rate of 120 frames per second (FPS)) than a refresh rate of the color camera (Col. 7, Ln. 59-65, FIG. 2, i.e. cameras 220 may include high quality, high resolution RGB video cameras … with a frame rate of 60 frames per second (FPS)).
It would have been obvious to a person having ordinary skill in the art at the time the invention’s effective date was filed to combine Pillans et al. and Maric et al. teaching method of head-mounted tracking user or object in an environment using plurality of cameras positioned behind windows with Stahl et al. teaching method of plurality of cameras in which monochrome camera having refresh rate higher than color camera to effectively accommodate designs, manufactures, and cost requirements utilizing monochrome camera having refresh rate higher than color camera (Stahl et al.’s Col. 7, Ln. 59-65).
Regarding Claim 5, (Currently Amended)
Pillans et al. and Maric et al. teach
the method of claim 3.
However, Pillans et al. and Maric et al. do not explicitly teach
wherein the color camera has a higher resolution than a resolution of the monochrome camera.
In the same field of endeavor, Stahl et al. teach
wherein the color camera has a higher resolution (Col. 9, Ln. 2-9, FIG. 2, i.e. light sensor 223 may include an RGB high dynamic range (HDR) video camera, for example a 500×500 pixel count camera) than a resolution of the monochrome camera (Col. 8, Ln. 45-55, FIG. 2, i.e. 400×400 pixel count … sensors 222 may provide low latency monochrome).
It would have been obvious to a person having ordinary skill in the art at the time the invention’s effective date was filed to combine Pillans et al. and Maric et al. teaching method of head-mounted tracking user or object in an environment using plurality of cameras positioned behind windows with Stahl et al. method of plurality of cameras in which color camera having a higher resolution than the monochrome camera to effectively render content in important regions with more detail/at higher resolution than content in less important regions (Stahl et al.’s Col. 23, Ln. 55-59).
Regarding Claim 14, (Currently Amended)
Pillans et al. and Maric et al. teach
the system of claim 13.
However, Pillans et al. and Maric et al. do not explicitly teach
wherein the monochrome camera has at least one of (i) a higher refresh rate than a refresh rate of the color camera, and/or (ii) a lower resolution than a resolution of the color camera.
In the same field of endeavor, Stahl et al. teach
wherein the monochrome camera has at least one of (i) a higher refresh rate (Col. 8, Ln. 45-55, FIG. 2, i.e. head pose sensors 222 … with a frame rate of 120 frames per second (FPS)) than a refresh rate of the color camera (Col. 7, Ln. 59-65, FIG. 2, i.e. cameras 220 may include high quality, high resolution RGB video cameras … with a frame rate of 60 frames per second (FPS)), and/or (ii) a lower resolution than a resolution of the color camera.
It would have been obvious to a person having ordinary skill in the art at the time the invention’s effective date was filed to combine Pillans et al. and Maric et al. teaching of head-mounted system tracking user or object in an environment using plurality of cameras positioned behind windows with Stahl et al. teaching system of plurality of cameras in which monochrome camera having refresh rate higher than color camera to effectively accommodate designs, manufactures, and cost requirements utilizing monochrome camera having refresh rate higher than color camera (Stahl et al.’s Col. 7, Ln. 59-65).
Response to Argument/Amendment
4. Applicants’ Request for Continued Examination, 08/14/2026, has been entered and made of record. Claim(s) 1, 3-5, 7, 11, 13-14, 16, and 20 is/are amended, Claim(s) 2, 6, 8-10, 12, 15, and 17 is/are cancelled, additional Claims 21-26 is/are new. Thus, Claim(s) 1, 3-5, 7, 11, 13-14, 16, and 20-26 is/are pending in this application.
Allowable Subject Matter
5. Claim(s) 18-19 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
6. The following is an examiner’s statement of reasons for allowance:
Pillans et al. (US Patent/PGPub. No. 20240089605) teach a head-mounted device may include one or more sensors for capturing images of a real-world environment and for determining where the head-mounted device is facing in the real-world environment. The sensors can be used to capture images for creating a lighting map that indicates where one or more light sources are located within the environment and for creating a physical map representing a geometry of the environment. The sensors can be used to predict a direction in which the head-mounted device will be facing in the real-world environment. One of the sensors can include an image sensor for capturing images that are displayed to a user of the head-mounted device. One or more settings of the image sensor can be adjusted based on the lighting map and based on the predicted direction of where the head-mounted device will be facing when capturing a future frame.
Maric et al. (US Patent/PGPub. No. 20210333823) teach a head-mounted device may have a head-mounted housing. The housing may include a frame with left and right openings that receive respective left and right optical modules that present images to a user's eyes. Each optical module may have a lens and display that presents an image through the lens. Camera support members may be coupled to respective left and right peripheral portions of the frame. Each camera support member may have openings configured to receive cameras. Antennas may be formed on a camera support member. The antennas may have metal traces on a surface of the camera support member, may have conductive structures embedded within the camera support member, and/or may have patterned metal traces on printed circuits attached to or embedded in the camera support member. The cameras may operate through portions of a display cover layer that covers an outwardly-facing display on the head-mounted housing.
The subject matter of the claim(s) that could neither be found/suggested nor obviously combinable in the prior arts of record. The subject matter was a device/method including
“…wherein the instructions when executed by the one or more processes further cause the system to:
colorize image data from the first type of camera using color information from the second type of camera.” (Claim 18),
“…wherein the instructions when executed by the one or more processes further cause the system to:
project structured light onto a controller operating with the head-wearable device; and
track a position of the controller using at least a portion of the first image data and the second image data.” (Claim 19),
in combination with the other elements (or steps) of the device or apparatus and method recited in the claims.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VINH T. LAM whose telephone number is (571) 270-3704. The examiner can normally be reached Monday to Friday 8:00 AM to 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nitin K Patel can be reached at (571) 272-7677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VINH T LAM/Primary Examiner, Art Unit 2628
Prosecution Insights