N O N - F I N A L A C T I O N
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 1/07/25 complies with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Closest Prior Art
The prior art (cited on PTO-892) is considered pertinent to applicant's disclosure. Among these, the following references are considered to be the closest, collectively disclosing the state of the art concerned with a wearable device comprising a digital camera and processor / controller for controlling focus based on a target object at a depth associated with a user’s gaze.
OH (US 12,003,691) – applied to 35 USC 102/103 rejection, see Abstract, Figures 1-2 (wearable HMD device), Fig. 3 (method per col. 4, line 37-67 & col. 8, lines 9-20), Fig. 4 (depth image per col. 5, lines 32-48), Fig. 5 (target object from user’s gaze per col. 5, line 49 – col. 6, line 32), Fig. 7 (camera focus adjustment method further using calibration data – col. 8, lines 21-51), wherein col. 8, lines 43-51, discloses “In some examples, the camera calibration is adjustable over time—e.g., to account for changes in lens performance caused by wear-and-tear and/or changes in environmental conditions, such as temperature. For instance, the camera calibration may be changed automatically by evaluating contrast levels in images captured after focus adjustment, and/or changed manually by a human user after deciding that the current calibration is not providing sufficiently focused images”.
CHEN TAO (CN 112507799) – applied to 35 USC 103 rejection, (English translation attached), see Abstract, Fig. 1A-1B, Fig. 4: S101-S105, and para [0255-322].
LU WEICHENG (CN 113325947) – (English translation attached), see Abstract, Fig. 1, Fig. 3A-3B, and para [0003, 0079-89].
OLLILA-1 (US 11,503,202) – see Abstract, and Figures 2-4.
OLLILA-2 (US 20210243384) – see Abstract, Fig. 4-5 (depth of field - DOF), and Fig. 12: steps 1202 (determine gaze direction), 1204 (target ROI) & 1206 (adjust focus).
NOTE: Examiner welcomes INTERVIEW(s) to discuss the instant application’s claimed invention as it corresponds to the specification embodiments, as well as, discussing the similarities/differences taught/not taught by prior art. In the interest of compact prosecution, Applicant’s arguments/amendments should not only address the cited closest art applied/relied on in the 35 USC 102/103 rejection (below), but also address the other cited closest art not applied/relied on.
Examiner cites particular columns and line numbers in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
Claim Rejections – 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-6, 10, 14-19, and 22 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by OH (US 12,003,691) -- hereafter, termed as shown “underlined”.
As per INDEPENDENT CLAIM 1, OH teaches a method for imaging (see method in Fig. 3: steps 302-312 as applied to wearable HMD device imaging in Fig. 1-2 in view of Fig. 4-7; also see method in Figures 8-9), comprising:
determining a target object associated with a gaze of a user from a surrounding environment in which the user is located (Fig. 1: target object 106; Gaze detection per Fig. 3: steps 304, 306 & 308; Fig. 5: target object 506/508 in view of col. 5, line 49 – col. 6, line 32);
determining a target depth of the target object with respect to a camera imaging the surrounding environment based on depth information of the surrounding environment with respect to the camera (see Fig. 3: steps 302, 304, 306 & 308; Fig. 4-5: depth image “plane” 400/506 with target object 406/508; Fig. 1: target object 106 identified as ROI 406 at target depth in Fig. 4 as discussed in col. 5, lines 32-48);
and controlling focusing of the camera based on the target depth to image the target object (Fig. 3: step 310 in view of col. 8, lines 9-20; Fig. 4-5: depth image “plane” 400/506 with target object 406/508).
As per CLAIM 2, OH teaches the method of claim 1, wherein controlling focusing of the camera based on the target depth to image the target object comprises: determining, in the surrounding environment, a target plane matching the target depth; and controlling the camera to focus onto the target plane to image the target object (see Fig. 3: steps 302, 304, 306, 308 in view of col. 4, line 37-67, and step 310 in view of col. 8, lines 9-20; see Fig. 4-5: depth image “plane” 400/506 with target object 406/508 as discussed in col. 5, lines 32-48 and discussed in col. 5, line 49 – col. 6, line 32).
As per CLAIM 3, OH teaches the method of claim 2, wherein determining, in the surrounding environment, a target plane matching the target depth comprises: determining, as the target plane, a plane in the surrounding environment having the target depth with respect to the camera (see Fig. 3: steps 302, 304, 306, 308 in view of col. 4, line 37-67, and step 310 in view of col. 8, lines 9-20; see Fig. 4-5: depth image “plane” 400/506 with target object 406/508 as discussed in col. 5, lines 32-48 and discussed in col. 5, line 49 – col. 6, line 32).
As per CLAIM 4, OH teaches the method of claim 2, wherein determining, in the surrounding environment, a target plane matching the target depth comprises: determining, from a plurality of predetermined depth ranges, a predetermined depth range in which the target depth is located, wherein the plurality of predetermined depth ranges correspond to a plurality of predetermined planes respectively; and determining, as the target plane, a predetermined plane corresponding to the determined predetermined depth range (The features discussed in claim 2 may further use pre-existing camera calibration depth map information per col. 8, lines 33-42, which states “The specific amount by which the lens is moved is in some cases determined by a camera calibration that maps distance ranges between the camera and focus target (e.g., as given by the depth values of the ROI depth pixels), to different positions of the lens within its housing, corresponding to different focus levels of the camera. In some cases, this is done based on a preexisting camera calibration that maps different distance ranges to different focus levels”).
As per CLAIM 5, OH teaches the method of claim 2, wherein controlling the camera to focus onto the target plane to image the target object comprises: determining configuration information for a lens of the camera based on a depth of the target plane with respect to the camera, and adjusting the lens based on the configuration information (see Fig. 3: steps 302, 304 & 310; Fig. 4-5: depth image “plane” 400/506; and adjusting lens using calibration configuration info per col. 8, lines 9-51).
As per CLAIM 6, OH teaches the method of claim 1, wherein determining a target object associated with a gaze of a user from a surrounding environment in which the user is located comprises at least one of: determining, based on tracking of the eyes of the user, an object at which the user is currently gazing as the target object, predicting an object to be gazed at by the user as the target object based on sensing information associated with the user, or determining, from the surrounding environment, an object for directing the gaze of the user as the target object (see Fig. 3: steps 302, 304, 306, 308 in view of col. 4, line 37-67, and step 310 in view of col. 8, lines 9-20; see Fig. 4-5: depth image “plane” 400/506 with target object 406/508 as discussed in col. 5, lines 32-48 and discussed in col. 5, line 49 – col. 6, line 32).
As per INDEPENDENT CLAIM 10, OH teaches a wearable device, comprising:
a camera configured to image a surrounding environment in which a user is located (Fig. 1-2: camera 204 & 206); a controller (Fig. 10: computer 1000) configured to: “determine a target object associated with a gaze of the user from the surrounding environment; determine a target depth of the target object with respect to the camera based on depth information of the surrounding environment with respect to the camera; and control focusing of the camera based on the target depth to image the target object” (The “underlined limitations” for the wearable device’s controller functions of claim 10 comprises similar / analogous limitations to the method of claim 1. Thus, for same reasons, the cited teachings over prior art discussed in claim 1 also applies to rejecting the “underlined limitations” in claim 10).
As per INDEPENDENT CLAIM 14, OH teaches a non-transitory computer readable storage medium, on which a computer program is stored, wherein the computer program is executable by a processor (Fig. 10: computer 1000 with memory 1004) to implement a method for imaging, comprising: “determining a target object associated with a gaze of a user from a surrounding environment in which the user is located; determining a target depth of the target object with respect to a camera imaging the surrounding environment based on depth information of the surrounding environment with respect to the camera; and controlling focusing of the camera based on the target depth to image the target object” (The “underlined limitations” for the computer program executable by a processor to implement a method of claim 14 comprises similar / analogous limitations to the method of claim 1. Thus, for same reasons, the cited teachings over prior art discussed in claim 1 also applies to rejecting the “underlined limitations” in claim 14).
As per CLAIM 15, OH teaches the wearable device of claim 10, wherein the controller is configured to: determine, in the surrounding environment, a target plane matching the target depth; and control the camera to focus onto the target plane to image the target object (see Fig. 3: steps 302, 304, 306, 308 in view of col. 4, line 37-67, and step 310 in view of col. 8, lines 9-20; see Fig. 4-5: depth image “plane” 400/506 with target object 406/508 as discussed in col. 5, lines 32-48 and discussed in col. 5, line 49 – col. 6, line 32).
As per CLAIM 16, OH teaches the wearable device of claim 15, wherein the controller is configured to: determine, as the target plane, a plane in the surrounding environment having the target depth with respect to the camera (see Fig. 3: steps 302, 304, 306, 308 in view of col. 4, line 37-67, and step 310 in view of col. 8, lines 9-20; see Fig. 4-5: depth image “plane” 400/506 with target object 406/508 as discussed in col. 5, lines 32-48 and discussed in col. 5, line 49 – col. 6, line 32).
As per CLAIM 17, OH teaches the wearable device of claim 15, wherein the controller is configured to: determine, from a plurality of predetermined depth ranges, a predetermined depth range in which the target depth is located, wherein the plurality of predetermined depth ranges correspond to a plurality of predetermined planes respectively; and determine, as the target plane, a predetermined plane corresponding to the determined predetermined depth range (The features discussed in claim 2 may further use pre-existing camera calibration depth map information per col. 8, lines 33-42, which states “The specific amount by which the lens is moved is in some cases determined by a camera calibration that maps distance ranges between the camera and focus target (e.g., as given by the depth values of the ROI depth pixels), to different positions of the lens within its housing, corresponding to different focus levels of the camera. In some cases, this is done based on a preexisting camera calibration that maps different distance ranges to different focus levels”).
As per CLAIM 18, OH teaches the wearable device of claim 15, wherein the controller is configured to: determine configuration information for a lens of the camera based on a depth of the target plane with respect to the camera; and adjust the lens based on the configuration information (see Fig. 3: steps 302, 304 & 310; Fig. 4-5: depth image “plane” 400/506; and adjusting lens using calibration configuration info per col. 8, lines 9-51).
As per CLAIM 19, OH teaches the wearable device of claim 10, wherein the controller is further configured to perform at least one of: determining, based on tracking of the eyes of the user, an object at which the user is currently gazing as the target object, predicting an object to be gazed at by the user as the target object based on sensing information associated with the user, or determining, from the surrounding environment, an object for directing the gaze of the user as the target object (see Fig. 3: steps 302, 304, 306, 308 in view of col. 4, line 37-67, and step 310 in view of col. 8, lines 9-20; see Fig. 4-5: depth image “plane” 400/506 with target object 406/508 as discussed in col. 5, lines 32-48 and discussed in col. 5, line 49 – col. 6, line 32).
As per CLAIM 22, OH teaches the medium of claim 14, wherein controlling focusing of the camera based on the target depth to image the target object comprises: determining, in the surrounding environment, a target plane matching the target depth; and controlling the camera to focus onto the target plane to image the target object (see Fig. 3: steps 302, 304, 306, 308 in view of col. 4, line 37-67, and step 310 in view of col. 8, lines 9-20; see Fig. 4-5: depth image “plane” 400/506 with target object 406/508 as discussed in col. 5, lines 32-48 and discussed in col. 5, line 49 – col. 6, line 32).
Claim Rejections – 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 7 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over OH (US 12,003,691) in view of CHEN TAO (CN 112507799) -- hereafter, termed as shown “underlined”.
As per CLAIM 7, OH teaches the method of claim 1. Regarding the limitation: “obtaining an ambient temperature where a lens of the camera is located; determining temperature compensation information for adjusting the lens based on the ambient temperature and a temperature-dependent focus feature of the lens which is pre- calibrated; controlling focusing of the camera based on the target depth to image the target object comprises: controlling the focusing of the camera based on the temperature compensation information and the target depth”, this feature is considered obvious over OH’s additional teachings to use calibration information as discussed in col. 8, lines 33-51, disclosing “In some examples, the camera calibration is adjustable over time—e.g., to account for changes in lens performance caused by wear-and-tear and/or changes in environmental conditions, such as temperature; AND in further view of prior art CHEN, as noted in “Closest Prior Art” section (see Abstract, Fig. 1A-1B, Fig. 4: S101-S105, and para [0255-322]), which discloses a wearable MR HMD 100 comprises a digital camera with adjustable focus based on gaze detection and depth, and discloses an additional sensor such as a temperature sensor by teaching MR device 100 associated with the front camera 130 or other sensor may be configured to evaluate and/or identify the external condition, including but not limited to a day of time, direction of illumination, environment, temperature.
Thus, when considering the collective knowledge bestowed by each applied prior art, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to COMBINE the teachings of CHEN into suitable modification with the teachings of OH to produce Applicant’s claimed invention with the structural arrangement / functional configuration stated in said underlined limitation for the MOTIVATED REASON to account for changes in lens performance caused by wear-and-tear and/or changes in environmental conditions over time in the analogous art of an electronic device configured with one/more cameras.
As per CLAIM 11, OH teaches the wearable device of claim 10. Regarding the limitation: “a temperature sensor configured to sense an ambient temperature where a lens of the camera is located, the controller is further configured to: determine temperature compensation information for adjusting the lens based on the ambient temperature and a temperature-dependent focus feature of the lens which is pre-calibrated ; and control the focusing of the camera based on the temperature compensation information and the target depth”, this feature is considered obvious over OH’s additional teachings to use calibration information as discussed in col. 8, lines 33-51, disclosing “In some examples, the camera calibration is adjustable over time—e.g., to account for changes in lens performance caused by wear-and-tear and/or changes in environmental conditions, such as temperature; AND in further view of prior art CHEN, as noted in “Closest Prior Art” section (see Abstract, Fig. 1A-1B, Fig. 4: S101-S105, and para [0255-322]), which discloses a wearable MR HMD 100 comprises a digital camera with adjustable focus based on gaze detection and depth, and discloses an additional sensor such as a temperature sensor by teaching MR device 100 associated with the front camera 130 or other sensor may be configured to evaluate and/or identify the external condition, including but not limited to a day of time, direction of illumination, environment, temperature.
Thus, when considering the collective knowledge bestowed by each applied prior art, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to COMBINE the teachings of CHEN into suitable modification with the teachings of OH to produce Applicant’s claimed invention with the structural arrangement / functional configuration stated in said underlined limitation for the MOTIVATED REASON to account for changes in lens performance caused by wear-and-tear and/or changes in environmental conditions over time in the analogous art of an electronic device configured with one/more cameras.
Claims 8 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over OH (US 12,003,691) in view of Official Notice -- hereafter, termed as shown “underlined”.
As per CLAIM 8, OH teaches the method of claim 1. Regarding the limitation: “wherein controlling focusing of the camera based on the target depth comprises: determining whether the camera is enabled to provide an image of the surrounding environment to the user; in response to determining that the camera is enabled to provide the image of the surrounding environment to the user, determining whether an autofocus function of the camera is enabled; and in response to the autofocus function being enabled, controlling the focusing of the camera based on the target depth”, OH perform these features by the method in Fig. 3 (steps 302-312) in what is considered an enabled state. However, OH does not explicitly teach to determine whether the camera is enabled / camera’s autofocus function is enabled before performing features by the method in Fig. 3 (steps 302-312).
However, in an imaging system, enabling/disabling the function according to the application scenario is considered to be an obvious conventional design and is a means customary in the art for power management reasons and for devices that have multiple cameras arranged at different viewpoints.
Therefore, Official Notice (MPEP § 2144.03) is taken that both the concepts and advantages of performing camera functions based on a determination of enablement / disablement is well known and expected in the art. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to perform camera functions based on a determination of enablement / disablement for power management reasons and for devices that have multiple cameras arranged at different viewpoints in the analogous art of an electronic device configured with one/more cameras.
As per CLAIM 20, OH teaches the wearable device of claim 10. Regarding the limitation: “wherein the controller is configured to: determine whether the camera is enabled to provide an image of the surrounding environment to the user; in response to determining that the camera is enabled to provide the image of the surrounding environment to the user, determine whether an autofocus function of the camera is enabled; and in response to the autofocus function being enabled, control the focusing of the camera based on the target depth”, OH perform these features by the method in Fig. 3 (steps 302-312) in what is considered an enabled state. However, OH does not explicitly teach to determine whether the camera is enabled / camera’s autofocus function is enabled before performing features by the method in Fig. 3 (steps 302-312).
However, in an imaging system, enabling/disabling the function according to the application scenario is considered to be an obvious conventional design and is a means customary in the art for power management reasons and for devices that have multiple cameras arranged at different viewpoints.
Therefore, Official Notice (MPEP § 2144.03) is taken that both the concepts and advantages of performing camera functions based on a determination of enablement / disablement is well known and expected in the art. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to perform camera functions based on a determination of enablement / disablement for power management reasons and for devices that have multiple cameras arranged at different viewpoints in the analogous art of an electronic device configured with one/more cameras.
Allowable Subject Matter
Claims 9 and 21 are objected to as being dependent upon a rejected base claim but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Contact Information
Any inquiry concerning this communication or earlier communications from the EXAMINER should be directed to AKSHAY TREHAN whose telephone number is (571) 270-5252. The examiner can normally be reached between the hours of 10am – 6pm during the weekdays Monday – Friday.
Interviews with the examiner are available via telephone AND video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant may contact the examiner via telephone OR use the USPTO Automated Interview Request (AIR), which can be found at: http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, TWYLER HASKINS can be reached on (571) 272-7406. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov.
Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AKSHAY TREHAN/
Examiner, Art Unit 2639
/TWYLER L HASKINS/Supervisory Patent Examiner, Art Unit 2639
Prosecution Insights