DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Information Disclosure Statement
This acknowledges that as of the date of this office action, no additional Information Disclosure Statement has been submitted by the applicant.
Response to Amendment
Claims 1-20 are currently pending.
Independent claim(s) 1, 7, 16 and 17, and dependent claims 2-6, 8-14 and 18 have been amended by applicant’s amendments received 16 March 2026. No new matter has been introduced.
Prior objections of the drawings have been overcome by amendment and are therefore withdrawn.
Prior objections of the specification have been overcome by amendment and are therefore withdrawn.
Prior objections of claims 6 and 13 have been overcome by amendment and are therefore withdrawn.
Response to Arguments
Applicant’s arguments, see Remarks, pgs. 10-12, filed 16 March 2026, with respect to the rejection(s) of claim(s) 1, 7, 16 and 17 under 35 U.S.C. 102(a)(1) and (a)(2) have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of a different interpretation of the previously applied reference, as well as newly found prior art as prior art of record.
Applicant’s Remarks indicate that the prior art cited in the rejection of independent claims 1, 7, 16 and 17, Price et al. (hereinafter Price, US 20180224545 A1), does not anticipate two sets of pixels where one set acquires phase-based images (indirect time-of-flight, iTOF) and the second set acquires image information based on other information (such as direct time-of-flight, dTOF). While Price does not explicitly anticipate this combination of sensor pixels, Price does teach an embodiment where the sensors may be duplicated and used for both iTOF and dTOF separately. This embodiment is described in the rejection as found below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 6-8, 13-14, 16-17, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Price et al. (hereinafter Price, US 20180224545 A1) alone.
Regarding claims 1 and 17, Price teaches an electronic device and method of operation ([0004]) comprising:
at least one processor including processing circuitry ([0028]; Fig. 2, electronic components such as a microprocessor (not shown) may be mounted within the assembly (22)),
memory storing instructions that, when executed by the at least one processor individually or collectively, cause the electronic device to ([0028], [0032]; where circuit boards may bear electronic components such as microprocessors and memory):
calculate a distance to a subject based on a phase difference between phase-based images acquired by first pixels included in a Rx sensor ([0022] - [0023], [0031], [0044]; Fig. 2, depth camera (34) with pixels used to capture images where the captured image has pixel values corresponding to distances based on phase differences);
adaptively determine a transmission power of a Tx light source based on the distance to the subject, and control the Tx light source to output a pulse according to the transmission power ([0042]; illumination power of the emitter (e.g. an IR light emitter) is dynamically adjusted based on distance to objects determined on contextual information from one or more previous frames),
wherein the Rx sensor comprises the first pixels configured to sense the phase-based images, or second pixels configured to sense image information based on the pulse signal ([0022] - [0023], [0031], [0044]; depth camera (34) includes a plurality of pixels used to capture images, where each pixel may be used to measure a phase delay of reflected light as in indirect time-of-flight, or a direct time-of-flight delay, which corresponds to distance values for each pixel).
Price does not explicitly teach two sets of sensors, which separately collect direct time-of-flight (dTOF) images/range information and indirect time-of-flight (iTOF) images based on phase difference information. However, Price does indicate explicitly that “Any or all of the features and functions described above can be combined with each other, except to the extent it may be otherwise stated above or to the extent that any such embodiments may be incompatible by virtue of their function or structure.” ([0091]).
Therefore, to one of ordinary skill in the art before the effective filing date of the claimed
invention, it would have been obvious prima facie to modify Price to utilize either two separate sensors, or a split sensor within the system as a combination of an embodiment which utilizes dTOF sensing and an embodiment which utilizes iTOF sensing with a reasonable expectation of success. As a combination is already disclosed in Price, and one of ordinary skill in the art would understand that this combination/ duplication of parts would not be incompatible or render the system inoperable, utilizing two sensors to operate in the two methods of Price simultaneously would be an obvious combination (See MPEP 2144.04(VI)(C) for supporting rationale under Reversal, Duplication, or Rearrangement of Parts).
Regarding claim 6, Price as modified above teaches the electronic device of claim 1, comprising at least one of:
smart glasses, a head-mounted device (HMD) comprising an augmented reality (AR) device, a HMD comprising a virtual reality (VR) device, or a HMD comprising a mixed reality (MR) device ([0022], [0032]; head-mounted device (HMD) or near-to-eye displays (NED) generates images which are overlaid to provide a virtual or augmented reality).
Regarding claim 7, Price teaches a wearable electronic device comprising ([0022], [0032]; Fig. 2, head-mounted device (HMD) (20)):
at least one processor including processing circuitry ([0028]; Fig. 2, electronic components such as a microprocessor (not shown) may be mounted within the assembly (22)),
memory storing instructions that, when executed by the at least one processor individually or collectively, cause the electronic device to ([0028], [0032]; where circuit boards may bear electronic components such as microprocessors and memory):
before driving a Tx light source, calculate a distance to a subject based on a phase difference between phase-based images acquired by first pixels included in a Rx sensor ([0022] - [0023], [0031], [0044]; Fig. 2, depth camera (34) with pixels used to capture images where the captured image has pixel values corresponding to distances based on phase differences);
and generate virtual content based on an image information and the distance to the subject ([0032]; the HMD includes electronics to generate images and overlay them on the user's 3D view based on the environment);
and a display configured to display the virtual content ([0027]; Fig. 2, left (26-1) and right (26-2) AR displays),
wherein the RX sensor comprises the first pixels configured to sense the phase-based images, or second pixels configured to sense image information based on a pulse signal output from the Tx light source ([0022] - [0023], [0031], [0044]; depth camera (34) includes a plurality of pixels used to capture images, where each pixel may be used to measure a phase delay of reflected light as in indirect time-of-flight, or a direct time-of-flight delay, which corresponds to distance values for each pixel).
Price does not explicitly teach two sets of sensors/pixels, which separately collect direct time-of-flight (dTOF) images/range information and indirect time-of-flight (iTOF) images based on phase difference information. However, Price does indicate explicitly that “Any or all of the features and functions described above can be combined with each other, except to the extent it may be otherwise stated above or to the extent that any such embodiments may be incompatible by virtue of their function or structure.” ([0091]).
Therefore, to one of ordinary skill in the art before the effective filing date of the claimed
invention, it would have been obvious prima facie to modify Price to utilize either two separate sensors, or a split sensor within the system as a combination of an embodiment which utilizes dTOF sensing and an embodiment which utilizes iTOF sensing with a reasonable expectation of success. As a combination is already disclosed in Price, and one of ordinary skill in the art would understand that this combination/ duplication of parts would not be incompatible or render the system inoperable, utilizing two sensors to operate in the two methods of Price simultaneously would be an obvious combination (See MPEP 2144.04(VI)(C) for supporting rationale under Reversal, Duplication, or Rearrangement of Parts).
Regarding claim 8, Price as modified above teaches the wearable electronic device of claim 7, wherein the instructions further cause the wearable electronic device to:
control the Tx light source to output the pulse signal according to a transmission power, wherein the transmission power is based on the distance to the subject ([0042]; illumination power of the emitter (e.g. an IR light emitter) is dynamically adjusted based on distance to objects determined on contextual information from one or more previous frames).
Claim 13 is similarly rejected to claim 6.
Regarding claim 14, Price as modified above teaches the wearable electronic device of claim 7, wherein
The instructions cause the wearable electronic device to adjust one or more attributes of the virtual content based on the distance to the subject ([0032] - [0033]; the HMD includes electronics to control, synchronize operations, and perform data processing such as generate images and overlay them on the user's 3D view).
Regarding claim 16, Price teaches a LiDAR system comprising ([0023], the HMD device may include a depth sensing system such as a ToF camera):
a Tx light source configured to output a pulse signal ([0031]; Fig. 2 illumination module(s) (36) may include laser diodes);
and an Rx sensor configured to receive pulse signals reflected by a subject based on the pulse signal ([0031], [0034]; Fig. 2, depth camera (34) where pixels may measure ToF or phase delay where the depth camera is capable of differentiating between differing emitted light, such as that emitted by illumination modules (36-1) to (36-4)),
wherein the Rx sensor comprises a plurality of second pixels configured to sense image information, or at least one pair of phase-based first pixels distributed among the second pixels, the at least one pair of phase-based pixels configured to obtain phase-based images, wherein a phase difference among the phase-based images is used to calculate a distance to the subject ([0022] - [0023], [0031], [0044]; depth camera (34) includes a plurality of pixels used to capture images, where each pixel may be used to measure a phase delay of reflected light as in indirect time-of-flight, or a direct time-of-flight delay, which corresponds to distance values for each pixel).
Price does not explicitly teach two sets of sensors, which separately collect direct time-of-flight (dTOF) images/range information and indirect time-of-flight (iTOF) images based on phase difference information. However, Price does indicate explicitly that “Any or all of the features and functions described above can be combined with each other, except to the extent it may be otherwise stated above or to the extent that any such embodiments may be incompatible by virtue of their function or structure.” ([0091]).
Therefore, to one of ordinary skill in the art before the effective filing date of the claimed
invention, it would have been obvious prima facie to modify Price to utilize either two separate sensors, or a split sensor within the system as a combination of an embodiment which utilizes dTOF sensing and an embodiment which utilizes iTOF sensing with a reasonable expectation of success. As a combination is already disclosed in Price, and one of ordinary skill in the art would understand that this combination/ duplication of parts would not be incompatible or render the system inoperable, utilizing two sensors to operate in the two methods of Price simultaneously would be an obvious combination (See MPEP 2144.04(VI)(C) for supporting rationale under Reversal, Duplication, or Rearrangement of Parts).
Regarding claim 20, Price as modified above teaches a non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 17 ([0070]).
Claim(s) 2-4, 9-11 and 18-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Price et al. (hereinafter Price, US 20180224545 A1) in view of Nakata (US 5448329 A).
Regarding claim 2, Price as modified above teaches the electronic device of claim 1, but does not explicitly teach a system further comprising a camera with auto-focusing.
Nakata teaches a focus detecting apparatus, which includes instructions which cause an electronic device to calculate the distance to the subject using an auto-focusing function (Col. 1, lines 22-38) of a camera (Col. 4, lines 49-64).
To one of ordinary skill in the art before the effective filing date of the claimed invention, it would have been obvious prima facie to modify Price to incorporate the teachings of Nakata to utilize a system where the detector is a camera with an auto-focusing function with a reasonable expectation of success. Nakata notes that auto-focusing a distance measurement device where the system looks for specific distance or phase measurements allows for more efficient and precise focusing, especially in moving objects (Col. 29, lines 30-47). Inclusion into the system of Price would have a similar increase in efficiency, leading to a predictable result of reductions in power consumption.
Regarding claim 3, Price as modified above teaches the electronic device of claim 1, but does not explicitly teach determining whether an object is an environment is in-focus or out-of-focus before determining a distance.
Nakata teaches a processor which follows instructions which are configured to determine whether the subject is in focus using the first pixels, and to calculate the distance to the subject only in response to the subject being in focus (Col. 6, lines 32-64, where if an object is determined to be in-focus, the object distance measuring operation is affected after a specific lapse of time).
To one of ordinary skill in the art before the effective filing date of the claimed invention, it would have been obvious prima facie to modify Price to incorporate the teachings of Nakata to utilize an in or out of focus determination before a distance measurement is made with a reasonable expectation of success. Nakata notes that auto-focusing a distance measurement device where the system looks for specific distance or phase measurements allows for more efficient and precise focusing, especially in moving objects (Col. 29, lines 30-47). Inclusion into the system of Price, specifically on the pixels which collect phase information, would have a similar increase in efficiency, leading to a predictable result of reductions in power consumption by not taking computational power or emitting light at a higher intensity if an object is found to be out of focus.
Regarding claim 4, Price as modified above teaches the electronic device of claim 3, but does not explicitly teach determining whether an object is an environment is in-focus or out-of-focus before determining a distance.
Nakata teaches in response to a subject not being in focus, the processor scans the surroundings by driving the first pixels instead of driving the Tx light source (Col. 13 line 59 -Col. 14 line 4; where in instances of out-of-focus objects, no further operation is affected, such as distance determination based on phase information).
To one of ordinary skill in the art before the effective filing date of the claimed invention, it would have been obvious prima facie to modify Price to incorporate the teachings of Nakata to utilize an in or out of focus determination before a distance measurement is made with a reasonable expectation of success. Using the ideas of Nakata, it would be logical to one of ordinary skill in the art to utilize the lowest power option, which in this instance are the autofocus/passive detector pixels over an emitter, to preserve power in a mobile system such as the HMD device of Price.
Claims 9-11 are similarly rejected to claims 2-4, respectively.
Regarding claim 18, Price as modified above teaches the method of claim 17, but does not explicitly teach calculating a distance based on a specific phase difference value.
Nakata teaches at least one of calculating a distance in which a phase difference among the phase based images becomes 0 (Col. 2 lines 10-16, Col. 19 lines 39-54, Col. 20 lines 10-29; where when a defocus amount (phase difference) is determined to be less than a threshold, it is marked as effective and operations continue),
and calculating a distance corresponding to a maximum product between phases corresponding to the phase-based images.
To one of ordinary skill in the art before the effective filing date of the claimed invention, it would have been obvious prima facie to modify Price to incorporate the teachings of Nakata to only calculate the distance when a specific phase difference condition has been met with a reasonable expectation of success. Price discusses determining phase differences for a plurality of pixels in the detector ([0022] - [0023], [0031], [0044]) and so using a specific phase difference, such as 0, as an instigating condition for calculations would have a predictable result of saving power as either the emitters only emit or distance is only calculated in situations where the phases of two pixels align, indicating that there is distance overlap in the two values.
Regarding claim 19, Price teaches the method of claim 17, but does not explicitly teach a system further comprising a camera with auto-focusing, where the distance calculation is completed for in-focus objects.
Nakata teaches an electronic device which comprises a camera with an auto-focusing function (Col. 1, lines 22-38),
and the calculating of the distance to the subject comprises acquiring the distance to the subject in focus, the subject determined to be in focus using the auto-focusing function of the camera (Col. 6, lines 32-64, where if an object is determined to be in-focus, the object distance measuring operation is affected after a specific lapse of time).
To one of ordinary skill in the art before the effective filing date of the claimed invention, it would have been obvious prima facie to modify Price to incorporate the teachings of Nakata to utilize a system where the detector is a camera with an auto-focusing function with a reasonable expectation of success. Nakata notes that auto-focusing a distance measurement device where the system looks for specific distance or phase measurements allows for more efficient and precise focusing, especially in moving objects (Col. 29, lines 30-47). Inclusion into the system of Price would have a similar increase in efficiency, leading to a predictable result of reductions in power consumption.
Claim(s) 5, 12, and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Price et al. (hereinafter Price, US 20180224545 A1) in view of Kimura et al. (hereinafter Kimura, US 20210142552 A1).
Regarding claim 5, Price as modified above teaches the electronic device of claim 1, wherein the instructions further cause the electronic device to:
adjust the transmission power of the Tx light source to a first power in response to the distance to the subject being equal to or less than a first distance ([0065] - [0066]; Fig. 7, a first depth range (44) exists between the user's hands and the HMD, and illumination power is varied depending on depth) ;
adjust the transmission power of the Tx light source to a second power greater than the first power in response to the distance to the subject being greater than the first distance and equal to or less than a second distance, which is greater than the first distance ([0065] - [0066]; Fig. 7, a second depth range (46) exists beyond the user's hands, and illumination power is varied depending on depth).
Price does not explicitly teach utilizing a third distance range, such as setting a far depth limit.
Kimura teaches an information processing device, as implemented in an HMD, where a parameter of the system’s display may be set based on distinct detection ranges, such as adjusting a parameter to a third value greater than the second value in response to the distance to the subject exceeding the second distance ([0115] – [0123]; Fig. 6A, where Step S102 determines if the surface is equal to or larger than distance alpha, sets a parameter based on that long range, step S108, where a distance is determined to be smaller than α and equal to or larger than β and a parameter set based on the mid-range, or step S114, where distance is smaller than β (and potentially larger than a minimum distance), and a parameter is set based on this short range.).
To one of ordinary skill in the art before the effective filing date of the claimed invention, it would have been obvious prima facie to modify Price to incorporate the teachings of Kimura to utilize two distinct distances, with three distance ranges to determine output power of an emitter with a reasonable expectation of success. As Price already discusses two ranges (closer than a user’s hand and farther than the hand) which have emitter power varied by detected depth, inclusion of a third range would have a predictable result of further utilizing dynamic power usage to mitigate excess power consumption. Both Price ([0004]) and Kimura ([0006]) note that in systems such as HMDs, unnecessary over-illumination or power consumption for graphics/processing should be avoided as they are battery powered.
Claim 12 is similarly rejected to claim 5.
Regarding claim 15, Price as modified above teaches the wearable electronic device of claim 14, but is silent on the specifics of the attributes of virtual content created and adjusted by the system.
Kimura discusses that one or more attributes which are standard in virtual content and which may be modified by the system are a color ([0186], where the system sets a color of a pixel for displaying), a shape ([0107]; the virtual object may have differing shapes due to the real-world location), a size ([0134]; objects displayed will have differing sizes depending on the distance of the surface), and an audio ([0099]; the speaker (18) may play back audio depending on information from the control unit (10)).
To one of ordinary skill in the art before the effective filing date of the claimed invention, it would have been obvious prima facie to modify Price to incorporate the teachings of Kimura to have variable attributes of created virtual content such as virtual object size, color, shape, or sounds associated with the virtual content, with a reasonable expectation of success. Attributes such as these are well known as variables within virtual content created by systems such as the HMD devices taught by Kimura and Price, and making modifications to them for augmented or virtual reality systems would have predictable variations to one of ordinary skill in the art.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Etou et al. (US 20200412994 A1) teaches an imaging element where a CMOS image sensor may act as a direct TOF distance sensor, and the system may display out to an augmented reality (AR) display.
Lapstun (US 20160139402 A1) teaches a light field display device, which incorporates both an image sensor, which can determine distances based on phase-differences, and a radiance detector, which may be configured for direct time-of-flight measurements, which all may incorporate an autofocus process.
Kobayashi et al. (US 20200409152 A1) teaches an image capturing and display apparatus as an image capturing unit, where a combination of phase difference information acquired by the ToF method and phase difference information acquired by the imaging plane phase difference method may be utilized.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Kara Richter whose telephone number is (571)272-2763. The examiner can normally be reached Monday - Thursday, 8A-5P EST, Fridays are variable.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Helal Algahaim can be reached at (571) 270-5227. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.M.R./Examiner, Art Unit 3645
/HELAL A ALGAHAIM/SPE , Art Unit 3645
Prosecution Insights