DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 18-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter as follows.
Claims 18-20 recite “a computer-readable medium”. However, in the state of the art, transitory signals are commonplace as a medium for transmitting computer instruction and thus, in the absence of any evidence to the contrary and give the broadest reasonable interpretation, the scope of a "computer readable medium" covers a signal per se."
A “magnetic, optical, electromagnetic, infrared, … or propagation medium” is neither a process nor a product, (i.e., a tangible “thing”) and therefore does not fall within one of the four statutory classes of § 101. Rather, a “magnetic, optical, electromagnetic, infrared, … or propagation medium” is a form of energy, in the absence of any physical structure or tangible material.
The Examiner suggests amending the claim to recite the “computer-readable medium” as “non-transitory computer-readable storage medium” to include tangible computer readable media, while at the same time excluding the intangible media such as signals, carrier waves, etc. Any amendment to the claim should be commensurate with its corresponding disclosure.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 6, 8 ,10-13, 16, and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Park (US 2024/0380961 A1 – hereinafter Park) and Dominguez Castro et al. (US 2015/0358571 A1 - hereinafter).
Regarding claim 1, Park discloses a method performed by an electronic device having a display configured to present image frames at a frame rate and an image sensor with a first processor and a second processor: performing a first readout of an image sensor to generate a first image frame comprising a sensor representation of a physical environment, wherein the first readout is a first type of readout of the image sensor ([0178]-[0183]; Figs. 12-13 - at step 1216, a first readout of the image sensor to provide light information as raw image data, the first readout is a type of readout that reads all rows or columns corresponding to a FOV as further described at least in [0171]); identifying, by the first processor, one or more first regions of interest in the first image frame, the one or more first regions of interest corresponding to one or more regions of the physical environment (Fig. 13; [0174] – determining ROIs by an NPU of the image sensor as shown in Fig. 12); performing a second readout to generate a second image frame, wherein the second image frame comprises the one or more first regions of interest ([0195] – performing a second readout to generate ROIs by a control and data logic as further shown in Figs. 12-13); and presenting the second image frame on the display ([0197] – presenting the ROIs to the user on a display).
Park does not disclose the first readout is a first type of readout of the image sensor; the second readout is a readout of the image sensor, the second readout is a second type of readout of the image sensor, and the first type of readout and the second type of readouts are different readout types.
Dominguez Castro discloses a first readout is a first type of readout of an image sensor ([0183]-[0185]; Figs. 9 - performing a fast non-destructive first readout of the image sensor to provide an entire pixel array for determining ROIs); a second readout is a readout of the image sensor, the second readout is a second type of readout of the image sensor, and the first type of readout and the second type of readouts are different readout types ([0183]-[0185]; Fig. 9 – determining ROIs by the processing unit of the first readout circuitry on the first read-out image, and performing a second destructive readout of the active array 1910 based on the determined ROIs).
One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to incorporate the teachings of Dominguez Castro into the method taught by Park to reduce the time required to recalculate and update the ROI (Dominguez Castro: [0182]).
Regarding claim 2, Park in view of Dominguez Castro also discloses the method of claim 1, wherein the first processor is an in-sensor processor (Park: Fig. 12 – the NPU of the image sensor, or Dominguez Castro: [0183] - the processing unit of the first readout circuitry), and the second processor is a device processor (Park: Fig. 13 – the second device, i.e. the control logic, is a device processor as shown in Fig. 12).
The motivation for incorporating Dominguez Castro into the method of Park has been discussed in claim 1 above.
Regarding claim 3, Park in view of Dominguez Castro also discloses the method of claim 1, wherein presenting the second image frame comprises: performing image processing techniques on the second image frame to produce a processed second image frame ([0197] – processing the ROIs); and presenting the processed second image frame on the display ([0197] – presenting the processed ROIs to the display).
Regarding claim 6, Park in view of Dominguez Castro also discloses the method of claim 1, wherein the electronic device is a head mounted display device (Fig. 5; [0046]; [0086]) comprising at least the display ([0100]; [0136]-[0137]), the image sensor ([0104]; [0108]; Fig. 11), a battery ([0092]), and a motion sensor ([0100]; [0135]).
Regarding claim 8, Park in view of Dominguez Castro also discloses the method of claim 6, wherein the display is a transparent display permitting a user of the electronic device to simultaneously view a displayed image frame and the physical environment ([0066] - smart glasses comprising a transparent display allowing users to simultaneously view a displayed image frame and the physical environment enabling, gesture-based user interface interactions with augmented reality and/or extended reality (AR/VR) objects).
Regarding claim 10, Park in view of Dominguez Castro also discloses the method of claim 1, wherein the second image frame is smaller than the display ([0070]).
Regarding claim 11, Dominguez Castro also discloses the method of claim 1, wherein the first type of readout is a non-destructive readout and the second type of readout is a destructive readout ([0183]-[0185] – the first type of readout is a non-destructive readout so that the image can be later read out by a second readout using the determined ROIs).
The motivation for incorporating Dominguez Castro into the method of Park has been discussed in claim 1 above.
Claim 12 is rejected for the same reason as discussed in claim 1 above in view of Park also disclosing a computing device (Fig. 12), comprising: one or more memories (Fig. 12; [0142]-[0143]; [0206]); and one or more processors in communication with the one or more memories and configured to execute instructions stored in the one or more memories to perform operations (Fig. 12; [0142]-[0143]; [0206]).
Claim 13 is rejected for the same reason as discussed in claim 3 above.
Claim 16 is rejected for the same reason as discussed in claim 6 above.
Claim 18 is rejected for the same reason as discussed in claim 1 above in view of Park also disclosing a computer-readable medium storing a plurality of instructions that, when executed by one or more processors of a computing device, cause the one or more processors to perform the recited operations (Fig. 12; [0142]-[0143]; [0206]).
Claim 19 is rejected for the same reason as discussed in claim 3 above.
Claims 4, 9, 14, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Park and Dominguez Castro as applied to claims 1-3, 6, 8, 10-13, 16, and 18-19 above, and further in view of Ozone et al. (US 2024/0056694 A1 – hereinafter Ozone).
Regarding claim 4, Park in view of Dominguez Castro also discloses the method of claim 1, further comprising: performing a third readout of the image sensor to generate a third image frame, wherein the third readout is the first type of readout of the image sensor ([0178]-[0183]; Figs. 12-13 - at step 1216, another readout of the image sensor to provide light information as raw image data just like the first readout for another image capturing, in view of Dominguez Castro disclosing the readout as the first type, i.e. non-destructive readout in [0185]), identifying one or more second regions of interest in the third image frame (Fig. 13; [0174] – determining ROIs in the light information corresponding to the third readout); performing a fourth readout of the image sensor to generate a fourth image frame, wherein the fourth readout is the second type of readout of the image sensor, and the fourth image frame comprises the one or more second regions of interest, wherein the fourth image frame is smaller than the display ([0195] – performing a fourth readout to generate new ROIs, each of the ROIs is smaller than the display, in view of Dominguez Castro disclosing the readout is performed to the image sensor 1910 shown in at least Fig. 19); generating a displayed image frame ([0195] – generating an image frame for display); and presenting the displayed image frame on the display ([0197] – presenting the image frame to the user on a display).
However, Park and Dominguez Castro do not disclose combining the second image frame and the fourth image frame to generate the displayed image frame.
Ozone discloses combining a second image frame and a fourth image frame to generate a displayed image frame ([0148] - generating an image by combining an entire angle of view image and a ROI image for display).
One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to incorporate the teachings of Ozone into the method taught by Park and Dominguez Castro to display high-quality image with reduced amount of data.
Regarding claim 9, see the teachings of Park and Dominguez Castro as discussed in claim 8 above. However, Park and Dominguez Castro do not disclose presenting the second image frame comprises: superimposing the second image frame on the one or more first regions of interest corresponding to the one or more regions of the physical environment.
Ozone discloses presenting a second image frame comprises: superimposing the second image frame on one or more first regions of interest corresponding to one or more regions of the physical environment ([0148] - generating an image by combining an entire angle of view image and a ROI image for display).
One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to incorporate the teachings of Ozone into the method taught by Park and Dominguez Castro to display high-quality image with reduced amount of data.
Claim 14 is rejected for the same reason as discussed in claim 4 above.
Claim 20 is rejected for the same reason as discussed in claim 4 above.
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Park, Dominguez Castro, and Ozone as applied to claims 1-4, 6, 8-14, 16, and 18-20 above, and further in view of Shin et al. (US 2023/0368343 A1 – hereinafter Shin).
Regarding claim 5, Park in view of Dominguez Castro also discloses identifying the one or more second regions of interest comprises: determining a first set of coordinates for the one or more objects in the first image frame (Fig. 13; [0174] – determining ROIs by an NPU of the image sensor as shown in Fig. 12, in view of Dominguez Castro disclosing determining ROI’s coordinates in at least [0175] and [0177]); determining a second set of coordinates for the one or more objects in the third image frame (Fig. 13; [0174] – determining ROIs by an NPU of the image sensor as shown in Fig. 12, in view of Dominguez Castro disclosing determining ROI’s coordinates in at least [0175] and [0177]); determining whether the one or more objects in the first image frame has/have moved ([0122]); and identifying the subset of the one or more objects as the one or more second regions of interest ([0122]).
The motivation for incorporating the teachings of Dominguez Castro into the method taught by Park has been discussed in claim 1 above.
However, Park, Dominguez Castro, and Ozone do not disclose comparing the first set of coordinates and the second set of coordinates to identify a subset of the one or more objects that have changed coordinates between the first image frame and the third image frame.
Shin discloses comparing the first set of coordinates and the second set of coordinates to identify a subset of the one or more objects that have changed coordinates between the first image frame and the third image frame ([0062] – comparing the two sets of coordinates to detect any change between frames).
One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to incorporate the teachings of Shin into the method taught by Park, Dominguez Castro, and Ozone to identify ROIs as object’s movements more accurately (Shin: [0006]).
Claim 15 is rejected for the same reason as discussed in claim 5 above.
Claims 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Park and Dominguez Castro as applied to claims 1-3, 6, 8, 10-13, 16, and 18-19 above, and further in view of Ozone and Son et al. (US 2021/0349528 A1 – hereinafter Son).
Regarding claim 7, Park in view of Dominguez Castro also discloses the method of claim 6, further comprising: identifying one or more second regions of interest, wherein the one or more second regions of interest are not present in the second image frame (Fig. 13; [0174] – determining ROIs by an NPU of the image sensor as shown in Fig. 12, assuming the ROIs are newly detected and were not present in the second image frame which was previously captured); performing a fourth readout of the image sensor to generate a fourth image frame, wherein the fourth readout is the second type of readout of the image sensor, and the fourth image frame comprises the one or more second regions of interest, wherein the fourth image frame is smaller than the display ([0195] – performing a fourth readout to generate new ROIs, each of the ROIs is smaller than the display, in view of Dominguez Castro disclosing the fourth readout is of the image sensor and of a second type, i.e. destructive manner); generating a displayed image frame ([0195] – generating an image frame for displ); and presenting the displayed image frame on the display ([0197] – presenting the image frame to the user on a display).
However, Park and Dominguez Castro do not disclose detecting, by the motion sensor, a change of a pose of the head mounted display device; identifying one or more second regions of interest based at least in part on the change of the pose; and combining the second image frame and the fourth image frame to generate a displayed image frame.
Ozone discloses combining a second image frame and a fourth image frame to generate a displayed image frame ([0148] - generating an image by combining an entire angle of view image and a ROI image for display).
One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to incorporate the teachings of Ozone into the method taught by Park and Dominguez Castro to display high-quality image with reduced amount of data.
Park, Dominguez Castro, and Ozone do not disclose detecting, by the motion sensor, a change of a pose of the head mounted display device; identifying one or more second regions of interest based at least in part on the change of the pose.
Son discloses detecting, by a motion sensor, a change of a pose of the head mounted display device ([0012] – detecting a change of a pose of the head mounted display device to identify ROIs); and identifying one or more second regions of interest based at least in part on the change of the pose ([0012] – detecting a change of a pose of the head mounted display device to identify ROIs).
One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to incorporate the teachings of Son into the method taught by Park, Dominguez Castro and Ozone to facilitate identifying ROIs based on user’s gaze point.
Claim 17 is rejected for the same reason as discussed in claim 7 above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUNG Q DANG whose telephone number is (571)270-1116. The examiner can normally be reached IFT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Thai Q Tran can be reached at 571-272-7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HUNG Q DANG/Primary Examiner, Art Unit 2484