Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “a depth calculating unit” in claim 1 and dependent claim(s) 2-9.
See paras. 25-26 “The control unit 11 includes at least one processor such as a CPU (central processing unit) or a GPU (graphical processing unit). The image data acquired by the imaging device 13 is provided to the control unit 11. The control unit 11 uses the image data to generate a depth map indicating the distance to an object.
The memory unit 12 includes a main memory unit and an auxiliary memory unit. For example, the main memory unit is a volatile memory such as RAM (random access memory), and the auxiliary memory unit is a non-volatile memory such as a ROM (read only memory), an EEPROM (electrically erasable and programmable read only memory), a flash memory, and a hard disk. The control unit 11 executes a program stored in the memory unit 12 to control the liquid crystal shutter 14 and calculates depths (distances to the object). Processing executed by the control unit 11 will be discussed later.”
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 5-8, 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Igeta et al. (US 20230236454 A1), hereinafter Igeta, in view of Yahata (US 20120148108 A1), hereinafter Yahata.
Regarding claim 1, Igeta teaches A system for generating a depth map, comprising: a liquid crystal shutter that forms a coded aperture part; (Abstract see "a camera module includes a liquid crystal panel, an image sensor and an optical system. The liquid crystal panel is configured to display a coded aperture pattern. The optical system is interposed between the liquid crystal panel and the image sensor. The liquid crystal panel includes a liquid crystal layer containing liquid crystal molecules and dichroic dye molecules aligned following the liquid crystal molecules." Para. 28 see "by using the above-described coded aperture technology, it is possible to calculate the distance to the subject on the basis of the image and create a depth map indicating the distance to the subject. Incidentally, processing of calculating the distance to the subject, processing of creating the depth map, and the like are executed by a controller (CPU) that controls an operation of the camera module or a controller of an electronic device connected to the camera module (electronic device on which the camera module is mounted)."). an imaging device that captures light transmitted through the coded aperture part; (Para. 31 see "when the liquid crystal panel PNL is in the transparent state, light transmitted through the liquid crystal panel PNL and the optical system OS enters the image sensor IS. As a result, the camera module CM can capture an image based on the light incident on the image sensor IS." Para. 32 see "when the liquid crystal panel PNL is in the absorption state, a coded aperture pattern is displayed on the liquid crystal panel PNL, and a large number of incident light control areas are formed. In other words, the light transmitted through the liquid crystal panel PNL in which the coded aperture pattern is displayed and the optical system OS is incident on the image sensor IS."). and a depth calculating unit that uses a captured image acquired by the imaging device to calculate a depth for each unit area of a depth map, (Para. 28 see "by using the above-described coded aperture technology, it is possible to calculate the distance to the subject on the basis of the image and create a depth map indicating the distance to the subject. Incidentally, processing of calculating the distance to the subject, processing of creating the depth map, and the like are executed by a controller (CPU) that controls an operation of the camera module or a controller of an electronic device connected to the camera module (electronic device on which the camera module is mounted)."). wherein the coded aperture part includes an aperture pattern that includes a first area, a second area, (Para. 34 see "The incident light control area PCA includes a light shielding area LSA that shields incident light incident on the image sensor IS, and a light transmissive area TA through which the incident light incident on the image sensor IS is transmitted."). the first area having a first light transmittance, (Para. 34 see "In FIGS. 2 and 3, dotted areas correspond to the light shielding area LSA, and other areas correspond to the light transmissive area TA. Incidentally, in FIG. 2, the dotted area may be referred to as a first light shielding area LSA 1, and an area other than the first light shielding area LSA 1 may be referred to as a first light transmissive area TA 1."). the second area having a second light transmittance lower than the first light transmittance, (Para. 34 see "In FIGS. 2 and 3, dotted areas correspond to the light shielding area LSA, and other areas correspond to the light transmissive area TA. Incidentally, in FIG. 2, the dotted area may be referred to as a first light shielding area LSA 1, and an area other than the first light shielding area LSA 1 may be referred to as a first light transmissive area TA 1." Examiner Note: The light shielding area LSA has low light transmittance and the transmissive area TA has high light transmittance.).
Igeta does not teach and a third area, the third area having a third light transmittance lower than the second light transmittance.
However, Yahata teaches and a third area, (Para. 38 see "FIG. 3D shows an example of an aperture formed by an aggregate of regions having different transmittances." Examiner Note: Fig. 3D shows four different regions with different transmittance levels.). the third area having a third light transmittance lower than the second light transmittance. (Para. 38 see "FIG. 3D shows an example of an aperture formed by an aggregate of regions having different transmittances." Examiner Note: Fig. 3D shows four different regions with different transmittance levels.).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Igeta to incorporate the teachings of Yahata to include a third area in the aperture pattern having a third light transmittance lower than the second light transmittance. Doing so would predictably improve depth estimation accuracy by providing various levels of light transmittance beyond minimum and maximum light transmittance. This would create more distinctive blur patterns at different object distances.
Regarding claim 2, Igeta in view of Yahata teaches The system for generating the depth map according to claim 1.
In addition, Igeta teaches wherein the first light transmittance is an upper limit of a light transmittance that is feasible by the liquid crystal shutter, (Para. 34 see "In FIGS. 2 and 3, dotted areas correspond to the light shielding area LSA, and other areas correspond to the light transmissive area TA. Incidentally, in FIG. 2, the dotted area may be referred to as a first light shielding area LSA 1, and an area other than the first light shielding area LSA 1 may be referred to as a first light transmissive area TA 1." Examiner Note: The light shielding area LSA is the lower limit.). the third light transmittance is a lower limit of a light transmittance that is feasible by the liquid crystal shutter, (Para. 34 see "In FIGS. 2 and 3, dotted areas correspond to the light shielding area LSA, and other areas correspond to the light transmissive area TA. Incidentally, in FIG. 2, the dotted area may be referred to as a first light shielding area LSA 1, and an area other than the first light shielding area LSA 1 may be referred to as a first light transmissive area TA 1." Examiner Note: The light shielding area LSA is the lower limit.).
Igeta does not teach and the second light transmittance is a transmittance between the first light transmittance and the third light transmittance.
However, Yahata teaches and the second light transmittance is a transmittance between the first light transmittance and the third light transmittance. (Para. 38 see "FIG. 3D shows an example of an aperture formed by an aggregate of regions having different transmittances." Examiner Note: Fig. 3D shows four different regions with different transmittance levels. There is an upper transmittance level, a lower transmittance level, and two in between transmittance level.).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Igeta and Yahata to incorporate the teachings of Yahata to include a second light transmittance that is an intermediate value between the upper and lower limits of a feasible transmittance range. Doing so would predictably improve depth map accuracy by measuring an additional light transmittance to provide a blur different from the first and third light transmittance, enabling more precise calculations of object distances.
Regarding claim 5, Igeta in view of Yahata teaches The system for generating the depth map according to claim 1.
In addition, Igeta teaches wherein the first area and the second area are circular with a same center. (Para. 34 see "The incident light control area PCA includes a light shielding area LSA that shields incident light incident on the image sensor IS, and a light transmissive area TA through which the incident light incident on the image sensor IS is transmitted. The incident light control area PCA has, for example, a circular shape, and the light shielding area LSA includes at least an annular part located at an outermost periphery of the incident light control area PCA. In FIGS. 2 and 3, dotted areas correspond to the light shielding area LSA, and other areas correspond to the light transmissive area TA. Incidentally, in FIG. 2, the dotted area may be referred to as a first light shielding area LSA 1, and an area other than the first light shielding area LSA 1 may be referred to as a first light transmissive area TA 1.").
Regarding claim 6, Igeta in view of Yahata teaches The system for generating the depth map according to claim 1.
In addition, Igeta teaches wherein the aperture pattern of the coded aperture part includes a plurality of the first areas. (Para. 32 see "when the liquid crystal panel PNL is in the absorption state, a coded aperture pattern is displayed on the liquid crystal panel PNL, and a large number of incident light control areas are formed." Para. 34 see "In FIGS. 2 and 3, dotted areas correspond to the light shielding area LSA, and other areas correspond to the light transmissive area TA. Incidentally, in FIG. 2, the dotted area may be referred to as a first light shielding area LSA 1, and an area other than the first light shielding area LSA 1 may be referred to as a first light transmissive area TA 1. In addition, in FIG. 3, the dotted areas may be referred to as second light shielding areas LSA 2, and an area other than the second light shielding areas LSA 2 may be referred to as a second light transmissive area TA 2.").
Regarding claim 7, Igeta in view of Yahata teaches The system for generating the depth map according to claim 1.
In addition, Igeta teaches wherein the aperture pattern of the coded aperture part includes a plurality of the second areas. (Para. 32 see "when the liquid crystal panel PNL is in the absorption state, a coded aperture pattern is displayed on the liquid crystal panel PNL, and a large number of incident light control areas are formed." Para. 34 see "In FIGS. 2 and 3, dotted areas correspond to the light shielding area LSA, and other areas correspond to the light transmissive area TA. Incidentally, in FIG. 2, the dotted area may be referred to as a first light shielding area LSA 1, and an area other than the first light shielding area LSA 1 may be referred to as a first light transmissive area TA 1. In addition, in FIG. 3, the dotted areas may be referred to as second light shielding areas LSA 2, and an area other than the second light shielding areas LSA 2 may be referred to as a second light transmissive area TA 2.").
Regarding claim 8, Igeta in view of Yahata teaches The system for generating the depth map according to claim 6.
In addition, Igeta teaches wherein the aperture pattern of the coded aperture part includes a plurality of the second areas. (Para. 32 see "when the liquid crystal panel PNL is in the absorption state, a coded aperture pattern is displayed on the liquid crystal panel PNL, and a large number of incident light control areas are formed." Para. 34 see "In FIGS. 2 and 3, dotted areas correspond to the light shielding area LSA, and other areas correspond to the light transmissive area TA. Incidentally, in FIG. 2, the dotted area may be referred to as a first light shielding area LSA 1, and an area other than the first light shielding area LSA 1 may be referred to as a first light transmissive area TA 1. In addition, in FIG. 3, the dotted areas may be referred to as second light shielding areas LSA 2, and an area other than the second light shielding areas LSA 2 may be referred to as a second light transmissive area TA 2.").
Regarding claim 11, Igeta teaches means for forming a coded aperture part with a use of a liquid crystal shutter; (Abstract see "a camera module includes a liquid crystal panel, an image sensor and an optical system. The liquid crystal panel is configured to display a coded aperture pattern. The optical system is interposed between the liquid crystal panel and the image sensor. The liquid crystal panel includes a liquid crystal layer containing liquid crystal molecules and dichroic dye molecules aligned following the liquid crystal molecules." Para. 28 see "by using the above-described coded aperture technology, it is possible to calculate the distance to the subject on the basis of the image and create a depth map indicating the distance to the subject. Incidentally, processing of calculating the distance to the subject, processing of creating the depth map, and the like are executed by a controller (CPU) that controls an operation of the camera module or a controller of an electronic device connected to the camera module (electronic device on which the camera module is mounted)."). means for capturing light transmitted through the coded aperture part; (Para. 31 see "when the liquid crystal panel PNL is in the transparent state, light transmitted through the liquid crystal panel PNL and the optical system OS enters the image sensor IS. As a result, the camera module CM can capture an image based on the light incident on the image sensor IS." Para. 32 see "when the liquid crystal panel PNL is in the absorption state, a coded aperture pattern is displayed on the liquid crystal panel PNL, and a large number of incident light control areas are formed. In other words, the light transmitted through the liquid crystal panel PNL in which the coded aperture pattern is displayed and the optical system OS is incident on the image sensor IS."). and means for using a captured image acquired by the imaging device to calculate a depth for each unit area of the depth map, (Para. 28 see "by using the above-described coded aperture technology, it is possible to calculate the distance to the subject on the basis of the image and create a depth map indicating the distance to the subject. Incidentally, processing of calculating the distance to the subject, processing of creating the depth map, and the like are executed by a controller (CPU) that controls an operation of the camera module or a controller of an electronic device connected to the camera module (electronic device on which the camera module is mounted)."). wherein the means for forming the coded aperture part forms an aperture pattern that includes a first area, a second area, (Para. 34 see "The incident light control area PCA includes a light shielding area LSA that shields incident light incident on the image sensor IS, and a light transmissive area TA through which the incident light incident on the image sensor IS is transmitted."). the first area having a first light transmittance, (Para. 38 see "FIG. 3D shows an example of an aperture formed by an aggregate of regions having different transmittances." Examiner Note: Fig. 3D shows four different regions with different transmittance levels.). the second area having a second light transmittance lower than the first light transmittance, (Para. 34 see "In FIGS. 2 and 3, dotted areas correspond to the light shielding area LSA, and other areas correspond to the light transmissive area TA. Incidentally, in FIG. 2, the dotted area may be referred to as a first light shielding area LSA 1, and an area other than the first light shielding area LSA 1 may be referred to as a first light transmissive area TA 1." Examiner Note: The light shielding area LSA has low light transmittance and the transmissive area TA has high light transmittance.).
Igeta does not teach A non-transitory information storage medium storing a program that causes a computer to function as: and a third area, the third area having a third light transmittance lower than the second light transmittance.
However, Yahata teaches A non-transitory information storage medium storing a program that causes a computer to function as: (Para. 30 see "A microprocessor (CPU) 115 executes programs stored in the read only memory (ROM) of a memory 116 and the like by using the random access memory (RAM) of the memory 116 as a work memory to control the respective components and execute various control operations and various processes via a system bus 120." Para. 31 see "The memory 116 holds information such as programs executed by the CPU 115, the imaging parameters output from the optical system control unit 114, optical characteristic information used for distance estimation processing, and noise parameters for the image sensing apparatus 100."). and a third area, (Para. 38 see "FIG. 3D shows an example of an aperture formed by an aggregate of regions having different transmittances." Examiner Note: Fig. 3D shows four different regions with different transmittance levels.). the third area having a third light transmittance lower than the second light transmittance. (Para. 38 see "FIG. 3D shows an example of an aperture formed by an aggregate of regions having different transmittances." Examiner Note: Fig. 3D shows four different regions with different transmittance levels.).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Igeta to incorporate the teachings of Yahata to include a third area in the aperture pattern having a third light transmittance lower than the second light transmittance. Doing so would predictably improve depth estimation accuracy by providing various levels of light transmittance beyond minimum and maximum light transmittance. This would create more distinctive blur patterns at different object distances. Furthermore, it would be obvious to include a non-transitory information storage medium storing a program to be executed by a processor. Doing so would predictably improve the robustness and flexibility of the system by allowing the depth estimation algorithm to be easily updated, maintained, distributed, and executed as opposed to fixed mechanical parts.
Claim 10 is rejected under the same analysis as claim 1 above.
Allowable Subject Matter
Claim(s) 3-4, 9 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claims 3 and 4, the prior art references of Igeta and Yahata do not disclose the specific transmittance percentages of the second transmittance.
Regarding claim 9, the prior art references of Igeta and Yahata do not disclose point spread functions using reference depths to calculate a deviation value for each unit area of the depth map using a plurality of reference depth restored images generated based on captured images and a PSF among a plurality of PSFs.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Han et al. (US20100201865A1) discloses an imaging method for use with a variable coded aperture filter and an imaging apparatus using the imaging method.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEXANDER J VAUGHN whose telephone number is (571) 272-5253. The examiner can normally be reached M-F 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ANDREW MOYER can be reached on (571) 272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALEXANDER JOSEPH VAUGHN/Examiner, Art Unit 2675
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights