DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on September 1, 2023 has been considered by the examiner and placed in the file.
Claim Interpretation
The claims in this application are given their broadest reasonable interpretation (BRI) using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art.
Some of the claim terms have been given BRIs below in light of the specification. These BRIs are used for purposes of searching for prior art, but cannot be incorporated into the claims. Should applicant wish different interpretations, Applicant should point to the portions of the specification that clearly support a different interpretation.
The BRI of a claim limitation is limited by the description in the specification when 35 U.S.C. 112(f), is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f), is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f), because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
A degradation unit in claim 1;
A restoration unit in claim 1;
A depth estimation unit in claim 1;
A sampling unit in claim 3;
A saturation determination unit in claim 4; and
A control information input unit in claim 6.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f), they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
The BRIs for all of these terms is a computer including a processor for performing the functions and memory for storing instructions that are executed by the processor to cause the computer to perform the corresponding functions, and equivalents thereof. Support for this interpretation is found in paras. [0006], [0067], [0068] and [0069] of the present specification.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f), applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 11 is also rejected under 35 U.S.C. 101 for the additional reason that it recites a computer program per se, which has been held to constitute non-eligible subject matter. (See Microsoft Corp. v. AT&T Corp., 550 U.S. 437, 449, 82 USPQ2d 1400, 1407 (2007); see also Benson, 409 U.S. 67, 175 USPQ2d 675 (An "idea" is not patent eligible); see also MPEP 2106.03 I).
Claims 1-9 fall into the statutory class of machine. Claim 10 falls into the statutory class of process. Claim 11, as drafted, does not fall into any of the statutory classes, but would fall into the statutory class of manufacture if amended to recite that the program is embedded on a non-transitory computer-readable medium, for which support is found in the present disclosure.
Independent claims 1, 10 and 11 recite a mathematical algorithm that processes a light reception image on the basis of a degradation characteristic to blur the image to produce light reception data, that processes the light reception data using a restoration characteristic that is the inverse of the degradation characteristic to restore the data and that estimates depth of a subject based on the restored data.
The USPTO has enumerated groupings of abstract ideas that are firmly rooted in Supreme Court precedent as well as Federal Circuit decisions interpreting that precedent (See MPEP 2106.04(a)). The enumerated groupings of abstract ideas are defined as:
1) Mathematical concepts – mathematical relationships, mathematical formulas or equations, mathematical calculations;
2) Certain methods of organizing human activity – fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions); and
3) Mental processes – concepts performed in the human mind (including an observation, evaluation, judgment, opinion).
The operations recited in claims 1, 10 and 11 are mathematical operations that fall under enumerated grouping 1).
Once it has been determined that the claim under examination recites an abstract idea, the claim must be further analyzed to determine whether any additional elements in the claim integrate the abstract idea into a practical application (See MPEP 2106.04(d)). The Supreme Court has distinguished between principles themselves, which are not patent eligible, and the integration of those principles into practical applications, which are patent eligible. See, e.g., Mayo Collaborative Servs. v. Prometheus Labs., Inc., 566 U.S. 66, 80, 84, 101 USPQ2d 1961, 1968-69, 1970 (2012).
Accordingly, after determining that a claim recites an abstract idea, the claim as a whole must be evaluated to determine whether it integrates the recited abstract idea into a practical application. To determine whether the claim as a whole integrates the abstract idea into a practical application, any additional elements of the claims (i.e., claim elements other than those that constitute the abstract idea) must be evaluated to determine whether they amount to significantly more than the judicial exception itself.
In claim 1, the only additional elements other than the abstract idea are the degradation unit, the restoration unit and the depth estimation unit. In determining whether the additional elements recited in the claim integrate the abstract idea into a practical application, a determination must made as to whether or not the additional elements amount to significantly more than the abstract idea. Simply appending well-understood, routine, conventional components or activities previously known to the industry, specified at a high level of generality, to the judicial exception does not amount to the claim reciting significantly more than the abstract idea.
As indicated above, the BRI for these elements is a computer including a processor and memory for performing the recited functions, and the equivalents thereof. These components are well-understood, routine, conventional devices previously known to the industry, specified at a high level of generality. Therefore, the recitation of these additional elements in claim 1 does not amount to claim 1 reciting significantly more than the abstract idea.
Claim 2 does not recite any elements in addition to those discussed above, but merely further details about the steps recited in claim 1. Therefore, claim 2 does not recite significantly more than the abstract idea.
Claims 3, 4 and 6 recite the additional elements of the sampling unit, the saturation determination unit and the control information input unit, respectively. As indicated above, the BRI for these elements is a computer including a processor and memory for performing the recited functions, and the equivalents thereof. These components are well-understood, routine, conventional devices previously known to the industry, specified at a high level of generality. Therefore, the recitation of these additional elements in claims 3, 4 and 6 does not amount to those claims reciting significantly more than the abstract idea.
Claims 5, 7, 8 and 9 do not recite any elements in addition to those discussed above, but merely further details about the steps recited in the claims from which they depend. Therefore, these claims do not recite significantly more than the abstract idea.
Claim 10 also does not recite any elements in addition to those discussed above with reference to claim 1, but merely further details about the functions recited in claim 1, which are the same as those recited in the steps of claim 10. Therefore, claim 10 does not recite significantly more than the abstract idea.
Claim 11 recites the same steps as recited in claim 1 and does not recite any elements in addition to those discussed above except for the program, which is a well-understood, routine, conventional part. Therefore, claim 10 does not recite significantly more than the abstract idea.
For all of these reasons, claims 1-11 are rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 102
Claim(s) 1 and 6-11 are rejected under 35 U.S.C. 102(a) as being anticipated by U.S. Publ. Appl. No. 2018/0143419 A1 to Bryll et al. (hereinafter referred to as “Bryll et al.”).
Regarding claim 1, Bryll et al. discloses and information processing apparatus (Fig. 1, para. [0018], machine vision inspection system 10) comprising:
a degradation unit (lens controller 271) that blurs a light reception image of reference light received by a light reception unit on a basis of a known degradation characteristic (the BRI for this limitation, based on para. [0085] of the present specification, is that the computer causes a focus position of the light reception image onto the SPAD to be shifted to produce a predetermined amount of blur in the image being detected by the SPAD; in Bryll et al., para. [0026], the lens controller 271 can be controlled by an operating program that controls modulation of lens 270 “to periodically modulate its optical power (e.g., sinusoidally) and thereby periodically modulate a focus position of the imaging system over a plurality of focus positions along a Z-height direction at a determined operating frequency. The periodically modulated VFL lens optical power defines a periodic focus modulation, which corresponds to a focus distance variation for the imaging system”; para. [0053] of Bryll et al. discloses that modulation of the focus position blurs the light reception image: “the imaging system 300 may provide a raw EDOF image, which is a blurred image including information acquired throughout a desired focus range during its exposure”; for each focus position, the degradation characteristic that produces the blur is known because it is based on the known characteristics of the optics system and on the known focus position of lens 270);
a restoration unit (Fig. 2, control system portion 120) that restores light reception data of the reference light using a restoration characteristic, the restoration characteristic being an inverse characteristic of the degradation characteristic (paras. [0046]-[0052], the light reception image, which is the “raw EDOF image” in Bryll et al., is processed by the control system portion 120 to remove defocus blur by using a restoration characteristic, namely, an integrated point spread function (IPSF), that is the inverse of the degradation characteristic of the optics system; paras. [0047]-[0048] and equations 1 and 2, the IPSF is used in a deconvolution process that is the inverse of a convolution that convolves a function representing the raw EDOF image acquired at different focus positions over time with the IPSF); and
a depth estimation unit that estimates a depth of a subject on a basis of the restored light reception data (paras. [0050]-[0053] and equations 4 and 5, the depth estimate is given in equation 5, which incorporates the Fourier transform inverse operation to produce the estimated depth of field for each focal position).
Regarding claim 6, Bryll et al. discloses that information processing apparatus comprises a control information input unit (Fig. 2, lens controller/interface 271) that inputs control information indicating required accuracy of depth and a degradation characteristic determination unit that determines the degradation characteristic on a basis of the control information (para. [0026], the operating program of the lens controller 271 controls the periodic modulation of the focus position of the lens 270 in a predetermined manner to select the timing of when the lens 270 is at predetermined Z-height positions during exposure, which controls the accuracy at which the depth of field estimates are calculated in equation 5; the degradation characteristic is determined on the basis of the modulation and focus position information because it is used to calculate the IPSF in equation 1, which is the “blur function” corresponding to the degradation characteristic, para. [0048]).
Regarding claim 7, the BRI for this claim, based on paras. [0063]-[0066] and [0079]-[0084] of the present specification, is that the control information input unit determines that accuracy of the depth measurement that is required for a particular situation and causes a degradation characteristic to be selected to achieve the required depth accuracy so that the restored image will have the required depth accuracy. For example, if a very accurate distance measurement is required, a degradation characteristic is selected that increases the blur of the light reception image.
Bryll et al. discloses that the control information input unit determines that accuracy of the depth measurement that is required for a particular situation and causes a degradation characteristic to be selected to achieve the required depth accuracy in the restored image (para. [0037], the focus position is modulated in a predetermined manner while displaying live images having particular depth accuracies to the user, but when a very accurate high-resolution image of a particular feature is required, a fixed focus position can be selected. The fixed focus position would produce a corresponding defocus blur in the raw EDOF image that would be removed using the inverse deconvolution process described above to produce the highly accurate restored image).
Regarding claim 8, Bryll et al. discloses that the control information input unit estimates a situation in which measurement is performed on a basis of sensor information, and determines the required accuracy of the depth on a basis of the estimated situation (paras. [0063]-[0064], Fig. 4, a live EDOF image processing and display manager 422 and an EDOF processing portion work in conjunction with one another to estimate a situation on the basis of sensor information output from camera 460 of the machine vision system 500, where the estimated situation corresponds to speed of movement of the workpiece; based on the estimated situation, the control information input unit (422/423) determines the accuracy of the of depth that is required and causes a particular level of processing to be performed to remove defocus blur to achieve a particular depth).
Regarding claim 9, Bryll et al. discloses that the degradation unit blurs the light reception image by shifting a focal position of a lens of the light reception unit (para. [0026], modulating the focus position of the lens 270) and that the restoration unit restores the light reception data on a basis of the restoration characteristic generated from a blur model according to the shift of the focal position of the lens (paras. [0046]-[0051] and equations 1-5, the IPSF is a blur model (“blur function”) that restores the light reception data on the basis of the restoration characteristic generated from the blur model according to the modulation of the focus position).
Regarding claim 10, the rejection of claim 1 applies mutatis mutandis to claim 10.
Regarding claim 11, to the extent that claim 11 recites the same limitations that are recited in claim 1, the rejection of claim 1 applies mutatis mutandis to claim 11. The only limitation that is recited in claim 11 that is not also recited in claim 1 is a program. The BRI for “program” is a computer program. Bryll et al. discloses a computer program for execution on a processor for performing the operations recited in claims 1 and 11 (paras. [0019], [0041] and claim 17).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Bryll et al. in view of U.S. Publ. Appl. No. 2019/0004533 A1 to Huang et al. (hereinafter referred to as “Huang et al.”).
Regarding claim 3, Bryll et al. does not explicitly disclose that the sampling unit that samples the light reception data up-samples the light reception data such that the restoration is performed on the up-sampled light reception data.
Huang et al., in the same field of endeavor, discloses up-sampling an light reception data (para. [0035], Fig. 2, data obtained by LiDAR unit 215) to increase its size by a factor (para. [0061], Fig, 4). It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the machine vision inspection system 10 of Bryll et al. to perform up-sampling of the light reception data as taught by Huang et al. One of ordinary skill in the art would have been motivated to make the modification to improve resolution of the restored image since up-sampling of image data is commonly performed for this purpose. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods (modifying the software executed by the control system portion 120 of Bryll et al.) to yield predictable results.
Allowable Subject Matter
Claims 2, 4 and 5 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims and to overcome the rejections under 35 U.S.C. 101.
The following is a statement of reasons for the indication of allowable subject matter.
Regarding claim 2, the BRI for this claim, based on paras. [0020]-[0023], is that the degradation characteristic causes the light reception image to be blurred, or stretched, such that the half-value width of its peak is twice or more the width of the sampling period during which the sensor elements of the light detector are sampled. None of the prior art teaches or suggests this limitation.
Regarding claim 4, the BRI for this limitation is that when saturated data is detected, unsaturated data that is correlated in time with the saturated data is used to correct the saturated data. None of the prior art teaches this limitation. U.S. Publ. Appl. No. 2019/0250257 A1 to Finkelstein et al. discloses detecting when saturation begins occurring or is about to occur and taking steps to ensure that saturated data is not used, para. [0154]. However, Finkelstein et al. does not teach or suggest using unsaturated data that is time-correlated with saturated data to correct the saturated data.
Regarding claim 5, it recites allowable subject matter due to its dependency from claim 4.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. Publ. Appl. No. 2019/0250257 A1 to Finkelstein et al. discloses Light Detection And Ranging (LIDAR) apparatus includes a pulsed light source to emit optical signals, a detector array comprising single-photon detectors to output respective detection signals indicating times of arrival of a plurality of photons incident thereon, and processing circuitry to receive the respective detection signals. The processing circuitry includes one or more of a recharge circuit configured to activate and deactivate subsets of the single photon detectors for respective strobe windows between pulses of the optical signals and at differing delays, a correlator circuit configured to output respective correlation signals representing detection of one or more of the photons having times of arrival within a predetermined correlation time relative to one another, and a time processing circuit comprising a counter circuit configured to increment a count value and a time integrator circuit configured to generate an integrated time value based on the respective correlation signals or detection signals. The processing circuitry detects when saturation begins occurring or is about to occur and takes steps to ensure that saturated data is not collected.
U.S. Publ. Appl. No. 2020/0116558 A1 discloses a ranging/imaging system that can scan a field of view using a multispectral sensor array, e.g., by rotating the sensor array about an axis transverse to the rows. During this motion, a given location in space can be successively imaged by each of the channel types, thereby providing a multispectral image set with inherent registration between imaging modalities (or channels). The spatial relationship of the channels in the array, optical properties of the imaging optics (e.g., a focal length distortion profile of a bulk imaging optic), and the imaging rate relative to motion (e.g., rotation) of the sensor array can be selected so that the data from different channels maps easily onto a uniform grid of pixels representing the field of view.
U.S. Publ. Appl. No. 2020/00551265 A1 discloses electronic device having a first and second camera deployed on one side of the electronic device, a memory, and at least one processor configured to acquire a plurality of first image frames for external objects using the first camera based on an input corresponding to a photographing signal, acquire a second image frame for the external objects using the second camera while acquiring parts of the first image frames, generate depth information for the external objects based on the image frame corresponding to the second image frame among the plurality of first image frames and the second image frame, generate a first corrected image by combining a plurality of designated image frames among the plurality of first image frames, and generate a second corrected image in which parts of the external objects included in the first corrected image are blurred based on the depth information.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL J SANTOS whose telephone number is (571)272-2867. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Bella can be reached at (571)272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL JOSEPH SANTOS/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667