DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the original application filed on 12/28/2023 and the Preliminary Amendment to the claims filed on 3/2/2024. Acknowledgment is made with respect to a claim of priority to Chinese Application CN202211694760.8 filed on 12/28/2022.
Claim Objections
Claims 13-26 are objected to because of the following informalities:
Claim 13 recites the limitation “wherein the method for training a neural network model for interferogram phase estimation including:” (emphasis added) which should read as “wherein the method for training a neural network model for interferogram phase estimation [[including]] includes:” (emphasis added) for better grammar. Dependent claims 14-26 depend on objected claim 13, and are also objected to by virtue of this dependency. Appropriate correction is required.
Claim 24 recites the limitation “rein the loss function value is a root mean square error loss function value, and is represented as:” (emphasis added) which should read as “wherein [[rein]] the loss function value is a root mean square error loss function value, and is represented as:” (emphasis added) to correct a typo. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 13-26 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 13 recites the limitations “inputting the interferogram for estimation to a neural network model trained based on a method for training a neural network model … wherein the method for training a neural network model for interferogram phase estimation including: obtaining a training interferogram and a true phase image of the training interferogram; inputting the training interferogram to a neural network model” (emphasis added).It is unclear if the claimed “a neural network model” referenced in the limitations above are one and the same. For examination purposes, the limitations will be interpreted to mean that the recited neural network model is the same model used throughout the claim for both the training and inference processes. Dependent claims 14-26 depend on indefinite claim 13, and are also rejected under 35 USC § 112(b) by virtue of this dependency. Appropriate correction is required.
Claim 23 recites the limitation “when image cropping is performed on the training interferogram, a quantity of pixels in a cropped image is sufficient to fit an ideal sphere” (emphasis added). First, the claim does not recite a positively recited step. The “image cropping” is phrased as a conditional statement. Second, the terms “sufficient” and “ideal” are relative terms of degree which render the claim indefinite. The terms “sufficient” and “ideal” are not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. For examination purposes, the limitation will be interpreted to mean “[[when]] performing image cropping [[is performed]] on the training interferogram[[,]] so that a quantity of pixels in a cropped image [[is sufficient to]] fits [[an ideal]] a sphere” (emphasis added). Appropriate correction is required.
Claim 25 recites an equation with undefined variables “m” and “n”. For examination purposes, the variables “m” and “n” in the equation of the claim will be interpreted to be matrix size dimensions as disclosed in claim 24. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 13, 17-19, 21, 24, and 25 are rejected under 35 U.S.C. § 103 as being obvious over Kando et al. (Kando et al., “Phase Extraction from Single Interferogram Including Closed-Fringe Using Deep Learning”, Aug. 28, 2019, Appl. Sci. 2019, 9, 3529; doi:10.3390/app9173529, pp. 1-13, hereinafter “Kando”) in view of Zhou et al. (Zhou et al., “UNet++: A Nested U-Net Architecture for Medical Image Segmentation”, Jul. 1, 2020, Deep Learn Med Image Anal Multimodal Learn Clin Decis Support (2018), pp. 1-12, hereinafter “Zhou”).
Regarding claim 13, Kando discloses [a]n interferogram phase estimation method, comprising: (Abstract; “In an optical measurement system using an interferometer, a phase extracting technique from interferogram is the key issue … To extract the phase from the interferogram including the closed-fringes we propose the use of deep learning”)
obtaining an interferogram for estimation of a measured object; and (Pages 5-6, §3; “In this study, the input, x, and the output, y, correspond to the interferograms and the phase-shift images, respectively … In the simulation of the interferogram, a white noise with normal distribution, inoise(r),was also superposed”, which discloses obtaining an interferogram that is used as an input for estimation; and Figure 7 Caption; “interferograms which are input to the phase extraction”; and Page 11, §4.2; “we applied the algorithms to interferograms which are obtained by an experiment to measure a 3D refractive index distribution in which the object is three candle flames”, wherein the measured object is the candle flame or gas sample being examined by the interferometer)
inputting the interferogram for estimation to a neural network model trained based on a method for training a neural network model for interferogram phase estimation, to obtain a phase image corresponding to the interferogram for estimation; (Abstract; “To extract the phase from the interferogram including the closed-fringes we propose the use of deep learning. A large amount of the pairs of the interferograms and phase-shift images are prepared, and the trained network, the input for which is an interferogram and the output a corresponding phase-shift image, is obtained using supervised learning … the trained network can be applicable to the interferogram including the closed-fringes”, wherein the trained u-net receives a single interferogram and outputs the corresponding phase-shift or phase image; and §3; “the output, y, correspond to the interferograms and the phase-shift images”; and §4; “The implementation was done using Chainer, which is one of the frameworks for neural networks”; and Figure 7 Caption; “The remained two columns are of the phase-shift images estimated by the deep learning, φDL”, wherein the output is explicitly a phase image)
wherein the method for training a neural network model for interferogram phase estimation including: obtaining a training interferogram and a true phase image of the training interferogram; (Pages 5-6, §3; “Since the problem is the inverse problem, the output of the deep learning system, y, which is the phase-shift image, φ(r), is defined first; then the input of the learning, x, which is the interferogram, i(r), is evaluated by the simulation of interference according to Equation (3) … In this study, 90,000 pairs of interferograms and corresponding phase-shift images were generated, of which 80,000 pairs were used as learning data and 10,000 pairs as test data…. The loss function during the training step was defined by the average of the root mean squared error of the estimated output image, fDL(r), from the ground truth, fGT(r)”, wherein the claimed true phase image corresponds to the ground truth of Kando)
inputting the training interferogram to a neural network model, (Page 5, §3; “This study uses supervised learning. Supervised learning is one of the machine learning methods and is a method for obtaining an optimal functional relation y = f (x) by learning a model using a large number of input data, x, and output data, y. In this study, the input, x, and the output, y, correspond to the interferograms and the phase-shift images, respectively”; and Figure 3; the figure discloses receiving the input images which are the interferograms)
obtaining a predicted phase image output by the neural network model; (Page 7, §3; “The training means the iteration of updating the network parameters so that the value of the loss function E becomes smaller”; and Figure 7 Caption; φDL is described as the deep-learning phase-estimated image)
calculating a loss function value between the predicted phase image and the true phase image; and (Page 6, §3 and Equation 12; “The loss function during the training step was defined by the average of the root mean squared error of the estimated output image, fDL(r), from the ground truth, fGT(r)”)
training, by minimizing the loss function value, the neural network model through gradient back propagation (Pages 6-7, §3; “As a learning method of the network, the back-propagation method was used … The training means the iteration of updating the network parameters so that the value of the loss function E becomes smaller”).
Kando fails to explicitly disclose but Zhou discloses wherein the neural network model has N convolutional layer branches, an ith convolutional layer branch in the N convolutional layer branches has a plurality of N+1−i convolutional layer branches that are cascaded, 1≤i≤N, an output feature map of a first convolutional layer in the ith convolutional layer branch is down-sampled and then input to a first convolutional layer in an (i+1)th convolutional layer branch, an output feature map of a jth convolutional layer in the (i+1)th convolutional layer branch is up-sampled and then input to a (j+1)th convolutional layer in the ith convolutional layer branch, 1≤j≤N+1−i, and an output feature map of each convolutional layer in the ith convolutional layer branch is input to each downstream convolutional layer of the convolutional layer; (Page 7, Figure 1; the figure discloses the neural network model with N branches that are cascaded, down sampling an output feature map, inputting the feature maps into a next convolutional layer in a branch, up-sampling the output feature map, and inputting the output feature map into a subsequent convolutional layer in the branch; and Page 3 §3 and §3.1; “UNet++ starts with an encoder sub-network or backbone followed by a decoder sub-network … the skip pathway between nodes X0,0 and X1,3 consists of a dense convolution block with three convolution layers where each convolution layer is preceded by a concatenation layer that fuses the output from the previous convolution layer of the same dense block with the corresponding up-sampled output of the lower dense block … Basically, nodes at level j = 0 receive only one input from the previous layer of the encoder; nodes at level j = 1 receive two inputs, both from the encoder sub-network but at two consecutive levels; and nodes at level j > 1 receive j + 1 inputs, of which j inputs are the outputs of the previous j nodes in the same skip pathway and the last input is the up-sampled output from the lower skip pathway”; and Equation 1).
Kando and Zhou are analogous art because both are concerned with U-net deep learning frameworks. Before the effective filing date of the claimed invention, it would have been obvious to one skilled in deep learning to combine the U-net architecture of Zhou with the phase estimation of Kando to yield to the predictable result of wherein the neural network model has N convolutional layer branches, an ith convolutional layer branch in the N convolutional layer branches has a plurality of N+1−i convolutional layer branches that are cascaded, 1≤i≤N, an output feature map of a first convolutional layer in the ith convolutional layer branch is down-sampled and then input to a first convolutional layer in an (i+1)th convolutional layer branch, an output feature map of a jth convolutional layer in the (i+1)th convolutional layer branch is up-sampled and then input to a (j+1)th convolutional layer in the ith convolutional layer branch, 1≤j≤N+1−i, and an output feature map of each convolutional layer in the ith convolutional layer branch is input to each downstream convolutional layer of the convolutional layer. The motivation for doing so would be to provide for a new, more powerful architecture for medical image segmentation (Zhou; Abstract).
Regarding claim 17, the rejection of claim 13 is incorporated and Kando further discloses imaging a predetermined object through a predetermined image acquisition device to obtain the training interferogram (§4.2; and see generally §3).
Regarding claim 18, the rejection of claims 13 and 17 are incorporated and Kando further discloses imaging the predetermined object through the predetermined image acquisition device based on a multi-step phase shifting method, to obtain a plurality of interferograms with a plurality of phases; and
obtaining a phase-shift-free interferogram as the training interferogram (§4.2; and see generally §3).
Regarding claim 19, the rejection of claim 13 is incorporated and Kando fails to explicitly disclose but Zhou discloses wherein a quantity of convolutional layer branches of the neural network model is 4, a quantity of convolution kernels is doubled from 32, and a size of a convolution kernel is 5×5 (see generally §4).
Regarding claim 21, the rejection of claim 13 is incorporated and Kando further discloses performing image cropping or image augmentation on the training interferogram (§5; “A large amount of interferograms and their corresponding phase-shift images were prepared by simulation, and deep neural networks with them as input and output, respectively, were trained by supervised learning”; and §3; “the pairs of the phase-shift images generated by simulation and the interferograms evaluated by them are prepared as training data”).
Regarding claim 24, the rejection of claim 13 is incorporated and Kando further discloses rein the loss function value is a root mean square error loss function value, and is represented as: … where F1 (i) and F2 (i) are respectively matrix representations of a predicted phase image and a true phase image that correspond to an ith interferogram in k interferograms in total, and matrix sizes of the predicted phase image and the true phase image are m×n (Page 6, Equation 12).
Regarding claim 25, the rejection of claim 13 is incorporated and Kando further discloses wherein the loss function value is a relative root mean square error loss function value, and is represented as … where F1 (i) a,b and F2 (i) a,b are respectively pixel values at a location (a, b) in a predicted phase image and a true phase image that correspond to an ith interferogram in k interferograms in total (Page 6, Equation 13).
Claims 14-16, 23, and 26 are rejected under 35 U.S.C. § 103 as being obvious over Kando in view of Zhou and further in view of Nguyen et al. (US 20020109831 A1, hereinafter “Nguyen”).
Regarding claim 14, the rejection of claim 13 is incorporated and Kando fails to explicitly disclose but Nguyen discloses wherein the measured object is an interferogram of the optical fiber end face (Abstract; “A method and apparatus is provided for mounting any one of a plurality of types of connectors upon the end portion of a fiber optic cable”; and [0030-0031]; “FIGS. 9A-9D depict four phase shift images (or interferrograms) with phase shifts of π/2, π, 3π/2 and 2π, respectively. … FIG. 9E is a composite phase shift image based upon the four phase shift images of FIGS. 9A-9D”; and Figures 9A-9E).
Kando, Zhou, and Nguyen are analogous art because all are concerned with image analysis. Before the effective filing date of the claimed invention, it would have been obvious to one skilled in image analysis to combine the optical fiber of Nguyen with the phase estimation of Kando and Zhou to yield to the predictable result of wherein the measured object is an interferogram of the optical fiber end face. The motivation for doing so would be to automatically inspect and classify optical fibers during a connectorization process (Nguyen; [0007]).
Regarding claim 15, the rejection of claims 13 and 14 are incorporated and Kando fails to explicitly disclose but Nguyen discloses wherein obtaining an interferogram for estimation of a measured object includes imaging the optical fiber end face through a predetermined image acquisition device to obtain the interferogram of the optical fiber end face ([0071]; “the imaging system 108 includes a scanning camera 110, an interferometer 111 and an associated micropositioner 112 for moving the camera in increments, such as 6 micron increments, in order to scan the surface of the end face 94 of the optical fiber 95 at different phase shift positions per exposure, such as π/2 phase shift positions per exposure”).
The motivation to combine Kando, Zhou, and Nguyen is the same as discussed above with respect to claim 14.
Regarding claim 16, the rejection of claims 13 and 14 are incorporated and Kando fails to explicitly disclose but Nguyen discloses providing an optical fiber connector to be determined qualified; and imaging the optical fiber end face of the optical fiber connector through a predetermined image acquisition device to obtain the interferogram of the optical fiber end face ([0084]; “the end face inspection unit 106 also preferably determines if an unacceptable end face can be corrected, such as by further polishing the end face, or if the optical fiber must be totally reworked or discarded”; and [0082]; “the features extracted from the composite image include: (1) radius of curvature of the end face, (2) spherical-fitting error, (3) fiber height (protruding or recessed), (4) fiber core diameter, (5) fiber cladding diameter, (6) concentricity of the end face, and (7) fringes within the Region of Interest (ROI) including, at least, the fiber end-face”).
The motivation to combine Kando, Zhou, and Nguyen is the same as discussed above with respect to claim 14.
Regarding claim 23, the rejection of claims 13 and 21 are incorporated and Kando fails to explicitly disclose but Nguyen discloses when image cropping is performed on the training interferogram, a quantity of pixels in a cropped image is sufficient to fit an ideal sphere ([0084]; “the end face inspection unit 106 also preferably determines if an unacceptable end face can be corrected, such as by further polishing the end face, or if the optical fiber must be totally reworked or discarded”; and [0082]; “the features extracted from the composite image include: (1) radius of curvature of the end face, (2) spherical-fitting error, (3) fiber height (protruding or recessed), (4) fiber core diameter, (5) fiber cladding diameter, (6) concentricity of the end face, and (7) fringes within the Region of Interest (ROI) including, at least, the fiber end-face”).
The motivation to combine Kando, Zhou, and Nguyen is the same as discussed above with respect to claim 14.
Regarding claim 26, the rejection of claim 13 is incorporated and Kando fails to explicitly disclose but Nguyen discloses calculating, based on the phase image, at least one of a radius of curvature, a vertex offset, and an optical fiber height of a predetermined object ([0082]; “the features extracted from the composite image include: (1) radius of curvature of the end face, (2) spherical-fitting error, (3) fiber height (protruding or recessed), (4) fiber core diameter, (5) fiber cladding diameter, (6) concentricity of the end face, and (7) fringes within the Region of Interest (ROI) including, at least, the fiber end-face”).
The motivation to combine Kando, Zhou, and Nguyen is the same as discussed above with respect to claim 14.
Claim 20 is rejected under 35 U.S.C. § 103 as being obvious over Kando in view of Zhou and further in view of Yuan et al. (Yuan et al., “High-accuracy phase demodulation method compatible to closed fringes in a single-frame interferogram based on deep learning”, Jan. 18, 2021, Optics Express, Vol. 29, Issue 2, pp. 2538-2554, hereinafter “Yuan”).
Regarding claim 20, the rejection of claim 13 is incorporated and Kando fails to explicitly disclose but Yuan discloses wherein each convolution layer comprises a ResBlock convolution block (§2.4; “The network’s main body is a four-layer U-Net structure, and the processing of each layer is realized through DenseBlock”; and Figure 3 and its caption; “(b) The DenseBlock used in (a) has a growth rate of 8. (c) The ConvBlock used in DenseBlock”, which discloses that the resblock or denseblock comprises a resblock convolution block or convblock).
Kando, Zhou, and Yuan are analogous art because all are concerned with phase estimation and deep learning. Before the effective filing date of the claimed invention, it would have been obvious to one skilled in deep learning to combine the ResBlock convolution block of Yuan with the phase estimation of Kando and Zhou to yield to the predictable result of wherein each convolution layer comprises a ResBlock convolution block. The motivation for doing so would be to provide for a neural network architecture for single-frame interferogram demodulation (Yuan; Abstract).
Claim 22 is rejected under 35 U.S.C. § 103 as being obvious over Kando in view of Zhou and further in view of Lin et al. (Lin et al., “Optical Fringe Patterns Filtering Based on Multi-stage Convolution Neural Network”, Jul. 2, 2020, arXiv:1901.00361v3, pp. 1-12, hereinafter “Lin”).
Regarding claim 22, the rejection of claim 13 is incorporated and Kando fails to explicitly disclose but Lin discloses when a quantity of convolution layer branches is less than or equal to a predetermined threshold, performing image cropping on the training interferogram; or when a quantity of convolution layer branches is greater than a predetermined threshold, performing image augmentation on the training interferogram (§4.1.1; “To avoid over fitting to some extent, the common data augmentation is also used, such as horizontal inversion, rotation transformation and other geometric transformation methods. Finally, 230400 clean-noisy patch pairs with patch size of 80 × 80 are cropped to train the proposed CNN model”).
Kando, Zhou, and Yuan are analogous art because all are concerned with deep learning. Before the effective filing date of the claimed invention, it would have been obvious to one skilled in deep learning to combine the image cropping or augmentation of Lin with the phase estimation of Kando and Zhou to yield to the predictable result of when a quantity of convolution layer branches is less than or equal to a predetermined threshold, performing image cropping on the training interferogram; or when a quantity of convolution layer branches is greater than a predetermined threshold, performing image augmentation on the training interferogram. The motivation for doing so would be to directly remove speckle from the input noisy fringe pattern (Lin; Abstract).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Brent Hoover whose telephone number is (303)297-4403. The examiner can normally be reached Monday - Friday 9-5 MST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Kawsar can be reached at 571-270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRENT JOHNSTON HOOVER/Primary Examiner, Art Unit 2127
Prosecution Insights