Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Objections
Claim 14 is objected to because of the following informalities: for improper claim structure reciting “in any of the preceding claims”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-4, 11-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 2018/0260937 A1 to Gadi et al., hereinafter, “Gadi” in view of Material classification in X-ray images based on multi-scale CNN to Benedykciuk et al., hereinafter, “Benedykciuk”.
Claim 1. (currently amended) Gadi teaches A computer-implemented method of processing one or more inspection images comprising a plurality of pixels, the method comprising: [Abstract] a method of denoising one or more inspection images comprising a plurality of pixels
obtaining an input inspection image generated by an inspection system configured to inspect one or more containers, [0028] the inspection images may be generated by an inspection system 1 configured to inspect one or more containers
wherein the inspection system is configured to inspect the container by transmission, [0029] the inspection system 1 may be configured to inspect the container by transmission, through the container, of inspection radiation (e.g. x-ray) and may be configured to detect the transmitted radiation on an inspection radiation
through the container of inspection radiation generated by an accelerator, [0051] The source 2 shown in FIG. 5 comprises a device for producing and accelerating an electron beam 12. The source 2 may further comprise a target 14 for the electron beam (accelerator)
and having an angular divergence from the accelerator to an inspection radiation receiver comprising a plurality of detectors; [0029] to detect the transmitted radiation on an inspection radiation receiver comprising a plurality of detectors…the inspection radiation may have an angular divergence from an inspection radiation source…
the higher noise comprising a Poisson-Gaussian noise whose variance is non-constant in the plurality of pixels, Gadi [0055] the inspection image 10 of FIG. 3A is corrupted by the Poisson-Gaussian noise and the variance of the noise is non-constant in the plurality of pixels 13 of the image 10
the input inspection image having a higher noise, [0032] the inspection
image 10 of FIG. 3A is corrupted by the Poisson-Gaussian noise
Gadi teaches the inspection image having noise (Poisson-Gaussian) but fails to explicitly teaches the input image having a higher noise. Benedykciuk, in the same field of inspecting cargo in x-ray images, teaches [page 1290] probably due to the high noise of the input data…
[page 1291] …the input data, ... images from X-ray scanners are often very noisy (higher noise).
and a lower resolution; [page 1291] …the input data, which is necessary due to the architecture of the investigated ImageNet networks. Unfortunately, images from X-ray scanners are often very noisy (higher noise). Interpolation of such images will cause this noise to be blurred and enlarged (lower resolution).
and processing the obtained input inspection image by applying, to the input inspection image, a trained machine learning algorithm for simultaneously increasing the lower resolution and decreasing the higher noise, [page 1292, line 7-col. 2 line 2] …was observed for the proposed network-based method… We want to try out deep neural networks (autoencoders), whose aim is to denoise images or simultaneously interpolate images with removing noise
to generate an output inspection image having a resolution higher than the lower resolution and a noise lower than the higher noise. [page 1292, col. 2 lines 1-2] …deep neural networks (autoencoders), whose aim is to denoise images or simultaneously interpolate images with removing noise
Gadi teaches denoising x-ray images. Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Gadi with the teachings of Benedykciuk [Abstract] to improve the accuracy of material recognition in x-ray images.
Claim 2. (original) Benedykciuk teaches wherein the machine learning algorithm comprises a deep learning algorithm. [page 1292, col. 2 lines 1-2] …deep neural networks (autoencoders)
Claim 3. (original) Benedykciuk teaches wherein the machine learning algorithm comprises a deep neural network, DNN. [page 1292, col. 2 lines 1-2] …deep neural networks (autoencoders)
Claim 4. (currently amended) Benedykciuk teaches wherein the machine learning algorithm is previously trained using training data as input inspection images. [page 1289, col 1, lines 1-2] Our CNN classifier has been trained on input data based on low (LE) and high (HE) energy X-ray readings.
[page 1289, col 2, lines 1-2] trained the classifier on a dataset comprised of over 1 million sample patches and over 100k test patches
Claim 11. (currently amended) Gadi teaches performed on a part of the input inspection image corresponding to a zone of interest. [0144] As shown in FIG. 11A, in an example, the received inspection image 10 discussed above corresponds to a selected zone of interest 114 in an image 120 generated by the inspection system.
Claim 12. (currently amended) Gadi teaches wherein the input inspection image is defined by a zone of interest in an inspection image. [0144] As shown in FIG. 11A, in an example, the received inspection image 10 discussed above corresponds to a selected zone of interest 114 in an image 120 generated by the inspection system.
Claim 13. (currently amended) Gadi teaches performed on a computer comprising a memory and a processor. [0037] a processor 51, a memory 52
Claim(s) 5-6 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 2018/0260937 A1 to Gadi et al., hereinafter, “Gadi” in view of Material classification in X-ray images based on multi-scale CNN to Benedykciuk et al., hereinafter, “Benedykciuk” and in further view of Unprocessing Images for Learned Raw Denoising to Brooks et al., hereinafter, “Brooks”.
Claim 5. (original) Gadi and Benedykciuk fail to explicitly teach the training data is previously generated by a synthetic data generator implementing a method to generate a plurality of input inspection images. Brooks, in the field of denoising images using neural networks, teaches wherein the training data is previously generated by a synthetic data generator implementing a method to generate a plurality of input inspection images, [Figure 1], [Figure 2] sRGB images unprocess noisy raw images
[2. Related Work] accurate models for denoising JPEG-compressed sRGB images, whereas we focus on denoising raw images.
[4.4. Training] To create our synthetic training data, we start with the 1 million images of the MIR Flickr extended dataset…
the implemented method comprising, for each inspection image: [4.4. Training] We downsample all images by 2× using a Gaussian kernel (σ =1) to reduce the effect of noise…then take random 128×128 crops of each image…
injecting a Poissonian noise to the inspection image, [Figure 2] add Shot and Read Noise, [3.1. Shot and Read Noise] Shot noise is a Poisson random variable whose mean is the true light intensity (measured in photoelectrons).
the Poissonian noise being dependent on an intensity of the inspection radiation; [3.1. Shot and Read Noise] Shot noise is a Poisson random variable whose mean is the true light intensity (measured in photoelectrons)
and adding a random Gaussian noise to the inspection image having the injected Poissonian noise. [Figure 2] add Shot and Read Noise, [3.1. Shot and Read Noise] Read noise is an approximately Gaussian random variable
Gadi teaches denoising x-ray images. Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Gadi with the teachings of Brooks [Introduction] to alleviate the discrepancy of unrealistic synthetic training data.
Claim 6. (original) Brooks further teaches wherein the training data is previously generated by the synthetic data generator further lowering a resolution of each image of the plurality of images of the generated training data. [4.4. Training] We downsample all images by 2× using a Gaussian kernel (σ =1) to reduce the effect of noise
Claim(s) 7-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 2018/0260937 A1 to Gadi et al., hereinafter, “Gadi” in view of Material classification in X-ray images based on multi-scale CNN to Benedykciuk et al., hereinafter, “Benedykciuk” and in further view of Nested Dense Attention Network for Single Image Super-Resolution to Qiu et al., hereinafter, “Qiu”.
Claim 7. (currently amended) Gadi fails to explicitly teach applying the machine learning algorithm. Benedykciuk, in the field of inspecting cargo in x-ray images and denoising images teaches wherein applying the machine learning algorithm comprises: [Introduction] Our aim is to employ a CNN approach
applying, to the input inspection image, a feature extractor; [Introduction] Our aim is to employ a CNN approach for the entire feature extraction
[page 1288, col 1, 31-33 - line – col 2, first line] to extract image features…CNN is not very complex, it allows for a good extraction 123 of features.
Gadi teaches denoising x-ray images. Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Gadi with the teachings of Benedykciuk [Abstract] to improve the accuracy of material recognition in x-ray images.
Qiu, in the field of denoising image using neural network (machine learning) teaches and applying, to a feature map resulting from the application of the feature extractor, [page 253, 3.3 Adaptive Channel Attention Module] Given input feature f ∈ RC×H×W which has C feature maps with size of H ×W. At first, f is fed into convolutional layer and activated by PReLU [7] to extract deep feature
the deep neural network, DNN, [page 254, 3.5 Hybrid Non-Local Up-sampler] all the features…fed into one convolutional layer
the DNN comprising multiple connections paths, [Figure 5] , [page 253, 3.3 Adaptive Channel Attention Module] Then the deep feature will be split into two branches f1, f2, where f1 = f2 are the transformed features of f . Then we feed f1 and f2 into different branches (paths).
a feature merger, [page 254, 3.5 Hybrid Non-Local Up-sampler] all the features will be concatenated and fed into one convolutional layer
and subpixel layer comprising a pixel shuffler configured to perform an upscaling of the pixels. [page 251, 2.1 Deep CNNs based SISR]… proposed sub pixel convolution to upscale the low-resolution images
[page 254, 3.5 Hybrid Non-Local Up-sampler]… Figure 5…sub-pixel convolution to design Hybrid Non-Local Up-sampler(HNLU) to upscale the feature at the end of the network to generate high-resolution image.
Gadi teaches denoising x-ray images. Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Gadi with the teachings of Qui [Abstract] to better integrate features of different levels extracted from different layers so that features are not lost.
Claim 8. (original) Qiu teaches wherein the DNN is a residual and densely connected. [page 251]… very deep model EDSR by stacking residual blocks which greatly improved the evaluation index and visual quality… are based on dense connections [10] to form deep networks
Claim 9. (currently amended) Gadi teaches further comprising applying, to the upscaled image, a clipping operation. [0145] the detection of the hidden objects may be enhanced, as the denoising (upscaled) may facilitate the selection of the part 115 to be zoomed as a zoomed zone (clipping) 116 as shown in FIG. 11B.
Claim 10. (currently amended) Qui teaches wherein the machine learning algorithm comprises a loss function L, wherein the loss function L of the machine learning algorithm is such that: L=α·LMS-SSIM+(1-α)·GσGM·Ll1wherein α is a weight learned by the machine learning algorithm to best map the lower resolution input inspection image to the higher resolution output inspection image, LMS-SSIM is a loss function associated with a Multi-Scale Structure Similarity Index Metric, the LMS-SSIM loss function being the loss function of the synthetic data generator, Ll1 is a loss function associated with an l1normalization, the Ll1 loss function being the loss function of the DNN, and GσGM is a function weighting the Ll1 loss given a Gaussian kernel. Qiu [page 252, col.2, 1st paragraph] including Equation 6
Gadi teaches denoising x-ray images. Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Gadi with the teachings of Qui [Abstract] to better integrate features of different levels extracted from different layers so that features are not lost.
Claim(s) 14-17 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Material classification in X-ray images based on multi-scale CNN to Benedykciuk et al., hereinafter, “Benedykciuk” in view of US 2018/0260937 A1 to Gadi et al., hereinafter, “Gadi” and Nested Dense Attention Network for Single Image Super-Resolution to Qiu et al., hereinafter, “Qiu”.
Claim 14. (currently amended) Benedykciuk teaches A computer-implemented method of training a machine learning algorithm used in any of the preceding claims, the method comprising comprising: [Introduction] Our aim is to employ a CNN approach
[page 1292, col. 2 lines 1-2] …deep neural networks (autoencoders), whose aim is to denoise images or simultaneously interpolate images with removing noise
applying, to an input inspection image, a feature extractor, [page 1288, col 1, 31-33 - line – col 2, first line] to extract image features…CNN is not very complex, it allows for a good extraction 123 of features.
wherein the input inspection image has a higher noise, [page 1290] probably due to the high noise of the input data…
[page 1291] …the input data, ...Unfortunately, images from X-ray scanners are often very noisy (higher noise).
and a lower resolution; [page 1291] …the input data, which is necessary due to the architecture of the investigated ImageNet networks. Unfortunately, images from X-ray scanners are often very noisy (higher noise). Interpolation of such images will cause this noise to be blurred and enlarged (lower resolution).
Benedykciuk fails to explicitly teach a Poisson-Gaussian noise whose variance is non-constant in the plurality of pixels. Gadi, in the field of denoising x-ray images teaches the higher noise comprising a Poisson-Gaussian noise whose variance is non-constant in the plurality of pixels, [0055] the inspection image 10 of FIG. 3A is corrupted by the Poisson-Gaussian noise and the variance of the noise is non-constant in the plurality of pixels 13 of the image 10
Benedykciuk teaches denoising x-ray images using neural networks. Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Benedykciuk with the teachings of Gadi [0002] to alleviate the difficulty for a user to detect hidden objects, such as weapons or dangerous material, in inspection images.
Benedykciuk fails to explicitly teach applying, to a feature map resulting from the application of the feature extractor. Qiu in the field of denoising image using neural network (machine learning) teaches and applying, to a feature map resulting from the application of the feature extractor, [page 253, 3.3 Adaptive Channel Attention Module] Given input feature f ∈ RC×H×W which has C feature maps with size of H ×W. At first, f is fed into convolutional layer and activated by PReLU [7] to extract deep feature
a deep neural network, DNN, [page 254, 3.5 Hybrid Non-Local Up-sampler] all the features…fed into one convolutional layer
comprising multiple connections paths, [Figure 5] [page 253, 3.3 Adaptive Channel Attention Module] Then the deep feature will be split into two branches f1, f2, where f1 = f2 are the transformed features of f . Then we feed f1 and f2 into different branches (paths).
a feature merger, [page 254, 3.5 Hybrid Non-Local Up-sampler] all the features will be concatenated and fed into one convolutional layer
and subpixel layer comprising a pixel shuffler configured to perform an upscaling. [page 251, 2.1 Deep CNNs based SISR]… proposed sub pixel convolution to upscale the low-resolution images
[page 254, 3.5 Hybrid Non-Local Up-sampler]… Figure 5…sub-pixel convolution to design Hybrid Non-Local Up-sampler(HNLU) to upscale the feature at the end of the network to generate high-resolution image.
Benedykciuk teaches denoising images using neural networks. Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Benedykciuk with the teachings of Qui [Abstract] to better integrate features of different levels extracted from different layers so that features are not lost.
Claim 15. (original) Qui teaches wherein the DNN is a residual and densely connected. [page 251]… very deep model EDSR by stacking residual blocks which greatly improved the evaluation index and visual quality… are based on dense connections [10] to form deep networks
Claim 16. (currently amended) Gadi teaches further comprising applying, to the upscaled image, a clipping operation. [0145] the detection of the hidden objects may be enhanced, as the denoising (upscaled) may facilitate the selection of the part 115 to be zoomed as a zoomed zone (clipping) 116 as shown in FIG. 11B.
Claim 17. (currently amended) Qui teaches wherein the machine learning algorithm comprises a loss function L, wherein the loss function L of the machine learning algorithm is such that: L=α·LMS-SSIM+(1-α)·GσGM·Ll1wherein α is a weight learned by the machine learning algorithm to best map the lower resolution input inspection image to the higher resolution output inspection image, LMS-SSIM is a loss function associated with a Multi-Scale Structure Similarity Index Metric, the LMS-SSIM loss function being the loss function of a synthetic data generator adding a Poisson-Gaussian noise to the input inspection image, Ll1 is a loss function associated with an l1normalization, the Ll1 loss function being the loss function of the DNN, and GσGM is a function weighting the Ll1 loss given a Gaussian kernel. Qiu [page 252, col.2, 1st paragraph] including Equation 6
Claim 20. (currently amended) A method of producing a device configured to process inspection images, the method comprising: obtaining a machine learning algorithm trained by the method of claim 14; Benedykciuk [page 1292, line 7-col. 2 line 2] …was observed for the proposed network-based method… We want to try out deep neural networks (autoencoders), whose aim is to denoise images or simultaneously interpolate images with removing noise
and storing the obtained trained machine learning algorithm in a memory of the device. Benedykciuk [page 1289, 4 Experiments and Results] The proposed model is implemented using Tensorflow and Keras libraries. All of our experiments are conducted on the Nvidia graphics processing unit GeForce 2080 (Turing microarchitecture).
Qiu [page 254, col.2, last line] We implement our model with Pytorch framework and train it on a V100GPU.
Reviewed and analyzed in the same way as claim 14. See the above analysis and rationale.
Claim(s) 18-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Material classification in X-ray images based on multi-scale CNN to Benedykciuk et al., hereinafter, “Benedykciuk” in view of US 2018/0260937 A1 to Gadi et al., hereinafter, “Gadi” and Nested Dense Attention Network for Single Image Super-Resolution to Qiu et al., hereinafter, “Qiu” in further view of Unprocessing Images for Learned Raw Denoising to Brooks et al., hereinafter, “Brooks”.
Claim 18. (currently amended) Benedykciuk fails to explicitly teach a synthetic data generator implementing a method to generate a plurality of input inspection images. Brooks in the field of denoising images using neural networks, wherein each input inspection image is previously generated by a synthetic data generator implementing a method to generate a plurality of input inspection images, [Figure 1], [Figure 2] sRGB images unprocess noisy raw images
[2. Related Work] accurate models for denoising JPEG-compressed sRGB images, whereas we focus on denoising raw images.
[4.4. Training] To create our synthetic training data, we start with the 1 million images of the MIR Flickr extended dataset [23]
the implemented method comprising, for each inspection image: [4.4. Training] We downsample all images by 2× using a Gaussian kernel (σ =1) to reduce the effect of noise…then take random 128×128 crops of each image…
injecting a Poissonian noise to the inspection image, [Figure 2] add Shot and Read Noise, [3.1. Shot and Read Noise] Shot noise is a Poisson random variable whose mean is the true light intensity (measured in photoelectrons).
the Poissonian noise being dependent on an intensity of the inspection radiation; [3.1. Shot and Read Noise] Shot noise is a Poisson random variable whose mean is the true light intensity (measured in photoelectrons)
and adding a random Gaussian noise to the inspection image having the injected Poissonian noise. [Figure 2] add Shot and Read Noise, [3.1. Shot and Read Noise] Read noise is an ap proximately Gaussian random variable
Benedykciuk teaches denoising images using neural networks. Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Benedykciuk with the teachings of Brooks [Introduction] to alleviate the discrepancy of unrealistic synthetic training data.
Claim 19. (original) Brooks further teaches wherein the inspection image is previously generated by the synthetic data generator further lowering a resolution of each image of the plurality of images. [4.4. Training] We downsample all images by 2× using a Gaussian kernel (σ =1) to reduce the effect of noise
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DELOMIA L GILLIARD whose telephone number is (571)272-1681. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DELOMIA L GILLIARD/Primary Examiner, Art Unit 2661
Prosecution Insights