DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-16 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Bauer et al. (CA 3162187).
Regarding claims 1 and 11, Bauer discloses an apparatus (fig. 1) for image enhancement, the apparatus comprising:
interface circuitry (112, 120, and 121 of fig. 1A, page 29, lines 1-3, the computer system 120 comprises one or more processors 112 configured to instantiate and run one or more software programs or modules 114, 118, 122, 126 involved in generating the trained model 132) configured to:
receive first image data representing a multispectral image of a scene (102 of fig. 1A, hyperspectral camera images, page 28, lines 1-7); and
receive second image data representing a first image of the scene in the visible light spectrum (104 of fig. 1, RGB camera images, page 28, lines 1-7); and
processing circuitry (122, 124, 126, and 132 of fig. 1) configured to generate, based on the multispectral image of the scene and the first image of the scene in the visible light spectrum (2nd training image 106 and 1st training image 108 of fig. 1A) and using a trained machine-learning model (132 of figs. 1A and 2, page 28, lines 20-29, the hyperspectral images 106 are used as second training images and the transferred RGB images 205 are used as first training images during the training of the ML-model; page 31, lines 3-6, the training data used for training the ML-model 132 comprises several hundred or preferably several thousand first training images and a corresponding number of second training images which are aligned to each other to form several hundred or preferably several thousand pairs of aligned training images, and figure 4 a flowchart of a method for providing a trained ML-model adapted to automatically label images acquired with a first image acquisition technique, e.g. RGB imaging) for image enhancement (page 8, lines 4-16, refining the alignment; page 38, lines 17-22, the refined alignment is for the image enhancement; 410 of fig. 4), a second image of the scene in the visible light spectrum with at least one enhanced image property compared to the first image of the scene in the visible light spectrum (206 of figs. 2 and 3, page 33, lines 4-page 34, line 11).
Regarding claim 2, Bauer discloses the apparatus of claim 1, wherein the processing circuitry is further configured to subject the first image data to demosaicing processing to obtain the multispectral image (104 and 106 of fig. 1A, page 12, lines 11-19, to obtain hyperspectral and multispectral images; page 24, lines 1-17, the hyperspectral image acquisition technique is used to obtain the multispectral image).
Regarding claim 3, Bauer discloses the apparatus of claim 2, wherein the trained machine-learning model is trained to perform the demosaicing processing (106 of fig. 1A, page 21, lines 19-21, discloses that the image analysis system can be used both for training the ML-model and for applying the trained ML-model on the one or more test images; page 24, lines 1-17, the hyperspectral image acquisition technique is used to obtain the multispectral image).
Regarding claim 4, Bauer discloses the apparatus of claim 1, wherein the processing circuitry is configured to subject the multispectral image of the scene and the first image of the scene in the visible light spectrum to image alignment processing to align the multispectral image of the scene and the first image of the scene in the visible light spectrum (122 and 124 of fig. 1A, page 30, lines 1-10), and wherein the second image of the scene in the visible light spectrum is generated based on the aligned multispectral image of the scene (206 of figs. 2 and 3, page 33, line 3-page 34, line 11).
Regarding claim 5, Bauer discloses the apparatus of claim 4, wherein the trained machine-learning model is trained to perform the alignment processing (122 of fig. 1A and 408 of fig. 4; page 30, lines 1-14, page 34, lines 12-14 and page 39, lines 9-14, a method for providing a trained ML-model to perform the alignment).
Regarding claim 6, Bauer discloses the apparatus of claim 4, wherein the processing circuitry is further configured to subject the aligned multispectral image of the scene to feature extraction processing to extract one or more features from the multispectral image of the scene (114 of fig. 1A, extract features, fig. 1B, page 31, lines 7-26), and wherein the second image of the scene in the visible light spectrum is generated by the trained machine-learning model based on the one or more extracted features (206 of figs. 2 and 3, page 33, line 3-page 29, line 11).
Regarding claim 7, Bauer discloses the apparatus of claim 6, wherein the trained machine-learning model is trained to perform the feature extraction processing (114 and 128 of fig. 1A, feature extraction, page 9, lines 5-16).
Regarding claim 8, Bauer discloses the apparatus of claim 1, wherein the multispectral image of the scene comprises a plurality of image layers depicting the scene at different wavelength ranges (fig. 6, discloses different wavelengths, page 22, lines 14-22 and page 24, lines 1-17, a wide range of wavelengths),
wherein at least one of the wavelength ranges is outside the visible light spectrum (page 14, lines 19-22, page 24, lines 1-17, page 25, lines 6-11).
Regarding claim 9, Bauer discloses the apparatus of claim 1, wherein the at least one enhanced image property is one or more of increased image resolution, refined colors, reduced noise, and increased dynamic range (page 38 lines 17-22, refining the alignment as a function of pixel intensity and/or color similarities (e.g. based on a greenness index) for providing the alignment of the first and second image of the pair).
Regarding claim 10, Bauer discloses the apparatus of claim 1, further comprising:
a multispectral imaging sensor (102 of fig. 1A) configured to capture the scene and generate the first image data, wherein the multispectral imaging sensor is sensitive to at least one of ultraviolet light and infrared light (page 24, lines 1-17); and
a visible imaging sensor (104 of fig. 1A) configured to capture the scene and generate the second image data, wherein the visible imaging sensor is sensitive to visible light (page 24, line 24-page 25, line 5).
Regarding claims 12 and 13, Bauer further discloses an apparatus (fig. 1) for training a machine-learning model for image enhancement (page 23, lines 14-27, page 31, lines 12- 15, the machine learning training software 126; and figure 4 for the machine learning model) the apparatus comprising processing circuitry configured to:
subject a first image of a scene output by the machine-learning model to image degradation processing to obtain a second image of the scene with at least one degraded image property compared to the first image (2nd training image 402 of fig. 4; page 15, lines 12-15, low-resolution hyperspectral images with high amount of spectral information are used to automatically predict labels which are then automatically aligned to high-resolution images with lower spectral information; page 17, lines 21-26, the trained machine-learning model is a neural network, in particular a neural network comprising at least one bottleneck layer. A bottleneck layer is a layer that contains few nodes compared to the previous layers. It can be used to obtain a representation of the input with reduced dimensionality. An example of this is the use of autoencoders with bottleneck layers for nonlinear dimensionality reduction. This disclosure suggests the image degradation processing);
modify the machine-learning model (410 of fig. 4, modify the model such that it learns to predict label(s) and label positions based on the first features) based on a difference (408 of 4, the alignment of the difference as the green indices are compared with each other for computing and estimating a displacement field, page 38, lines 17-22) between the first image (402 of fig. 4) and a third image output by the machine-learning model (1st training image 406 of fig. 4, a third image) based on the second image of the scene (the reduced 2nd training image 402 of fig. 4 or the low resolution training image 402 of fig. 4) and a multispectral image of the scene (402 of fig. 4, page 13, lines 17-18, and page 24, lines 1-23, using hyperspectral technology with its higher detection capabilities due to higher number of spectral bands can be used for almost any problem encountered in the field of precision farming and related fields).
Regarding claim 14, Bauer further discloses a non-transitory machine-readable medium having stored thereon a program having a program code for performing the method according to claim 11, when the program is executed on a processor or a programmable hardware (fig. 1A, page 21, lines 4-9, page 29, lines 1-3).
Regarding claim 15, Bauer further discloses a program having a program code for performing the method according to a program having a program code for performing the method according to when the program is executed on a processor or a programmable hardware (fig. 1A, page 21, lines 4-9, page 29, lines 1-3).
Regarding claim 16, Bauer further discloses a mobile phone comprising an apparatus for image enhancement according to claim 1 (page 14, lines 28-30, page 29, lines 1-3).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Mojaver et al. (US 20220132052 A1) discloses the device may include different types of light filters for its various cameras with different sensitivities to different parts of the electromagnetic spectrum to optimize its response.
Vercauteren et al. (US 20230239583 A1) discloses a method and system that allows parameters of a desired target image to be determined from hyperspectral imagery of scene. The parameters may be representative of various aspects of the scene being imaged, particularly representative of physical properties of the scene.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TUNG T VO whose telephone number is (571)272-7340. The examiner can normally be reached Monday-Friday 6:30 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Brian Pendleton can be reached at 571-272-7527. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
TUNG T. VO
Primary Examiner
Art Unit 2425
/TUNG T VO/Primary Examiner, Art Unit 2425
Prosecution Insights