Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
DETAILED ACTION
Response to Arguments
Applicant’s amendments have led to the withdrawal of the pending 112f interpretations and 112b rejections. There is a new 112b rejection based on the new wording in claim 13.
The examiner disagrees with Applicant’s assertion that the cited art does not disclose the amended limitations for the reasons set forth in the office action below.
Claim Objections
Claims 1-6, 8 are objected to for the verb agreement within the claims. In particular, the initial verbs are in simple present tense while subsequent uses are in continuous present tense.
Claim 8 appears to have a typo with respect to “processors are further to.”
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 13-14 are rejected under 35 USC 112(b) or pre-AIA 35 U.S.C. 112, second paragraph as being indefinite because the claims recite a system that comprises method steps. The claim is therefore indefinite because it is unclear whether infringement occurs when one creates the system or performs the recited method steps.
A single claim which claims both an apparatus and the method steps of using the apparatus is indefinite under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph. See In re Katz Interactive Call Processing Patent Litigation, 639 F.3d 1303, 97 USPQ2d 1737 (Fed. Cir. 2011). In Katz, a claim directed to “[a] system with an interface means for providing automated voice messages…to certain of said individual callers, wherein said certain of said individual callers digitally enter data” was determined to be indefinite because the italicized claim limitation is not directed to the system, but rather to actions of the individual callers, which creates confusion as to when direct infringement occurs. Katz, 639 F.3d at 1318 (citing IPXL Holdings v. Amazon.com, Inc., 430 F.2d 1377, 1384, 77 USPQ2d 1140, 1145 (Fed. Cir. 2005), in which a system claim that recited “an input means” and required a user to use the input means was found to be indefinite because it was unclear “whether infringement … occurs when one creates a system that allows the user [to use the input means], or whether infringement occurs when the user actually uses the input means.”); Ex parte Lyell, 17 USPQ2d 1548 (Bd. Pat. App. & Inter. 1990) (claim directed to an automatic transmission workstand and the method of using it held ambiguous and properly rejected under 35 U.S.C. 112, second paragraph). MPEP 2173.05(p).
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-7, 11-13, and 15-16 as best understood are rejected under 35 USC 102 as being anticipated by US 2022/0366532 Vadivel, et al. (hereafter Vadivel)
1. An image processing device that uses a neural network trained based on a plurality of images to improve image quality of an input image obtained by image capture, wherein the image processing device comprises:
A memory unit for storing a tendency information indicating a tendency in the plurality of images used to train the neural network; and (Vadivel ¶39-40 data about brightness level of the training images)
one or more processors, wherein the one or more processors: (Vadivel ¶23 processor)
acquire an image processing parameter corresponding to the tendency information from the input image; (Vadivel ¶41-42 obtains an input image to the preprocessor; note an image has brightness and further discusses HDR/SDR images)
calculate an adjustment value for aligning the image processing parameter acquired from the input image with the tendency of the image processing parameter indicated by the tendency information; (Vadivel ¶43-44 scaling factor)
adjust the input image based on the adjustment value that has been calculated; and (Vadivel ¶50 adjusts the image)
input an adjusts input image to the neural network, and output an image which has been adjusted by the adjustment unit and of which an image quality has been improved by the neural network. (Vadivel ¶49-50 receives the preprocessed image and then outputs the new image)
Claims 15 and 16 are rejected similarly.
2. The image processing device according to claim 1, wherein calculating, the adjustment value comprises calculating a gain adjustment value acquired from the input image. (Vadivel ¶43-44 adjustment value)
3. The image processing device according to claim 1, wherein adjusting the input image based on the adjustment value that adjusts a brightness of the input image based on the adjustment value, (Vadivel ¶43 black level, which is one way to measure brightness in an image)
and outputting an image in which a brightness has been adjusted and of which the image quality has been improved by the neural network. (Vadivel ¶50 outputs the adjusted image)
4. The image processing device according to claim 3, wherein adjusting a brightness of the input image comprises multiplying an adjusted gain with the input image. (Vadivel ¶44 divide by scaling factor K which is equivalent to multiplying by 1/K)
5. The image processing device according to claim 1, wherein adjusting the input image based on the adjustment value comprises subtracting a black level of the input image based on the adjustment value that has been calculated, and outputting an image in which the black level has been subtracted and of which the image quality has been improved by the neural network. (Vadivel ¶43 black level subtractor; ¶50 output image)
6. The image processing device according to claim 3, wherein adjusting the input image based on the adjustment value comprises subtracting a black level of the input image based on the adjustment value that has been calculated, and subtracting a black level based on the adjustment value from a brightness-adjusted image in which the brightness has been. (Vadivel ¶43 black level subtractor; ¶50 output image)
11. An image processing system comprising a training device that trains the neural network based on multiple images; and (Vadivel ¶7 multiple images)
an image processing device according to claim 1. (See claim 1)
12. The image processing system according to claim 11, wherein the training device trains the neural network by teacher-based training. (Vadivel ¶13 uses a training set that is paired with ground truth images)
13. The image processing system according to claim 11, wherein the training device acquires the tendency information, and (Vadivel ¶27 obtains information about the images)
edits images before training based on the acquired tendency information. (Vadivel ¶28 edits images prior to training)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 7, 9-10, 14 are rejected under 35 USC 103 as being unpatentable over Vadivel in view of US 2021/0133943, Lee et al. (hereafter Lee)
Vadivel does not disclose
7. The image processing device according to claim 1, wherein the tendency information is information regarding an average brightness of multiple images used to train the neural network.
Lee ¶26 uses average values. It would have been obvious to modify the system of Vadivel to use average values as to reduce an illumination difference between consecutive images as taught by Lee.
Vadivel does not disclose
9. The image processing device according to claim 1, wherein the input image is a frame included in moving image data, and the tendency information is generated based on multiple consecutive frames included in the moving image data. (Lee ¶132 video training)
It would have been obvious to modify Vadivel and Lee to use video frames as video is merely a series of individual image frames.
10. The image processing device according to claim 9, wherein a number of frames used for generating the tendency information is determined in accordance with a frame rate of the moving image data. (Note that each individual frame and image is based on the frame rate, for example at a frame rate of 24 frames per second, there would be 24 images per second in video data)
Vadivel does not disclose
14. The image processing system according to claim 13, wherein the tendency information is information regarding variation in an average brightness of multiple images used for training the neural network, and
the training device edits the images before training if the variation in the average brightness in the tendency information is not within a prescribed range.
Lee ¶26 uses average values, ¶169 discloses editing the images if the illumination difference is too great. It would have been obvious to modify the system of Vadivel to edit if the variation in illumination is too great for the purposes of reducing the illumination difference between consecutive frames.
Claim 8 is rejected under 35 USC 103 as being unpatentable over Vadivel and Lee in view of US 2014/0253545, Sullivan et al. (hereafter Sullivan)
Vadivel and Lee do not disclose
8. The image processing device according to claim 1, wherein the one or more processors are further to quantize the input image to a number of tones based on a lookup table (LUT), wherein quantizing the input image comprises using, among multiple LUTs, an LUT in accordance with the adjustment value.
Sullivan ¶28 discloses the use of look up tables for image enhancement. It would have been obvious to modify the system of Vadivel and Lee to include the use of look up tables for the purposes of enabling brighter, higher contrast, and more colorful image displays as taught by Sullivan.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ming Shui whose telephone number is (303)297-4247. The examiner can normally be reached on 7-5 Pacific Time, M-Th.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Greg Morse can be reached on 571-272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ming Shui/
Primary Examiner, Art Unit 2663
Prosecution Insights