DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged that application claims priority to foreign application with application number KR10-2022-0102817 dated 8/17/2022. Copies of certified papers required by 37 CFR 1.55 have been received. Priority is acknowledged under 35 USC 119(e) and 37 CFR 1.78.
Information Disclosure Statement
The IDS dated 12/24/2024 and 12/4/2025 has been considered and placed in the application file.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim(s) 4, 6, 10 are rejected under 35 U.S.C. 112(b), as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claim 4 recites “wherein the first neural network model is used for real-time image enhancement of cameras of the first class.” It is unclear what the Applicant is claiming when stating “real-time” image enhancement. By definition, real-time is something that happens without delay and immediately after a certain event, and the Applicant has not claimed any delay or lag as defined. For examination purposes, the examiner will interpret this claim as the neural network model being used for image enhancement.
Claim 6 recites “wherein the image enhancement software is software configured to generate the enhanced images from the sample images in non-real time”. Similar to claim 4, it is unclear what the Applicant is claiming when stating “non-real time”, as by definition, non-real means something can happen without a time limit. or examination purposes, the examiner will interpret this claim as the image enhancement software is something similar to photoshop and the like.
Claim 10 also recites “the first neural network model is used for real-time image enhancement of cameras of the first class, and the second neural network model is used for real-time image enhancement of cameras of the second class” and thus is indefinite for the same reasons as claim 4.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 5, 7-9 are rejected under 35 U.S.C. 103 as being unpatentable over Ji (US 20220261965 A1) in view of Omi (US 20230245274 A1).
Regarding claim 1, Ji discloses a training method comprising: acquiring sample images of various qualities (Ji, paragraph [0034], "First, in step S201, the image processing device obtains a sample image set, the sample image set including a first number of sample images"),
generating enhanced images of at least some of the sample images using image enhancement software having an image enhancement function (Ji, paragraph [0042], " A resolution of the high-resolution training image 306 obtained through the downsampling may be lower than a resolution of the training image 302. In addition to the downsampling, in an embodiment, denoising and/or deblurring (not shown) may be performed on the training image 302 in step S312 to obtain the high-resolution training image 306 with a resolution lower than the resolution of the training image 302, and/or with noise and/or blurriness lower than noise and/or blurriness of the training image 302", a high-resolution image is downsampled through software),
constructing, from the sample images and the enhanced images, training data that forms pairs of input data and target data (Ji, paragraph [0041], " In an embodiment, each training image pair may include one training target image and one training input image (for example, as shown in FIG. 3, the training target image is a high-resolution training image 306, and the training input image is a low-resolution training image 307), a resolution of the training input image being lower than a resolution of the training target image").
While Ji teaches using the training data to output an enhanced output image in response to a low-quality input image being input and output a corresponding quality output image in response to a high-quality input image being input (Ji, paragraph [0054], "Specifically, as shown in FIG. 3, the training input image in the training image pair, for example the low-resolution training image 307 (or a region of a specific size cropped from the low-resolution training image 307), may be inputted into the super-resolution model 309 to be trained, and the super-resolution model 309 may output a model-generated image 308", a variety of training images of various qualities are inputted), they do not explicitly teach “performing, using the training data, supervised learning of a first neural network model to output an enhanced output image in response to a low-quality input image being input and output a corresponding quality output image in response to a high-quality input image being input”.
However, Omi teaches performing, using the training data, supervised learning of a first neural network model to output an enhanced output image in response to a low-quality input image being input and output a corresponding quality output image in response to a high-quality input image being input (Omi, paragraph [0025], "The following embodiments illustrate building of a learning model based on supervised learning using a convolutional neural network (CNN) in which a low-resolution medical image, which is input data, and a high-resolution medical image, which serves as correct data, are used as training data").
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to use supervised learning on Ji’s training images, as taught by Omi.
The suggestion/motivation for doing so would have been to reduce unexpected results and define an unambiguous goal.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Ji in view of Omi to obtain the invention as specified in claim 1.
Regarding claim 2, Ji in view of Omi discloses the training method of claim 1, wherein the sample images comprise first sample images captured by a first camera of a first class and second sample images captured by a second camera of the first class (Ji, paragraph [0022], "For example, a real image may be the original image actually captured by a camera, a smartphone, or another device."),
and the enhanced images comprise first enhanced images corresponding to at least some of the first sample images and second enhanced images corresponding to at least some of the second sample images (Ji, paragraph [0041], " In an embodiment, each training image pair may include one training target image and one training input image (for example, as shown in FIG. 3, the training target image is a high-resolution training image 306, and the training input image is a low-resolution training image 307), a resolution of the training input image being lower than a resolution of the training target image", images captured from a camera, smartphone, or other device will be enhanced accordingly based on their respective images).
Regarding claim 5, Ji in view of Omi discloses the training method of claim 1, wherein the performing of supervised learning comprises adjusting parameters of the first neural network model to reduce a difference between the target data and output data corresponding to an output of the first neural network model according to an input of the input data (Ji, paragraph [0054], "In an embodiment, as shown in FIG. 3, the loss function of the super-resolution model 309 may be calculated in step S314 based on the model-generated image 308 and the training target image (for example, the high-resolution training image 306) in the corresponding training image pair, and the network parameter of the super-resolution model 309 may be optimized based on the loss function").
Regarding claim 7, Ji in view of Omi discloses a computer program stored in a computer-readable recording medium to execute the method of claim 1 in combination with hardware (Ji, paragraph [0082], "The modules/units as disclosed herein can be implemented in the form of hardware (e.g., processing circuitry and/or memory) or in the form of software functional unit(s) (e.g., developed using one or more computer programming languages), or a combination of hardware and software.").
Claims 8-9 correspond to claims 1-2, additionally reciting a training apparatus (Ji, paragraph [0072], “FIG. 7 is a schematic diagram of an image processing apparatus 700 according to an embodiment of this application”),
A processor and a memory comprising instructions executable by the processor (Ji, paragraph [0076], “As shown in FIG. 8, the image processing device 800 according to an embodiment of this application may include a processor 801 and a memory 802, where the processor 801 and the memory 802 may be interconnected via a bus 803”). The remaining limitations of the claims are rejected for the same reasons of obviousness as corresponding claims 1-2.
Claim(s) 3-4, 10 are rejected under 35 U.S.C. 103 as being unpatentable over Ji (US 20220261965 A1) in view of Omi (US 20230245274 A1) and in further view of Shcherbinin (US 20200387750 A1).
Regarding claim 3, Ji in view of Omi discloses the training method of claim 2, wherein the sample images comprise third sample images captured by a third camera of a second class (Ji, paragraph [0022], "For example, a real image may be the original image actually captured by a camera, a smartphone, or another device."),
the enhanced images comprise third enhanced images corresponding to at least some of the third sample images (Ji, paragraph [0041], " In an embodiment, each training image pair may include one training target image and one training input image (for example, as shown in FIG. 3, the training target image is a high-resolution training image 306, and the training input image is a low-resolution training image 307), a resolution of the training input image being lower than a resolution of the training target image", similar to claim 2, images taken from each respective devices will be enhanced accordingly).
While Ji in view of Omi teaches using training data to output an enhanced output image in response to a low-quality input image being input and output a corresponding quality output image in response to a high-quality input image being input (Ji, paragraph [0041], " In an embodiment, each training image pair may include one training target image and one training input image (for example, as shown in FIG. 3, the training target image is a high-resolution training image 306, and the training input image is a low-resolution training image 307), a resolution of the training input image being lower than a resolution of the training target image"), they do not explicitly teach “the training method further comprises performing, using training data according to the third sample images and the third enhanced images, supervised learning of a second neural network model to output an enhanced output image in response to a low-quality input image being input and output a corresponding quality output image in response to a high-quality input image being input”.
However, Shcherbinin teaches the training method further comprises performing, using training data according to the third sample images and the third enhanced images, supervised learning of a second neural network model to output an enhanced output image in response to a low-quality input image being input and output a corresponding quality output image in response to a high-quality input image being input (Shcherbinin, paragraph [0011], “The neural network model training apparatus includes a memory configured to store one or more instructions, and at least one processor configured to execute the one or more instructions stored in the memory to obtain a low quality input image patch and a high quality input image patch, obtain a low quality output image patch by inputting the low quality input image patch to a first neural network model, obtain a high quality output image patch by inputting the high quality input image patch to a second neural network model, and train the first neural network model based on a loss function set to reduce a difference between the low quality output image patch and the high quality input image patch, and a difference between the high quality output image patch and the high quality input image patch, wherein the second neural network model is identical to the first neural network model.”).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to have a second neural network model identical to Ji’s (in view of Omi) first model to process the third sample images, as taught by Shcherbinin.
The suggestion/motivation for doing so would have been to divide and conquer, thus reducing the total computational time to process all the images.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Ji in view of Omi and in further view of Shcherbinin to obtain the invention as specified in claim 3.
Regarding claim 4, Ji in view of Omi and Shcherbinin discloses the training method of claim 3, wherein the first neural network model is used for real-time image enhancement of cameras of the first class (Ji, paragraph [0053], "As described above, the image processing model may include a trained neural network model (for example, a trained super-resolution model 309) that can perform image super-resolution processing"), and the second neural network model is used for real-time image enhancement of cameras of the second class (Shcherbinin, paragraph [0103], “The first neural network model and the second neural network model are the same neural network model”, each neural network model is set to enhance their respective input images).
Claim 10 corresponds to claim 3-4, additionally reciting a training apparatus (Ji, paragraph [0072], “FIG. 7 is a schematic diagram of an image processing apparatus 700 according to an embodiment of this application”). The remaining limitations of the claims are rejected for the same reasons of obviousness as corresponding claims 3-4.
Claim(s) 6 are rejected under 35 U.S.C. 103 as being unpatentable over Ji (US 20220261965 A1) in view of Omi (US 20230245274 A1) and in further view of Monikandan (US 8767020 B1).
Regarding claim 6, Ji in view of Omi discloses the training method of claim 1.
Ji in view of Omi does not teach “wherein the image enhancement software is software configured to generate the enhanced images from the sample images in non-real time”.
However, Monikandan teaches wherein the image enhancement software is software configured to generate the enhanced images from the sample images in non-real time (Monikandan, Col. 2, Lines 3-7, "For example, a user may wish to create a media object, such as a web page, that includes a web page graphic created with one application, such as Adobe Illustrator, and an enhanced digital image created with another application, such as Adobe Photoshop").
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to use photoshop to enhance Ji’s (in view of Omi) images, as taught by Monikandan.
The suggestion/motivation for doing so would have been to manually touch up on images that the neural networks may not have correctly enhanced.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Ji in view of Omi and in further view of Monikandan to obtain the invention as specified in claim 6.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WAYNE ZHANG whose telephone number is (571) 272-0245. The examiner can normally be reached Monday-Friday 10:00-6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ms. Sumati Lefkowitz can be reached on (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WAYNE ZHANG/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights