Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1 – 6 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 1, the phrase "as much as" renders the claim(s) indefinite because it is unclear whether 0 correction value is intended to be included, thereby rendering the scope of the claim(s) unascertainable. See MPEP § 2173.05(d).
Claims 2 – 6 depends on claim 1, thus, they are rejected accordingly.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1 – 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nachlieli et al. (US Patent Application Publication 2008/0298704, IDS), hereinafter referred as Nachlieli, in view of Pathak et al. (US Patent Application Publication 2022/0202491), hereinafter referred as Pathak, and in further view of Umeda et al. (US Patent Application Publication 2014/0037208), hereinafter referred as Umeda.
Regarding claim 1, Nachlieli discloses a method of improving quality of a first image (Fig. 1), the method comprising:
receiving the first image (Fig. 1, [0028], receive an input image 24);
generating first class inference information by inferring classes to which respective pixels of the first image belong and calculating a first confidence for the first class inference information ([0004, 0006], consider belief map as class inference information and first confidence);
determining a first correction effect to be applied to each pixel of the first image and a first correction value in which correction values are determined according to classes and confidences ([0006, 0034, 0060 - 0062], where "the enhancement level that varies pixel-by-pixel in accordance with the respective face probability values" corresponds to the recited correction effects and values. See also [0062] where the enhancement type depends on the region type); and
generating an enhanced image by applying the first correction effect to each pixel as much as the respective first correction value ([0034, 0061 - 0062]).
However, Nachlieli fails to explicitly disclose the method wherein inferring classes by using a trained neural network model.
However, in a similar field of endeavor Pathak discloses an image classification method (abstract, [0032]). In addition, Pathak discloses the method inferring classes by using a trained neural network model ([0032]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Nachlieli, and inferring classes by using a trained neural network model. The motivation for doing this is that the process can be more powerful by using an AI technology.
However, Nachlieli fails to explicitly disclose the method wherein determining correction value based on a table.
However, in a similar field of endeavor Umeda discloses a method for image pixel correction (abstract). In addition, Umeda discloses the system determining correction value based on a table ([0081], correct based on 3D LUT).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Nachlieli, and determining correction value based on a table. The motivation for doing this is that looking up a value requires minimal processing memory and time.
Regarding claim 2 (depends on claim 1), Nachlieli discloses the method wherein inferring classes correspond to correction effects to be applied to each pixel of the image ([0034, 0061 - 0062]).
However, Nachlieli fails to explicitly disclose the method further comprising: training a neural network model by using a training image and correct answer classes labeled to respective pixels of the training image as training data to obtain the trained neural network model, wherein the correct answer classes correspond to correction effects to be applied to each pixel of the training image.
However, in a similar field of endeavor Pathak discloses an image classification method (abstract, [0032]). In addition, Pathak discloses the method further comprising:training a neural network model by using a training image and correct answer classes labeled to respective pixels of the training image as training data to obtain the trained neural network model, wherein the correct answer classes correspond to training image with corrected labels ([0032]).
Nachlieli combine Pathak teaches training a neural network model by using a training image and correct answer classes labeled to respective pixels of the training image as training data to obtain the trained neural network model, wherein the correct answer classes correspond to correction effects to be applied to each pixel of the training image.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Nachlieli, and training a neural network model by using a training image and correct answer classes labeled to respective pixels of the training image as training data to obtain the trained neural network model, wherein the correct answer classes correspond to correction effects to be applied to each pixel of the training image. The motivation for doing this is that the AI model is fine tuned so that it will be more accurate.
Regarding claim 3 (depends on claim 1), Nachlieli discloses the method wherein the calculating of the first confidence comprises:
generating a low-resolution image by reducing a resolution of the first image ([0029], process a sub sampled version of the original full-sized image);
calculating second class inference information for the low-resolution image and a second confidence for the second class inference information ([0006, 0034, 0060 - 0062]); and
obtaining the first class inference information and the first confidence based on the second class inference information and the second confidence ([0029], the attribute extraction modules 12 process a sub sampled version of the original full-sized image, and the image enhancement module 22 processes is the original full image 24).
Regarding claim 4 (depends on claim 1), Nachlieli discloses the method wherein the determining of the first correction effect and the first correction value comprises generating a first correction map comprising the first correction value and having a same size as the first image and generating a second correction map having a same size as the first image and includes a second correction value for each pixel of the second correction map indicating an intensity of a second correction effect ([0034 – 0035, 0084 - 0088], tone correction for face, skin and other areas).
Regarding claim 5 (depends on claim 4), Nachlieli discloses the method wherein the generating the enhanced image comprises applying the first correction effect to the first image based on the first correction map through a first correcting circuit and applying the second correction effect to the first image based on the second correction map through a second correcting circuit (Fig. 1, [0028 – 0035, 0061 - 0062], applying face correction through face map module, applying skin correction through skin map module).
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nachlieli in view of Pathak, in further view of Umeda and Lin et al. (US Patent Application Publication 2019/0147570), hereinafter referred as Lin.
Regarding claim 6 (depends on claim 1), Nachlieli fails to explicitly disclose the method further comprising encoding the enhanced image using an encoder.
However, in a similar field of endeavor Lin discloses an image enhancement method (Fig. 4). In addition, Lin discloses the method further comprising further comprising encoding the enhanced image using an encoder ([0032]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Nachlieli, and further comprising encoding the enhanced image using an encoder. The motivation for doing this is that the image can be suitable for preview and display.
Claim(s) 7 – 12, 14 – 15 and 17 – 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nachlieli in view of Kamon (US Patent Application Publication 2021/0158100, IDS).
Regarding claim 7, Nachlieli discloses a system for generating an enhanced image by correcting a first image (Fig. 1), the system comprising:
a first circuit configured to generate first class inference information for each pixel of the first image and a first confidence for the respective first class inference information ([0004, 0006], consider belief map as class inference information and first confidence); and
a second circuit configured to a determine correction value for each respective pixel of the first image based on the respective first class inference information and the respective first confidence for each pixel and generate the enhanced image by applying correction effects corresponding to the respective correction values to the respective pixels of the first image ([0006, 0034, 0060 - 0062], where "the enhancement level that varies pixel-by-pixel in accordance with the respective face probability values" corresponds to the recited correction effects and values. See also [0062] where the enhancement type depends on the region type).
However, Nachlieli fails to explicitly disclose the system is system-on-chip (SoC)
wherein inferring classes by using a trained neural network model.
However, in a similar field of endeavor Kamon discloses an image system (Fig. 2). In addition, Kamon discloses the system is system-on-chip (SoC) ([0075]); wherein inferring classes by using a trained neural network model ([0023 – 0024, 0081 - 0083]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Nachlieli, and apply system-on-chip (SoC) wherein inferring classes by using a trained neural network model. The motivation for doing this is that the process can be more powerful by using an AI technology.
Regarding claim 8 (depends on claim 7), Nachlieli discloses wherein the system is configured to process relationships between a plurality of classes classified to have different correction effects and pixels ([0006, 0034, 0060 - 0062], where "the enhancement level that varies pixel-by-pixel in accordance with the respective face probability values" corresponds to the recited correction effects and values. See also [0062] where the enhancement type depends on the region type (face or skin)).
However, Nachlieli fails to explicitly disclose the system is system-on-chip (SoC)
wherein the process is using a neural network model to learn.
However, in a similar field of endeavor Kamon discloses an image system (Fig. 2). In addition, Kamon discloses the system is system-on-chip (SoC) ([0075]); wherein process is using a trained neural network model ([0023 – 0024, 0081 - 0083]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Nachlieli, and apply system-on-chip (SoC) wherein inferring classes by using a trained neural network model. The motivation for doing this is that the process can be more powerful by using an AI technology.
Regarding claim 9 (depends on claim 8), Nachlieli discloses the SoC wherein the classes have different weights for at least one of a denoise effect, a color correction effect, and a sharpening effect ([0065 – 0078]).
Regarding claim 10 (depends on claim 9), Nachlieli discloses the SoC wherein the classes comprise at least one of a face class, a skin class, a sky class, a detail class, an eye class, an eyebrow class, and a hair class ([0065 – 0078]).
Regarding claim 11 (depends on claim 10), Nachlieli discloses the SoC wherein the detail class comprises at least one of a grass class, a sand class, and a branch class ([0035, 0066]).
Regarding claim 12 (depends on claim 7), Nachlieli discloses the SoC wherein the first circuit generates a 2-dimensional segmentation map comprising the first class inference information and having a same size as the first image and a 2-dimensional confidence map comprising the first confidence and having a same size as the first image ([0004, 0006], consider belief map as class inference information and first confidence).
Regarding claim 14 (depends on claim 7), Nachlieli discloses the SoC wherein the first circuit is included in any one of a central processing unit (CPU), a neural processing unit (NPU), a digital signal processor (DSP), and a graphics processing unit (GPU) (Fig. 13, CPU 142).
Regarding claim 15 (depends on claim 7), Nachlieli discloses the SoC wherein the second circuit is included in any one of a central processing unit (CPU), a digital signal processor (DSP), and a graphics processing unit (GPU) (Fig. 13, CPU 142).
Regarding claim 17, Nachlieli discloses a system for correcting a first image, the system comprising:
a segmentation circuit configured to receive the first image and generate a segmentation map comprising class inference information corresponding to each pixel of the first image and a confidence map comprising confidence for the class inference information for each respective pixel of the first image ([0004, 0006], consider belief map as segmentation map and a confidence map); and
an image processing circuit configured to generate a correction map by determining correction effects to be applied to each pixel of the first image based on the segmentation map and the confidence map and apply the correction effects to the first image based on the correction map ([0006, 0034, 0060 - 0062], where "the enhancement level that varies pixel-by-pixel in accordance with the respective face probability values" corresponds to the correction map. See also [0062] where the enhancement type depends on the region type).
However, Nachlieli fails to explicitly disclose the system is system-on-chip (SoC)
wherein inferring classes by using a trained neural network model.
However, in a similar field of endeavor Kamon discloses an image system (Fig. 2). In addition, Kamon discloses the system is system-on-chip (SoC) ([0075]); wherein inferring classes by using a trained neural network model ([0023 – 0024, 0081 - 0083]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Nachlieli, and apply system-on-chip (SoC) wherein inferring classes by using a trained neural network model. The motivation for doing this is that the process can be more powerful by using an AI technology.
Regarding claim 18 (depends on claim 17), Nachlieli discloses the SoC wherein, to apply a first correction effect to a first region comprising a first pixel classified as a first class in the first image, the image processing circuit uses at least one of a denoise circuit configured to reduce noise in the first region, a color correction circuit configured to adjust a color value of the first region, and a sharpen circuit configured to increase sharpness of the first region ([0065 – 0078]).
Regarding claim 19 (depends on claim 18), Nachlieli discloses the SoC wherein the image processing circuit adjusts intensity of the first correction effect applied to the first pixel based on the confidence map ([0034 – 0035, 0084 - 0088], tone correction for face, skin and other areas).
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nachlieli in view of Kamon, in further view of Sridhara et al. (US Patent 9,837,115), hereinafter referred as Sridhara.
Regarding claim 13 (depends on claim 7), Nachlieli fails to explicitly disclose the SoC wherein the first class inference information comprises n bits, and the first confidence includes m bits (a value of m being greater than a value of n).
However, in a similar field of endeavor Sridhara discloses a detection and decoding process (Fig. 2). In addition, Sridhara discloses the process wherein a first information comprises n bits, and a first confidence includes m bits (col. 8, line 57 to col. 9, line 1, Bit values and the corresponding likelihood or confidence information may be referred to as “soft” values or information, as compared to hard values where a sequence of bits is provided without likelihood estimates.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Nachlieli, and wherein the first class inference information comprises n bits, and the first confidence includes m bits. The motivation for doing this is that using decoding for detecting can simplify the process.
Sridhara discloses the claimed invention except for a value of m being greater than a value of n.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Sridhara, since it has been held that where the general conditions of a claim are disclosed in the prior art, discovering .the optimum or working ranges involves only routine skill in the art. In re·
Aller, 105 USPQ 233.
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nachlieli in view of Kamon, in further view of Lin.
Regarding claim 16 (depends on claim 7), Nachlieli fails to explicitly disclose the SoC further comprising an encoder configured to encode the enhanced image.
However, in a similar field of endeavor Lin discloses an image enhancement method (Fig. 4). In addition, Lin discloses the method further comprising an encoder configured to encode the enhanced image ([0032]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Nachlieli, and further comprising an encoder configured to encode the enhanced image. The motivation for doing this is that the image can be suitable for preview and display.
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nachlieli in view of Kamon, in further view of Ceccaldi et al. (US Patent Application Publication 2019/0287292, IDS), hereinafter referred as Ceccaldi.
Regarding claim 20 (depends on claim 17), Nachlieli fails to explicitly disclose the SoC further comprising: a first processor and a second processor different from the first processor, wherein the segmentation circuit is included in the first processor and the image processing circuit is included the second processor.
However, in a similar field of endeavor Ceccaldi discloses an image processing system (Fig. 8). In addition, Ceccaldi discloses the system further comprising a first processor (Fig. 8, processor 22) and a second processor different from the first processor (Fig. 8, server 28), wherein the segmentation circuit is included in the first processor ([0072]) and the image processing circuit is included the second processor ([0072], the other image processing circuit is included the second processor).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Nachlieli, and further comprising: a first processor and a second processor different from the first processor, wherein the segmentation circuit is included in the first processor and the image processing circuit is included the second processor. The motivation for doing this is that a distributed computing technique can be achieved to enhance the computing power.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to QIAN YANG whose telephone number is (571)270-7239. The examiner can normally be reached on Monday-Thursday 8am-6pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached on 571-270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/QIAN YANG/
Primary Examiner, Art Unit 2677
Prosecution Insights