Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant's claim for foreign priority based on applications filed in Korea on 09/04/2023, 12/14/2023, and 01/16/2024. It is noted, however, that applicant has not filed a certified copy of the 10-2024-006753 filed application as required by 37 CFR 1.55. Further, it is noted that an attempt by the Office to electronically retrieve, under the priority document exchange program, the foreign application 10-2024-006753, 10-2023-0117238, and 10-2023-0182370 to which priority is claimed has FAILED on 02/04/2025. Useful information is provided at the Electronic Priority Document Exchange (PDX) Program Website (https://www.uspto.gov/patents/basics/international-protection/electronic-priority-document-exchange-pdx).including practice tips for priority document exchange (https://www.uspto.gov/patents/basics/international-protection/electronic-priority-document-exchange-pdx#Practice2tips). For further questions or assistance, please contact the Patent Electronic Business Center (EBC) toll-free at 1-866-217-9197 or locally at 571-272-4100, open M-F from 6AM to Midnight EST or at PDX@uspto.gov.
Information Disclosure Statement
The information disclosure statements (IDS(s)) submitted on 10/04/2024, 03/03/2025, and 11/17/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Specification
The disclosure is objected to because of the following informalities:
In [0061], "to. a Charge" should read "to: a charge".
In [0082], "RGV values" should read "RGB values".
In [0249], “included..” should read “included.”.
Appropriate correction is required.
Claim Objections
Claims 3-5, 7, 9, and 11 are objected to because of the following informalities:
In Claim 3, "comprises" should read "further comprises".
In Claim 4, "comprises" should read "further comprises".
In Claim 5, "comprises" should read "further comprises".
In Claim 7, "comprises" should read "further comprises".
In Claim 9, "comprises" should read "further comprises".
In Claim 11, "processor ," should read "processor,".
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "obtaining an image" in line 3 after reciting “partial area of an image” in line 1. It is unclear whether this refers to the same image or a new one, therefore, there is insufficient antecedent basis for this limitation in the claim. For the sake of further prosecution, Examiner will interpret these images to be one and the same.
Claim Rejections - 35 USC § 101
Claim 10 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because Claim 20 claims a "computer-readable recording medium”.
The broadest reasonable interpretation of a claim drawn to a computer-readable recording medium typically covers forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of computer-readable recording medium. See MPEP 2111.01. When the broadest reasonable interpretation of a claim covers a signal per se, the claim must be rejected under 35 U.S.C. 101 as covering non-statutory subject matter. The USPTO recognizes that applicants may have claims directed to computer-readable recording medium that cover signals per se, which the USPTO must reject under 35 U.S.C. 101 as covering both non-statutory subject matter and statutory subject matter. A claim drawn to such a computer-readable recording medium that covers both transitory and non-transitory embodiments may be amended to narrow the claim to cover only statutory embodiments to avoid a rejection under 35 U.S.C. 101 by adding the limitation "non-transitory" to the claim. Such an amendment would typically not raise the issue of new matter, even when the specification is silent because the broadest reasonable interpretation relies on the ordinary and customary meaning that includes signals per se.
Applicant’s specification in paragraphs [0309-0311] recites “Examples of the computer-readable recording medium include a magnetic medium such as a hard disk, a floppy disk, or a magnetic tape, an optical medium such as a compact disc read-only memory (CD-ROM) or a digital versatile disc (DVD), a magneto-optical medium such as a floptical disk, and a hardware device specially configured to store and execute program commands such as a ROM, a random-access memory (RAM), or a flash memory” Since Applicant’s disclosure does not limit the definition of “computer-readable recording medium”, it could be a signal.
As an additional note, a non-transitory computer-readable recording medium having executable programming instructions stored thereon is considered statutory as non-transitory computer readable media excludes transitory data signals.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 11-14, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Kanazawa et al. (US 2025/0232411 A1), hereinafter referenced as Kanazawa in view of Liu et al. (US 2024/0153194), hereinafter referenced as Liu.
Regarding Claim 1, Kanazawa discloses a method of generating a partial area of an image by using a generative model (Kanazawa: [Fig. 1], illustrates a flowchart <method> where a partial area of an image is generated using a machine learning model <because generation is happening, the machine learning models in this method are interpreted as generative ML models>; [0002], discloses image inpainting as a generative modification), the method comprising:
obtaining an image comprising information of the partial area (Kanazawa: [0029], discloses a computer system obtaining a lower resolution version of an input image; [Fig. 1], illustrates the lower resolution input image, reference character 16 <interpreted as the obtained input image> comprising one or more image elements, at reference character 14 <interpreted as the partial area>);
PNG
media_image1.png
794
1024
media_image1.png
Greyscale
obtaining an intermediate generated image by inputting the image into a first generative model, the intermediate generated image comprising first image information corresponding to the partial area (Kanazawa: [Fig. 1], illustrates a created intermediate image, at reference character 22, by inputting the image, at reference character 16, into a first machine learning model, the intermediate generated image comprises a generated element, at reference character 24, which corresponds with the one or more image elements, at reference character 14 <interpreted as the partial area>; [0029], discloses the undesirable image element 14 is replaced using inpainting <generative technique>);
and obtaining a final generated image comprising second image information by inputting (Kanazawa: [Fig. 1], illustrates creating a final generated image by inputting a modified intermediate image, at reference character 26, into a second machine learning model, where the output, at reference character 30, is different than the first generated image information, at reference character 22; [0041], discloses the second-machine learned inpainting model generates the refined portion that modified at least a portion of the inpainted data).
Kanazawa fails to disclose
inputting the image and the intermediate generated image to a second generative model
However, Liu discloses
inputting the image and the intermediate generated image to a second generative model (Liu: [0032], discloses inputting a final first stage image <image> and an updated second stage image <intermediate generated image> to a second stage diffusion model <generative>; [Fig. 3], illustrates the final first stage image, at reference character 330, and the updated second stage image, at reference character 338, inputted into a diffusion model, at reference character 304)
PNG
media_image2.png
550
524
media_image2.png
Greyscale
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply and/or modify the method disclosed by Kanazawa by supplementally inputting an initial image in along with the modified image to a generative model as taught by Liu. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to make this modification to enhance control, quality, and consistency in AI image generation.
Regarding Claim 11, it recites similar limitations to Claim 1, but as an electronic device. As shown in the rejection, the combination of Kanazawa and Liu disclose the method of Claim 1. They further disclose
An electronic device (Kanazawa: [0062], discloses a computing device) comprising:
memory storing one or more instructions (Kanazawa: [0062], discloses the computing device comprising a memory; [0075], discloses a program <has instructions> loaded into a memory);
and at least one processor, wherein the at least one processor executes the one or more instructions stored in the memory to cause the electronic device (Kanazawa: [0062], discloses the computing device comprising a processor; [0075], discloses a program <has instructions> loaded into a memory and executed by one or more processors) to: …
Regarding Claim 20, it recites similar limitations to Claims 1 and 11, but as a computer-readable recording medium. As shown in the rejection, the combination of Kanazawa and Liu disclose the method and electronic device of Claims 1 and 11 respectively. They further disclose
A computer-readable recording medium having recorded thereon a program for performing a method (Kanazawa: [0075], discloses a tangible computer-readable storage medium comprising one or more sets of computer-executable instructions) comprising: …
Regarding Claims 2 and 12, the combination of Kanazawa and Liu disclose the method and electronic device of Claims 1 and 11 respectively. They further disclose wherein the obtaining of the image comprising the information of the partial area comprises:
obtaining a mask map that distinguishes the partial area from an entire area of the image (Kanazawa: [0034], discloses processing <must be obtained to be processed> a mask that identifies the one or more image elements <partial area, by identifying it, it distinguishes it from the entire area of the image);
and concatenating the mask map to the image (Kanazawa: [Fig. 1], illustrates a mask, at reference character 18, and an downscaled image <interpreted as the initial image>, at reference character 16, inputted into a first machine learning model and the model outputting one image; [0034], discloses the mask is used to identified image elements with the first machine learning model to generate an augmented image modifying the mask-identified elements using inpainting <interpreted as concatenation>).
PNG
media_image3.png
170
360
media_image3.png
Greyscale
Regarding Claims 3 and 13, the combination of Kanazawa and Liu disclose the method and electronic device of Claims 1 and 11 respectively. They further disclose wherein the obtaining of the final generated image comprises:
encoding the intermediate generated image (Liu: [Fig. 3], illustrates an initial image, at reference character 302, modified to create image, at reference character 310 <interpreted as the intermediate generated image> inputted into an image encoder, at reference character 316);
and obtaining the final generated image by inputting the image and the encoded intermediate generated image to the second generative model (Liu: [Fig. 3], illustrates obtaining an output image, at reference character 346, after inputting the initial image, at reference character 302, and updated initial image, at reference character 328, which has passed though the image encoder, at reference character 316, into the diffusion model, at reference character 304).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply and/or modify the method or electronic device disclosed by the combination of Kanazawa and Liu by encoding a modified image and inputting the encoded modified image and original image into a generative model as further taught by Liu. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to make this modification to improve convergence and numeric stability.
Regarding Claims 4 and 14, the combination of Kanazawa and Liu disclose the method and electronic device of Claims 1 and 11 respectively. They further disclose wherein the obtaining of the final generated image comprises:
obtaining a text input (Liu: [0029], discloses receiving input text);
encoding the text input (Liu: [0029], discloses a text encoder is used to generate a text embedding using the input text);
and obtaining the final generated image by inputting the image and the intermediate generated image and the encoded text input to the second generative model (Liu: [Fig. 3], illustrates obtaining the output image <final generated image>, at reference character 346, by inputting the initial image, at reference character 302, and the updated initial image, at reference character 326, into the diffusion model, at reference character 304. The updated initial image is created using text input, at reference character 116, that is encoded by the text encoder, at reference character 318).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply and/or modify the method or electronic device taught by the combination of Kanazawa and Liu by using encoded text input as an additional input as disclosed by Liu. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to make this modification because text input allows a user to communicate desired image refinements, enabling precise semantic control.
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Kanazawa and Liu in view of Xiao et al. (US 2023/0095092 A1), hereinafter referenced as Xiao.
Regarding Claims 5 and 15, the combination of Kanazawa and Liu disclose the method and electronic device of Claims 1 and 11 respectively. They further disclose wherein the obtaining of the final generated image comprises:
the intermediate generated image (Kanazawa: [Fig. 1], illustrates an intermediate image at reference character 22)
second generative model (Kanazawa: [Fig. 1], illustrates a second machine learning model at reference character 28; [0041], discloses the second-machine learned inpainting model generates the refined portion that modified at least a portion of the inpainted data <meaning the second machine learning model would be a generative model>)
and obtaining the final generated image by inputting (Kanazawa: [Fig. 1], illustrates an outputted <obtained> final generated image at reference character 32, after the intermediate image, at reference character 22 is processed and inputted into the second machine learning model, at reference character 28)
They do not disclose
obtaining a denoising strength for the image
adding noise to the
and obtaining the final generated image by inputting the image
However, Xiao discloses
obtaining a denoising strength for the (Xiao: [0109] discloses recording the amount of noise <reads on denoising strength>)
adding noise to the (Xiao: [0110], discloses adding additional noise to a first output <intermediate generated image> to form second generated output);
and obtaining the final generated image by inputting the (Xiao: [0110], discloses adding additional noise to a first output <intermediate generated image> to form second generated output, every time noise is added to an image into a DDGAN it becomes a new input for the model; [Fig. 3B], illustrates a final generated output, at reference character 358, by inputting the image with added noise, at reference character 356, to the generative model, at reference character 206).
PNG
media_image4.png
480
536
media_image4.png
Greyscale
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply and/or modify the method or electronic device disclosed by the combination of Kanazawa and Liu by adding noise to an image before inputting it into a generative model as disclosed by Xiao. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to make this modification because to prevent overfitting, enhance and boost regularization and generalization.
Claims 6-8 and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Kanazawa, Liu, and Xiao in view of Smirnov et al. (US 20230289923 A1), hereinafter referenced as Smirnov.
Regarding Claim 6, the combination of Kanazawa, Liu, and Xiao disclose the method of Claim 5. They further disclose wherein the obtaining of the denoising strength for the intermediate generated image comprises:
obtaining an output from the second machine learning model is based on the intermediate generated image (Kanazawa: [Fig. 1], illustrates an intermediate image at reference character 22, the output of the second machine learning model is based on this intermediate generated image)
They do not disclose
obtaining a predicted confidence value based on the
and determining the denoising strength based on at least one of the predicted confidence value, a size of the partial area, or a shape of the partial area
However, Smirnov discloses
obtaining a predicted confidence value based on the intermediate generated image (Smirnov: [0085], discloses receiving confidence value <must be calculated from a set of data> associated with a pixel of a downscaled image <interpreted as intermediate generated, therefore the predicted confidence value is based on part of the intermediate generated image>);
and determining the denoising strength based on at least one of the predicted confidence value, (Smirnov: [0085], discloses a noise reduction circuit used to obtain a denoise image, it further discloses that the noise reduction performed is based on the confidence values of the received images, where higher confidence value indicates less noise reduction).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply and/or modify the method as taught by the combination of Kanazawa, Liu, and Xiao by determining the denoising strength based on the confidence value as taught by Smirnov. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to make this modification to prevent overfitting, enhance and boost regularization and generalization.
Regarding Claim 16, the combination of Kanazawa, Liu and Xiao disclose the electronic device of Claim 15. They further disclose wherein the obtaining of the denoising strength for the intermediate generated image comprises:
obtaining an output from the second machine learning model is based on the intermediate generated image (Kanazawa: [Fig. 1], illustrates an intermediate image at reference character 22, the output of the second machine learning model is based on this intermediate generated image)
They do not disclose
obtaining a predicted confidence value based on the
and determine the denoising strength based on the predicted confidence value
However, Smirnov discloses
obtaining a predicted confidence value based on the (Smirnov: [0085], discloses receiving confidence value <must be calculated from a set of data> associated with a pixel of a downscaled image <interpreted as intermediate generated, therefore the predicted confidence value is based on part of the intermediate generated image>);
and determine the denoising strength based on the predicted confidence value (Smirnov: [0085], discloses a noise reduction circuit used to obtain a denoise image, it further discloses that the noise reduction performed is based on the confidence values of the received images, where higher confidence value indicates less noise reduction).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply and/or modify the electronic device as taught by the combination of Kanazawa, Liu, and Xiao by determining the denoising strength based on the confidence value as taught by Smirnov. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to make this modification to prevent overfitting, enhance and boost regularization and generalization.
Regarding Claims 7 and 17, the combination of Kanazawa, Liu, Xiao, and Smirnov, disclose the method and electronic device of Claims 6 and 16 respectively. They further disclose wherein the obtaining of the final generated image comprises:
obtaining current noise information; concatenating the image and the current noise information (Xiao: [0083], discloses a forward diffusion process where an intermediate image is generated with more noise; [Fig. 3B], illustrates the image x0, at reference character 352, with the current noise information concatenated at xt-1, reference character 354 <the current noise information must be obtained to be concatenated with the image x0>);
inputting the concatenated image to the second generative model (Xiao: [Fig. 3B], illustrates the intermediate image, at reference character 354, is further processed then inputted to the generator <second generative model>, at reference character 206);
and obtaining next noise information from the second generative model (Xiao: [Fig. 3B], illustrates performing posterior sampling on the output of the generator, at reference character 206, where the new intermediate generated image, at referenced character 360 has noise added back <noise information must be obtained to add it I to the image>).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply and/or modify the method and electronic device disclosed by the combination of Kanazawa, Liu, Xiao, and Smirnov by performing obtaining and adding noise as further taught by Xiao. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to make this modification to improve model robustness and prevent overfitting.
Regarding Claims 8 and 18, the combination of Kanazawa, Liu, Xiao, and Smirnov disclose the method and electronic device of Claims 7 and 17 respectively. They further disclose wherein
the current noise information corresponds to the intermediate generated image with the added noise (Xiao: [Fig. 3B], illustrates intermediate image, at reference character 354, with noise <current noise information> added to it <therefore they correspond through a direct mathematical relationship that modifies pixels based on statistical distributions>).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply and/or modify the method or electronic device as disclosed by the combination of Kanazawa, Liu, Xiao, and Smirnov by having the current noise information correspond to the image with said added noise as further taught by Xiao. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to make this modification to improve model robustness and prevent overfitting.
Claims 9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Kanazawa, Liu, Xiao, and Smirnov in view of Ho et al. (“Denoising Diffusion Probabilistic Models”, 2020), hereinafter referenced as Ho.
Regarding Claims 9 and 19, the combination of Kanazawa, Liu, Xiao, and Smirnov disclose the method and electronic device of Claims 8 and 18 respectively. They further disclose
an output from the second machine learning model corresponds to the intermediate generated image (Kanazawa: [Fig. 1], illustrates an intermediate image at reference character 22, the output of the second machine learning model corresponds to this intermediate generated image)
They do not disclose
determining a target denoising order corresponding to the
and setting a denoising order of the current noise information as the determined target denoising order
However, Ho discloses
determining a target denoising order corresponding to the (Ho: [Background and Fig. 2], illustrates a forward and reversed diffusion process using a Markov chain where noise is iteratively added or reduced, see Fig. 2. When referencing the denoising order, the forward diffusion order has already taken place, so from xt to xt-1 would be the target denoising order corresponding to the image with added noise, based on the predefined order xT, xt, xt-1, x0, based on denoising strength where xT has the most noise and x0 has the least);
PNG
media_image5.png
160
714
media_image5.png
Greyscale
and setting a denoising order of the current noise information as the determined target denoising order (Ho: [Fig. 2], illustrates denoising from xT to x0 <interpreted as the target denoising order>, by actually denoising the image it sets the denoising order of the current noise information at any point in the process).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply and/or modify the method or electronic device taught by the combination of Kanazawa, Liu, Ho, Smirnov, and Xiao by having a set denoising order as further taught by Ho. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to make this modification to maintain consistency across the forward and reverse diffusion process.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Kanazawa, Liu, Xiao, Smirnov, and Ho in view of Wu et al. (US 2025/0022457 A1), hereinafter referenced as Wu.
Regarding Claim 10, the combination of Kanazawa, Liu, Xiao, Smirnov, and Ho disclose the method of Claim 9. Wu discloses wherein
the first generative model is a generative adversarial network (GAN) model, and the second generative model is a diffusion model (Wu: [Claim 5], recites where first model is an unsupervised diffusion generative adversarial network <GAN> model and the second model is a diffusion model).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply and or modify the method disclosed by Kanazawa, Liu, Xiao, Smirnov, and Ho by using a GAN followed by a diffusion model as disclosed by Wu. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to make this modification to utilize the speed of a GAN then refine the image using the diffusion model to output realistic inpaintings and maintain user satisfaction.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Bolsée et al. (EP 34870782 A1) discloses noisy confidence.
Li et al. (US 20240256831 A1) discloses a two-model system using a GAN for the first model then a diffusion model for the second model.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ISABELLA OCHSNER whose telephone number is (571)272-9322. The examiner can normally be reached 7:30 - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona Faulk can be reached at (571) 272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/I.O./Examiner, Art Unit 2618
/DEVONA E FAULK/Supervisory Patent Examiner, Art Unit 2618
Prosecution Insights