DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-10 and 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1 and 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being incomplete for omitting essential elements, such omission amounting to a gap between the elements. See MPEP § 2172.01. The omitted elements are: a memory for storing the obtained images and providing stored edit prompt(s) (Specification Para 58 and 65).
Claims 2-10 are rejected based on dependency from a rejected base claim.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Eran Levin et al., “Differential Diffusion: Giving Each Pixel Its Strength”, February 28, 2024, https://arxiv.org/pdf/2306.00950.
Independent claim 1, Levin discloses a method performed by an electronic device, of editing an image, the method comprising:
obtaining an image (i.e. input image – p. 2, col. 1, Para 2; Fig. 1 “image + map”);
obtaining an edit prompt for the image (i.e. given prompt – p. 2, Para 2; Fig. 1);
generating an edited image by using a diffusion model that uses the image and the edit prompt as input data (i.e. user diffusion model that take image as input and use text to guide the editing process – p. 2, col. 2, sec. 2.2, Para 1); and
outputting the edited image (Fig. 1 “output”),
wherein the generating of the edited image comprises applying different image generation strengths to a plurality of regions in the image, based on a segmentation map representing the plurality of regions (i.e. changes different regions of an image in different amounts -Fig. 1l; uses a change map or segmentation map – p. 8, col. 1, Para 1).
Claim 2, Levin discloses the method of claim 1, wherein the different image generation strengths are determined based on values of defined hyperparameters, and wherein the defined hyperparameters comprise a first hyperparameter indicating a degree to which an image condition is reflected and a second hyperparameter indicating a degree to which a text condition is reflected (i.e. changes different regions of an image in different amounts -Fig. 1; Fig. 9, 11; As we are the first method to allow change maps with an arbitrary number of strengths – p. , col. 1, sec. 5.2.3, Para 1).
Claim 3, Levin discloses the method of claim 2, wherein the first hyperparameter and the second hyperparameter correspond to each region of the plurality of regions, and the first hyperparameter and the second hyperparameter have different values for each region of the plurality of regions (i.e. changes different regions of an image in different amounts -Fig. 1; Fig. 9, 11; As we are the first method to allow change maps with an arbitrary number of strengths – p. , col. 1, sec. 5.2.3, Para 1).
Claim 4, Levin discloses the method of claim 1, wherein the generating of the edited image comprises: obtaining the segmentation map by segmenting an object region within the image (i.e. uses a change map or segmentation map – p. 8, col. 1, Para 1); and identifying the plurality of regions by using the segmentation map (i.e. To simplify this process, we propose a new visualization tool called “Strength Fan”. This fan is a modified image created by dividing it into columns, with each column undergoing editing at a different strength level. This allows users to observe multiple strength settings simultaneously, thereby simplifying the task of comparing and tuning edit strengths (Figure 7). – p. 5, col. 1, sec. 4.2 Strength Fan).
Claim 5, Levin discloses the method of claim 4, wherein the segmentation map includes a plurality of segment levels, and wherein the generating of the edited image comprises applying the different image generation strengths to the plurality of segment levels (i.e. To simplify this process, we propose a new visualization tool called “Strength Fan”. This fan is a modified image created by dividing it into columns, with each column undergoing editing at a different strength level. This allows users to observe multiple strength settings simultaneously, thereby simplifying the task of comparing and tuning edit strengths (Figure 7). – p. 5, col. 1, sec. 4.2 Strength Fan).
Claim 6, Levin discloses the method of claim 1, wherein the generating of the edited image comprises: generating an initial noise (i.e. We first noise the encoded original image according to the current timestep (z′ t). – p. 4, col. 1, sec. 3.3. Algorithm, Para 1); and generating the edited image by repeating a noise prediction process and a predicted noise removal for each time step, starting from the initial noise (i.el. the down-sampled map (µs) aligns with the positions of the latent pixels in the latent tensor. The denoising loop is changed for each time step t… We first noise the encoded original image according to the current timestep (z′ t). - p. 4, col. 1, sec. 3.3. Algorithm, Para 1; Finally, the U-Net denoises the result (zmix t ) - p. 4, col. 1, sec. 3.3. Algorithm, Para 2), wherein the noise prediction process uses classifier-free guidance (CFG) that combines conditional prediction and unconditional prediction, and wherein conditions for the CFG comprise an image condition with the image as a condition and a text condition with the edit prompt as a condition (i.e. Diffusion models have revolutionized image generation and editing, producing state-of-the-art results in conditioned and unconditioned image synthesis – abstract; Given an image, a mono-channel change map representing the desired change amount of each pixel, and a text prompt, our goal is to edit the image to produce a high-quality result that satisfies the desired change and adheres to the prompt. – p. 3, col. 1, second 3. Method).
Claim 7, Levin discloses the method of claim 6, wherein the noise prediction process comprises predicting a first noise corresponding to a first region of the image (i.e. Given an image, a mono-channel change map representing the desired change amount of each pixel, and a text prompt, our goal is to edit the image to produce a high-quality result that satisfies the desired change and adheres to the prompt. – p. 3, col. 1, sec. 3.1. Preliminaries; Algorithm 1) and a second noise corresponding to a second region of the image (i.e. Let N,n be the number of timesteps of Σ,σ, the noise levels in σ’s intermediate images match those of an inference chain – p. 4, col. 1, sec. 3.2. Observations: 1. The Suffix Principle).
Claim 8, Levin discloses the method of claim 7, wherein the noise prediction process comprises, for each single time step, predicting the first noise and the second noise together within the corresponding single time step, and predicting noise corresponding to the single time step by combining the first noise with the second noise (i.e. begins with an image with added Gaussian noise, then in an iterative process, the noise is gradually removed. This inference process creates a series of images (Intermediate Images), where each is the result of the denoising operation of the previous one (The Inference Chain) – p. 3, col. 1-2, sec. 3.1. Preliminaries; Algorithm 1).
Claim 9, Levin discloses the method of claim 8, wherein the generating of the edited image comprises: using third input data as the input data for the diffusion model, and wherein the noise prediction process comprises, for each single time step, predicting the noise corresponding to the single time step by further combining third noise corresponding to the third input data (i.e. begins with an image with added Gaussian noise, then in an iterative process, the noise is gradually removed. This inference process creates a series of images (Intermediate Images), where each is the result of the denoising operation of the previous one (The Inference Chain) – p. 3, col. 1-2, sec. 3.1. Preliminaries; Algorithm 1).
Claim 10, Levin discloses the method of claim 1, wherein the edited image is generated such that the edit prompt is reflected less in an object region of the edited image than in a remaining region thereof. (i.e. we would not want to make abrupt and complete transformations such as replacing all the trees with burnt stumps. Instead, we would like to introduce different amounts of fire into different regions on the photo, in a controllable manner - Figure 1 bottom-right).
Independent claim 11, the claim is similar in scope to claim 1. Therefore, the rationale as applied in the rejection of claim 1 applies herein.
Claims 12-19, the corresponding rationale as applied in the rejection of claims 2-10 apply herein.
Independent claim 20, the claim is similar in scope to claim 1. Therefore, the rationale as applied in the rejection of claim 1 applies herein.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHANTE HARRISON whose telephone number is (571)272-7659. The examiner can normally be reached Monday - Friday 8:00 am to 5:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at 571-272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHANTE E HARRISON/Primary Examiner, Art Unit 2615
Prosecution Insights