Detailed Actions
Claims 1-20 are currently presented for examination.
This application claims priority to U.S. Provisional Patent Application No. 63/465,613 filed May 11, 2023, and U.S. Provisional Patent Application No. 63/471,389 filed June 6, 2023.
Claims 7 and 14 are rejected under 35 U.S.C. 112(b) due to improper trademark usage under MPEP 2173.05(u).
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to judicial exception, an abstract idea, and it has not been integrated into practical application.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Brack et al. “The Stable Artist: Steering Semantics in Diffusion Latent Space”.
This action is made Non-Final.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 08/17/2023 and 08/20/2025 have been considered by the examiner.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed. The invention shares the same name with U.S. Nonprovisional Patent Application No. 18/451,214 “System And Method For Vehicle Design”.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 7 and 14 contains the trademark/trade name “Stable Diffusion” under the language wherein the diffusion model is a stable diffusion model. This trademark was filed by Stability AI Ltd on October 20, 2022 and registered on July 28, 2026. Where a trademark or trade name is used in a claim as a limitation to identify or describe a particular material or product, the claim does not comply with the requirements of 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph. See Ex parte Simpson, 218 USPQ 1020 (Bd. App. 1982). The claim scope is uncertain since the trademark or trade name cannot be used properly to identify any particular material or product. A trademark or trade name is used to identify a source of goods, and not the goods themselves. Thus, a trademark or trade name does not identify or describe the goods associated with the trademark or trade name. In the present case, the trademark/trade name is used to identify/describe a limitation to the diffusion model used in independent claims 1 and 8. Accordingly, the identification/description is indefinite. See MPEP 2173.05(u).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e. abstract idea) without anything significantly more.
Step 1: Claims 1-7 are directed to a system, which is a process, which is a statutory category of invention. Claims 8-14 are directed to a method, which is a process, which is a statutory category of invention. Claims 15-20 are directed to a non-transitory computer-readable medium, which is a manufacture, which is a statutory category of invention. Thus, claims 1-20 are directed to patent eligible categories of invention.
Step 2A, Prong 1: Claim 1 recites generate an image based on a reference image and a text description of the reference image using a diffusion model; and. This limitation does not distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that falls under the “Mental Process” grouping of abstract ideas, more specifically making an evaluation about how to create the image based on the references provided.
Claim 1 also recites based on whether the image meets a predetermined constraint: generate an intermediate image based on the image and the predetermined constraint using an analysis model, the intermediate image being incrementally closer to meeting the predetermined constraint than the image; This limitation does not distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, the limitation is a process step that falls under the “Mental Process” grouping of abstract ideas, more specifically making a judgement about how well the intermediate image follows the predetermined constraint and then remaking the image to be more closely aligned with the predetermined constraint.
Claim 1 also recites regenerate the image based on the intermediate image using the diffusion model; and This limitation does not distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, the limitation is a process step that falls under the “Mental Process” grouping of abstract ideas, more specifically making an observation about how close the intermediate image came to fulfilling the predetermined constraint.
Claim 1 also recites output the image This limitation does not distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, the limitation is a process step that falls under the “Mental Process” grouping of abstract ideas, more specifically making an observation about the created image.
In regard to claims 8 and 15, since both claims feature the same exact limitations as claim 1, the reasoning used for claim 1’s limitations apply to claims 8 and 15’s limitations unless previously specified as belonging to a specific claim.
Dependent claims 2-7, 9-14, and 16-20 further narrow the abstract ideas, identified in the independent claims.
Step 2A, Prong 2: Regarding claim 1, the limitation of generate an image based on a reference image and a text description of the reference image using a diffusion model; and amounts to mere instructions to apply an exception, where the claim invokes computers or other machinery merely as a tool to perform an existing process. See MPEP 2106.05(f)(2).
Also, the limitation based on whether the image meets a predetermined constraint: generate an intermediate image based on the image and the predetermined constraint using an analysis model, the intermediate image being incrementally closer to meeting the predetermined constraint than the image; amounts to mere instructions to apply an exception, where the claim invokes computers or other machinery as a tool to perform the existing process. See MPEP 2106.05(f)(2).
Also, the limitation regenerate the image based on the intermediate image using the diffusion model; and amounts to mere instructions to apply an exception, where the claim invokes computers or other machinery as a tool to perform the existing process. See MPEP 2106.05(f)(2).
Also, the limitation output the image amounts to mere instructions to apply an exception, where it is an insignificant extra-solution activity, more specifically selecting a particular data source or type of data to be manipulated. See MPEP 2106.05(g).
In regard to claims 8 and 15, both independent claims feature the same limitations as claim 1, so the reasoning used for claim 1’s limitations apply to claims 8 and 15’s limitations unless previously specified as belonging to a specific independent claim.
Further, claims 1 and 15 include the additional element of a processor. The processor is recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a computer and/or a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B: Regarding claim 1, the limitation of generate an image based on a reference image and a text description of the reference image using a diffusion model; and amounts to mere instructions to apply an exception, where the claim invokes computers or other machinery merely as a tool to perform an existing process. See MPEP 2106.05(f)(2).
Also, the limitation of based on whether the image meets a predetermined constraint: generate an intermediate image based on the image and the predetermined constraint using an analysis model, the intermediate image being incrementally closer to meeting the predetermined constraint than the image; amounts to mere instructions to apply an exception, where the claim invokes computers or other machinery as a tool to perform the existing process. See MPEP 2106.05(f)(2).
Also, the limitation of regenerate the image based on the intermediate image using the diffusion model; and amounts to mere instructions to apply an exception, where the claim invokes computers or other machinery as a tool to perform the existing process. See MPEP 2106.05(f)(2).
Also, the limitation of output the image is shown to reflect the court decisions of Versata Dev. Group, Inc. v. SAP Am., Inc. iv. Storing and retrieving information in memory, shown in MPEP 2106.05(d)(II).
Further, claim 8 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Therefore, the claims as a whole do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements, when considered alone or in combination, do not amount to significantly more than the judicial exception.
Further, claims 1 and 15 do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of the processor amounts to no more than mere instructions to apply to the exception using a generic computer component that does not impose any meaningful limits on practicing the abstract idea and therefore cannot provide an inventive concept. See MPEP 2106.05(b). Therefore, the claims as a whole do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements, when considered alone or in combination, do not amount to significantly more than the judicial exception.
The dependent claims include the same abstract ideas as recited in the independent claims and merely incorporate additional details that narrow the abstract ideas and fail to add significantly more to the claims.
Dependent claims 2, 9, and 16 recite “wherein the image is a representation of a vehicle”. This limitation is directed to further defining the object of the image generated, which further narrows the abstract idea identified in the independent claims, which is directed to “Mental Process”.
Dependent claims 3, 10, and 17 recite wherein the intermediate image is in one of: a two-dimensional format; a three-dimensional format; or an audio format. This limitation amounts to mere instructions to apply an exception, where the claim invokes computers or other machinery as a tool to perform the existing process. See MPEP 2106.05(f)(2).
Dependent claims 4, 11, and 18 recite wherein the predetermined constraint is based on one of: a drag coefficient; a manufacturability criterion; a vehicle dimension; a vehicle structural strength; a vehicle weight distribution; or a differentiable function of the image. This limitation does not distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with pencil paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Dependent claim 5 recites wherein the machine-readable instructions further include instructions that when executed by the processor cause the processor to: based on whether the image meets a second predetermined constraint: generate a second intermediate image based on the image and the second predetermined constraint using a second analysis model, the second intermediate image being incrementally closer to meeting the second predetermined constraint than the image;. This limitation does not distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, the limitation is a process step that falls under the “Mental Process” grouping of abstract ideas.
Additionally, dependent claim 5 recites regenerate the image further based on the second intermediate image using the diffusion model; and. This limitation amounts to mere instructions to apply an exception, where the claim invokes computers or other machinery as a tool to perform the existing process. See MPEP 2106.05(f)(2).
Additionally, dependent claim 5 recites output the image. This limitation amounts to mere instructions to apply an exception, where it recites the idea of a solution. The limitation does not indicate how the output is outputted. See MPEP 2106.05(f)(1).
Also, the claim limitations included the additional element of a processor. The processor is recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a computer and/or a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Dependent claim 6 recites wherein the machine-readable instructions further include instructions that when executed by the processor cause the processor to: determine whether the image meets the predetermined constraint using the analysis model. This limitation does not distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Also, the claim limitations included the additional element of a processor. The processor is recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a computer and/or a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Dependent claim 7 recites wherein the diffusion model is a stable diffusion model. This limitation amounts to mere instructions to apply an exception, where it invokes computers or other machinery merely as a tool to perform an existing process. Merely specifying the diffusion model is not enough. See MPEP 2106.05(f)(2).
Dependent claim 12 recites based on whether the image meets a second predetermined constraint: generating a second intermediate image based on the image and the second predetermined constraint using a second analysis model, the second intermediate image being incrementally closer to meeting the second predetermined constraint than the image; This limitation does not distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, the limitation is a process step that falls under the “Mental Process” grouping of abstract ideas.
Additionally, dependent claim 12 recites regenerating the image further based on the second intermediate image using the diffusion model; and. This limitation amounts to mere instructions to apply an exception, where the claim invokes computers or other machinery as a tool to perform the existing process. See MPEP 2106.05(f)(2).
Additionally, dependent claim 12 recites outputting the image. This limitation amounts to mere instructions to apply an exception, where it recites the idea of a solution. The limitation does not indicate how the output is outputted. See MPEP 2106.05(f)(1).
Dependent claim 13 recites determining whether the image meets a predetermined constraint using the analysis model. This limitation does not distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Dependent claim 19 recites wherein the instructions further include instructions that when executed by the processor cause the processor to: based on whether the image meets a second predetermined constraint: generate a second intermediate image based on the image and the second predetermined constraint using a second analysis model, the second intermediate image being incrementally closer to meeting the second predetermined constraint than the image; This limitation does not distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, the limitation is a process step that falls under the “Mental Process” grouping of abstract ideas.
Additionally, dependent claim 19 recites regenerate the image further based on the second intermediate image using the diffusion model; and. This limitation amounts to mere instructions to apply an exception, where the claim invokes computers or other machinery as a tool to perform the existing process. See MPEP 2106.05(f)(2).
Additionally, dependent claim 19 recites output the image. This limitation amounts to mere instructions to apply an exception, where it recites the idea of a solution. The limitation does not indicate how the output is outputted. See MPEP 2106.05(f)(1).
Also, the claim limitations included the additional element of a processor. The processor is recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a computer and/or a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Dependent claim 20 recites wherein the instructions further include instructions that when executed by the processor cause the processor to: determine whether the image meets the predetermined constraint using the analysis model. This limitation does not distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Also, the claim limitations included the additional element of a processor. The processor is recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a computer and/or a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Accordingly, claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e. an abstract idea) without anything significantly more.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Brack et al. “The Stable Artist: Steering Semantics in Diffusion Latent Space”.
(Examiner’s Note: The paper showcased has three submission histories showcased on arxiv.org: 12/12/2022 for v1, 12/30/2022 for v2, and 05/31/2023 for v3. The examiner could not find any difference between versions 2 and 3, however to account for the filing date of the provisional cases, version 2 will be the one used going forward. Even if the argument made is that the latest version was published after the earliest provisional case was filed, the information inside the latest version remains the same as the information found within version 2, so no new discoveries or changes were made. Based on examiner’s research, the change in versions was done to facilitate a minor font change.)
Regarding claim 1, Brack anticipates a processor; and a memory storing machine-readable instructions that, when executed by the processor, cause the processor to: (Section 1 Algorithm 1 Semantic Guidance (SEGA),
PNG
media_image1.png
539
340
media_image1.png
Greyscale
Examiner’s Note: While not outright stating the use of a processor, a POSITA would be able to understand that under the broadest reasonable interpretation this pseudocode would be performed using a processor and would be considered memory storing machine-readable instructions that would be executed by a processor.)
generate an image based on a reference image and a text description of the reference image using a diffusion model; and (Abstract and Section 2, “Let us now devise the Stable Artist, a semantic image editing technique for latent diffusion models. Roughly speaking, it generalizes the generative diffusion process by combining text conditioning through classifier-free guidance with editing concepts targeting dedicated parts of the image. In doing so, it substantially extends the combination of latents introduced in composable diffusion and is a general-purpose advancement of Safe Latent Diffusion (SLD); while SLD suppresses one inappropriate direction during generation, the Stable Artist features a variable number of directions that can be suppressed or enforced.”)
Based on whether the image meets a predetermined constraint: generate an intermediate image based on the image and the predetermined constraint using an analysis model, the intermediate image being incrementally closer to meeting the predetermined constraint than the image; (Section 2.3 and Figures 2-3, Section 4 and Figure 4, “Putting everything together, we start with an image generated from a prompt. Then, we iteratively perform adjustments with SEGA, enforcing or suppressing concepts, thus steering the overall semantics in diffusion latent space. Fig. 2 provides a 2-dimensional, visual explanation of se-mantic guidance. Intuitively, we can understand the latent space as a composition of arbitrary sub-spaces representing semantic concepts. The unconditioned noise estimate (black dot) starts us off at some random point in the latent space without semantic grounding. The guidance corresponding to the prompt “a portrait of a king” represents a vector (blue vector) moving us into a portion of the latent space where the concepts ‘male’ (human) and royal overlap, resulting in an image of a king. We can now further manipulate the generation process using SEGA. From the unconditioned starting point, we get the directions of ‘male’ and ‘female’ (orange/green lines) using estimates conditioned on the respective prompts. If we subtract this inferred ‘male’ direction from our prompt guidance and add the ‘female’ one, we now reach a point in latent space at the intersection of the ‘royal’ and ‘female’ sub-spaces, i.e. a queen. This vector represents the final direction (red vector) resulting from semantic guidance.”, “SEGA targets the relevant portions of the image while not changing other aspects. Furthermore, the Stable Artist offers continuous control over the strength of the applied concept semantically reflected in the image. It is worth noting that we achieve this without masking the image, be it user-provided masks or those implicitly inferred from attention maps. Instead, we produce results of competitive quality by targeting the relevant latent dimensions alone. This level of control can be seen throughout the images shown in the present work. Considering the top right example in Fig. 1, we make two local changes to the image composition and a global style change affecting the entire image. All three operations are performed simultaneously in the same latent space, and nonetheless, the majority of the image remains unchanged down to small details like the structure of clouds, the bush in the foreground, the castle, or the hill it stands on. We can observe a similar level of control in Fig. 3, where the fore- and background remain largely unaffected by changes to the car and vice versa. We investigated this further in Fig. 4, together with possible linear continuity of image alterations. Starting from the original image in the top left corner, we remove one concept from the image following the x- or y-axis. In both cases, the Stable Artist continually reduces the number of people or trees until the respective concept is removed entirely.”
Examiner’s Note: The specification describes the predetermined constraint as “can be a constraint that can be derived from the image” ([0020]). As such, the examiner interprets the predetermined constraint as a listed parameter in the desired image output. As such, the text prompts given would qualify as predetermined constraints. In addition, the specification lists an analysis model as “any suitable machine learning models” ([0017]). As Stable Artist is capable of adjusting the input image to suit the prompt requirements, a POSITA would recognize Stable Artist as an analysis model in this context.)
Regenerate the image based on the intermediate image using the diffusion model; and (Section 4 and figure 4, “We investigated this further in Fig. 4, together with possible linear continuity of image alterations. Starting from the original image in the top left corner, we remove one concept from the image following the x- or y-axis. In both cases, the Stable Artist continually reduces the number of people or trees until the respective concept is removed entirely. Again, the rest of the image remains fairly consistent, especially the concept that is not targeted. Going even further, a similar level of control is also possible with an arbitrary mixture of applied concepts. Let us consider the second row of Fig. 4, for example. The number of trees is kept at a medium level in that the row of trees on the left side of each image is always removed, but the larger tree on the right remains. While keeping the number of trees stable, we can still gradually remove the crowd at the same time.”, “Figure 4. The Stable Artist offers strong control over the latent space and can gradually perform edits at the desired strength. All images generated from the same initial noise latent using the prompt ‘a crowded boulevard’. Editing prompts denoted in blue and are gradually increased in strength from left to right and top to bottom. (Best viewed in color)”
Examiner’s Note: Figure 4 shows the Stable Artist working to fulfill the textual constraints given. Within the figure includes in progress pictures that show the Stable Artist filtering out the constraints of the original image. A POSITA would be able to recognize this as image regeneration given the context of the case.)
Output the image. (Section 8, “We presented the Stable Artist for directly interacting with concepts in DM’s latent space. To this end, we introduced semantic guidance (SEGA) that allows one to in fluence/steer the diffusion process along several directions. We demonstrated that the Stable Artist using SEGA offers fine-grained control over the generated image for performing sophisticated image composition and editing.”
Examiner’s Note: Generating an image would be considered output of an image.)
In regards to claim 8, it is the method embodiment of claim 1 with the same limitations to claim 1 except the limitation about the processor and the memory storing machine-readable instructions as such language is not present in claim 8, and is rejected using the same reasoning found in claim 1.
In regards to claim 15, it is the computer-readable medium embodiment of claim 1 with the same limitations to claim 1 except the limitation about the processor and the memory storing machine-readable instructions as such language is not present in claim 15, and is rejected using the same reasoning found in claim 1.
Regarding claim 2, Brack anticipates the limitations of claim 1. Brack also anticipates wherein the image is a representation of a vehicle (Figure 3, “Image editing, performed using semantic guiding of the Stable Artist. All images generated from the same initial noise latent using the prompt ‘a picture of a car’. Editing prompts denoted in blue. Arrows indicate the editing direction. The Stable Artist can act on explicit edits for local and global changes of the
PNG
media_image2.png
385
628
media_image2.png
Greyscale
image, as well as abstract editing concepts. (Best viewed in color)”)
In regards to claim 9, it is the method embodiment of claim 2 with the same limitations to claim 2, and is rejected using the same reasoning found in claim 2.
In regards to claim 16, it is the computer-readable medium embodiment of claim 2 with the same limitations to claim 2, and is rejected using the same reasoning found in claim 2.
Regarding claim 3, Brack anticipates the limitations of claim 1. Brack also anticipates wherein the intermediate image is in one of: a two-dimensional format; a three-dimensional format; or an audio format. (Section 1, “We present the Stable Artist, an iterative approach for
guiding a generated image toward the desired output. The Stable Artist can change aspects of the initial image using Semantic Guidance (SEGA) that provides fine-grained control of the image generation process by leveraging sophisticated operations in the model‘s latent space. This enables subtle edits to images, changes in composition and style, as well as optimization of the overall artistic conception. Furthermore, SEGA allows for probing the latent space of diffusion models to gain insights into how abstract concepts are represented by the model and how their interpretation reflects on the generated image. SEGA also facilitates advanced arithmetics between concepts that were previously only observed for natural language embeddings [1, 6, 8].”
Examiner’s Note: While not explicitly stated, diffusion models that utilize latent space always keep the image in a 2D format, only shrinking it for faster computations. The Examiner interprets this as meaning the intermediate image is always in a two-dimensional format, and thus meets the limitations of claim 3.)
In regards to claim 10, it is the method embodiment of claim 3 with the same limitations to claim 3, and is rejected using the same reasoning found in claim 3.
In regards to claim 17, it is the computer-readable medium embodiment of claim 3 with the same limitations to claim 3, and is rejected using the same reasoning found in claim 3.
Regarding claim 4, Brack anticipates the limitations of claim 1. Brack also anticipates wherein the predetermined constraint is based on one of: a drag coefficient; a manufacturability criterion; a vehicle dimension; a vehicle structural strength; a vehicle weight distribution; or a differentiable function of the image. (Figure 3, “Image editing, performed using semantic guiding of the Stable Artist. All images generated from the same initial noise latent using the prompt ‘a picture of a car’. Editing prompts denoted in blue. Arrows indicate the editing direction. The Stable Artist can act on explicit edits for local and global changes of the image, as well as abstract editing concepts. (Best viewed in color)”
Examiner’s Note: The specification mentions that the predetermined constraint can be “a constraint that can be derived from the image 140”. As such, any editing prompts listed in Figure 3 could be considered a predetermined constraint as per examiner interpretation.)
In regards to claim 11, it is the method embodiment of claim 4 with the same limitations to claim 3, and is rejected using the same reasoning found in claim 4.
In regards to claim 18, it is the computer-readable medium embodiment of claim 4 with the same limitations to claim 4, and is rejected using the same reasoning found in claim 4.
Regarding claim 5, Brack anticipates the limitations of claim 1. Brack also anticipates based on whether the image meets a second predetermined constraint: generate a second intermediate image based on the image and the second predetermined constraint using a second analysis model, the second intermediate image being incrementally closer to meeting the second predetermined constraint than the image; (Section 2.3 and Figure 4, “Now, the main idea is to influence the diffusion process along several directions. To achieve this, SEGA substantially extends the principles introduced in classifier-free guidance. The idea is to use multiple editing prompts ei targeting arbitrary concepts of the generated image, in addition to the text prompt p.”
Examiner’s Note: As Stable Artist is able to handle multiple predetermined constraints and is able to generate multiple intermediate pictures as seen in Figure 4, it covers the claim limitation.)
Regenerate the image further based on the second intermediate image using the diffusion model; and (Section 4 and figure 4, “We investigated this further in Fig. 4, together with possible linear continuity of image alterations. Starting from the original image in the top left corner, we remove one concept from the image following the x- or y-axis. In both cases, the Stable Artist continually reduces the number of people or trees until the respective concept is removed entirely. Again, the rest of the image remains fairly consistent, especially the concept that is not targeted. Going even further, a similar level of control is also possible with an arbitrary mixture of applied concepts. Let us consider the second row of Fig. 4, for example. The number of trees is kept at a medium level in that the row of trees on the left side of each image is always removed, but the larger tree on the right remains. While keeping the number of trees stable, we can still gradually remove the crowd at the same time.”, “Figure 4. The Stable Artist offers strong control over the latent space and can gradually perform edits at the desired strength. All images generated from the same initial noise latent using the prompt ‘a crowded boulevard’. Editing prompts denoted in blue and are gradually increased in strength from left to right and top to bottom. (Best viewed in color)”)
Output the image. (Section 8, “We presented the Stable Artist for directly interacting with concepts in DM’s latent space. To this end, we introduced semantic guidance (SEGA) that allows one to in fluence/steer the diffusion process along several directions. We demonstrated that the Stable Artist using SEGA offers fine-grained control over the generated image for performing sophisticated image composition and editing.”
Examiner’s Note: Generating an image would be considered output of an image.)
In regards to claim 12, it is the method embodiment of claim 5 with the same limitations to claim 5, and is rejected using the same reasoning found in claim 5.
In regards to claim 19, it is the computer-readable medium embodiment of claim 5 with the same limitations to claim 5, and is rejected using the same reasoning found in claim 5.
Regarding claim 6, Brack anticipates the limitations of claim 1. Brack also anticipates determine whether the image meets the predetermined constraint using the analysis model. (Section 2.3 and Figures 2-3, “Putting everything together, we start with an image generated from a prompt. Then, we iteratively perform adjustments with SEGA, enforcing or suppressing concepts, thus steering the overall semantics in diffusion latent space. Fig. 2 provides a 2-dimensional, visual explanation of se-mantic guidance. Intuitively, we can understand the latent space as a composition of arbitrary sub-spaces representing semantic concepts. The unconditioned noise estimate (black dot) starts us off at some random point in the latent space without semantic grounding. The guidance corresponding to the prompt “a portrait of a king” represents a vector (blue vector) moving us into a portion of the latent space where the concepts ‘male’ (human) and royal overlap, resulting in an image of a king. We can now further manipulate the generation process using SEGA. From the unconditioned starting point, we get the directions of ‘male’ and ‘female’ (orange/green lines) using estimates conditioned on the respective prompts. If we subtract this inferred ‘male’ direction from our prompt guidance and add the ‘female’ one, we now reach a point in latent space at the intersection of the ‘royal’ and ‘female’ sub-spaces, i.e. a queen. This vector represents the final direction (red vector) resulting from semantic guidance.”)
In regards to claim 13, it is the method embodiment of claim 6 with the same limitations to claim 6, and is rejected using the same reasoning found in claim 6.
In regards to claim 20, it is the computer-readable medium embodiment of claim 6 with the same limitations to claim 6, and is rejected using the same reasoning found in claim 6.
Regarding claim 7, Brack anticipates the limitations of claim 1. Brack also anticipates wherein the diffusion model is a stable diffusion model. (Section 2 Footnote 1, “In this context, ‘Stable’ is not only a nod to Stable Diffusion that builds the base of our implementation. Additionally, the Stable Artist behaves stable with respect to its control over the generated image.”)
In regards to claim 14, it is the method embodiment of claim 7 with the same limitations to claim 7, and is rejected using the same reasoning found in claim 7.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ma et al. “Pose Guided Person Image Generation”: Also teaches the regeneration of the intermediate image.
Kim et al. “Automatic Authorization Based On Synthetic Biometric Data and Non-Biometric Data” (US-9430628-B2): Also teaches the use of audio formats as valid intermediate image formats.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEXANDER JOSEPH MEKOVSKY whose telephone number is (571)272-0428. The examiner can normally be reached 7:30am-5pm M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Pitaro can be reached at (571) 272-4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALEXANDER JOSEPH MEKOVSKY/Examiner, Art Unit 2188
/RYAN F PITARO/Supervisory Patent Examiner, Art Unit 2188