Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The United States Patent & Trademark Office appreciates the application that is submitted by the inventor/assignee. The United States Patent & Trademark Office reviewed the following application and has made the following comments below.
Priority
This application claims benefit of foreign priority under 35 U.S.C. 119(a)-(d) of CN-202311794444, filed in China on 12/22/2023
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(B) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 8, 18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention. The Examiner strongly suggested that appropriate corrections be made to clarify the claim scope.
Claim 8 recites the limitation "the initial coding representation" on line 8. There is insufficient antecedent basis for this limitation in the claim.
Claim 18 recites the limitation "the initial coding representation" on line 8. There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained through the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-20 are rejected under 35 U.S.C 103 as being anticipated by Voynov et al (U.S. Patent Pub. No 2026/0268553 A1, hereinafter referred to as Voynov) in view of Yoon (U.S. Patent Pub. No 2023/0394319 A1, hereinafter referred to as Yoon)
In regards to claim 1, Voynov teaches a method for image processing, comprising: obtaining a first feature representation of an input image to be processed (paragraph 57, Voynov teaches obtaining a first conditioning input for an input image, examiner interprets this as a first feature representation); generating, based on the first feature representation (paragraph 57, Voynov teaches a first conditioning input, examiner interprets this as a first feature representation) and prompt information corresponding to a target visual task (paragraph 90, Voynov teaches an intermediate representation of the output image, examiner interprets this as prompt information corresponding to a target visual task), a second feature representation of the input image by using a diffusion model (paragraph 90, Voynov teaches using a diffusion model to generate an updated intermediate representation, examiner interprets this as generating a second feature representation), (Paragraph 27, paragraph 87, Voynov teaches generating an output image based on the updated intermediate representation).
Voynov does not explicitly disclose the prompt information being obtained by training of the diffusion model.
Yoon is in the same field of the art of image processing using a diffusion model.
Further, Yoon teaches the prompt information obtained by training of the diffusion model (paragraph 37, paragraph 71, Yoon teaches obtaining prompt data from training a diffusion model)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Voynov by modifying the prompt information corresponding to a target visual task in Voynov to be obtained by training of the diffusion model taught by Yoon, to make the invention that uses prompt information (Voynov) that is obtained by training of the diffusion model (Yoon); Thus, one of ordinary skill in the art would be motivated to combine the references since achieving conditional generation typically demands substantial quantity of labeled data (paragraph 3, Yoon).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to claim 2, Voynov in view of Yoon discloses wherein, generating the result of the input image with respect to the target visual task (paragraph 21, Voynov teaches generating a output image based on a input image and a target visual task) comprises: converting the second feature representation by using the prompt information (paragraph 86, Voynov teaches updating the intermediate representation using the prompt information from each step), to obtain a converted second feature representation (paragraph 84, Voynov teaches obtaining a new updated feature representation); and determining, based on the converted second feature representation (paragraph 28, Voynov teaches determining a output, based on the updated feature representation from the last step), the result by using a decoder for the target visual task (paragraph 28, Voynov teaches using a decoder to determine the output image).
In regards to claim 3, Voynov in view of Yoon discloses wherein, wherein the second feature representation (paragraph 90, Voynov teaches an updated intermediate representation, examiner interprets this as a second feature representation) comprises features of a plurality of scale (paragraph 76, Voynov teaches the feature representation has multiple features), and obtaining the converted second feature representation (paragraph 84, Voynov teaches obtaining a new updated feature representation) comprises: respectively converting the features of the plurality of scales by using the prompt information (paragraph 96, Voynov teaches converting the features using predicted noise, examiner interprets this as using prompt information), to obtain converted features of the plurality of scales (paragraph 96, Voynov teaches features that have been converted with noise); and combining the converted features of the plurality of scales into the converted second feature representation (paragraph 96, Voynov teaches combining the prediction noise to the second feature representation).
In regards to claim 4, Voynov in view of Yoon discloses wherein, wherein the decoder is trained (paragraph 28, Voynov teaches a decoder that is trained by the image generation system) together with the diffusion model for the target visual task (paragraph 24, Voynov teaches the Diffusion model is trained by the image generation system for visual task, the examiner interprets the decoder and Diffusion model both being trained by the image generation system as being trained).
In regards to claim 5, Voynov in view of Yoon discloses wherein, wherein generating the second feature representation of the input image comprises a plurality of steps, and a given step in the plurality of steps (Figure 6, Voynov teaches using multiple steps to generate the second feature representation) comprises: determining an input feature representation of the given step based on the first feature representation (paragraph 90, Voynov teaches determining a feature representation based on a first feature representation) or an output of a previous step of the given step (paragraph 91, Voynov teaches using the output of a previous step as the input); and generating, by using the diffusion model ,an output feature representation (paragraph 91, Voynov teaches using a diffusion model to generate a feature representation), of the given step from the input feature representation by taking the prompt information as a condition (paragraph 91, Voynov teaches using the input feature representation and prompt information to generate an output feature representation).
In regards to claim 6, Voynov in view of Yoon discloses wherein, generating the output feature representation (Paragraph 27, paragraph 87, Voynov teaches generating an output image) of the given step comprises: generating, by using the diffusion model, the output feature representation (paragraph 91, Voynov teaches using a diffusion model to generate a feature representation) from the input feature representation by taking the prompt information as a condition (paragraph 91, Voynov teaches using the input feature representation and prompt information to generate an output feature representation) and taking model modulation information for the given step as a time step coding (paragraph 90, Voynov teaches modulating information based on the time step), wherein the model modulation information is obtained by training of the diffusion model (paragraph 39, Yoon teaches generating the model modulation based on training the model).
In regards to claim 7, Voynov in view of Yoon discloses wherein, the prompt information is obtained by the following: generating an initial prompt representation for the prompt information (paragraph 34, paragraph 70, Yoon teaches generating initial prompt representation from prompt data; determining, based on the initial prompt representation, a training loss (paragraph 77, Voynov teaches determining a training loss value based on the prompt representation) for the target visual task (paragraph 116, Yoon teaches a target visual task) by using the diffusion model (paragraph 71, Yoon teaches using a Diffusion model as the generative model); updating the diffusion model and the initial prompt representation based on the training loss (paragraph 40-42, paragraph 101-102, Yoon teaches updating the generative model and the prompt representation based on the training loss value), until a predetermined condition is met (paragraph 40-42, paragraph 101-102, Yoon teaches updating the model and the prompt until the a predetermined termination requirement is reached); and determining the updated initial prompt representation as the prompt information (paragraph 72, Yoon teaches updating synthetic data as the updated prompt representation).
In regards to claim 8, Voynov in view of Yoon discloses wherein, generating the second feature representation of the input image comprises a plurality of steps (Figure 6, Voynov teaches using multiple steps to generate the second feature representation), and in a given step in the plurality of steps, model modulation information is taken as a time step coding (paragraph 90, Voynov teaches modulating information based on the time step), of the diffusion model (paragraph 90, Voynov teaches generating a time step from a layer in the diffusion model) , and the model modulation information is obtained by: generating an initial modulation representation for the model modulation information (paragraph 34, Yoon teaches a generating a latent vector, the examiner interprets this as the initial modulation representation); determining, by using the diffusion model (paragraph 71, Yoon teaches the generative model is a diffusion model) , the training loss based on the initial prompt representation and the initial modulation representation (paragraph 77, Yoon teaches determining the training loss value based on the prompt information and the visual task); updating the diffusion model, the initial coding representation, and the initial modulation representation based on the training loss (paragraph 40-42, paragraph 101-102, Yoon teaches updating the generative model, the prompt representation based on the training loss value, and the coding representation, the examiner interprets the coding representation as the parameters), until the predetermined condition is met (paragraph 40-42, paragraph 101-102, Yoon teaches updating the model and the prompt until the a predetermined termination requirement is reached); and determining the updated initial modulation representation as the model modulation information (paragraph 72, Yoon teaches updating synthetic data as the updated prompt representation).
In regards to claim 9, Voynov in view of Yoon discloses wherein, generating the second feature representation of the input image comprises: applying cross-attention to the first feature representation (paragraph 90, paragraph 93, Voynov teaches applying cross-attention to the first feature representation) and the prompt information to obtain an attention map (paragraph 90, paragraph 93, Voynov teaches obtaining an attention map based on the first feature representation and prompt information); and determining the second feature representation based on the attention map (paragraph 90, Voynov teaches determining the second feature representation based on the vectors from the cross-attention mechanism, examiner interprets this as an attention map).
In regards to claim 10, Voynov in view of Yoon discloses wherein, the prompt information comprises a plurality of prompt representations having the same dimension (paragraph 41, Voynov teaches prompt information has predetermined dimensions), and the number of the plurality of prompt representations is associated with the target visual task (paragraph 51-54, Voynov teaches the prompt representations is associated with a target visual task).
In regards to claim 11, Voynov discloses an electronic device, comprising: at least one processing unit (paragraph 104, Voynov teaches a processing unit); at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, wherein the instructions, when executed by the at least one processing unit, cause the electronic device to perform at least (paragraph 97-98, paragraph 104, Voynov teaches a memory unit, stored instructions to execute the processing unit): obtaining a first feature representation of an input image to be processed (paragraph 57, Voynov teaches obtaining a first conditioning input for an input image, examiner interprets this as a first feature representation); generating, based on the first feature representation (paragraph 57, Voynov teaches a first conditioning input, examiner interprets this as a first feature representation) and prompt information corresponding to a target visual task (paragraph 90, Voynov teaches an intermediate representation of the output image, examiner interprets this as prompt information corresponding to a target visual task), a second feature representation of the input image by using a diffusion model (paragraph 90, Voynov teaches using a diffusion model to generate an updated intermediate representation, examiner interprets this as generating a second feature representation(Paragraph 27, paragraph 87, Voynov teaches generating an output image based on the updated intermediate representation).
Voynov does not explicitly disclose the prompt information being obtained by training of the diffusion model.
Yoon is in the same field of the art of image processing using a diffusion model. Further,
Yoon teaches the prompt information obtained by training of the diffusion model (paragraph 37, paragraph 71, Yoon teaches obtaining prompt data from training a diffusion model).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Voynov by modifying the prompt information corresponding to a target visual task in Voynov to be obtained by training of the diffusion model taught by Yoon, to make the invention that uses prompt information (Voynov) that is obtained by training of the diffusion model (Yoon); Thus, one of ordinary skill in the art would be motivated to combine the references since achieving conditional generation typically demands substantial quantity of labeled data (paragraph 3, Yoon).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to claim 12, Voynov in view of Yoon discloses wherein, generating the result of the input image with respect to the target visual task comprises (paragraph 21, Voynov teaches generating a output image based on a input image and a target visual task): converting the second feature representation by using the prompt information (paragraph 86, Voynov teaches updating the intermediate representation using the prompt information from each step), to obtain a converted second feature representation (paragraph 84, Voynov teaches obtaining a new updated feature representation); and determining, based on the converted second feature representation (paragraph 28, Voynov teaches determining a output, based on the updated feature representation from the last step), the result by using a decoder for the target visual task (paragraph 28, Voynov teaches using a decoder to determine the output image).
In regards to claim 13, Voynov in view of Yoon discloses wherein, the second feature representation (paragraph 90, Voynov teaches an updated intermediate representation, examiner interprets this as a second feature representation) comprises features of a plurality of scale (paragraph 76, Voynov teaches the feature representation has multiple features), and obtaining the converted second feature representation (paragraph 84, Voynov teaches obtaining a new updated feature representation) comprises: respectively converting the features of the plurality of scales by using the prompt information (paragraph 96, Voynov teaches converting the features using predicted noise, examiner interprets this as using prompt information), to obtain converted features of the plurality of scales (paragraph 96, Voynov teaches features that have been converted with noise); and combining the converted features of the plurality of scales into the converted second feature representation (paragraph 96, Voynov teaches combining the prediction noise to the second feature representation).
In regards to claim 14, Voynov in view of Yoon discloses wherein, the decoder is trained (paragraph 28, Voynov teaches a decoder that is trained by the image generation system) together with the diffusion model for the target visual task (paragraph 24, Voynov teaches the Diffusion model is trained by the image generation system for visual task, the examiner interprets the decoder and Diffusion model both being trained by the image generation system as being trained).
In regards to claim 15, Voynov in view of Yoon discloses wherein, generating the second feature representation of the input image comprises a plurality of steps, and a given step in the plurality of steps (Figure 6, Voynov teaches using multiple steps to generate the second feature representation) comprises: determining an input feature representation of the given step based on the first feature representation (paragraph 90, Voynov teaches determining a feature representation based on a first feature representation) or an output of a previous step of the given step (paragraph 91, Voynov teaches using the output of a previous step as the input); and generating, by using the diffusion model, an output feature representation (paragraph 91, Voynov teaches using a diffusion model to generate a feature representation) of the given step from the input feature representation by taking the prompt information as a condition (paragraph 91, Voynov teaches using the input feature representation and prompt information to generate an output feature representation).
In regards to claim 16, Voynov in view of Yoon discloses wherein, generating the output feature representation (paragraph 27, paragraph 87, Voynov teaches generating an output image) of the given step comprises: generating, by using the diffusion model, the output feature representation (paragraph 91, Voynov teaches using a diffusion model to generate a feature representation) from the input feature representation by taking the prompt information as a condition (paragraph 91, Voynov teaches using the input feature representation and prompt information to generate an output feature representation) and taking model modulation information for the given step as a time step coding (paragraph 90, Voynov teaches modulating information based on the time step), wherein the model modulation information is obtained by training of the diffusion model (paragraph 39, Yoon teaches generating the model modulation based on training the model).
In regards to claim 17, Voynov in view of Yoon discloses wherein, the prompt information is obtained by the following: generating an initial prompt representation for the prompt information (paragraph 34, paragraph 70, Yoon teaches generating initial prompt representation from prompt data); determining, based on the initial prompt representation, a training loss (paragraph 77, Voynov teaches determining a training loss value based on the prompt representation) for the target visual task (paragraph 116, Yoon teaches a target visual task) by using the diffusion model (paragraph 71, Yoon teaches using a Diffusion model as the generative model); updating the diffusion model and the initial prompt representation based on the training loss (paragraph 40-42, paragraph 101-102, Yoon teaches updating the generative model and the prompt representation based on the training loss value), until a predetermined condition is met; and determining the updated initial prompt representation as the prompt information (paragraph 72, Yoon teaches updating synthetic data as the updated prompt representation).
In regards to claim 18, Voynov in view of Yoon discloses wherein, generating the second feature representation of the input image comprises a plurality of steps (figure 6, Voynov teaches using multiple steps to generate the second feature representation), and in a given step in the plurality of steps, model modulation information is taken as a time step coding (paragraph 90, Voynov teaches modulating information based on the time step) of the diffusion model (paragraph 90, Voynov teaches generating a time step from a layer in the diffusion model), and the model modulation information is obtained by: generating an initial modulation representation for the model modulation information (paragraph 34, Yoon teaches a generating a latent vector, the examiner interprets this as the initial modulation representation); determining, by using the diffusion model (paragraph 71, Yoon teaches the generative model is a diffusion model), the training loss based on the initial prompt representation and the initial modulation representation (paragraph 77, Yoon teaches determining the training loss value based on the prompt information and the visual task); updating the diffusion model, the initial coding representation, and the initial modulation representation based on the training loss (paragraph 40-42, paragraph 101-102, Yoon teaches updating the generative model, the prompt representation based on the training loss value, and the coding representation, the examiner interprets the coding representation as the parameters), until the predetermined condition is met (paragraph 40-42, paragraph 101-102, Yoon teaches updating the model and the prompt until the a predetermined termination requirement is reached); and determining the updated initial modulation representation as the model modulation information (paragraph 72, Yoon teaches updating synthetic data as the updated prompt representation).
In regards to claim 19, Voynov in view of Yoon discloses wherein, generating the second feature representation of the input image comprises: applying cross-attention to the first feature representation (paragraph 90, paragraph 93, Voynov teaches applying cross-attention to the first feature representation) and the prompt information to obtain an attention map (paragraph 90, paragraph 93, Voynov teaches obtaining an attention map based on the first feature representation and prompt information); and determining the second feature representation based on the attention map (paragraph 90, Voynov teaches determining the second feature representation based on the vectors from the cross-attention mechanism, examiner interprets this as an attention map).
In regards to claim 20, Voynov discloses a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement at least: obtaining a first feature representation of an input image to be processed (paragraph 57, Voynov teaches obtaining a first conditioning input for an input image, examiner interprets this as a first feature representation); generating, based on the first feature representation (paragraph 57, Voynov teaches a first conditioning input, examiner interprets this as a first feature representation) and prompt information corresponding to a target visual task (paragraph 90, Voynov teaches an intermediate representation of the output image, examiner interprets this as prompt information corresponding to a target visual task), a second feature representation of the input image by using a diffusion model (paragraph 90, Voynov teaches using a diffusion model to generate an updated intermediate representation, examiner interprets this as generating a second feature representation(paragraph 27, paragraph 87, Voynov teaches generating an output image based on the updated intermediate representation).
Voynov does not explicitly disclose the prompt information being obtained by training of the diffusion model.
Yoon is in the same field of the art of image processing using a diffusion model.
Further, Yoon teaches the prompt information obtained by training of the diffusion model (paragraph 37, paragraph 71, Yoon teaches obtaining prompt data from training a diffusion model).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Voynov by modifying the prompt information corresponding to a target visual task in Voynov to be obtained by training of the diffusion model taught by Yoon, to make the invention that uses prompt information (Voynov) that is obtained by training of the diffusion model (Yoon); Thus, one of ordinary skill in the art would be motivated to combine the references since achieving conditional generation typically demands substantial quantity of labeled data (paragraph 3, Yoon).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Pertinent Art
The prior art made of record and not relied upon is considered pertinent to the applicant’s disclosure:
Atsmon U.S Patent Publication No. 20260057565
Abati et al. U.S Patent Publication N0. 20240157207
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL ANTHONY DAVIS whose telephone number is (571)272-0170. The examiner can normally be reached M-F 7:30am – 4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached at (313)446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL ANTHONY DAVIS III/Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights