DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 01/29/2026 have been fully considered but they are not persuasive.
In response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, applicant argues that replacing the convolutional neural network disclosed by “534 application with the ‘429 application would be contrary to the goal and operation of the ‘534 application because the ‘429 diffusion would instead produce a new or modified model. However, applicant has amended the independent claims to include “a trained image transformation machine learning model that comprises at least one trained generative model.” Application ‘429 teaches that “generative models (e.g., generative adversarial models (GANs), diffusion models, and the like) have been trained to generate new output data (e.g., images or text) based on input prompts.” See paragraph [0002]. Therefore, applicant’s argument is moot.
In addition, the amended features are also taught by ‘429.
Accordingly, the rejection is maintained.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) is/are rejected under 35 U.S.C. 103 as being unpatentable over Tadross (Pub. No.: US 2021/0383534) in view of Showalter et al. (Pub. No.: US 2025/0200429).
Consider claims 1, 16, 17, Tadross discloses a computer system (Fig. 1, image processing device 102) comprising at least one processor (Fig. 1, processor 104) and at least one memory (Fig. 1, non-transitory memory) comprising a set of computer readable instructions which when executed by the at least one processor cause the system to (paragraph [0024], image processing device 102 includes a processor 104 configured to execute machine readable instructions):
obtain a two-dimensional ultrasound image (paragraph [0022], receive and process images acquired via ultrasound wherein the image may comprise two-dimensional (2D) or three-dimensional (3D), see paragraph [0034]);
derive from input data, classification information for each of a plurality of features in the two-dimensional ultrasound image, the input data comprising at least one of the two-dimensional ultrasound image or three-dimensional ultrasound data corresponding to the two-dimensional ultrasound image (paragraph [0022], determine a segmentation map for one or more ROIs present within said images and/or determine a standard view classification of the one or more images);
derive a rendered image (paragraph [0087], Fig. 7, operation 704) by supplying to an image transformation machine learning model (paragraph [0085], Fig. 7, a training data pair is fed to an input layer of a reduced depth CNN (i.e., convolutional neural network module 108, see Fig. 1)):
the two-dimensional ultrasound image as an input image; and
the classification information for each of a plurality of features in the input image (paragraph [0085], Fig. 7, operation 702, the training data pair comprises an image and a corresponding ground truth segmentation map of one or more ROIs, see paragraph [0084]).
Tadross does not specifically disclose a trained image transformation machine learning model that comprises at least one trained generative model and wherein the rendered image is a photo-realistic version of the two-dimensional ultrasound image.
Showalter discloses a trained image transformation machine learning model that comprises at least one trained generative model (paragraph [0019], Fig. 1, input text 105 and image 110 are provided to a generative machine learning model) and wherein the rendered image is a photo-realistic version of the two-dimensional ultrasound image (paragraph [0051], target image may comprise a real image (e.g., an actual photograph)).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to replace the machine learning model as disclosed by Tadross with the machine learning model as taught by Showalter to provide a basis for the generation process (Showalter, paragraph [0019]).
Consider claim 2, the combination of Tadross and Showalter discloses wherein the classification information comprises at least one of:
a segmentation map; and
pose information (paragraph [0022], segmentation map).
Consider claim 3, the combination of Tadross and Showalter discloses discloses wherein the two-dimensional ultrasound image is a first two-dimensional ultrasound image (paragraph [0054], a time series of 2D images), and the rendered image is a first rendered image (paragraph [0056], first convolutional layer of the trained reduced depth CNN), wherein the computer readable instructions when executed by the at least one processor cause the system to:
obtain a time series of two-dimensional ultrasound images including the first two- dimensional ultrasound image (paragraph [0054], Fig. 3, operation 302, receiving a time series of 2D images);
obtain classification information for features belonging to each of the two-dimensional ultrasound images in the time series (paragraph [0059], Fig. 3, operation 310, maps the one or more identified features, identified at operation 308, to a segmentation map); and
derive a time series of rendered images by supplying to the image transformation machine learning model, the time series of two-dimensional ultrasound images and the classification information for the features belonging to each of the two-dimensional ultrasound images in the time series, the time series of rendered images including the first rendered image (paragraphs [0056] to [0060], Fig. 3, operations 306 to 312).
Consider claim 4, Tadross does not specifically disclose wherein the image transformation machine learning model comprises a diffusion model.
Showalter discloses wherein the image transformation machine learning model comprises a diffusion model (paragraph [0016], diffusion model).
Therefore, in order to mitigate, eliminate, or at least reduce mode collapse while encouraging generative diversity, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have applied the same technique as suggested by Showalter wherein the image transformation machine learning model comprises a diffusion model, see teaching found in Showalter, paragraph [0016].
Consider claim 5, Tadross does not specifically disclose wherein the image transformation machine learning model comprises an additional machine learning model configured to process the classification information, wherein the computer readable instructions when executed by the at least one processor cause the system to:
generate by the diffusion model, the rendered image in dependence upon the result of processing the classification information by the additional machine learning model.
Showalter discloses wherein the image transformation machine learning model comprises an additional machine learning model configured to process the classification information, wherein the computer readable instructions when executed by the at least one processor cause the system to:
generate by the diffusion model, the rendered image (paragraph [0092], Fig. 6, trained machine learning models for image generation) in dependence upon the result of processing the classification information by the additional machine learning model (paragraph [0096], image classification).
Therefore, in order to accelerate the performance of common machine learning tasks, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have applied the same technique as suggested by Showalter wherein the image transformation machine learning model comprises an additional machine learning model configured to process the classification information, wherein the computer readable instructions when executed by the at least one processor cause the system to: generate by the diffusion model, the rendered image in dependence upon the result of processing the classification information by the additional machine learning model, see teaching found in Showaleter, paragraph [0096].
Consider claim 6, the combination of Tadross and Showalter discloses wherein the diffusion model comprises a denoiser network (Showalter, paragraph [0039], Fig.1, denoising backbone) comprising a plurality of encoders (Showalter, paragraph [0039], Fig. 2, sequence of encoder blocks 230A-B) and a plurality of decoders (Showalter, paragraph [0039], [0040], Fig. 2, sequence of decoder blocks 230D-E), wherein the additional machine learning model comprises a copy of the plurality of encoders with different model parameters (Showalter, paragraphs [0039], Fig. 2, each of the backbone encoder blocks 230A-B may perform operations such as convolution operations, downsampling operations, and the like), wherein the step of generating the rendered image comprises:
applying the outputs of the copy of the plurality of encoders to modify the outputs of the decoders (Showalter, paragraph [0047], Fig. 2, after the desired number of iterations are performed, the latent tensor 245 generated during the last iteration can be used to generate the output image from the model).
Consider claim 7, Tadross does not specifically disclose wherein the image transformation machine learning model comprises a generator model trained as part of a generative adversarial network.
Showalter discloses wherein the image transformation machine learning model comprises a generator model trained as part of a generative adversarial network (paragraph [0019], Fig. 1, generative machine learning model).
Therefore, in order to provide a basis for the generation process, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have applied the same technique as suggested by Showalter wherein the image transformation machine learning model comprises a generator model trained as part of a generative adversarial network, see teaching found in Showalter, paragraph [0019].
Consider claim 8, the combination of Tadross and Showalter discloses wherein the computer readable instructions, when executed by the at least one processor cause the system to: supply the classification information as conditioning information to the image transformation machine learning model (paragraph [0058], Fig. 3, operation 308, filters identify/extract patterns by computing a dot product between the filter weights of a convolutional filter and the pixel intensity values of the downsampled image over a receptive field of the filter).
Consider claims 9, 15, the combination of Tadross and Showalter discloses wherein the computer readable instructions, when executed by the at least one processor cause the system to: obtain the two-dimensional ultrasound image by performing volume rendering on the three-dimensional ultrasound data (paragraph [0040], the receptive fields of the first plurality of convolutional filters may be set to a rectangle (in the case of 2D images) or rectangular solid (in the case of 3D images).
Consider claim 10, the combination of Tadross and Showalter discloses wherein the computer readable instructions when executed by the at least one processor cause the system to:
obtain a depth map for the two-dimensional ultrasound image; and
derive the rendered image by supplying to the image transformation machine
learning model, the depth map (paragraph [0063], Fig. 3, operations 316 and 318, determining a depth based on the refined segmentation map).
Consider claim 12, the combination of Tadross and Showalter discloses wherein the computer readable instructions, when executed by the at least one processor cause the system to: perform a validation check by supplying the rendered image to a validation machine learning model configured to output a quality indication for the rendered image (paragraph [0026], accuracy/validation score of a trained CNN); and
in response to the rendered image failing the validation check, generate a third image corresponding to the three-dimensional ultrasound data by re-applying the two- dimensional ultrasound image as an input image to the image transformation machine learning model (paragraph [0092]).
Consider claims 11, 14, Tadross discloses a keyboard configured to enable a user to interact with and manipulate data within image processing system 100 (see paragraph [0030]).
Tadross does not specifically disclose wherein the computer readable instructions when executed by the at least one processor cause the system to derive the rendered image by supplying to the image transformation machine learning model, a text prompt.
Showalter discloses wherein the computer readable instructions when executed by the at least one processor cause the system to derive the rendered image by supplying to the image transformation machine learning model, a text prompt (paragraph [0019], an input text 105 and image 110 are provided to a generative machine learning model).
Therefore, in order to indicate desired modification(s) to the image, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have applied the same technique as suggested by Showalter wherein the computer readable instructions when executed by the at least one processor cause the system to derive the rendered image by supplying to the image transformation machine learning model, a text prompt, see teaching found in Showalter, paragraph [0020].
Consider claim 13, Tadross does not specifically disclose wherein the image transformation machine learning model is a diffusion model configured to apply a set of noise to the two- dimensional ultrasound image to generate the rendered image, wherein the generating the third image comprises re-applying the diffusion model to the two-dimensional ultrasound image as the input image with a different set of noise applied to the two-dimensional ultrasound image.
Showalter discloses wherein the image transformation machine learning model is a diffusion model (paragraph [0016], diffusion model) configured to apply a set of noise to the two-dimensional ultrasound image to generate the rendered image, wherein the generating the third image comprises re-applying the diffusion model to the two-dimensional ultrasound image as the input image with a different set of noise applied to the two-dimensional ultrasound image (paragraph [0025], denoising backbone is used to iteratively denoise (using Gaussian random noise) input latent tensors to generate an output image).
Therefore, in order to denoise input latent tensors to generate an output image, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have applied the same technique as suggested by Showalter wherein the image transformation machine learning model is a diffusion model configured to apply a set of noise to the two- dimensional ultrasound image to generate the rendered image, wherein the generating the third image comprises re-applying the diffusion model to the two-dimensional ultrasound image as the input image with a different set of noise applied to the two-dimensional ultrasound image, see teaching found in Showalter, paragraph [0025].
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Tadross and Showalter in view of Xu et al. (Pub. No.: US 2018/0374245).
Consider claim 18, the combination of Tadross and Showalter discloses wherein the rendered image is a first rendered image (paragraph [0022], receive and process images acquired via ultrasound wherein the image may comprise two-dimensional (2D) or three-dimensional (3D), see paragraph [0034]),
The combination of Tadross and Showalter does not specifically disclose wherein the computer-readable instructions, when executed by the at least one processor, cause the system to:
determine whether or not the first rendered image satisfies a predetermined requirement; and
when it is determined that the first rendered image does not satisfy the predetermined requirement, derive a second rendered image by supplying as the inputs to the trained image transformation machine learning model:
the two-dimensional ultrasound image;
a different set of noise that is applied to the two-dimensional ultrasound image; and
the classification information.
Xu discloses wherein the computer-readable instructions, when executed by the at least one processor, cause the system to:
determine whether or not the first rendered image satisfies a predetermined requirement; and when it is determined that the first rendered image does not satisfy the predetermined requirement, derive a second rendered image (paragraph [0079], does not satisfy the predetermined criteria, then model parameters of the deep learning model can be updated based on the error map and another batch of training data can be selected from the patient images and expected results and the deep learning model can be further trained) by supplying as the inputs to the trained image transformation machine learning model:
the two-dimensional ultrasound image, a different set of noise that is applied to the two-dimensional ultrasound image, and the classification information (paragraph [0083], including 2D images, noise and classification (see paragraph [0082])).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to replace the processor as disclosed by the combination of Tadross and Showalter with the processor as taught by Xu in order that a deep learning model can be applied to the patient images to provide estimated results, which can then be compared to the expected results (Xu, paragraph [0079]).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GERALD JOHNSON whose telephone number is (571)270-7685. The examiner can normally be reached Monday-Friday 8am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Carey Michael can be reached at (571)270-7235. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Gerald Johnson/
Primary Examiner, Art Unit 3797