Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This communication is a Final office action on merit. Claim 30 was canceled. Claims 1-29, 31, after amendment, are presently pending and have been considered below.
Request for Continued Examination
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 10/16/2025 has been entered.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 5/6/2021, 1/20/2023, 9/18/2023, 11/13/2023, and 10/17/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-29, 31 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1 recites: “randomly select one or more image transformations from among a plurality of image transformations according to a respective probability associated with each of a plurality of image transformations to apply to the initial set of images to generate a training set including transformed images simulating domain shifts across different imaging equipment or across different imaging protocols”.
Examiner fails to find sufficient support for the claimed limitations in original specification. Original specification appears to teach randomly select aspect value/parameter with a range of such values/parameters to perform image transformation and such aspect value or parameter being generated/selected according to a probability distribution (Fig 3, Aspect Parameters Table showing different aspect types and associated aspect parameter ranges; pars 0064, 0066). Therefore, a transformer may generate one output corresponding to the aspect value selected. A different transformer outputs will be rendered when a different aspect values being selected. This is different from randomly select one or more image transformers from among a plurality of image transformations as recited.
Claims 9, 15 and 23 recite similar limitations and are rejected with the same reason. All dependent claims, depending from their respective base claims are rejected the same.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-29, and 31 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites: “randomly select one or more image transformations from among a plurality of image transformations according to a respective probability associated with each of a plurality of image transformations to apply to the initial set of images to generate a training set including transformed images simulating domain shifts across different imaging equipment or across different imaging protocols”.
Recited limitation is ambiguous and unclear as to what exactly it is claiming. On one hand, the limitation indicates one or more image transformations from among a plurality of image transformations being selected randomly, however, the same limitation also indicates such selection being according to a respective probability associated with each of a plurality of image transformations, which is a definite selection criterion not a random one. The claimed limitation appears to have contradictive selection processes. In addition, claim limitation recites “a plurality of image transformations” more than once. These two do not exist any antecedent basis and from the context of claim limitation, the antecedent basis exists. Claim 1 therefore renders indefinite.
Claims 9, 15, and 23 recite similar limitations and they are rejected with the same reason. All dependent claims, depending from their respective base claims, are rejected the same.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or
nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-6, 9-11, 15, 23-24, 28 are rejected under 35 U.S.C. 103 as being unpatentable over “Train Here, Deploy There: Robust Segmentation in Unseen Domains”, 2018 IEEE Intelligent Vehicles Symposium (IV), June 26-30, 2018, Romera et al., submitted in IDS (hereinafter Romera) in view of US 2019/0392038 A1, Goutal et al. (hereinafter Goutal) and further in view of US 2010/0309287 A1, Rodriguez (hereinafter Rodriguez).
As to claim 1, Romera discloses a processor comprising:
one or more circuits to obtain an initial set of images (Abstract; II-III., produce realistic images); wherein a number of transformed images in the training set is greater than a number of images in the initial set (III. Pages 1829-1930, random translation of pixels and image resizing, regions cropping, scaling with diverse aspect ratios, as well as random augmentations; augmentations following uniform distribution and generating more transformed images than original images); and
train one or more neural networks, based, at least in part, on the generated training set (Fig 1, bottom portion, images with a number of different domains being trained individually; Fig 1, top portion; data augmentation being performed prior to the training process; augmentation (transformation) being based on known information or expected differences, see section III, image transformation via Geometric augmentations and Texture augmentations by randomly selecting an value following some statistic distribution ranges)
Romera does not expressly disclose randomly select one or more image transformations from among a plurality of image transformations according to a respective probability associated with each of a plurality of image transformations to apply to the initial set of images to generate a training set including transformed images simulating domain shifts across different imaging equipment or across different imaging protocols.
Goutal, in the same or similar field of endeavor, additionally teaches wherein the one or more transformations are randomly selected from among a plurality of transformations according to a respective probability associated with each of the plurality of transformations (Figs 5-6; pars 0120-0121, transformation being selected randomly based on its probability distribution among pluralities of transformations).
Rodriguez, in the same or similar field of endeavor, additionally teaches including transformed images simulating domain shifts across different imaging equipment or across different imaging protocols (pars 0005, 0007, 0015, 0025-0027, 0032-0034, 0047, 0058, 0066).
Therefore, consider Romera, Goutal, and Rodriguez’s teachings as a whole, it would have been obvious to one of skill in the art before the filing date of invention to incorporate Goutal’s teachings and Rodriguez’s teachings in Romera’s processor to perform image transformation based on its respective probability distribution as well as
model simulation.
As to claim 2, Romera as modified discloses the processor of claim 1, further comprising: first storage to store a set of images from a second domain; a transformer to transform a first image of the set of images from the second domain according to an image aspect, to form a first transformed image (Romera: Fig 1, top portion; section III, image augmentations prior to performing CNN training based on characteristic of images); and second storage to store the first transformed image, for use in training the one or more neural networks (Romera: Fig 1, top portion; section III, performing CNN training after augmentations). Note storage of augmented images for training is implied/inherent and/or a necessity in order to facilitate and perform the training afterwards; Goutal: pars 0142, 0153, storing the augmented text).As to claim 3, Romera as modified discloses the processor of claim 2, wherein the image aspect comprises one or more of quality aspect, appearance aspect, or spatial configuration aspect (Romera: section III, geometric augmentations and text augmentations aiming at improving the quality and appearance of the images and/or spatially augmented).As to claim 4, Romera as modified discloses the processor of claim 3, wherein the transformer includes logic for selecting an image aspect value for the image aspect among a range of aspect values, to be used for transforming the first image according to the image aspect and the image aspect value (Romera: section III, range of augmentation values including Scaling and Cropping (0.5-1.0), Aspect ratio, Rotation, Brightness, etc.).As to claim 5, Romera as modified discloses the processor of claim 2, further comprising segmentation storage for storing segmentation data of the first image (see rejection in claim 2).As to claim 6, Romera as modified discloses the processor of claim 5, wherein the image aspect comprises the spatial configuration aspect and wherein the transformer modifies the first image according to spatial configuration aspect parameters and modifies the segmentation data of the first image according to the spatial configuration aspect parameters (Romera: section III, geometric and texture augmentations with defined augmentation ranges and parameters to provide a range of segmentation data based on the way images being augmented geographically (e.g. spatial configuration)).
As to claim 9, it is a device (a processor) claim encompassed claim 1. Rejection of claim 1 is therefore incorporated herein.
As to claim 10, it is rejected with the same reason as set forth in claim 2.
As to claim 11, it is rejected with the same reason as set forth in claim 3.
As to claim 15, it is a method claim necessitated claim 1. Rejection of claim 1 is therefore incorporated herein.
As to claim 23, it is a method claim essentially necessitated by claim 1. Rejection of claim 1 is therefore incorporated herein.
As to claim 24, it is rejected with the same reason as set forth in claim 10.
As to claim 28, it is rejected with the same reason as set forth in claim 3.
As to claim 31, Romera as modified discloses the processor of claim 1, wherein the one or more neural networks are trained on one or more randomly selected types of transformations of one or more images (Romera: section III, different types of augmentations; Goutal: Figs 5-6; pars 0120-0121).
Claims 7-8, 12-14, 16-22, 25-27, 29 are rejected under 35 U.S.C. 103 as being unpatentable over Romera in view of Goutal and further in view of Rodriguez and US 10,169,864 B1, Bagherinia et al. (Bagherinia).
As to claim 7, Romera as modified discloses the processor of claim 2, wherein the image aspect comprises a spatial configuration aspect and wherein the first image is a volume image, the processor further comprising: an image cropper, to crop the first image into sub-volume images (section III, cropping operation to crop different regions in the image), but does not expressly disclose wherein sub-volume images are processed separately. Bagherinia, in the same or similar field of endeavor, additionally teaches cropping each individual volume image patch, or total, partial, or pates of the OCT data (Fig 4; col 7, lines 40-49; col 8, lines 10-12; col 11, lines 49-62). Therefore, consider Romera as modified and Bagherinia’s teachings as a whole, it would have been obvious to one of skill in the art before the filing date of invention to incorporate Bagherinia’s teachings on flexible cropping operations in Romera’s teachings to improve cropping flexibility as desired.As to claim 8, Romera as modified discloses the processor of claim 7, wherein the image cropper is a cropper that interpolates within a minimal cuboid containing a 3D coordinate grid (Bagherinia: col 11, line 49-col 12, line 11; col 19, lines 52-66, segmentation image with cropping and interpolation or upsampling with 3D volumetric images).
As to claim 12, Romera as modified discloses the processor of claim 9, but does not expressly discloses wherein the one or more images are from a first domain and comprise medical images.
An ordinary skill in the art, however, would understand and appreciate that Romera as modified’s teachings on image augmentation and training can be readily applied to medical image as well. Nevertheless, Bagherinia, in the same or similar field of endeavor, additionally teaches image augmentation and training in medical image applications (Figs 1, 4-8, retinal image segmentation and training). Therefore, consider Romera as modified and Bagherinia’s teachings as a whole, it would have been obvious to one of skill in the art before the filing date of invention to incorporate Bagherinia’s teachings in Romera as modified ’s teachings to improve medical image quality.As to claim 13, Romera as modified discloses the processor of claim 12, wherein the first set of images are images obtained using a first medical device and are different from one or more images are images obtained using a second medical device different from the first medical device (Bagherinia: Figs 2-4, 14; col 20, lines 52-65; col 21, lines 4-34; col 22, lines 38-39, one or more input devices, user interface devices, video capture devices, processors for receiving and processing medical images).
As to claim 14, Romera as modified discloses the processor of claim 9, wherein the first set of images comprises volumetric images (see rejection in claim 7).
As to claim 16, Romera as modified discloses the method of claim 15, further comprising: obtaining a first set of images, comprising at least a first image (Romera: Fig 1); obtaining a segmentation of the first image; determining a transform aspect parameter (Romera: section III), wherein the transform aspect parameter corresponds to at least one of the expected differences between a first domain and a second domain (Romera: Figs 1-2; Table II, section III); determining a transform aspect parameter value (Romera: section III, providing a range of parameter values for augmentations); transforming the first image based on the transform aspect parameter value to form a transformed first image (section III, various parameters corresponding to geometric and texture augmentations (e.g. transformation), including flip, translation, scaling and cropping, rotation, etc.; training the first neural network with the transformed first image (Romera: Figs 1-2).
Romera as modified does not expressly disclose the segmentation represents boundaries of objects depicted in the first image.
Bagherinia, in the same or similar field of endeavor, further teaches the segmentation represents boundaries of objects depicted in the first image (Figs 3-4; col 2, lines 4-28; col 8, lines 1-33). Therefore, consider Romera as modified and Bagherinia’s teachings as a whole, it would have been obvious to one of skill in the art before the filing date of invention to incorporate Bagherinia’s teachings in Romera as modified ’s teachings to improve medical image quality by properly perform image segmentation including segmenting the boundary of the object. As to claim 17, Romera as modified discloses the method of claim 16, further comprising: determining whether the first image can be transformed as a whole using a memory (Romera: section III, cropping operation; Bagherinia: Figs 3-4; col 7, lines 40-49; col 8, lines 10-13; col 11, lines 49-62, cropping the total, partial, or patches of the OCT data); and cropping the first image into a plurality of sub-volumes for loading into the memory separately (Bagherinia: Figs 3-4; col 7, lines 40-49; col 8, lines 10-13; col 11, lines 49-62).
As to claim 18, Romera as modified discloses the method of claim 16, further comprising generating a plurality of transformed images from the first image, using a plurality of transform aspect parameters (Romera: section III, various parameters corresponding to geometric and texture augmentations (e.g. transformation), including flip, translation, scaling and cropping, rotation, etc.).
As to claim 19, it is rejected with the same reason as set forth in claim 3.
As to claim 20, Romera as modified discloses the method of claim 16, wherein the transform aspect parameter comprises a spatial configuration aspect parameter (Romera: section III, various spatial related parameters corresponding to geometric and texture augmentations, including flip, translation, scaling and cropping, rotation, etc.), the method further comprising: modifying the first image according to the spatial configuration aspect parameter (Romera: section III); and modifying the segmentation of the first image according to the spatial configuration aspect parameter (Romera: section III).
As to claim 21, Romera as modified discloses the method of claim 20, wherein modifying the first image according to the spatial configuration aspect parameter comprises cropping sub-volumes of the first image randomly for loading into a memory to apply the transform aspect parameter value to the first image (Romera: section III, various parameters to be chosen for different geometric and texture augmentations; Bagherinia: Fig 9a; col 5, lines 61-64, segmenting OCT volumetric data into sub-volumes of given size).
As to claim 22, Romera as modified discloses the method of claim 16, further comprising: further comprising training the first neural network over a plurality of training epochs, using a distinct transform aspect parameter for each of the plurality of training epochs (Romera: section IV, Table II, there exist variance of each augmentation, training may require significant more epochs for some than others until convergence).
As to claims 25-26, they are rejected with the same reason as set forth in claims 12-13, respectively.
As to claim 27, it is rejected with the same reason as set forth in claim 14.
As to claim 29, Romera as modified discloses the processor of claim 1, wherein the one or more neural networks are trained on a plurality of partial images cropped from the one or more images (Romera: Fig 1; section III, image scaling and cropping with respect to the regions of interest prior to input for training using CNNs; Bagherinia: Figs 3-4; col 7, lines 40-49; col 8, lines 1-25; col 11, lines 49-62, cropping of the total, partial, or patches of the OCT data (e.g. a plurality of cropped images or image segments/regions/patches being trained)).
30. (Canceled)
Response to Arguments
Applicant’s arguments have been fully considered but they are moot in light of new grounds of rejection.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Examiner’s Note
Examiner has cited particular column, line number, paragraphs and/or figure(s) in the reference(s) as applied to the claims for the convenience of the Applicant. Although the specified citations are representative of the teachings of the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the reference(s) in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Qun Shen whose telephone number is (571) 270-7927. The examiner can normally be reached on Mon-Friday from 9:00-5:00. If attempts to reach the examiner by telephone are unsuccessful, the examiner's Supervisor, Amandeep Saini can be reached on (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/QUN SHEN/
Primary Examiner, Art Unit 2662