DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicants
2. This communication is in response to the application filled on 02/18/2025.
3. Claims 1-8 are pending.
4. Limitations appearing inside {} are intended to indicate the limitations not taught by said prior art(s)/combinations.
Claim Rejections - 35 USC § 103
5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
6. Claims 1-8 are rejected under 35 U.S.C. 103 as being unpatentable over “DTR-GAN: An Unsupervised Bidirectional Translation Generative Adversarial Network for MRI-CT Registration” to Yang et al. (hereinafter Yang), and further in view of U.S. Publication No. 2024/0085304 to Li et al. (hereinafter Li).
7. Regarding Claim 1, Yang discloses a system for translation of images between two distinct heterogenous {cameras}, comprising ([pg. 1, Abstract, par. 1, ln. 1-19] “Medical image registration is a fundamental and indispensable element in medical
image analysis, which can establish spatial consistency among corresponding anatomical structures across various medical images. Since images with different modalities exhibit different features, it remains a challenge to find their exact correspondence. Most of the current methods based on image-to-image translation cannot fully leverage the available information, which will affect the
subsequent registration performance. To solve the problem, we develop an unsupervised multimodal image registration method named DTR-GAN.Firstly, we design a multimodal registration framework via a bidirectional translation network to transform the multimodal image registration into a unimodal registration, which can effectively use the complementary information of different modalities. Then, to enhance the quality of the transformed images in the translation network, we design a multiscale encoder–decoder network that effectively captures both local and global features in images. Finally, we propose a mixed similarity loss to encourage the warped image to be closer to the target image in deep features… The results indicate that DTR-GAN obtains a competitive performance compared to other methods in MRI-CT registration. Compared with DFR, DTR-GAN has not only obtained performance improvements of 2.35% and 2.08% in the dice similarity coefficient (DSC) of MRI-CT registration and CT-MRI registration on the Learn2Reg dataset but has also decreased the average symmetric surface distance (ASD) by 0.33 mm and 0.12 mm on the Learn2Reg dataset.”, [pg. 10, 4.2. Implementation Details, par. 1, ln. 1-3] “We utilize the Pytorch framework to implement DTR-GAN, which runs on a hardware environment equipped with a GeForce RTX 3080Ti GPU and 12 GB RAM. The network is trained for 300 epochs with a batch size of 1.”):
a source {camera} configured to capture a source image in communication with a first generative adversarial network (first GAN), the first GAN comprising a source encoder and at least a source generator ([pg. 4, Figure 1, see (B)], [pg. 6, Figure 3], [pg. 5, 3.2. Dual Contrastive Translation Network, par. 1, ln. 1 to pg. 6, par. 3, ln. 9] “Since it is difficult to perform multimodal image registration with significant intensity differences using a single registration network, we use the bidirectional translation network to ensure a consistent shape in the image-to-image translation. It contributes to aligning two modalities well in the subsequent registration network… To overcome these constraints, we design a bidirectional translation network DT-GAN. Specifically, we combine an improved CycleGAN with contrastive learning, which replaces the cycle-consistency loss with PacthNCE loss to avoid shape distortion in image-to-image translation. DT-GAN includes translation networks TP and TR, which comprise two generators
G
x
→
y
and
G
y
→
x
and two discriminators
D
X
and
D
Y
, the structure of the TP is shown in Figure 3. Generator
G
x
→
y
has an encoder
G
e
n
c
x
→
y
and a decoder
G
d
e
c
x
→
y
. And generator
G
y
→
x
is made up of an encoder
G
e
n
c
y
→
x
and a decoder
G
d
e
c
y
→
x
. When the image pairs X and Y are used as input, we obtain transformed images Y’ and X’ from generator
G
x
→
y
and generator
G
y
→
x
during the translation process, respectively.”, [pg. 6, 3.2.1 Generator, par. 1, ln. 1-8] “Considering the existence of domain gaps between different domains, we use different embeddings (
H
X
and
H
Y
) to extract features of image X and image Y to capture variability in both two image domains. The generator module consists of
G
x
→
y
and
G
y
→
x
in translation networks TP and TR. Generator
G
x
→
y
and generator
G
y
→
x
use the same structure but represent translation in different directions, where
G
x
→
y
enables the mapping
X
→
Y
and
G
y
→
x
enables the mapping
Y
→
X
. The generator
G
x
→
y
and generator
G
y
→
x
are both composed of the encoder, the decoder, nine residual blocks, and the multiclass feature fusion (MFF) module…”);
a target {camera} configured to capture target images in communication with a second generative adversarial network (second GAN), the second GAN comprising a target encoder and at least a target generator ([pg. 4, Figure 1, see (C)], [pg. 6, Figure 3], [pg. 5, 3.2. Dual Contrastive Translation Network, par. 1, ln. 1 to pg. 6, par. 3, ln. 9], [pg. 6, 3.2.1 Generator, par. 1, ln. 1-8]);
a transformation prediction module configured to receive intermediate outputs of the source encoder and the target encoder as input ([pg. 4, Figure 1, see R and STN spatial transforms], [pg. 4, 3. Methods, par. 2, ln. 1-12] “The bidirectional translation network DT-GAN includes TP and TR networks, which represent the two mapping directions from the source image X to the target image Y and the target image Y to the source image X, respectively. Inspired by VoxelMorph, we employ a registration network that consists of U-net [29] and a Spatial Transformer Network (STN) [30] to warp the source image into the warped image. When the source image X and target image Y are used as input, the registration network R predicts an optimal mapping ϕ to achieve an excellent alignment of the warped image X′ w with the target image Y. Meanwhile, translation networks TP and TR take X and Y as inputs, respectively, to output the transformed images Y′ and X′. The STN transforms the input images into warped images, where the source image X is warped into the registered image
X
w
'
and the translated image Y′ is warped into the registered image
Y
w
'
…”, [pg. 5, Figure 2, see registration network, output ϕ], [pg. 5, 3.1 Registration Network, par. 1, ln. 1 to par. 2, ln. 6] “Figure 2 depicts the architecture of the registration network R, which is made up of U-net and the STN. U-net is utilized to capture the mapping between the input image pairs, while the STN transforms the source image into the warped image… The registration network takes X and Y as input image pairs and produces the deformation field ϕ as output, in which ϕ = R(X,Y) and ϕ is a 2D vector. The STN resamples the source image X and the transformed image Y′ into the warped images
X
w
'
and
Y
w
'
, respectively… see Equations (2), (3)…”, [pg. 7, 3.3.1 Registration Loss, par. 1, ln. 1 pg. 8, par. 4, ln. 2] “(1) Mixed similarity loss The existing multimodal similarity measures utilized as loss functions in the network remain inefficient, thereby impacting the registration performance of the training process. For instance, the NMI is computationally difficult and unsuitable for gradient-based methods. Similarly, the NCC is dependent on the domain and cannot be generalized to all modalities. The translation-based registration framework can alleviate the problem of hand-crafted multimodal similarity measures. However, during the image-to-image translation process, there may be a distribution mismatch, especially when dealing with images that have a complex appearance. Therefore, it is unreliable to rely solely on translation algorithms to transform multimodal registration into unimodal registration. Hence, to reduce the impact of mismatched images generated during the translation process, we propose a mixed similarity loss that focuses on the structural information of the images, aiming to maximize the similarity between the registered image and the target image. To penalize appearance discrepancies between target and warped images, we propose a similarity loss
L
r
e
g
(
R
)
.
L
r
e
g
(
R
)
is defined as
L
r
e
g
R
=
H
Y
P
G
e
n
c
y
→
x
Y
-
H
X
P
(
G
e
n
c
x
→
y
(
X
w
'
)
)
1
s
u
m
(6)… where sum represents summing them up together. We utilize separate embeddings
H
X
P
and
H
Y
P
to extract the 256-dimensional feature stacks from the input image pairs and then project them onto 64-dimensional vectors.
H
X
P
and
H
Y
P
consist of 2-layer MLP, alongside convolutional, ReLU, average pooling, linear transformation, ReLU, and final linear transformation layers. To explore the potential mapping relationships in the feature space, we extract features shared between the warped image and the target image to guide the accurate registration. To minimize the appearance dissimilarity between the warped image of the translated image
Y
w
'
and the target image Y, L(TP,R) is defined as
L
T
P
,
R
=
Y
w
'
-
Y
1
(7)… So the mixed similarity loss
L
m
s
is computed by
L
m
s
=
L
r
e
g
R
+
L
(
T
P
,
R
)
”); and
a warping module in communication with the transformation prediction module, the output of the warping module being fed to {the source generator and} the target generator ([pg. 4, Figure 1, see R and STN spatial transforms], [pg. 4, 3. Methods, par. 2, ln. 1-12], [pg. 5, Figure 2, see registration network, output ϕ], [pg. 5, 3.1 Registration Network, par. 1, ln. 1 to par. 2, ln. 6] see Equations (2) and (3) which define “warping” module based on output of registration network ϕ).
However, while Yang discloses translation between heterogenous domains, Yang does not specifically disclose cameras, or wherein the output of the warping module is also fed to the source generator. Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that while Yang does not specifically disclose cameras, the heterogenous domain transfer as taught in Yang would effectively encompass two heterogenous imaging systems, since CT and MRI are acquired through different means (CT is X-rays, MRI is magnetic and radio waves).
However, Li specifically discloses wherein an analogous dual GAN contrastive learning architecture translation may be performed between heterogenous cameras ([Fig. 3, 6], [par. 0096, ln. 1-20] “These high-precision microscopes are typically expensive and bulky. Thus, they are commonly used in a lab environment handled by expert human operators (e.g., a pathologist). In contrast, image-based assays where the images are captured by cameras already existing in mass mobile devices (e.g., smart phones) may provide a low-cost solution for areas such as low-cost collection and analysis of blood samples of the public. This may be particularly useful in certain healthcare situations, such as point-of-care (POC), where a large amount of assays needs to be processed quickly and economically. However, the captured assay images using the low-cost mobile device tend to be low-quality containing variations caused by the wide range of variations in imaging systems from the mobile devices employed to capture the images. Factors such as the resolutions and magnifying powers of different cameras and the distortions from the imaging systems or lenses behind the cameras can all vary widely, thereby severely limiting the accuracy and reliability of the image-based assaying based on these mobile devices”, [par. 0098, ln. 1-6] “In this disclosure, the term “System L” may refer to a low-quality image system (e.g. due to inconsistence in lighting/camera optics/imaging system sensor array (e.g. photodetector array), spherical distortions, high noise level in captured images, lower magnification, lower resolutions etc.).”, [par. 0099, ln. 1-8] “The term “System H” refers to a high-quality image system that meets the current regulations for deployment in commercial settings. The System H commonly possesses a superior image quality compared to System L (e.g., in terms of a top line optical system with professional lighting/camera optics/imaging system sensor array, low noise level in captured images, higher magnification power, higher resolutions etc.).”, [par. 0101, ln. 1-16] “…each of the factors such as light source, optics, and imaging system may also vary across different Systems L. This variation may be caused by lack of calibrations among different types and/or individual Systems L. The statistical distributions of these variations among different Systems L may also vary. To improve the performance of Systems L, machine learning models may be employed to compensate the variations in a specific System L. The machine learning model such as a neural network model may be custom trained for each individual System L. The custom-trained machine learning model can then be used to process the images captured by the corresponding individual System L during assaying. Although the custom-rained machine learning model may improve the performance of each individual System L, it is not suitable for deployment in a mass market because such training of each individual System L (also termed “device” or “assay device”) is inefficient, time consuming, and expensive, and is therefore not practical in real world applications.”), and wherein the output of the warping module is also fed to the source generator ([par. 0130, ln. 1-16] “At 210, processing devices 102 may train the machine learning model using each of transformed regions in the first image and each of transformed regions in the second image. The machine learning model 108 may be trained using the training dataset including the transformed regions of the first assay images and their corresponding regions of the second assay images, where both the transformed regions of the first assay images and their corresponding regions of the second assay images are mapped into the true dimension space to mitigate the distortions existing in Systems L and Systems H. In this way, the individual system variations are substantially removed, and the regions of the first assay images and regions of the second assay images are mapped to a common true dimension space using the standard information embedded in the assay images.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would specifically recognize Yang and Li as within the same field of image-to-image translation using dual GAN contrastive learning architectures, and as analogous to the claimed invention. The motivation to combine would have been obvious to one of ordinary skill in the art, and is further discloses in Li, wherein by incorporating warping results for both the source generator and target generator you allow for a more accurate mapping between the source and target image for the generators ([par. 0130, ln. 1-16]). With regard to motivation to combine the cameras of Li with the system of Yang, one of ordinary skill in the art, before the effective filling date of the claimed invention, would specifically recognize the architecture as used in Yang would likewise be applicable to the simpler image-to-image translation between heterogenous cameras as taught in Li, which would expand the model performance improvement as taught in Yang ([pg. 1, Abstract, par. 1, ln. 1-19]) to other modalities (e.g., RGB-to-RGB). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the system of Yang with the cameras and feeding of warping module outputs to the source generator as taught in Li, through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art would have combined the system of Yang with the cameras and feeding of warping module outputs to the source generator as taught in Li such that the model architecture of Yang was trained on heterogenous cameras as taught in Li and further included feeding the warping output to the generator for the source image as taught in Li.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the system of Yang with the cameras and feeding of warping module outputs to the source generator as taught in Li to obtain the invention as specified in claim 1.
8. Regarding Claim 2, a combination of Yang and Li teaches the system of claim 1. Yang further discloses wherein the source generator is configured to generate a Translated Target Image ([pg. 4, Figure 1, see (B)], [pg. 6, Figure 3, see Y’], [pg. 5, 3.2. Dual Contrastive Translation Network, par. 1, ln. 1 to pg. 6, par. 3, ln. 9], [pg. 6, 3.2.1 Generator, par. 1, ln. 1-8]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the system of Yang with the cameras and feeding of warping module outputs to the source generator as taught in Li to obtain the invention as specified in claim 2.
9. Regarding Claim 3, a combination of Yang and Li teaches the system of claim 1. Yang further discloses wherein the target generator is configured to generate a Translated Source Image ([pg. 4, Figure 1, see (C)], [pg. 6, Figure 3, see Y’, which would correspond to X’ in the mirrored architecture of TR in Figure 1], [pg. 5, 3.2. Dual Contrastive Translation Network, par. 1, ln. 1 to pg. 6, par. 3, ln. 9], [pg. 6, 3.2.1 Generator, par. 1, ln. 1-8]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the system of Yang with the cameras and feeding of warping module outputs to the source generator as taught in Li to obtain the invention as specified in claim 3.
10. Regarding Claim 4, a combination of Yang and Li teaches the system of claim 1. Yang discloses wherein the transformation prediction module is trained to optimize the loss between the translated source image vis-a-vis target image and translated target image vis-a-vis source image ([pg. 4, Figure 1, see R and STN spatial transforms], [pg. 4, 3. Methods, par. 2, ln. 1-12], [pg. 5, Figure 2, see registration network, output ϕ], [pg. 5, 3.1 Registration Network, par. 1, ln. 1 to par. 2, ln. 6], [pg. 7, 3.3.1 Registration Loss, par. 1, ln. 1 pg. 8, par. 4, ln. 2], [pg. 8, par. 5, ln. 1-5] “(2) Smooth loss To prevent discontinuity in the deformation field ϕ when making the warped image closer to the target image, the smooth loss
L
s
m
o
o
t
h
, which is similar to the TV loss [34], is employed to constrain ϕ to avoid excessive distortion of the warped image.”). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the system of Yang with the cameras and feeding of warping module outputs to the source generator as taught in Li to obtain the invention as specified in claim 4.
11. Regarding Claim 5, the claim language is analogous to claim 1 with further limitations. Specifically, Yang discloses a method for translation of images between two distinct heterogenous {cameras} ([pg. 1, Abstract, par. 1, ln. 1-19]),
wherein a source {camera} is configured to capture a source image in communication with a first generative adversarial network (first GAN), the first GAN comprising a source encoder and at least a source generator ([pg. 4, Figure 1, see (B)], [pg. 6, Figure 3], [pg. 5, 3.2. Dual Contrastive Translation Network, par. 1, ln. 1 to pg. 6, par. 3, ln. 9]), and
wherein a target {camera} is configured to capture target images in communication with a second generative adversarial network (second GAN), the second GAN comprising a target encoder and at least a target generator ([pg. 4, Figure 1, see (C)], [pg. 6, Figure 3], [pg. 5, 3.2. Dual Contrastive Translation Network, par. 1, ln. 1 to pg. 6, par. 3, ln. 9], [pg. 6, 3.2.1 Generator, par. 1, ln. 1-8]), the method comprising:
providing output of the source encoder and the target encoder along with source image and target image as input to a transformation prediction module ([pg. 4, Figure 1, see R, also Source: X and Target: Y into R], [pg. 4, 3. Methods, par. 2, ln. 1-12], [pg. 5, Figure 2, see registration network, output ϕ], [pg. 5, 3.1 Registration Network, par. 1, ln. 1 to par. 2, ln. 6], [pg. 7, 3.3.1 Registration Loss, par. 1, ln. 1 pg. 8, par. 4, ln. 2], [pg. 8, par. 5, ln. 1-5]);
executing a warping module with outputs from the transformation prediction module, source encoder and at least the target encoder ([pg. 4, Figure 1, see R and STN spatial transforms], [pg. 4, 3. Methods, par. 2, ln. 1-12], [pg. 5, Figure 2, see registration network, output ϕ], [pg. 5, 3.1 Registration Network, par. 1, ln. 1 to par. 2, ln. 6] see Equations (2) and (3) which define “warping” module based on output of registration network ϕ, [pg. 7, 3.3.1 Registration Loss, par. 1, ln. 1 pg. 8, par. 4, ln. 2]);
feeding the output of the warping module as input to {both the source generator and} the target generator ([pg. 4, Figure 1, see R and STN spatial transforms], [pg. 4, 3. Methods, par. 2, ln. 1-12], [pg. 5, Figure 2, see registration network, output ϕ], [pg. 5, 3.1 Registration Network, par. 1, ln. 1 to par. 2, ln. 6] see Equations (2) and (3) which define “warping” module based on output of registration network ϕ);
running the source generator and target generator to generate a translated target image and a translated source image respectively ([pg. 4, Figure 1, see (B) and (C)], [pg. 6, Figure 3], [pg. 5, 3.2. Dual Contrastive Translation Network, par. 1, ln. 1 to pg. 6, par. 3, ln. 9]);
defining a loss function comprising loss between the translated source image vis-a-vis target image and translated target image vis-a-vis source image ([pg. 4, Figure 1, see R and STN spatial transforms], [pg. 4, 3. Methods, par. 2, ln. 1-12], [pg. 5, Figure 2, see registration network, output ϕ], [pg. 5, 3.1 Registration Network, par. 1, ln. 1 to par. 2, ln. 6], [pg. 7, 3.3.1 Registration Loss, par. 1, ln. 1 pg. 8, par. 4, ln. 2], [pg. 8, par. 5, ln. 1-5]);
optimizing the loss function to configure the transformation prediction module ([pg. 4, Figure 1, see R and STN spatial transforms], [pg. 4, 3. Methods, par. 2, ln. 1-12], [pg. 5, Figure 2, see registration network, output ϕ], [pg. 5, 3.1 Registration Network, par. 1, ln. 1 to par. 2, ln. 6], [pg. 7, 3.3.1 Registration Loss, par. 1, ln. 1 pg. 8, par. 4, ln. 2], [pg. 8, par. 5, ln. 1-5]); and
running the configured transformation prediction module to enable translation of images between the two distinct heterogenous {cameras} ([pg. 4, Figure 1, see R and STN spatial transforms], [pg. 4, 3. Methods, par. 2, ln. 1-12], [pg. 5, Figure 2, see registration network, output ϕ], [pg. 5, 3.1 Registration Network, par. 1, ln. 1 to par. 2, ln. 6] see Equations (2) and (3) which define “warping” module based on output of registration network ϕ, [pg. 7, 3.3.1 Registration Loss, par. 1, ln. 1 pg. 8, par. 4, ln. 2], [pg. 8, par. 5, ln. 1-5]).
Rejections analogous to claim 1 are further applicable to claim 5. Specifically, while Yang discloses translation between heterogenous domains, Yang does not specifically disclose cameras, or wherein the output of the warping module is also fed to the source generator. Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that while Yang does not specifically disclose cameras, the heterogenous domain transfer as taught in Yang would effectively encompass two heterogenous imaging systems, since CT and MRI are acquired through different means (CT is X-rays, MRI is magnetic and radio waves).
However, Li specifically discloses wherein an analogous dual GAN contrastive learning architecture translation may be performed between heterogenous cameras ([Fig. 3, 6], [par. 0096, ln. 1-20], [par. 0098, ln. 1-6], [par. 0099, ln. 1-8], [par. 0101, ln. 1-16]) and wherein the output of the warping module is also fed to the source generator ([par. 0130, ln. 1-16]). The motivations to combine remains analogous to claim 1. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Yang with the cameras and feeding of warping module outputs to the source generator as taught in Li, through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art would have combined the method of Yang with the cameras and feeding of warping module outputs to the source generator as taught in Li such that the model architecture of Yang was trained on heterogenous cameras as taught in Li and further included feeding the warping output to the generator for the source image as taught in Li.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Yang with the cameras and feeding of warping module outputs to the source generator as taught in Li to obtain the invention as specified in claim 5.
12. Regarding Claim 6, a combination of Yang and Li teaches the method of claim 5. Yang further discloses wherein the transformation prediction module is a neural network ([pg. 4, Figure 1, see R and STN spatial transforms], [pg. 4, 3. Methods, par. 2, ln. 1-12], [pg. 5, Figure 2, see registration network, output ϕ], [pg. 5, 3.1 Registration Network, par. 1, ln. 1 to par. 2, ln. 6], [pg. 7, 3.3.1 Registration Loss, par. 1, ln. 1 pg. 8, par. 4, ln. 2], [pg. 8, par. 5, ln. 1-5]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Yang with the cameras and feeding of warping module outputs to the source generator as taught in Li to obtain the invention as specified in claim 6.
13. Regarding Claim 7, a combination of Yang and Li teaches the method of claim 5. Yang further discloses wherein optimizing the loss function comprises tuning the network parameters and hyperparameters of the neural network ([pg. 4, Figure 1, see R and STN spatial transforms], [pg. 4, 3. Methods, par. 2, ln. 1-12], [pg. 5, Figure 2, see registration network, output ϕ], [pg. 5, 3.1 Registration Network, par. 1, ln. 1 to par. 2, ln. 6], [pg. 7, 3.3.1 Registration Loss, par. 1, ln. 1 pg. 8, par. 4, ln. 2], [pg. 8, par. 5, ln. 1-5]). The examiner further notes this would have been obvious in view of the nature of neural network loss, since during backpropagation and gradient descent weights within the neural networks are necessarily altered to minimize the loss. This would have been apparent to one of ordinary skill in the art in view of the widespread usage of neural networks. The examiner further notes that such parameter alteration during training is disclosed in Li ([par. 0132, ln. 1-7] “Machine learning model 108 can be any suitable model including but not limited to deep neural network (DNN), convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), graph neural network (GNN) etc. All of these models may include parameters that can be adjusted during training based on training dataset or examples.”, [par. 0136, ln 1-20] “During training, the first discriminator loss function may be used in a backpropagation to train the discriminator 306B, and the first discriminator loss function may be used in a backpropagation to train the discriminator 308B. The parameters of generator 306 may be adjusted in a backpropagation based on the first generator loss function from discriminator 306B so that generator 306A may produce generated assay images in domain H considered by discriminator 306B as “real.” Similarly, the parameters of generator 308A may be adjusted in a backpropagation based on the second generator loss function from discriminator 308B so that generator 308A may produce generated assay images in domain L considered by discriminator 306B as “real” …discriminator 306B, 308B may be trained in several epochs in step 1, and then generator 306A, 308A may be trained in subsequent several epochs in step 2. The steps 1 and 2 may be repeated alternatively during training until each of the GANs 302, 304 converges.”). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Yang with the cameras and feeding of warping module outputs to the source generator as taught in Li to obtain the invention as specified in claim 7.
14. Regarding Claim 8, a combination of Yang and Li teaches the method of claim 5. Yang further discloses wherein the transformation prediction module is configured to derive a transformation parameter ([pg. 4, Figure 1, see R and STN spatial transforms], [pg. 4, 3. Methods, par. 2, ln. 1-12], [pg. 5, Figure 2, see registration network, output ϕ], [pg. 5, 3.1 Registration Network, par. 1, ln. 1 to par. 2, ln. 6], [pg. 7, 3.3.1 Registration Loss, par. 1, ln. 1 pg. 8, par. 4, ln. 2], [pg. 8, par. 5, ln. 1-5]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Yang with the cameras and feeding of warping module outputs to the source generator as taught in Li to obtain the invention as specified in claim 8.
Conclusion
15. The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. See PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAULO ANDRES GARCIA whose telephone number is (703)756-5493. The examiner can normally be reached Mon-Fri, 8-4:30PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached on (571)272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAULO ANDRES GARCIA/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights