Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "the performance of one or more training operations" in the final two lines of the claim. There is insufficient antecedent basis for this limitation in the claim. Examiner treats “the performance…” as “performance…”
Claims 2-10 are rejected for failing to cure the deficiencies of claim 1.
Claim 3 recites the limitation "the performance of one or more prediction-based actions" in line 7. There is insufficient antecedent basis for this limitation in the claim. Examiner treats “the performance…” as “performance…”
Claims 11-12 are rejected because they recite the same indefinite limitation as method claims 1 and 3, respectively. Claims 12-16 are rejected for failing to cure the deficiencies of claim 11.
Claim 17 recites the limitation "the performance of one or more prediction-based actions" in line the final two lines of the claim. There is insufficient antecedent basis for this limitation in the claim. Examiner treats “the performance…” as “performance…”
Claims 18-20 are rejected for failing to cure the deficiencies of claim 17.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claims 1-10 and 17-20 recite a method, claims 11-16 recite a system comprising memory and a processor. A method and a system each falls within one of the four statutory categories of patent eligible subject matter.
Claim 1
Step 2A Prong 1: Identify an observation of a training modality for a data element of the plurality of data elements is an observation mental process which can reasonably be performed in the human mind with the aid of pencil and paper.
Generating,
Generating,
Generating,
Initiating, or more training operations comprise optimizing a loss using a loss function. The claim recites an abstract idea.
Step 2A Prong 2: Receiving, by one or more processors, training data comprising a plurality of data elements that comprise (i) a plurality of data values associated with a plurality of training modalities and (ii) a plurality of modality observation variables amounts to mere data-gathering, which is an insignificant extra-solution activity under MPEP 2106.05(g). One or more processors amount to generic computer components for applying the abstract ideas on a generic computer under MPEP 2106.05(f).
Using a modality-agnostic latent variable encoder of a multi-modal generative machine learning model amounts to mere instructions to apply the abstract ideas on a generic computer under MPEP 2106.05(f).
Using a modality-specific latent variable encoder of the multi-modal generative machine learning model amounts to mere instructions to apply the abstract ideas on a generic computer under MPEP 2106.05(f).
The additional elements as disclosed above, alone or in combination, do not integrate the abstract ideas into a practical application as they are mere insignificant extra solution activities as disclosed in combination with generic computer functions that are implemented to perform the abstract ideas disclosed above. The claim is directed to an abstract idea.
Step 2B: Receiving, by one or more processors, training data comprising a plurality of data elements that comprise (i) a plurality of data values associated with a plurality of training modalities and (ii) a plurality of modality observation variables is analogous to receiving data over a network, which the courts have recognized as well-understood, routine, conventional activity under MPEP 2106.05(d)(II). One or more processors amount to generic computer components for applying the abstract ideas on a generic computer under MPEP 2106.05(f).
Using a modality-agnostic latent variable encoder of a multi-modal generative machine learning model amounts to mere instructions to apply the abstract ideas on a generic computer under MPEP 2106.05(f).
Using a modality-specific latent variable encoder of the multi-modal generative machine learning model amounts to mere instructions to apply the abstract ideas on a generic computer under MPEP 2106.05(f).
The additional elements as disclosed above, in combination with the abstract ideas, are not sufficient to amount to significantly more than the abstract ideas as they are well-understood, routine and conventional activities as disclosed in combination with generic computer functions that are implemented to perform the abstract ideas disclosed above. The claim is not patent eligible.
Claim 2 incorporates the rejection of claim 1.
Step 2A Prong 1: The abstract ideas of claim 1 are incorporated. Identify
Step 2A Prong 2 and Step 2B: The multi-modal generative machine learning model comprises a variational autoencoder architecture amounts to a mere field of use and technological environment under MPEP 2106.05(h). The claim is not patent eligible.
Claim 3 incorporates the rejection of claim 1.
Step 2A Prong 1: The abstract ideas of claim 1 are incorporated. Generating, using the multi-modal
Initiating the performance of one or more prediction-based actions based on the one or more modality predictions is a judgement mental process which can reasonably be performed in the human mind with the aid of pencil and paper. Paragraph [0099] discloses the one or more prediction-based actions may comprise, for example, generating a diagnostic report.
Step 2A Prong 2 and Step 2B: Receiving, using the multi-modal generative machine learning model, an input data record comprising a plurality of input data elements amounts to mere instructions to apply the abstract ideas on a generic computer under MPEP 2106.05(f). The claim is not patent eligible.
Claim 4 incorporates the rejection of claim 3.
Step 2A Prong 1: The abstract ideas of claim 3 are incorporated. The one or more modality predictions comprise a synthetic modality element imputed for the missing modality element is a judgement mental process which can reasonably be performed in the human mind with the aid of pencil and paper.
Step 2A Prong 2 and Step 2B: An input data element of the plurality of input data elements comprises a missing modality element amounts to a mere field of use and technological environment under MPEP 2106.05(h). The claim is not patent eligible.
Claim 5 incorporates the rejection of claim 3.
Step 2A Prong 1: The abstract ideas of claim 3 are incorporated.
Step 2A Prong 2 and Step 2B: The one or more modality predictions comprise one or more synthetic data records, each comprising a plurality of synthetic modality elements amounts to a mere field of use and technological environment under MPEP 2106.05(h). The claim is not patent eligible.
Claim 6 incorporates the rejection of claim 1.
Step 2A Prong 1: The abstract ideas of claim 1 are incorporated. The one or more training operations comprises optimizing the loss using the loss function. Equation 1 at paragraph [0086] discloses generating a loss using a loss function.
Step 2A Prong 2 and Step 2B: The claim does not recite any additional elements which, alone or in combination, would integrate the abstract ideas into a practical application or which, in combination with the abstract ideas, would be sufficient to amount to significantly more than the abstract ideas. The claim is not patent eligible.
Claim 7 incorporates the rejection of claim 6.
Step 2A Prong 1: The abstract ideas of claim 6 are incorporated. The loss function defines an aggregate loss comprising a modified evidence lower bound (ELBO) loss that is based on an impute loss is a mathematical calculation. Paragraphs [0086]-[0088] disclose the loss functions.
Step 2A Prong 2 and Step 2B: The claim does not recite any additional elements which, alone or in combination, would integrate the abstract ideas into a practical application or which, in combination with the abstract ideas, would be sufficient to amount to significantly more than the abstract ideas. The claim is not patent eligible.
Claim 8 incorporates the rejection of claim 7.
Step 2A Prong 1: The abstract ideas of claim 7 are incorporated. The modified ELBO loss is defined by an expectation operator, a probability distribution, an approximate posterior distribution, the one or more modality-agnostic latent variables, and the one or more modality-specific latent variables is a mathematical calculation. Paragraph [0087] discloses the ELBO loss function.
Step 2A Prong 2 and Step 2B: The claim does not recite any additional elements which, alone or in combination, would integrate the abstract ideas into a practical application or which, in combination with the abstract ideas, would be sufficient to amount to significantly more than the abstract ideas. The claim is not patent eligible.
Claim 9 incorporates the rejection of claim 7.
Step 2A Prong 1: The abstract ideas of claim 7 are incorporated. The impute loss comprises a reward that incentivizes an imputation of a subset of missing modalities using a subset of observed modalities within a training dataset is a mathematical calculation. Paragraph [0088] disclose the impute loss function.
Step 2A Prong 2 and Step 2B: The claim does not recite any additional elements which, alone or in combination, would integrate the abstract ideas into a practical application or which, in combination with the abstract ideas, would be sufficient to amount to significantly more than the abstract ideas. The claim is not patent eligible.
Claim 10 incorporates the rejection of claim 9.
Step 2A Prong 1: The abstract ideas of claim 9 are incorporated. The impute loss is optimized over a plurality of iterations and, at each iteration of the plurality of iterations, the subset of missing modalities is randomly chosen is a mathematical calculation. Paragraph [0088] disclose the impute loss function.
Step 2A Prong 2 and Step 2B: The claim does not recite any additional elements which, alone or in combination, would integrate the abstract ideas into a practical application or which, in combination with the abstract ideas, would be sufficient to amount to significantly more than the abstract ideas. The claim is not patent eligible.
Claim 11 recites a system which implements the same features as the method of claim 1 and is therefore rejected for at least the same reasons.
In Step 2A Prong 2 and Step 2B, a computing system comprising memory and one or more processors communicatively coupled to the memory, the one or more processors configured to [execute the method of claim 1] amounts to generic computer components for applying the abstract ideas on a generic computer under MPEP 2106.05(f). The claim is not patent eligible.
Claims 12-16 each recites a system which implements the same features as the method of claims 3-4 and 6-8, respectively, and are therefore rejected for at least the same reasons.
Claim 17
Step 2A Prong 1: Generating,
One or more modality predictions [being] based on (i) one or more modality-agnostic latent variables that are based on a plurality of data values associated with a plurality of training modalities and (ii) one or more modality-specific latent variables that are based on a plurality of modality observation variables and the one or more modality-agnostic latent variables are mathematical calculations. Paragraph [0087] discloses an ELBO loss function that incentivizes the multi-modal generative machine learning model to encode data into a lower-dimensional space. Encoding data into a lower-dimensional space is a mathematical calculation.
Initiating,
Step 2A Prong 2: Receiving, by one or more processors, an input data record comprising a plurality of input data elements amounts to mere data-gathering, an insignificant pre-solution activity under MPEP 2106.05(g). Processors amount to generic computer components for applying the abstract ideas on a generic computer under MPEP 2106.05(f).
A multi-modal generative machine learning model that is applied to the input data record amounts to mere instructions to apply the abstract ideas on a generic computer under MPEP 2106.05(f).
The additional elements as disclosed above, alone or in combination, do not integrate the abstract ideas into a practical application as they are mere insignificant extra solution activities as disclosed in combination with generic computer functions that are implemented to perform the abstract ideas disclosed above. The claim is directed to an abstract idea.
Step 2B: Receiving, by one or more processors, an input data record comprising a plurality of input data elements is analogous to receiving data over a network, which the courts have recognized as a well-understood, routine, conventional activity under MPEP 2106.05(d)(II).
Processors amount to generic computer components for applying the abstract ideas on a generic computer under MPEP 2106.05(f).
A multi-modal generative machine learning model that is applied to the input data record amounts to mere instructions to apply the abstract ideas on a generic computer under MPEP 2106.05(f).
The additional elements as disclosed above, in combination with the abstract ideas, are not sufficient to amount to significantly more than the abstract ideas as they are well-understood, routine and conventional activities as disclosed in combination with generic computer functions that are implemented to perform the abstract ideas disclosed above. The claim is not patent eligible.
Claims 18-19 each recites a method which implements the same features as the method of claim 4-5, respectively, and are therefore rejected for at least the same reasons.
Claim 20 incorporates the rejection of claim 17.
Step 2A Prong 1: The abstract ideas of claim 17 are incorporated. The multi-modal
Step 2A Prong 2 and Step 2B: The multi-modal generative machine learning model amounts to mere instructions to apply the abstract ideas on a generic computer under MPEP 2106.05(f). The claim is not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3, 6, 11-12, 14, 17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Gonzalez et al. (US 20190135300 A1) in view of Li et al. (US 20210312296 A1).
Regarding claim 1, Gonzalez teaches: A computer-implemented method comprising: receiving, by one or more processors, training data comprising a plurality of data elements that comprise (i) a plurality of data values associated with a plurality of training modalities and (ii) a plurality of modality observation variables that each identify an observation of a training modality for a data element of the plurality of data elements; ([0049] and [0050], lines 1-8, and [0074], lines 6-10, where a plurality of data values corresponds to the plurality of sensor signals, and a plurality of modality observation variables corresponds to the identifiers i = 1, 2, n-1, n.)
generating, by the one or more processors and using a modality-agnostic latent variable encoder of a multi-modal generative machine learning model, one or more modality-agnostic latent variables based on the plurality of data values; ([0045], [0053] on page 7, col. 2, lines 1-9, [0054], and Fig. 7 discloses that a global encoder sub-net 703 encodes multiple sensor signals into a contextual fused sensor data representation 716. A “modality-agnostic latent variable” is the contextual fused sensor data representation 716. It is a latent variable since this data is represented in a lower dimensional space than the sensor signals.)
generating, by the one or more processors and using a modality-specific
generating, by the one or more processors and using a loss function, a loss for the multi-modal generative machine learning model based on the one or more modality-agnostic latent variables and the one or more modality-specific
initiating, by the one or more processors, the performance of one or more training operations based on the loss. ([0046], [0069]-[0071] discloses training the global encoder sub-net 703 and decoder sub-nets 705a-d in an end-to-end manner.)
However, Gonzalez does not explicitly teach: generating, … using a modality-specific latent variable encoder of the multi-modal generative machine learning model, one or more modality-specific latent variables
generating a loss based on the one or more modality-specific latent variables
But Li teaches: generating, … using a modality-specific latent variable encoder of the
generating a loss based on the one or more modality-specific latent variables ([0034]-[0035])
Li teaches applying a loss function on latent variables. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have applied Li’s contrastive loss function on latent variables after Gonzalez’s decoder sub-net 705a. A motivation for the combination is to compare the similarity of a pair of data signals. (Li, [0034])
Regarding claim 2, the combination of Gonzalez and Li teaches: The computer-implemented method of claim 1,
Gonzalez teaches: wherein the multi-modal generative machine learning model comprises a
However, Gonzalez does not explicitly teach: a variational autoencoder architecture.
But Li teaches: a variational autoencoder architecture. ([0037])
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated Li’s variational encoder-decoder structure into the combination of Gonzalez and Li. A motivation for the combination is to approximate inference in a latent Gaussian model where the approximate posterior and model likelihood are parametrized by neural networks. (Gonzalez, [0037])
Regarding claim 3, the combination of Gonzalez and Li teaches: The computer-implemented method of claim 1 further comprising:
Gonzalez teaches: receiving, using the multi-modal generative machine learning model, an input data record comprising a plurality of input data elements; ([0050], lines 1-8)
generating, using the multi-modal generative machine learning model, one or more modality predictions based on the one or more modality-specific more modality-agnostic latent variables; and ([0053] on page 7, col. 2, lines 1-5, [0054], [0056], lines 1-15 discloses reconstructing input sensor data, which are modality predictions.)
initiating the performance of one or more prediction-based actions based on the one or more modality predictions. ([0072], [0083] and [0084], lines 1-3 disclose generating an anomaly notification based on a deviation between input sensor data and reconstructed input sensor data.)
However, Gonzalez does not explicitly teach: the one or more modality-specific latent variables
But Li teaches: the one or more modality-specific latent variables ([0015], [0027], lines 1-10, all of [0033])
A motivation for the combination is the same as the motivation given for claim 1.
Regarding claim 6, the combination of Gonzalez and Li teaches: The computer-implemented method of claim 1,
Gonzalez teaches: wherein the one or more training operations comprises optimizing the loss using the loss function. ([0044, lines 12-16, [0046], [0069]-[0071] discloses using an engineered cost function to train global encoder sub-net 703 and decoder sub-nets 705a-d in an end-to-end manner. Optimizing the loss means minimizing a difference between collected sensor data and reconstructed sensor data.)
Claim 11 recites a system which implements the same features as the method of claim 1 and is therefore rejected for at least the same reasons.
Gonzalez teaches: A computing system comprising memory and one or more processors communicatively coupled to the memory, the one or more processors configured to [execute the method of claim 1] ([0089], lines 1-5 and [0095])
Claims 12 and 14 each recites a system which implements the same features as the method of claims 3 and 6, respectively, and are therefore rejected for at least the same reasons.
Regarding claim 17, Gonzalez teaches: A computer-implemented method comprising: receiving, by one or more processors, an input data record comprising a plurality of input data elements; ([0050], lines 1-8 and [0074], lines 6-10, where a plurality of input data elements are a plurality of sensor data)
generating, by the one or more processors and via a multi-modal generative machine learning model that is applied to the input data record, one or more modality predictions based on (i) one or more modality-agnostic latent variables that are based on a plurality of data values associated with a plurality of training modalities and (ii) one or more modality-specific “modality prediction” and “modality-specific variables” are reconstructed input sensor data. The decoder sub-nets 705a-d transform (encode) the contextual fused sensor data representation 716 into reconstructed input sensor data for each modality. Reconstructing input sensor data is based on the identifiers i = 1, 2, n-1, n because each decoder sub-net corresponds to a particular type of sensor data.)
initiating, by the one or more processors, the performance of one or more prediction-based actions based on the one or more modality predictions. ([0072], [0083] and [0084], lines 1-3 disclose generating an anomaly notification based on a deviation between input sensor data and reconstructed input sensor data.)
However, Gonzalez does not explicitly teach: one or more modality-specific latent variables
But Li teaches: generating, [via a] machine learning model that is applied to the input data record, one or more modality predictions based on… one or more modality-specific latent variables ([0015], [0027], lines 1-10, all of [0033] and [0038], lines 1-6. A “modality-specific latent variable” is a first emotion latent vector 133-1 generated by encoder 132-1, which is specific to one modality, EEG signals. A modality prediction includes a decoder reconstructing corresponding individual and emotion latent vectors to output reconstructed data.)
Li teaches applying a loss function on latent variables, and Li’s signal pairs S1 and S2 (disclosed at [0045], lines 7-11) are analogous to Gonzalez’s contextual fused sensor data representation 716. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have applied Li’s encoder 132-1 after Gonzalez’s global encoder sub-net 703, and to have applied Li’s contrastive loss function on latent variables after Gonzalez’s decoder sub-net 705a. A motivation for the combination is to separate subject-dependent and subject-independent factors from sensor data. This provides a solution to bias in subsequent classification and allows the classification to be more precise and accurate. (Li, [0015], [0018])
Regarding claim 20, the combination of Gonzalez and Li teaches: The computer-implemented method of claim 17,
Gonzalez teaches: wherein the multi-modal generative machine learning model is trained by optimizing a loss using a loss function that is based on an impute loss. (Examiner treats an impute loss as any loss based on a prediction. [0046], lines 18-19 and [0069]-[0071] discloses using an engineered cost function to train global encoder sub-net 703 and decoder sub-nets 705a-d in an end-to-end manner. The outputs of the decoder sub-nets are imputations and predictions.)
Claims 4-5, 13, and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Gonzalez et al. (US 20190135300 A1) in view of Li et al. (US 20210312296 A1) and Choi et al. (US 20190276041 A1).
Regarding claim 4, the combination of Gonzalez and Li teaches: The computer-implemented method of claim 3,
However, Gonzalez and Li do not explicitly teach: wherein an input data element of the plurality of input data elements comprises a missing modality element and the one or more modality predictions comprise a synthetic modality element imputed for the missing modality element.
But Choi teaches: wherein an input data element of the plurality of input data elements comprises a missing modality element and the one or more modality predictions comprise a synthetic modality element imputed for the missing modality element. ([0005], [0036]-[0037], [0087] and Fig. 5 discloses a multi-modal deep auto-encoder (MMDAE) structure for predicting an output of a missing input modality. In Fig. 5, the plurality of input data elements include “RGB image,” “LIDAR sensor data,” and “Ultrasound sensory data”. The LIDAR sensor data is a missing modality element, and a predicted output for this data is a synthetic modality element.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated Choi’s technique of imputing a missing input modality to a multi-modal auto-encoder into the combination of Gonzalez and Li. A motivation for the combination is to enable an autonomous vehicle equipped with sensors to continue operating after a sensor has failed. (Choi, [0007]-[0010])
Regarding claim 5, the combination of Gonzalez and Li teaches: The computer-implemented method of claim 3,
However, Gonzalez and Li do not explicitly teach: wherein the one or more modality predictions comprise one or more synthetic data records, each comprising a plurality of synthetic modality elements.
But Choi teaches: wherein the one or more modality predictions comprise one or more synthetic data records, each comprising a [0036]-[0037], [0087] and Fig. 5 discloses a multi-modal deep auto-encoder (MMDAE) structure for predicting an output (i.e., a synthetic reconstruction) of a missing input modality, “LIDAR sensory data”. In Fig. 5, a synthetic data record is a combination of reconstructed/output values of the RGB image, LIDAR sensor data, and Ultrasound sensory data because it contains the synthetic reconstruction for LIDR sensor data, which is also a synthetic modality element.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated Choi’s technique of imputing a missing input modality to a multi-modal auto-encoder into the combination of Gonzalez and Li. It would have been further obvious to have imputed two different missing input modalities at the same time to generate “a plurality of synthetic modality elements” as claimed. A motivation for the combination is to enable an autonomous vehicle equipped with sensors to continue operating after a sensor has failed. (Choi, [0007]-[0010])
Claim 13 recites a system which implements the same features as the method of claim 4.
Claims 18-19 each recites a system which implements the same features as the method of claims 4-5, respectively, and are therefore rejected for at least the same reasons.
Claims 7-9 and 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Gonzalez et al. (US 20190135300 A1) in view of Li et al. (US 20210312296 A1) and Zhang et al. (US 20210406765 A1).
Regarding claim 7, the combination of Gonzalez and Li teaches: The computer-implemented method of claim 6,
However, Gonzalez and Li do not explicitly teach: wherein the loss function defines an aggregate loss comprising a modified evidence lower bound (ELBO) loss that is based on an impute loss.
But Zhang teaches: wherein the loss function defines an aggregate loss comprising a modified evidence lower bound (ELBO) loss that is based on an impute loss. ([0136], [0137], lines 1-3, [0144], and [0145], lines 1-4 discloses maximizing variational lower bound (ELBO) loss. The formula for ELBO is an “aggregate loss” because the argument of Eq_φ is a summation of terms. ELBO is also based on an impute loss because the pre-training learns to impute unobserved features based on the term log pθ (xtu|zt).)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated Zhang’s ELBO loss into the combination of Gonzalez and Li. A motivation for the combination is that an ELBO function is one strategy to train an autoencoder while imputing missing features. (Zhang, [0067], [0072], [0145])
Regarding claim 8, the combination of Gonzalez, Li, and Zhang teaches: The computer-implemented method of claim 7,
Gonzalez teaches: wherein the ([0046], lines 18-19 and [0069]-[0071] discloses using an engineered cost function to train global encoder sub-net 703 (“modality-agnostic latent variables”) and decoder sub-nets 705a-d (“modality specific variables”) in an end-to-end manner.)
However, Gonzalez does not explicitly teach: wherein the modified ELBO loss is defined by an expectation operator, a probability distribution, an approximate posterior distribution, the one or more modality-agnostic latent variables, and the one or more modality-specific latent variables.
But Li teaches: wherein the
A motivation for the combination is the same as the motivation given for claim 1.
However, Gonzalez and Li do not explicitly teach: wherein the modified ELBO loss is defined by an expectation operator, a probability distribution, an approximate posterior distribution,
But Zhang teaches: wherein the modified ELBO loss is defined by an expectation operator, a probability distribution, an approximate posterior distribution, ([0071], lines 5-13, 17-18 discloses p(x|z) is a probability distribution and q(z|x) is an approximate posterior distribution. Note that “p(z|x)” disclosed in lines 17-18 is assumed to be a typo for “p(x|z)”. [0131], lines 4-6 disclose that “E” represents an expectation. In [0144], the ELBO loss is defined by an Eq_φ, p(x|z) and q(z|x).)
It would have been obvious to a person having ordinary skill in the art before the effective filing date for the claimed invention to have incorporated Zhang’s ELBO loss into the combination of Gonzalez and Li. A motivation for the combination is the same as the motivation given for claim 7.
Regarding claim 9, the combination of Gonzalez, Li, and Zhang teaches: The computer-implemented method of claim 7,
However, Gonzalez and Li do not explicitly teach: wherein the impute loss comprises a reward that incentivizes an imputation of a subset of missing modalities using a subset of observed modalities within a training dataset.
But Zhang teaches: wherein the impute loss comprises a reward that incentivizes an imputation of a subset of missing modalities using a subset of observed modalities within a training dataset. (A subset of missing modalities are missing values within a modality, and a subset of observed modalities are observed values within a modality. [0163], lines 4-8 discloses four different measurement modalities. Based on [0072], [0073], lines 1-13, [0144]-[0145], maximizing the variational lower-bound uses a gradient that which is a reward for converging and learning missing values.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date for the claimed invention to have incorporated Zhang’s ELBO loss into the combination of Gonzalez and Li. A motivation for the combination is the same as the motivation given for claim 7.
Claims 15-16 each recites a system which implements the same features as the method of claims 7-8, respectively, and are therefore rejected for at least the same reasons.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Gonzalez et al. (US 20190135300 A1) in view of Li et al. (US 20210312296 A1), Zhang et al. (US 20210406765 A1), and Sun et al. (US 20240045994 A1).
Regarding claim 10, the combination of Gonzalez, Li, and Zhang teaches: The computer-implemented method of claim 9,
However, Gonzalez and Li do not explicitly teach: wherein the impute loss is optimized over a plurality of iterations and, at each iteration of the plurality of iterations, the subset of missing modalities is randomly chosen.
But Zhang teaches: wherein the impute loss is optimized over a plurality of iterations and, at each iteration of the plurality of iterations, the subset of [data points is]
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have applied Zhang’s number of learning steps to the pre-training task of maximizing the variational lower-bound. A motivation for the combination is the same as the motivation given for claim 7.
However, Gonzalez, Li, and Zhang do not explicitly teach: the subset of missing modalities is randomly chosen.
But Sun teaches: each iteration of the plurality of iterations, the subset of missing modalities is randomly chosen. ([0081], [0084], [0149]-[0150])
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have applied Sun’s technique of masking a random modality in each iteration of training to the combination of Gonzalez, Li, and Zhang. A motivation for the combination is to train the model to recover any input modality that might become corrupted. (Sun, [0081])
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Asher H. Jablon whose telephone number is (571)270-7648. The examiner can normally be reached Monday - Friday, 9:00 am - 6:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Al Kawsar can be reached at (571)270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.H.J./Examiner, Art Unit 2127
/JEREMY L STANLEY/Examiner, Art Unit 2127
Prosecution Insights