DETAILED ACTION
This final action is in response to the amendment and remarks filed on 09/22/2026 for application 18/477,359.
Claims 1, 3-5, 8, and 10-20 have been amended.
Claims 1-20 thereby remain pending in the application. Claims 1, 10, and 15 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-10, and 12-16. and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Poulain et al. (“Transformer-based Multi-target Regression on Electronic Health Records for Primordial Prevention of Cardiovascular Disease”, available conference 2021), hereinafter Poulain, in view of Ren et al. (“Generative Adversarial Networks Enhanced Pre-training for Insufficient Electronic Health Records Modeling”, published August 2022), hereinafter Ren, and Li et al. (“BEHRT: Transformer for Electronic Health Records”, published 28 Apr 2020), hereinafter Li.
Regarding claim 1, Poulain teaches A computer-implemented method (“In this study, we propose a multi-target regression model leveraging transformers to learn the bidirectional representations of EHR data and predict the future values of 11 major modifiable risk factors of cardiovascular disease (CVD). Inspired by the proven results of pre-training in natural language processing studies, we apply the same principles on EHR data, dividing the training of our model into two phases: pre-training and fine-tuning. We use the finetuned transformer model in a “multi-target regression” theme. Following this theme, we combine the 11 disjoint prediction tasks by adding shared and target-specific layers to the model and jointly train the entire model. We evaluate the performance of our proposed method on a large publicly available EHR dataset. Through various experiments, we demonstrate that the proposed method obtains a significant improvement (12.6% MAE on average across all 11 different outputs) over the baselines” [Poulain Abstract]) comprising:
storing, by one or more processors, a set of one or more initialized weights in association with one or more layers of a machine learning model, (“We pre-trained our model using the same approach as BEHRT, itself based on BERT using the masked language modeling (MLM) approach, where a percentage of the input tokens are masked at random and then predicted by the model. Following this approach, during the tokenization process of the input sequences, we randomly selected 15% of the medical codes to be predicted by the model. We then replaced 80% of these codes (12% of all observed medical codes) with the MASK token, 10% (1.5% of all observed medical codes) with another randomly picked medical code, and left the remaining 10% (1.5% of all observed medical codes) of the codes unchanged. This process adds noise to the input data, forcing the model to fight through the noise during the pre-training process. The modified input tokens will then be embedded and fed into the transformer layers and the output of these layers will be used to perform the MLM prediction and be trained using the cross-entropy loss…Following a transfer-learning approach, after the pre-training process, we initialized the embeddings and the transformer’s weights of the fine-tuned model with the pre-trained ones, allowing the model to use the knowledge acquired during the MLM process” [Poulain page 729 Pre-training]) wherein:
(i) the set of one or more initialized weights is generated based on a set of one or more training embeddings, (see Fig. 1 – “EHR data representation for a fictional patient with three total visits. The CLS, SEP, and END tokens denote the beginning of the patient’s record, the separation between two visits, and the end of the record, respectively. For the input embeddings, C1, C2, and C3 refer to the first, second, and third conditions of the vocabulary…The sex, race, and ethnicity embeddings are numerical values that refer to their respective categories as described in our GitHub. The segment embeddings alternate between 0 and 1 from one visit to the other and the position embeddings denotes the visit number each token belongs to. The seven different embeddings are then summed up to create the embeddings output sequence before being fed into the model as the input to serve as the full representation of a patient’s record” [Poulain page 728])
(ii) the set of one or more initialized weights is pre-trained based on a pre-training dataset comprising a set of one or more training temporal sequence of encounters data records including an extra-record encounter data element, ((“We pre-trained our model using the same approach as BEHRT, itself based on BERT using the masked-language modeling (MLM) approach, where a percentage of the input tokens are masked at random and then predicted by the model. Following this approach, during the tokenization process of the input sequences, we randomly selected 15% of the medical codes to be predicted by the model. We then replaced 80% of these codes (12% of all observed medical codes) with the MASK token, 10% (1.5% of all observed medical codes) with another randomly picked medical code, and left the remaining 10% (1.5% of all observed medical codes) of the codes unchanged. This process adds noise to the input data, forcing the model to fight through the noise during the pre-training process. The modified input tokens will then be embedded and fed into the transformer layers and the output of these layers will be used to perform the MLM prediction and be trained using the cross-entropy loss” [Poulain page 729 Pre-training]; A portion of input tokens are replaced with a MASK token (i.e. marked as extra-record data element) and then predicted as a pre-training task) and
(iii) the set of one or more training embeddings comprises a descriptive embedding, a sequential ordering embedding, an age/time embedding, a locale embedding, or an encounter number embedding (“see Fig. 1 – “EHR data representation for a fictional patient with three total visits. The CLS, SEP, and END tokens denote the beginning of the patient’s record, the separation between two visits, and the end of the record, respectively. For the input embeddings, C1, C2, and C3 refer to the first, second, and third conditions of the vocabulary…The sex, race, and ethnicity embeddings are numerical values that refer to their respective categories as described in our GitHub. The segment embeddings alternate between 0 and 1 from one visit to the other and the position embeddings denotes the visit number each token belongs to” [Poulain page 728])
generating, by the one or more processors and based on the extra-record encounter data element a set of one or more fine-tuned weights, wherein the set of one or more fine-tuned weights are generated by updating at least a portion of the set of one or more initialized weights using a fine-tuning dataset associated with a target classification (“We use the finetuned transformer model in a “multi-target regression” theme. Following this theme, we combine the 11 disjoint prediction tasks by adding shared and target-specific layers to the model and jointly train the entire model” [Poulain Abstract]; “As shown in the DeepMTR paper, having layers with shared parameters composed of maxout units [24] and target-specific layers increases the overall performance of the MTR task. We have, therefore, added three shared linear layers composed of maxout units with a parameter k (number of hidden units) of 5 and three target-specific linear layers for each of our 11 target outputs. A schematic representation of the model’s architecture is shown in Fig. 3, where, during the fine-tuning process, the pooled output (extraction of the CLS token representation) of transformers’ layers is used as input to the model’s first shared layer. The shared layers are processed sequentially, and the output of the last layer is fed into Ng (11 in our case) targetspecific sub-networks… We compute the loss for every target output by multiplying the squared error of each patient with its corresponding masking vector mg to only account for the non-missing values for each target output. We divide the total loss for each target output by the total number of non-missing values that are recorded among all patients for that target output. We then define the overall loss as the average loss across every target output” [Poulain page 729 Fine-tuning]; Shared layers and task-specific layers are added, and the entire model architecture is then jointly trained (i.e., weights are updated, including that of the transformer model) to be task-aware based on calculated prediction loss for target outputs (i.e., target classification))
providing, by the one or more processors, [an] input temporal sequence of encounters data record to the machine learning model to produce a prediction score for [a] prediction encounter data element associated with the target classification (“We used the fine-tuning process of the model to predict the 11 CVD risk factors through MTR… To account for the fact that some target outputs are more prevalent in the dataset than others (the data imbalance), we defined our fine-tuning loss function as the average mean squared error (MSE) between the ground truth yg and the predicted value ^ yg of every target output g as shown in Eq. 2” [Poulain page 729 Fine-tuning]; Within the target output, each predicted value ^yg represents a predicted score by the model for the corresponding risk factor (see also ^y1…^yNg as output of Target Specific Layers in Fig. 3 [Poulain page 729]); see also Embeddings Output Sequence in Fig. 3 [Poulain page 729]).
However, Poulain does not expressly teach the training temporal sequence of encounters data records including a re-ordered encounter data element during the pre-training phase.
In the same field of endeavor, Ren teaches a means of leveraging electronic health records data for disease prediction through a transformer-based machine learning model (“In recent years, automatic computational systems based on deep learning are widely used in medical fields, such as automatic diagnosing and disease prediction. Most of these systems are designed for data sufficient scenarios… Many data augmentation methods have been proposed to solve the data insufficiency problem, such as using GAN (Generative Adversarial Networks) to generate training data. However, the augmented data usually contains lots of noise. Directly using them to train sensitive medical models is very difficult to achieve satisfactory results. To overcome this problem, we propose a novel deep model learning method for insufficient EHR (Electronic Health Record) data modeling, namely GRACE, which stands GeneRative Adversarial networks enhanCed prE-training. In the method, we propose an item-relation-aware GAN to capture changing trends and correlations among data for generating high-quality EHR records. Furthermore, we design a pre-training mechanism consisting of a masked records prediction task and a real-fake contrastive learning task to learn representations for EHR data using both generated and real data. After the pre-training, only the representations of real data is used to train the final prediction model” [Ren Abstract]) wherein the training temporal sequence of encounters data records includ[es] a re-ordered encounter data element during the pre-training phase (“The generator aims to generate fake visit sequences from random vectors. Since the target are visit sequences, our model first generates a hidden state vector ˜𝒔𝜏 from a random vector 𝒛 as
PNG
media_image1.png
37
319
media_image1.png
Greyscale
, where
PNG
media_image2.png
34
143
media_image2.png
Greyscale
and
PNG
media_image3.png
30
102
media_image3.png
Greyscale
are learnable parameters, p(z) denotes a random distribution (i.e., Gaussian distribution). Then, the GAN model uses a Transformer decoder to decode the hidden state vector as a visit sequences, i.e.,
PNG
media_image4.png
33
486
media_image4.png
Greyscale
… Since the EHR data is very complicated and contains rich semantic information, if we direct use random parameters to initialize the Transformer decoder, it is very hard to accomplish the sequence generation task. To overcome this issue, we design an AutoEncoder initialization mechanism to set the parameter of the Transformer decoder in Eq. (3). The AutoEncoder contains an encoder and a decoder. The encoder is responsible for encoding a real visit sequence data as a dense vector… Once the AutoEncoder has been trained, we initialize the TansformerDecoder in Eq. (3) with the TansformerDecoder of the AutoEncoder (in Eq. (6)). In this way, the data characteristics of the real visit sequence are memorized by the TansformerDecoder, and therefore it can be used to generate higher quality fake data” [Ren page 3812 EHR Generator]; “Although the item-relation-aware GAN can effectively capture the characteristics of EHR data and generate reasonable data, there is also a gap between the fake data and the real data. We believe a representation that can distinguish real and fake samples is more effective for real data modeling. Based on this insight, we design a real-fake contrastive task to train the representation learning model. Specifically, for a fake sample in a mini-batch, we use the representations of sequences from the fake data as positive samples and the rest representations as negative samples. We employ a contrastive loss function, called InfoNCE [31], to classify positive and negative samples” [Ren page 3813 Pre-training task #2: Real-Fake Contrastive Learning]; Because the parameters of the TransformerDecoder are learned directly based on structural patterns of real visit sequences, the fake visit sequences generated by the GAN model are broadly interpretable as a re-ordering of real visit sequence patterns via transformation of noise vector z through learned latent distribution parameters. The pre-training task then attempts to detect the fake visit sequences (i.e., re-ordered encounter data elements) in the overall mini-batch of samples).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the training temporal sequence of encounters data records including a re-ordered encounter data element during the pre-training phase as taught by Ren into Poulain because they are both directed towards leveraging electronic health records data for disease prediction through transformer-based machine learning models. Incorporating the teachings of Ren would enable the resulting model to better handle insufficient / low sample data records and avoid overfitting [Ren Abstract].
However, the combination does not expressly teach improving predictive accuracy of the machine learning model in predicting an inclusion of a prediction encounter data element in an input temporal sequence of encounters data record at a future time, and produc[ing] a prediction score for the prediction encounter data element associated with the target classification being included in the input temporal sequence of encounters data record at the future time.
In the same field of endeavor, Li teaches a means of leveraging electronic health records data for disease prediction through a transformer-based machine learning model (“The latest developments in machine learning (including deep learning) provides a great opportunity to address this unmet need. In this study, we introduce BEHRT: A deep neural sequence transduction model for electronic health records (EHR), capable of simultaneously predicting the likelihood of 301 conditions in one’s future visits. When trained and evaluated on the data from nearly 1.6 million individuals, BEHRT shows a striking improvement of 8.0–13.2% (in terms of average precision scores for different tasks), over the existing state-of-the-art deep EHR models” [Li Abstract]) that improv[es] predictive accuracy of the machine learning model in predicting an inclusion of a prediction encounter data element in an input temporal sequence of encounters data record at a future time, (“We demonstrated this property of the model by training and testing it on CPRD - one of the largest linked primary care EHR systems – for predicting the next mostly likely diseases in one’s future visits. Based on our results, BEHRT outperformed the best deep EHR models in the literature by more than ~8% (absolute improvement) in predicting for a range of more than 300 diseases” [Li page 9 Discussion]) and produc[es] a prediction score for the prediction encounter data element associated with the target classification being included in the input temporal sequence of encounters data record at the future time (“In this study, we aim to use a given patient’s past EHR to predict his/her future diagnoses (if any), as a multi-label classification problem (i.e., simultaneously predicting a probability for each and every disease); this will result in a single predictive model that scales across a range of diseases (as opposed to needing to train one predictive model per disease)” [Li page 3 BEHRT: A Transformer-based Model for EHR]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated improving predictive accuracy of the machine learning model in predicting an inclusion of a prediction encounter data element in an input temporal sequence of encounters data record at a future time, and produc[ing] a prediction score for the prediction encounter data element associated with the target classification being included in the input temporal sequence of encounters data record at the future time as taught by Li into the combination because both Poulain and Li are directed towards leveraging electronic health records data for disease prediction through a transformer-based machine learning model. Given that Poulain expressly discusses the BEHRT model of Li and utilizes a synonymous pre-training task (“We pre-trained our model using the same approach as BEHRT, itself based on BERT using the masked-language modeling (MLM) approach” [Poulain page 4 Pre-training]), one of ordinary skill in the art would recognize the value of incorporating the downstream future-condition prediction task as taught by Li to thereby enable prediction of conditions in future patient visits, facilitating earlier intervention and disease management [Li Abstract].
Regarding claim 2, the combination of Poulain, Ren, and Li teaches the limitations of parent claim 1, and Poulain further teaches wherein the machine learning model comprises a transformer machine learning model architecture (“First, we present a transformer-based model modifying the original BERT architecture that receives EHR data to generate representations of the data. Second, we propose a novel architecture that connects the output of our transformer model to a DNN for multi-target regression to allow our model to accurately predict the value of the targeted medical measurements” [Poulain pages 726-727 Introduction]).
Regarding claim 3, the combination of Poulain, Ren, and Li teaches the limitations of parent claim 1, and Poulain further teaches wherein the machine learning model comprises a plurality of transformer layers, wherein each of the plurality of transformer layers comprises self-attention and a feedforward network (see Bidirectional Transformer Layers in Fig. 3 [Poulain page 729]; “Among the most successful examples of related studies reconciling natural language processing (NLP) and EHR data are the attention-based models [14], originally designed for the NLP tasks, that have been shown very effective at capturing the longitudinal aspects of EHR data…Li et al. extended BERT [13] to EHR data by formulating longitudinal EHR data as an NLP task to predict the diagnoses at a given visit [11]. They named the new architecture, BEHRT (referring to BERT + EHR)” [Poulain page 727 Related Work]; “We refer the reader to the BERT [13] and BEHRT [11] papers for additional information about the transformer’s architecture and the pretraining process” [Poulain page 729 Pre-training]; Standard transformer architectures (e.g., BERT/BEHRT) implicitly consist of layers comprising self-attention and feedforward network components (see also Alammar, “The Illustrated Transformer” [pages 1-5]))
Regarding claim 5, the combination of Poulain, Ren, and Li teaches the limitations of parent claim 1, and Poulain further teaches wherein the set of one or more initialized weights are further generated by pre-training the set of one or more initialized weights by:
determining one or more extra-record encounter data elements in one or more first training temporal sequence of encounters data records from the pre-training dataset (“We pre-trained our model using the same approach as BEHRT, itself based on BERT using the masked-language modeling (MLM) approach, where a percentage of the input tokens are masked at random and then predicted by the model. Following this approach, during the tokenization process of the input sequences, we randomly selected 15% of the medical codes to be predicted by the model. We then replaced 80% of these codes (12% of all observed medical codes) with the MASK token, 10% (1.5% of all observed medical codes) with another randomly picked medical code, and left the remaining 10% (1.5% of all observed medical codes) of the codes unchanged. This process adds noise to the input data, forcing the model to fight through the noise during the pre-training process. The modified input tokens will then be embedded and fed into the transformer layers and the output of these layers will be used to perform the MLM prediction and be trained using the cross-entropy loss” [Poulain page 729 Pre-training]; A portion of input tokens are replaced with a MASK token (i.e. marked as extra-record data element) and then predicted as a pre-training task).
Ren further teaches determining one or more re-ordered encounter data elements in one or more second training temporal sequence of encounters data records from the pre-training dataset (“The generator aims to generate fake visit sequences from random vectors. Since the target are visit sequences, our model first generates a hidden state vector ˜𝒔𝜏 from a random vector 𝒛 as
PNG
media_image1.png
37
319
media_image1.png
Greyscale
, where
PNG
media_image2.png
34
143
media_image2.png
Greyscale
and
PNG
media_image3.png
30
102
media_image3.png
Greyscale
are learnable parameters, p(z) denotes a random distribution (i.e., Gaussian distribution). Then, the GAN model uses a Transformer decoder to decode the hidden state vector as a visit sequences, i.e.,
PNG
media_image4.png
33
486
media_image4.png
Greyscale
… Since the EHR data is very complicated and contains rich semantic information, if we direct use random parameters to initialize the Transformer decoder, it is very hard to accomplish the sequence generation task. To overcome this issue, we design an AutoEncoder initialization mechanism to set the parameter of the Transformer decoder in Eq. (3). The AutoEncoder contains an encoder and a decoder. The encoder is responsible for encoding a real visit sequence data as a dense vector… Once the AutoEncoder has been trained, we initialize the TansformerDecoder in Eq. (3) with the TansformerDecoder of the AutoEncoder (in Eq. (6)). In this way, the data characteristics of the real visit sequence are memorized by the TansformerDecoder, and therefore it can be used to generate higher quality fake data” [Ren page 3812 EHR Generator]; “Although the item-relation-aware GAN can effectively capture the characteristics of EHR data and generate reasonable data, there is also a gap between the fake data and the real data. We believe a representation that can distinguish real and fake samples is more effective for real data modeling. Based on this insight, we design a real-fake contrastive task to train the representation learning model. Specifically, for a fake sample in a mini-batch, we use the representations of sequences from the fake data as positive samples and the rest representations as negative samples. We employ a contrastive loss function, called InfoNCE [31], to classify positive and negative samples” [Ren page 3813 Pre-training task #2: Real-Fake Contrastive Learning]; Because the parameters of the TransformerDecoder are learned directly based on structural patterns of real visit sequences, the fake visit sequences generated by the GAN model are broadly interpretable as a re-ordering of real visit sequence patterns via transformation of noise vector z through learned latent distribution parameters. The pre-training task then attempts to detect the fake visit sequences (i.e., re-ordered encounter data elements) in the overall mini-batch of samples).
Regarding claim 6, the combination of Poulain, Ren, and Li teaches the limitations of parent claim 5, and Poulain further teaches wherein determining the one or more extra-record encounter data elements further comprises:
replacing a training encounter data element in at least one of the one or more first training temporal sequence of encounters data records with a substitute training encounter data element; (“We then replaced 80% of these codes (12% of all observed medical codes) with the MASK token, 10% (1.5% of all observed medical codes) with another randomly picked medical code, and left the remaining 10% (1.5% of all observed medical codes) of the codes unchanged” [Poulain page 729 Pre-training]) and
determining, using the machine learning model, the substitute training encounter data element as an extra-record encounter data element. ([Poulain page 729 Pre-training] as detailed in claim 5 above; The MLM prediction task attempts to predict the input tokens (i.e., extra-record elements) that were replaced with the substitute tokens).
Regarding claim 7, the combination of Poulain, Ren, and Li teaches the limitations of parent claim 5, and Ren further teaches wherein determining one or more re-ordered encounter data elements further comprises:
re-arranging one or more training encounter data elements in at least one of the one or more second training temporal sequence of encounters data records; ([Ren page 3812 EHR Generator] as detailed in claim 5 above; Because the parameters of the TransformerDecoder are learned directly based on structural patterns of real visit sequences, the fake visit sequences generated by the GAN model are broadly interpretable as a re-ordering of real visit sequence patterns via transformation of noise vector z through learned latent distribution parameters) and
determining, using the machine learning model, the one or more training encounter data elements have been re-arranged. ([Ren page 3813 Pre-training task #2: Real-Fake Contrastive Learning] as detailed in claim 5 above; The pre-training task then attempts to detect the fake visit sequences (i.e., re-ordered encounter data elements) in the overall mini-batch of samples)
Regarding claim 8, the combination of Poulain, Ren, and Li teaches the limitations of parent claim 1, and Poulain further teaches wherein the input temporal sequence of encounters data record, the pre-training dataset, or the fine-tuning dataset comprises structured data from one or more electronic data records (“In this study, we used the EHR portion of the All of Us Research Program [21], which is a publicly available dataset collected from data donations of over one million adults (18 yr+) participants in the US1…Our final cohort included 6,993 patients (1,498 males and 5,495 females). For these patients, we extracted the demographics, conditions, prescriptions, and laboratory measurements from the original dataset” [Poulain page 727 Materials and Methods])
Regarding claim 9, the combination of Poulain, Ren, and Li teaches the limitations of parent claim 1, and Poulain further teaches wherein the pre-training dataset comprises one or more of codes, code types, temporal information, location type information, sequential ordering information, or utilization information (“To reduce the number of condition and prescription codes, we grouped them using the “IsA” relationship from the concept relationship table. This process reduced the number of codes from 15,375 to 574 and 14,118 to 495, for the conditions and prescriptions, respectively” [Poulain page 727 Materials and Methods])
Regarding claims 10 and 12-14, they are apparatus claims that substantially corresponds to the method of claims 1 and 5-7, which are already taught by Poulain, Ren, and Li as detailed above. Poulain further teaches A computing system comprising memory and one or more processors communicatively coupled to the memory, the one or more processors configured to perform the claimed functions (“We evaluate the performance of our proposed method on a large publicly available EHR dataset. Through various experiments, we demonstrate that the proposed method obtains a significant improvement (12.6% MAE on average across all 11 different outputs) over the baselines”) [Poulain Abstract]; Evaluation of the proposed method on a large dataset implicitly requires a computer (i.e., processor coupled to memory) with adequate processing power for execution of model training and evaluation). Consequently, claim 10 is rejected for the same reasons as claims 1 and 5-7.
Regarding claims 15, 16, and 18-20, they are product claims that substantially correspond to the method of claims 1, 3, and 5-7, which are already taught by Poulain, Ren, and Li as detailed above. Poulain further teaches One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processors, cause the one or more processors to: perform the claimed functions (“We evaluate the performance of our proposed method on a large publicly available EHR dataset. Through various experiments, we demonstrate that the proposed method obtains a significant improvement (12.6% MAE on average across all 11 different outputs) over the baselines” [Poulain Abstract]; Evaluation of the proposed method on a large dataset implicitly requires memory (i.e., non-transitory computer-readable storage media) coupled to a processor with adequate power for execution of model training and evaluation). Consequently, claims 15 and 16 are rejected for the same reasons as claims 1, 3, and 5-7.
Claims 4, 11, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Poulain, Ren, and Li, as applied to claims 1, 10, and 15 above, further in view of Solberg et al. (“Detection of Outliers in Reference Distributions: Performance of Horn’s Algorithm”, published 2005) hereinafter Solberg.
Regarding claim 4, the combination of Poulain, Ren, and Li teaches the limitations of parent claim 1, and Poulain further teaches tokenizing at least a portion of the pre-training dataset by:
determining a plurality of deciles based on a plurality of scores; (“We formatted our data by following a similar method used in other transformer architectures for EHR data and define the entire record of each patient as a document, the visits as sentences, and prescriptions, measurements, and conditions as words. For the measurements, we discretized the values of each variable into deciles” [Poulain page 728 Materials and Methods]; For measurements in the dataset, the range of values (i.e., scores) is broken up into deciles)
determining a plurality of maximum decile values associated with the plurality of deciles; ([Poulain page 728 Materials and Methods] as detailed above; Determining deciles for a range of measurements inherently requires determination of a minimum and maximum measurement value for each decile) and
assigning one or more pre-training score feature values to one or more of the plurality of deciles based on the plurality of maximum decile values (“For example, an SBP value between the 3rd and the 4th decile would be transformed into the word SBP_30. This process created a measurement vocabulary M of size 10 _ Nm, with Nm being the number of unique measurements in the dataset used by our model” [Poulain page 728 Materials and Methods]; Based on established range (i.e., min value to max value) of values for each decile, feature values are transformed into the corresponding word (i.e., pre-training score feature value)).
However, Poulain does not expressly teach determining one or more maximum score feature values for a scoring identifier of a plurality of scoring identifiers associated with the plurality of scores based on an exponential function comprising the plurality of maximum decile values.
In the same field of endeavor, Solberg discloses a means of pre-processing medical data to enable further analytics (“Medical laboratory reference data may be contaminated with outliers that should be eliminated before estimation of the reference interval. A statistical test for outliers has been proposed by Paul S. Horn and coworkers (Clin Chem 2001;47:2137– 45). The algorithm operates in 2 steps: (a) mathematically transform the original data to approximate a gaussian distribution; and (b) establish detection limits (Tukey fences) based on the central part of the transformed distribution” [Solberg Abstract]) that determining one or more maximum score feature values for a scoring identifier of a plurality of scoring identifiers associated with the plurality of scores (“The algorithm operates in 2 steps: In the first step, the original data are transformed to approximate a gaussian distribution, to the extent this is possible in the presence of outliers. Horn et al. (8 ) used for this purpose the Box–Cox function (9 ). In the second step, 2 detection limits (fences) are established based on the middle 50% of the transformed distribution, as suggested earlier by Tukey (10 ). Possible outliers are identified as the values located outside of these fences” [Solberg pages 2326-2327]; “Pseudo-random data were generated for the distributions described above (Table 1). All experiments presented here were based on distributions with n = 1000 values each… We analyzed each data set by applying 2 (sensitivity study) or 3 (specificity study) of the versions of Horn’s algorithm for outlier detection that were presented above” [Solberg page 2328 Simulation Experiments]; The disclosed algorithm identifies, for a distribution of values (wherein the distribution may be representative of a given feature/property of medical data (i.e., scoring identifier)), fence values determining a minimum and maximum (i.e., maximum score feature value) possible value, and identifies as outliers all values outside those fences) based on an exponential function comprising the plurality of maximum decile values (““The algorithm operates in 2 steps: In the first step, the original data are transformed to approximate a gaussian distribution, to the extent this is possible in the presence of outliers. Horn et al. (8 ) used for this purpose the Box–Cox function (9 )” [Solberg pages 2326]; “Mathematical functions can transform data of nongaussian distributions to approximate the theoretical gaussian distribution. Three functions of this kind were used in our study. One of these, the Box–Cox transformation function (9 ), which was used in the original Horn’s algorithm for outlier detection (8 ), is as follows…The 2 other transformations considered in the present study are those of the 2-stage normalization procedure recommended by the IFCC for parametric estimation of reference limits (1 ). Manly’s exponential function (11 ) corrects for nongaussian skewness:… We also studied 2 modifications of Horn’s algorithm. In the first step of the algorithm, we replaced the Box–Cox transformation either with the exponential transformation or with a 2-stage transformation consisting of the exponential transformation followed by the modulus transformation” [Solberg pages 2327-2328 Transformations and Horn’s Algorithm and its Modifications]; An exponential function may be utilized to capture the original distribution of the data in the first step of the disclosed algorithm – such a function that represents an entire distribution further implicitly represents its decile values).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated determining one or more maximum score feature values for a scoring identifier of a plurality of scoring identifiers associated with the plurality of scores based on an exponential function comprising the plurality of maximum decile values as taught by Solberg into the combination because both Poulain and Solberg are directed towards means of pre-processing medical data to enable further analytics. Incorporating the teachings of Solberg would allow for the detection and elimination of outliers in measurement data prior to tokenization, thereby preventing downstream prediction tasks from being skewed by noise or extreme/contaminated values [Solberg Abstract].
Regarding claims 11 and 17, they are apparatus and product claims that substantially correspond to the method of claim 4, which is already taught by the combination of Poulain, Ren, Li and Solberg as detailed above. Consequently, they are rejected for the same reasons as claim 4.
Response to Amendment and Arguments
The amendment filed 09/22/2026 has been entered.
Applicant’s amendment to the claims with respect to resolving objections and indefiniteness rejections under 35 U.S.C. 112(b) have been considered, and the objections and rejections are consequently withdrawn.
The remarks filed 09/22/2026 have been fully considered.
Applicant’s remarks traversing the non-eligible subject matter rejections under 35 U.S.C. 101 set forth in the office action mailed 06/26/2026, in view of claims 1-20 as amended, have been considered and are persuasive.
Consequently, the rejections have been withdrawn.
Applicant's remarks traversing the prior art rejections under 35 U.S.C. 102 and 35 U.S.C. 103 set forth in the office action mailed 06/26/2026, in view of claims 1-20 as amended, have been considered.
Although a new ground of rejection has been applied, the examiner has determined a response necessary for the portion of the remarks ([Remarks page 21 para. 2]) wherein the reference(s) applied in the prior rejection of record is/are still being relied upon in the new grounds of rejection set forth above to teach or suggest the subject matter being specifically challenged in applicant’s argument.
The remaining remarks ([Remarks page 21 para.3 – Remarks page 23]), while having been considered, are moot because the new grounds of rejection set forth above does not rely on the reference(s) applied in the prior rejection of record for the subject matter being specifically challenged in applicant’s argument.
Applicant alleges that Poulain’s MLM masking procedure “neither inserts elements from one input sequence into another input sequence no re-arranges the temporal ordering of encounters within an input sequence”, and therefore fails to teach or suggest "the set of one or more initialized weights is pre-trained based on a pre-training dataset comprising a set of one or more training temporal sequence of encounters data records including an extra-record encounter data element and a re-ordered encounter data element”.
The examiner at least disagrees that Poulain does not teach the “set of one or more training temporal sequence of encounters data records including an extra-record encounter data element”. The scope of claim elements are determined in light of their broadest reasonable interpretation, and mere recitation of the term “extra-record encounter data element” does not require insertion of elements from one input sequence into another input sequence. Limitations from the specification are not read into the claims.
Applicant has not presented further arguments with respect to the dependent claims. As such, amended claims 1-20 stand rejected under 35 U.S.C. 103 in light of the new grounds of rejection detailed above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VIJAY M BALAKRISHNAN whose telephone number is (571) 272-0455. The examiner can normally be reached 10am-5pm EST Mon-Thurs.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JENNIFER WELCH can be reached on (571) 272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/V.M.B./
Examiner, Art Unit 2143
/JENNIFER N WELCH/Supervisory Patent Examiner, Art Unit 2143
Prosecution Insights