DETAILED ACTION
This Action is responsive to Claims filed 04/29/2026.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/29/2026 has been entered.
Status of the Claims
Claims 1-8 and 15 were previously cancelled. Claims 10-11 and 14 have been cancelled. Claims 9, 12-13, and 16 are currently pending.
Response to Arguments
Applicant's arguments, see Pages 5-9, filed 04/29/2026 have been fully considered but they are not persuasive.
The Examiner fails to understand the distinction the Applicant attempts to draw between their interpretation of the cited reference Song and the instant claims. The Applicant delineates, on Page 7, between Song’s indexing of CAN IDs, and sorting a magnitude of CAN IDs converted into a value between 0 and K-1, where there are K allowed CAN IDs. The Examiner fails to see the distinction when, as cited, Song maps the CAN IDs to a sorted list of indices from 0 to the number of CAN IDs. Without further amendments clarifying structure or implementation differentiating these features, the Examiner submits Song reasonably reads on the generic recitation of this feature.
As to the Applicant’s arguments regarding the combination of the cited references: The Examiner maintains Chen is relevant to the current claims, as Chen highlights the delineation of anomalous versus allowed traffic with non-zero values, at least, would have been known in the art at the time of the Applicant’s filing. See the reiterated motivation to combine Song and Chen below. The Examiner maintains Debjit remains relevant to the instant Application, at least in order to demonstrate the overall application and use of a forwards and backwards GPT-2 network would have been known in the art at the time of the Applicant’s filing. See the reiterated motivation to combine Song, Chen, and Debjit below.
In the absence of further claim amendments highlighting structure or implementation differentiating the instant claims from the previously cited mapping, see the updated 103 Rejection(s) below.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 9, 12-13, and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Song et al. (Self-Supervised Anomaly Detection for In-Vehicle Network Using Noised Pseudo Normal Data, 2021), hereinafter Song; Chen et al. (DATA CURATION AND QUALITY ASSURANCE FOR MACHINE LEARNING-BASED CYBER INTRUSION DETECTION, 2021), hereinafter Chen; and Debjit et al. (COINS: Dynamically Generating COntextualized Inference Rules for Narrative Story Completion, 2021), hereinafter Debjit.
In regards to claim 9: The present invention claims: “An intrusion detection apparatus for a controller area network (CAN), the intrusion detection apparatus comprising: a processor; a memory storing one or more instructions executable by the processor, and a transceiver connected to a bus of the CAN to perform communications, wherein when executed by the processor, the one or more instructions cause the intrusion detection apparatus to:” Song teaches “Thus, the objectives of this paper are to provide an anomaly detection method in such a limited data environment [CAN bus and CAN traffic] where attack data is deficient while normal data is relatively easy to obtain.” (Page 1098, right column).
“collect CAN identifiers (IDs) from the CAN in an order of occurrence;” Song teaches “In this study, we use only CAN ID sequence for modeling sequential patterns of CAN traffic except payload of messages.” (Page 1103, left column).
“generate a sequence of a predetermined number L of CAN IDs from the collected CAN IDs, L being an integer greater than or equal to 1;” Song teaches “The input to the generator model is a CAN ID or a sequence of CAN IDs.” (Page 1103, right column).
“input the sequence into a… network and calculating a value of a loss function corresponding to the sequence…” Song teaches “The input to the generator model is a CAN ID or a sequence of CAN IDs. The LSTM-based model is trained to guess which CAN ID is the most probable as the following CAN ID at each time step based on a given CAN ID or a sequence of CAN IDs.” (Page 1103, right column) and “The categorical cross-entropy loss function is used because the last dense layer outputs a probability over the CAN IDs. The categorical cross-entropy can be implemented by adding Softmax activation before calculating the cross-entropy.” (Page 1104, left column).
“wherein when a number of allowed CAN IDs is K, the allowed CAN IDs in the sequence are sorted in an order of magnitude and converted into values 0 to K - 1,” Song teaches “For the generator model, each CAN ID represented as a hexadecimal string is mapped to an integer representation as an indexes from 0 to the number of CAN IDs in CAN traffic.” (Page 1103, left column).
“wherein the intrusion detection apparatus is included in a head unit or a gateway of a vehicle, and is implemented as a diagnostic device existing inside or outside the vehicle,” Song teaches “This paper proposes a novel method of training anomaly detection model based on self-supervised learning for in-vehicle network security.” (Introduction, right column).
“…wherein the probability matrix represents probabilities that CAN IDs included in the sequence respectively correspond to one of the allowed CAN IDs represented by the value K or the disallowed CAN ID,” Song teaches their method determining an anomalous versus allowed CAN ID based on prediction and classification over distribution of a given CAN ID. Song reasonably reads on predicting a probability in “The categorical cross-entropy loss function is used because the last dense layer outputs a probability over the CAN IDs.” (Page 1104, left column).
“wherein the value of the loss function is an anomaly score for the sequence based on the probability matrix,” Per the Applicant’s Specification (Equation 5), the output of this loss is based on a probability matrix, which Song reads on in “The categorical cross-entropy loss function is used because the last dense layer outputs a probability over the CAN IDs. The categorical cross-entropy can be implemented by adding Softmax activation before calculating the cross-entropy. The softmax activation normalizes a C-dimensional vector s to a C-dimensional vector σ(s) in the range (0, 1) of which the sum is 1, which is calculated as:” (Page 1104, left column).
While Song teaches an LSTM as the generator model, Song fails to explicitly teach:
“bi-directional generative pretrained transformer 2 (GPT2)…”
and
“CAN IDs that are not allowed in the sequence are converted to a common K, K being an integer equal to or greater than 1.” However, Chen, in a similar field of endeavor of anomaly detection, teaches “Since the goal of this case study is to develop a host-based intrusion detection system (HIDS), we conduct machine learning experiments on the UNM, MIT, and ADFA-LD datasets. A detailed introduction of these datasets can be found in our GitHub repository. Regardless of the slight difference in ADFA and UNM data formats, we use similar data cleaning and augmentation techniques to create a dataset for each class. Processing data for pre-trained language models vectors, such as BERT [62] and GPT-2 [63], is similar to normal machine learning algorithms.” (GPT2, Page 9, Section 4.1.1) and “row with normal sequence is labeled 0, whereas the one with intrusion sequence is labeled 1. We use normal data and intrusion data from each dataset to create a sample pool.” (Page 9, Section 4.1.1, given the Applicant’s Specification recites the “converted to K” step without specifics, the Examiner interprets this as a highly general means of delineating an abnormal sequence, which Chen reads on).
Chen highlights the need to improve the data quality when building a machine learning based intrusion detection system (Abstract). It would have been obvious to one of ordinary skill in the art at the time of the Applicant’s filing to incorporate the improved data-work teachings of Chen in an intrusion detection system like Song’s in order to improve the overall model quality.
“and wherein the processor is further configured to compare the anomaly score to a threshold and, when the anomaly score is equal to or greater than the threshold, determine a period corresponding to the sequence as a period in which an intrusion exists.” Per the Applicant’s Specification (Equation 7), the output of this loss can be interpreted as an anomaly score, which Song reasonably reads on in “The categorical cross-entropy loss function is used because the last dense layer outputs a probability over the CAN IDs.” (Page 1104, left column) and Chen reasonably reads on in “We use small GPT-2 to train on processed data from both classes of each dataset using 16 epochs and 16 batch size. Unlike BERT, GPT-2 does not require an additional output layer; instead, it outputs the sequence’s likelihood in each class.” (Page 11, GPT-2, mapping the output of a probability to a general anomaly score).
The combination of Song and Chen fails to explicitly teach “the bidirectional GPT2 network including a forward GPT module, a backward GPT module, a fully-connected layer, and a softmax function layer” However, Debjit teaches “We adopt a pretrained GPT-2 (base)…transformer model with multiple Transformer blocks of multi-head self-attention and fully connected layers.” (Page 4, Inference Step) and “While we designed our sentence generation model in such a way that it can utilize inference rules from both forward and backward directions for each sentence…” (Page 5, left column, inference rules in both directions would require the network to handle both directions (modules)). Debjit teaches both concatenation and the use of a softmax layer in the Inference Step Section on Pages 4 and 5.
Debjit highlights the strengths of GPT2 networks on sequential inference tasks (Introduction). It would have been obvious to one of ordinary skill in the art at the time of the Applicant’s filing to utilize these known GPT2 structures or functionalities if they were to use GPT2 in a combination of Song and Chen to realize the sequential inferencing power of GPT2.
“wherein the sequence is input to the forward GPT module in an original order and to the backward GPT module in a reverse order,” Debjit teaches “While we designed our sentence generation model in such a way that it can utilize inference rules from both forward and backward directions for each sentence…” (Page 5, left column).
“wherein an output of the forward GPT module and an output of the backward GPT module are concatenated to generate a matrix, the matrix is converted into a (K+1)xL matrix by the fully-connected layer, the (K+1)xL matrix is transformed into a probability matrix by the softmax function layer,” Debjit teaches “We concatenate the generated inference rules (Ii = Ei ⊕ Ci)7 and store the last hidden representation in MemIR 2 IRN_L_H, where N is the number of sentences, L the maximum inference sequence length and H the hidden state dimensions.” (Page 5, right column) and “The softmax layer defines the model to output the most probable target sequence: the most likely inference rules (Ei and Ci) for each relation type (cf. Algorithm Line 4-5).” (Page 5, left column). Song also teaches “The categorical cross-entropy loss function is used because the last dense layer outputs a probability over the CAN IDs. The categorical cross-entropy can be implemented by adding Softmax activation before calculating the cross-entropy.” (Page 1104, left column).
In regards to claim 12: The present invention claims: “wherein to calculate the value of the loss function, the one or more instructions further cause the intrusion detection apparatus to input to the forward GPT module embedding vectors corresponding to CAN IDs from 0-th CAN ID to (L-2)-th CAN ID belonging to the sequence, and the forward GPT module outputs E-dimensional vectors having a same dimensionality as the embedding vectors, which correspond to 1st to (L-1)-th CAN IDs.” Song teaches “The embedding layer is the input layer that maps a given index, an integer representation of a CAN ID, to a vector of fixed-length, which is set to 256 in this study. The converted CANID vector is fed to the next LSTM layer with 256 units. The LSTM layer extracts the context of a given sequence. Finally, the dense layer then outputs logits predicting the log-likelihood of the next CAN ID. Therefore, the output size of the dense layer is the same as the number of CAN IDs.” (Pages 1103-1104).
In regards to claim 13: The present invention claims: “wherein to calculate the value of the loss function, the one or more instructions further cause the intrusion detection apparatus to input to the forward GPT module embedding vectors corresponding to CAN IDs from (L-1)-th CAN ID to 1st CAN ID belonging to the sequence, and the backward GPT module outputs E-dimensional vectors having a same dimensionality as the embedding vectors, which correspond to (L-2)-th to 0-th CAN IDs.” Song teaches “The embedding layer is the input layer that maps a given index, an integer representation of a CAN ID, to a vector of fixed-length, which is set to 256 in this study. The converted CANID vector is fed to the next LSTM layer with 256 units. The LSTM layer extracts the context of a given sequence. Finally, the dense layer then outputs logits predicting the log-likelihood of the next CAN ID. Therefore, the output size of the dense layer is the same as the number of CAN IDs.” (Pages 1103-1104) and Debjit teaches “While we designed our sentence generation model in such a way that it can utilize inference rules from both forward and backward directions for each sentence…” (Page 5, left column). A combination of Song, Chen, and Debjit would reasonably read on vector embeddings and outputs in the forwards and backwards directions.
In regards to claim 16: The present invention claims: “wherein the loss function is defined as
PNG
media_image1.png
65
260
media_image1.png
Greyscale
xf") is an l-th variable of an m-th CAN ID sequence corresponding to a detection target sequence, and y,"n is an actual generated CAN ID for xi", which is a ground truth value.” Per the Applicant’s Specification (Equation 7), the output of this loss can be interpreted as an anomaly score, which Song reasonably reads on in “The categorical cross-entropy loss function is used because the last dense layer outputs a probability over the CAN IDs.” (Page 1104, left column) and Chen reasonably reads on in “We use small GPT-2 to train on processed data from both classes of each dataset using 16 epochs and 16 batch size. Unlike BERT, GPT-2 does not require an additional output layer; instead, it outputs the sequence’s likelihood in each class.” (Page 11, GPT-2, mapping the output of a probability to a general anomaly score).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GRIFFIN T BEAN whose telephone number is (703)756-1473. The examiner can normally be reached M - F 7:30 - 4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li Zhen can be reached at (571) 272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GRIFFIN TANNER BEAN/Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights