Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 9 is objected to because of the following informalities:
In claim 9, “second type if impact” should be “second type of impact” (to correct a typographical error).
For purposes of examination, the claims have been interpreted as having the meaning of the suggested corrections.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 9 and 12-19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
In claim 9, the limitations of “the event type associated with the first type of impact is a collision” and “the event type associated with the second type if impact is no collision” lack clear antecedent basis because this claim recites two event types, whereas the preceding claim 8 only recites “an event type” without distinction between different types of event types. To overcome this rejection, the Examiner suggests amending claim 8 to recite “…are associated with a first event type and a second event type, respectively” and amending claim 9 to use the term “first event type” and “second event type.” For purposes of Examination, claim 9 has been interpreted in this manner.
In claim 12, the limitation of “the first model” at the middle of the claim (in the paragraph beginning with “input”) lacks antecedent basis. For purposes of Examination, this term has been interpreted to mean “a first model.”
Dependent claims 13-19 are also rejected on the above grounds because these claims incorporate the indefinite limitation of claim 12 due to their dependencies.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
1. Claims 1-2, 6-9, 12-14, and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Nag et al. (US 2025/0303994 A1) (“Nag”) in view of Kwon et al., “Driving Behavior Classification and Sharing System Using CNN-LSTM Approaches and V2X Communication,” Approaches and V2X Communication. Appl. Sci. 2021, 11, 10420 (“Kwon”).
As to claim 1, Nag teaches a method comprising:
inputting, in a machine-learning collision prediction model, a set of one or more kinematic variables associated with a trigger event based on movement of a mobile device, wherein the set of one or more kinematic variables are recorded over a duration of time, [[0031]: “In a non-limiting example, the computing device can determine a vehicle event based on inertial sensor data and speed from at least one sensor in a housing inside a cabin of a vehicle and classify the vehicle event as a collision event or a non-collision event based on the inertial sensor data.” [0066]: “If the inertial sensor measurements exceed the threshold, a vehicle event can be detected, and a feature extraction process, such as the feature extraction process 218, can be performed on the sensor data 122. The feature extraction process 218 can generate inertial features 225A, GPS speed features 225B, and vehicle class information 225C (e.g., the features 225).” Here, speed is a kinematic variable, as well as inertial sensor data (see [0039], which teaches accelerometers). The data is recorded. See [0018]: “FIG. 4 shows an example time-series graph showing windows of sensor data before, during, and after a potential collision event, according to an embodiment.” The limitation of a “machine-learning collision prediction model” is taught in the form of a set of machine learning models that includes an “initial classifier 304” ([0069]), which is a trained machine learning model (see [0068]: “The initial classifier 304 can be trained prior to performing the operations of the flow diagram 300, for example, using a supervised learning process…”).] the machine-learning collision prediction model including at least a convolutional neural network (CNN) model layer, […], and a prediction model layer, [[0068]: “The initial classifier 304 can include any number of layers, such as fully connected layers, convolutional layers, activation layers, softmax layers, or other types of neural network layers.” Note that the output of the model implies a prediction layer. See [0069]: “The output values for the initial classifier 304 can indicate, for example, whether the detected vehicle event is a collision or a non-collision. The initial classifier 304 may be further trained to generate a confidence value, which may indicate the likelihood that the prediction of the collision or the non-collision is accurate.”] […];
generating a prediction score using the machine-learning collision prediction model, the prediction score generated based on the set of one or more kinematic variables and associated with the event; [[0069]: “The initial classifier 304 may be further trained to generate a confidence value, which may indicate the likelihood that the prediction of the collision or the non-collision is accurate.” The output is based on the kinematic variables. See [0066]: “The features 225 can be extracted for a number of time windows 215 in the sensor data 122, as described herein. Upon extracting the features that correspond to the vehicle event, the features can be provided as input to the initial classifier 304.”]
generating a prediction that the event is a type of impact, the prediction generated based on the prediction score; [[0023]: “The systems and methods described herein provide techniques to both detect and classify various collision events and non-collision events (e.g., that may indicate a false positive potential collision event) …Such approaches cannot detect and classify non-collision events or collision events into sub-categories (e.g., types or classes of non-collision or collision events involving the vehicle).” That is, non-collision/collision is an event type. Note that as defined in this application (see paragraph 6 of the specification), non-collision is considered to be a type of impact. See also [0076]: “The collision classifier 314 can generate a prediction of a collision subclass, which can include a front-end collision, a rear-end (e.g., a back-end) collision, a left collision, a right collision, a low-clearance (e.g., top) collision, a ground clearance collision (e.g., an undercarriage collision), a bird or animal collision, or a vehicle topple event, among others.” This is based on the prediction score. See [0075]: “However, if the confidence value of the predicted collision event is greater than or equal to the threshold, the features 225 (e.g., which may have undergone a transformation process 230) can be provided as input to the collision classifier 314.”] and
outputting the prediction, wherein the outputted prediction is time-oriented. [The generation of the prediction also constitutes outputting. Furthermore, the prediction is also outputted as a notification, as disclosed in [0077]: “The notification may include a type of subclass and in some scenarios, the notification may further include a query and requests a response from the driver.” Furthermore, the prediction is time-oriented because it is a prediction for a certain time window. See, e.g., [0088]: “Determining a vehicle event can include identifying a time period or time window”; [0066]: “The features 225 can be extracted for a number of time windows 215 in the sensor data 122, as described herein.”]
Nag does not explicitly teach the model having “a long short-term memory (LSTM) model layer” and “wherein the CNN model extracts features for time segments of the duration of time, and the respective features associated with respective time segments are fed into the LSTM model layer.”
Kwon teaches “a long short-term memory (LSTM) model layer” and “wherein the CNN model extracts features for time segments of the duration of time, and the respective features associated with respective time segments are fed into the LSTM model layer.” [§ 2, paragraph 2: “In the state-of-the-art approaches, a combined network of CNN-LSTM is applied rather than one kind of network of CNN or LSTM. The CNN-LSTM network extracts the features of the input signal through the CNN and uses it as the input of the LSTM. The combination of CNN and LSTM improves recognition performance by utilizing both spatial and temporal information.” Page 6, bottom paragraph: “The CNN-LSTMs have been applied as an approach to predict text descriptions using image sequences. The CNN-LSTM architecture uses a CNN layer in the feature extraction process of the input data integrated with LSTM to support sequence prediction, as shown in Figure 6.” As shown in FIG. 6 (As well as FIGS. 3-4), the CNN output is fed into the LSTM.]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Nag with the teachings of Kwon by implementing a CNN-LSTM model architecture, so as to arrive at the features of “a long short-term memory (LSTM) model layer” and “wherein the CNN model extracts features for time segments of the duration of time, and the respective features associated with respective time segments are fed into the LSTM model layer” as recited in the instant claim. The motivation would have been to implement a type of model that “improves recognition performance by utilizing both spatial and temporal information” (§ 2, paragraph 2, part quoted above).
As to claim 2, the combination of Nag and Kwon teaches the method of claim 1, comprising:
generating the machine-learning collision prediction model by training a first collision prediction algorithm, wherein the training is based on training data including a training dataset comprising a first set of events associated with a first type of data and a second set of events associated with a second type of data, wherein the first type of data and the second type of data are indicative of a same type of impact. [Nag, [0068]: “The initial classifier 304 can be trained prior to performing the operations of the flow diagram 300, for example, using a supervised learning process. The initial classifier 304 can be trained using historic records of vehicle events, and features extracted therefrom (e.g., the historical data 265).” Nag, [0061]: “The historical data 265 can be utilized as training data to train one or more of the artificial intelligence models 230, for example, using a suitable supervised machine-learning algorithm. The historical data 265 can include sets of historical features 225 captured during operation of one or more vehicles. The historical data 265 can include transformed features 225 from several vehicles. In some implementations, ground-truth labels are assigned to each set of features 225 (e.g., which each correspond to a respective vehicle event) in the historical data 265, using a label assignment process… The ground-truth labels can identify the class (e.g., collision event or non-collision event) and sub-class of the vehicle event to which the set of historical features 225 corresponds.” That is, the historical data for various vehicles are different sets of events, along with their corresponding data being of first and second types. Furthermore, data points that are all of either a collision or non/collision classification meets the limitation of data indicative of a same type of impact. Since training data is disclosed in the generic sense, and it is understood that training data includes multiple examples, the use of training data wherein the first type of data and the second type of data are indicative of a same type of impact is implied. Alternatively, if the applicant disagrees, the Examiner submits that the use of training data that meets this limitation would have been obvious as a combination of prior art elements according to known methods to yield predictable results, namely the combination of training data having different characteristics to yield the predictable result of training the model to reflect the data.]
As to claim 6, the combination of Nag and Kwon teaches the method of claim 1, wherein the set of one or more kinematic variables include at least one of global positioning system (GPS) speed variables, GPS altitude variables, or accelerometer magnitude variables. [Nag, [0039]: “The one or more sensors 135 can include any type of sensor that is capable of capturing information about the operations of the vehicle 110, including but not limited to accelerometers, gyroscopes, magnetometers, inertial measurement units (IMU), GPS receivers, or any other type of inertial sensor.” Note that GPS includes “GPS speed” ([0058]). See also Nag, [0056]: “Examples of the outputs of wavelet transform techniques applied to accelerometer and gyroscope sensor data is shown in FIG. 5.”; [0057]: “The plots represents 6-axis time-series signals from accelerometer and gyroscope sensors” Therefore, the alternatives of GPS speed variables and accelerometer magnitude variables is taught.]
As to claim 7, the combination of Nag and Kwon teaches the method of claim 1, wherein the prediction is generated based on a value of the prediction score being at or above a threshold value. [Nag, [0075]: “However, if the confidence value of the predicted collision event is greater than or equal to the threshold, the features 225 (e.g., which may have undergone a transformation process 230) can be provided as input to the collision classifier 314. The confidence threshold may be predetermined based on previously analyzed collision data.”]
As to claim 8, the combination of Nag and Kwon teaches the method of claim 1, wherein the type of impact is a first type of impact or a second type of impact, and wherein the first type of impact and the second type of impact are associated with an event type. [Nag, [0023]: “The systems and methods described herein provide techniques to both detect and classify various collision events and non-collision events (e.g., that may indicate a false positive potential collision event) …Such approaches cannot detect and classify non-collision events or collision events into sub-categories (e.g., types or classes of non-collision or collision events involving the vehicle).” That is, non-collision/collision is an event type. Note that as defined in this application (see paragraph 6 of the specification), non-collision is considered to be a type of impact.]
As to claim 9, the combination of Nag and Kwon teaches the method of claim 8, wherein the event type associated with the first type of impact is a collision and the event type associated with the second type if impact is no collision. [Nag, [0023]: “The systems and methods described herein provide techniques to both detect and classify various collision events and non-collision events (e.g., that may indicate a false positive potential collision event) …Such approaches cannot detect and classify non-collision events or collision events into sub-categories (e.g., types or classes of non-collision or collision events involving the vehicle).”]
As to claim 12, Nag teaches a system comprising:
one or more processors; [[0009]: “Another embodiment is directed to a system for detecting a vehicle collision. The system can include at least one processor coupled to a non-transitory memory.”]
a prediction system, wherein the prediction system includes a machine-learning collision prediction model comprising at least a first layer […] and a third layer; [The limitation of a “machine-learning collision prediction model” is taught in the form of a set of machine learning models that includes an “initial classifier 304” ([0069]), which is a trained machine learning model (see [0068]: “The initial classifier 304 can be trained prior to performing the operations of the flow diagram 300, for example, using a supervised learning process…” [0068]: “The initial classifier 304 can include any number of layers, such as fully connected layers, convolutional layers, activation layers, softmax layers, or other types of neural network layers.” The limitations of both a first and a third layer are met by the teaching of plural layers.] and
a memory storing computer-executable instructions that, when executed by the one or more processors, cause the system to: [[0095]: “When implemented in software, the functions may be stored as one or more instructions or code on a non-transitory computer-readable or processor-readable storage medium.”]
input, into the machine-learning collision prediction model, a set of one or more kinematic variables associated with a trigger event based on movement of a mobile device, wherein the set of one or more kinematic variables are recorded over a duration of time, [[0031]: “In a non-limiting example, the computing device can determine a vehicle event based on inertial sensor data and speed from at least one sensor in a housing inside a cabin of a vehicle and classify the vehicle event as a collision event or a non-collision event based on the inertial sensor data.” [0066]: “If the inertial sensor measurements exceed the threshold, a vehicle event can be detected, and a feature extraction process, such as the feature extraction process 218, can be performed on the sensor data 122. The feature extraction process 218 can generate inertial features 225A, GPS speed features 225B, and vehicle class information 225C (e.g., the features 225).” Here, speed is a kinematic variable, as well as inertial sensor data (see [0039], which teaches accelerometers). The data is recorded. See [0018]: “FIG. 4 shows an example time-series graph showing windows of sensor data before, during, and after a potential collision event, according to an embodiment.”] wherein the first model extracts features for time segments of the duration of time, and […]; [[0006]: “The method may include generating a set of features by executing a feature extraction function using at least one of the inertial sensor data or the speed captured during the time period.” [0066]: “If the inertial sensor measurements exceed the threshold, a vehicle event can be detected, and a feature extraction process, such as the feature extraction process 218, can be performed on the sensor data 122. The feature extraction process 218 can generate inertial features 225A, GPS speed features 225B, and vehicle class information 225C (e.g., the features 225).” The feature extraction function/process corresponds to a first model.]
generate, by the machine-learning collision prediction model, a prediction score, wherein the prediction score is based on the set of one or more kinematic variables and associated with the event; [[0069]: “The initial classifier 304 may be further trained to generate a confidence value, which may indicate the likelihood that the prediction of the collision or the non-collision is accurate.” The output is based on the kinematic variables. See [0066]: “The features 225 can be extracted for a number of time windows 215 in the sensor data 122, as described herein. Upon extracting the features that correspond to the vehicle event, the features can be provided as input to the initial classifier 304.”]
determine a prediction, according to the prediction score, that the event is a type of impact; [[0023]: “The systems and methods described herein provide techniques to both detect and classify various collision events and non-collision events (e.g., that may indicate a false positive potential collision event) …Such approaches cannot detect and classify non-collision events or collision events into sub-categories (e.g., types or classes of non-collision or collision events involving the vehicle).” That is, non-collision/collision is an event type. Note that as defined in this application (see paragraph 6 of the specification), non-collision is considered to be a type of impact. See also [0076]: “The collision classifier 314 can generate a prediction of a collision subclass, which can include a front-end collision, a rear-end (e.g., a back-end) collision, a left collision, a right collision, a low-clearance (e.g., top) collision, a ground clearance collision (e.g., an undercarriage collision), a bird or animal collision, or a vehicle topple event, among others.” This is based on the prediction score. See [0075]: “However, if the confidence value of the predicted collision event is greater than or equal to the threshold, the features 225 (e.g., which may have undergone a transformation process 230) can be provided as input to the collision classifier 314.”] and
output the prediction, wherein the outputted prediction is time-oriented. [The generation of the prediction also constitutes outputting. Furthermore, the prediction is also outputted as a notification, as disclosed in [0077]: “The notification may include a type of subclass and in some scenarios, the notification may further include a query and requests a response from the driver.” Furthermore, the prediction is time-oriented because it is a prediction for a certain time window. See, e.g., [0088]: “Determining a vehicle event can include identifying a time period or time window”; [0066]: “The features 225 can be extracted for a number of time windows 215 in the sensor data 122, as described herein.”]
Nag does not explicitly teach the model having specifically “a second layer” wherein “the respective features associated with respective time segments are fed into the second layer”
Kwon teaches “a second layer” and “the respective features associated with respective time segments are fed into the second layer.” [§ 2, paragraph 2: “In the state-of-the-art approaches, a combined network of CNN-LSTM is applied rather than one kind of network of CNN or LSTM. The CNN-LSTM network extracts the features of the input signal through the CNN and uses it as the input of the LSTM. The combination of CNN and LSTM improves recognition performance by utilizing both spatial and temporal information.” Page 6, bottom paragraph: “The CNN-LSTMs have been applied as an approach to predict text descriptions using image sequences. The CNN-LSTM architecture uses a CNN layer in the feature extraction process of the input data integrated with LSTM to support sequence prediction, as shown in Figure 6.” As shown in FIG. 6 (As well as FIGS. 3-4), the CNN output is fed into the LSTM (a second layer), whose output is then fed into other layers.]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Nag with the teachings of Kwon by implementing a CNN-LSTM model architecture, so as to arrive at the claimed invention. The motivation would have been to implement a type of model that “improves recognition performance by utilizing both spatial and temporal information” (§ 2, paragraph 2, part quoted above).
As to claim 13, the combination of Nag and Kwon teaches the system of claim 12, wherein the machine-learning collision prediction model includes at least a convolutional neural network (CNN) model layer, […], and a prediction model layer. [Nag, [0068]: “The initial classifier 304 can include any number of layers, such as fully connected layers, convolutional layers, activation layers, softmax layers, or other types of neural network layers.” Note that the output of the model implies a prediction layer. See Nag, [0069]: “The output values for the initial classifier 304 can indicate, for example, whether the detected vehicle event is a collision or a non-collision. The initial classifier 304 may be further trained to generate a confidence value, which may indicate the likelihood that the prediction of the collision or the non-collision is accurate.”]
Kwon further teaches “a long short-term memory (LSTM) model layer” [§ 2, paragraph 2: “In the state-of-the-art approaches, a combined network of CNN-LSTM is applied rather than one kind of network of CNN or LSTM. The CNN-LSTM network extracts the features of the input signal through the CNN and uses it as the input of the LSTM. The combination of CNN and LSTM improves recognition performance by utilizing both spatial and temporal information.” Page 6, bottom paragraph: “The CNN-LSTMs have been applied as an approach to predict text descriptions using image sequences. The CNN-LSTM architecture uses a CNN layer in the feature extraction process of the input data integrated with LSTM to support sequence prediction, as shown in Figure 6.” As shown in FIG. 6 (As well as FIGS. 3-4), the CNN output is fed into the LSTM (a second layer), whose output is then fed into other layers.]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Nag and Kwon so as to have also arrived at the claimed invention of the instant dependent claim. The motivation for doing so is covered by the motivation given for Kwon in the rejection of the parent independent claim.
As to claim 14, the combination of Nag and Kwon teaches the system of claim 12, wherein the computer-executable instructions further configure the one or more processors to cause the system to:
generate the machine-learning collision prediction model by training a first collision prediction algorithm, wherein the training is based on training data including a training dataset comprising a first set of events associated with a first type of data and a second set of events associated with a second type of data, wherein the first type of data and the second type of data are indicative of a same type of impact. [Nag, [0068]: “The initial classifier 304 can be trained prior to performing the operations of the flow diagram 300, for example, using a supervised learning process. The initial classifier 304 can be trained using historic records of vehicle events, and features extracted therefrom (e.g., the historical data 265).” Nag, [0061]: “The historical data 265 can be utilized as training data to train one or more of the artificial intelligence models 230, for example, using a suitable supervised machine-learning algorithm. The historical data 265 can include sets of historical features 225 captured during operation of one or more vehicles. The historical data 265 can include transformed features 225 from several vehicles. In some implementations, ground-truth labels are assigned to each set of features 225 (e.g., which each correspond to a respective vehicle event) in the historical data 265, using a label assignment process… The ground-truth labels can identify the class (e.g., collision event or non-collision event) and sub-class of the vehicle event to which the set of historical features 225 corresponds.” That is, the historical data for various vehicles are different sets of events, along with their corresponding data being of first and second types. Furthermore, data points that are all of either a collision or non/collision classification meets the limitation of data indicative of a same type of impact. Since training data is disclosed in the generic sense, and it is understood that training data includes multiple examples, the use of training data wherein the first type of data and the second type of data are indicative of a same type of impact is implied. Alternatively, if the applicant disagrees, the Examiner submits that the use of training data that meets this limitation would have been obvious as a combination of prior art elements according to known methods to yield predictable results, namely the combination of training data having different characteristics to yield the predictable result of training the model to reflect the data.]
As to claim 17, the combination of Nag and Kwon teaches the system of claim 12, wherein the type of impact is a first type of impact or a second type of impact, and wherein the first type of impact and the second type of impact are associated with an event type. [Nag, [0023]: “The systems and methods described herein provide techniques to both detect and classify various collision events and non-collision events (e.g., that may indicate a false positive potential collision event) …Such approaches cannot detect and classify non-collision events or collision events into sub-categories (e.g., types or classes of non-collision or collision events involving the vehicle).” That is, non-collision/collision is an event type. Note that as defined in this application (see paragraph 6 of the specification), non-collision is considered to be a type of impact.]
As to claim 18, the combination of Nag and Kwon teaches the system of claim 17, wherein the event type associated with the first type of impact is a collision and the event type associated with the second type of impact is no collision. [Nag, [0023]: “The systems and methods described herein provide techniques to both detect and classify various collision events and non-collision events (e.g., that may indicate a false positive potential collision event) …Such approaches cannot detect and classify non-collision events or collision events into sub-categories (e.g., types or classes of non-collision or collision events involving the vehicle).”]
2. Claims 3 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Nag in view of Kwon, and further in view of Yang et al., “Short-Term Wind Power Prediction Based on CEEMDAN and Parallel CNN-LSTM,” 2022 IEEE/IAS Industrial and Commercial Power System Asia (I&CPS Asia), Shanghai, China, 2022, pp. 1166-1172 (“Yang”).
As to claim 3, the combination of Nag and Kwon teaches the method of claim 1, as set forth above.
Kwon teaches “comprising: […] and inputting the altered outputs into a prediction layer to output the prediction” [As shown in FIG 6, the altered outputs (inputs processed by and up to the Fully Connected Layer) are fed into the softmax layer, which corresponds to a prediction layer that is used to generate the prediction.]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Nag and Kwon so as to have also arrived at the above limitations of the instant dependent claim. The motivation for doing so is covered by the motivation given for Kwon in the rejection of the parent claim.
The combination of references thus far does not teach the further limitations of the instant dependent claim.
Yang teaches “comprising: generating a high frequency CNN model and a low frequency CNN model in the CNN model layer, wherein the high frequency CNN model and the low frequency CNN model feed into respective LSTM models in the LSTM model layer;” [§ II.A, paragraphs 1 and 4: “The overall framework of short-term WPP method based on CEEMDAN-FTC and parallel CNN-LSTM is shown in Fig.1… Total four CNN-LSTMs are established, in which the input feature is high frequency component, low frequency component, trend component, and original wind speed, respectively. The input of one CNN-LSTM is the original wind speed in four height and the input of rest three CNN-LSTMs are the three datasets derived in Step 3. Those four CNN-LSTMs are parallel to each other and are connected by a fully connected layer between the LSTM hidden layer and output layer.” As shown in FIG. 1, each CNN feeds into a respective LSTM.] “concatenating outputs of the respective LSTM models together; inputting the concatenated outputs into a dense layer to alter one or more dimensions of the concatenated outputs;” [As shown in FIG. 1 and described in the parts quoted above, the LSTM are fed into a fully connected layer, which corresponds to a “dense layer” since fully connected and dense are synonyms in this context. Furthermore, the output from the LSTMs is considered to be “concatenated” because they are fed into a single input layer as shown in FIG. 1; thus, by being fed into a single input layer, the outputs of the LSTMs are concatenated into a single sequential structure by being adjacent to each other in a sequence. The limitation of “to alter” is met because the output of the fully connected layer has one node, i.e., one dimensions, compared to the plurality of input nodes, i.e., a plurality of dimensions.]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of the references combined thus far with the teachings of Yang by implementing the use of a parallel CNN-LSTMs, so as to arrive at the claimed invention of the instant dependent claim. The motivation for doing so would have been to specialize different CNN-LSTMs models for data of different frequencies for improved prediction accuracy, as suggested by Yang (abstract: “In the second stage, a novel parallel CNN-LSTM neural network architecture is proposed as WPP model, in which the input feature consists both three frequency components derived in the first stage and the original wind speed sequence in different height. The results show that the proposed method is an effective short-term WPP method which improves the prediction accuracy.”).
As to claim 15, the combination of Nag and Kwon teaches the system of claim 13, as set forth above.
Kwon further teaches “wherein the computer-executable instructions further configure the one or more processors to cause the system to: […] input the altered outputs into the prediction layer to output the prediction” [As shown in FIG 6, the altered outputs (inputs processed by and up to the Fully Connected Layer) are fed into the softmax layer, which corresponds to a prediction layer that is used to generate the prediction.]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Nag and Kwon so as to have also arrived at the above limitations of the instant dependent claim. The motivation for doing so is covered by the motivation given for Kwon in the rejection of the parent independent claim.
The combination of references thus far does not teach the further limitations of the instant dependent claim.
Yang teaches “generate a high frequency CNN model and a low frequency CNN model in the first layer, wherein the high frequency CNN model and the low frequency CNN model feed into respective LSTM models in the second layer;” [§ II.A, paragraphs 1 and 4: “The overall framework of short-term WPP method based on CEEMDAN-FTC and parallel CNN-LSTM is shown in Fig.1… Total four CNN-LSTMs are established, in which the input feature is high frequency component, low frequency component, trend component, and original wind speed, respectively. The input of one CNN-LSTM is the original wind speed in four height and the input of rest three CNN-LSTMs are the three datasets derived in Step 3. Those four CNN-LSTMs are parallel to each other and are connected by a fully connected layer between the LSTM hidden layer and output layer.” As shown in FIG. 1, each CNN feeds into a respective LSTM.] “concatenate outputs of the respective LSTM models together; input the concatenated outputs into a dense layer to alter one or more dimensions of the concatenated outputs;” [As shown in FIG. 1 and described in the parts quoted above, the LSTM are fed into a fully connected layer, which corresponds to a “dense layer” since fully connected and dense are synonyms in this context. Furthermore, the output from the LSTMs is considered to be “concatenated” because they are fed into a single input layer as shown in FIG. 1; thus, by being fed into a single input layer, the outputs of the LSTMs are concatenated into a single sequential structure by being adjacent to each other in a sequence. The limitation of “to alter” is met because the output of the fully connected layer has one node, i.e., one dimensions, compared to the plurality of input nodes, i.e., a plurality of dimensions.]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of the references combined thus far with the teachings of Yang by implementing the use of a parallel CNN-LSTMs, so as to arrive at the claimed invention of the instant dependent claim. The motivation for doing so would have been to specialize different CNN-LSTMs models for data of different frequencies for improved prediction accuracy, as suggested by Yang (abstract: “In the second stage, a novel parallel CNN-LSTM neural network architecture is proposed as WPP model, in which the input feature consists both three frequency components derived in the first stage and the original wind speed sequence in different height. The results show that the proposed method is an effective short-term WPP method which improves the prediction accuracy.”).
3. Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Nag in view of Kwon, and further in view of Taghia et al. (US 2025/0363384 A1) (“Taghia”).
As to claim 4, the combination of Nag, Kwon, and Yang teaches the method of claim 3, but does not teach the further limitations of the instant dependent claim.
Taghia teaches “wherein the prediction layer outputs the prediction using transfer learning.” [[0002]: “Transfer learning is a machine learning technique focusing on transferring knowledge between different but similar domains. One could, for example, train a model on Task A (e.g., Task A may be prediction of average read latency given data collected from a data center) and then transfer what has been learned to solve Task B, which is a task that is somewhat related to but not the same as Task A (e.g., Task B may be prediction of average write latency given data collected from a data center). By taking advantage of/applying what has been learned from similar domains to the target domain, increased performance, better generalization, and the reduced need for target domain data can be achieved.” The examiner notes that “using” in the instant claim does not require any specific manner of use.]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of the references combined thus far with the teachings of Taghia by implementing the use of transfer learning for the machine learning collision prediction model, so as to arrive at the limitations of the instant dependent claim. The motivation would have been to take advantage of what has been learned for increased performance, better generalization, and the reduced need for target domain data, as suggested by Taghia (see parts quoted above).
4. Claims 5 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Nag in view of Kwon, and further in view of Simoncini et al. (US 2025/0085109 A1) (“Simoncini”) and Rahmes et al. (US 2025/0349219 A1) (“Rahmes”).
As to claim 5, the combination of Nag, Kwon, and Yang teaches the method of claim 3, and the use of “global positioning system (GPS) speeds” [Nag, [0066]: “If the inertial sensor measurements exceed the threshold, a vehicle event can be detected, and a feature extraction process, such as the feature extraction process 218, can be performed on the sensor data 122. The feature extraction process 218 can generate inertial features 225A, GPS speed features 225B, and vehicle class information 225C (e.g., the features 225).”] but does not teach the further limitations of the instant dependent claim.
Simoncini teaches “wherein the high frequency CNN model receives accelerometer magnitude and interpolated global positioning system (GPS) speeds to a matching frequency that is convoluted together,” [[0016]: “In some implementations, the first CNN model may resample the GPS data and the IMU data, via interpolation, so that the GPS data and the IMU data have the same quantity of samples (e.g., that is a multiple θ=3 of the quantity of video data timestamps). The first CNN model may apply a set of convolutional operations, and may utilize a max pooling operation of size θ to align the sampling rates of the GPS data and the IMU data with the timestamps of the video data. The first CNN model may apply the same convolutional operation to process each signal independently to learn filters to be applied to a generic signal and to extract features describing a temporal evolution and preserving individual signal semantic meaning.” Note that aligning the sampling rates of both the GPS and the IMU data with the timestamps of the video data constitutes matching their frequencies. Applying the same convolution operation to the signals and processing them in the same model constitutes convolving them together, especially since the CNN is multilayered (see [0023]). Furthermore, note that “IMU data” refers to an accelerometer. See [0012]: “The IMU data may include data identifying acceleration measurements and angular velocity measurements of the vehicles over time.” In regards to the limitation of GPU speeds. While “speed” is not explicitly stated, this limitation is already taught in the existing combination of references, and Simoncini teaches that the GPU data includes location and time (“The GPS data may include data identifying GPS locations of the vehicles over time.”). Thus, while Simoncini does not explicitly mention “speed,” its teachings are consistent with the use of GPS data that is already processed in the form of speed data.]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of the references combined thus far with the teachings of Simoncini so as to arrive at the above-discussed features of the instant claim. The motivation for doing so would have been to prepare data in a manner that enables utilization of both GPS and IMU data to classify vehicle activity (see Simoncini, [0008]: “current techniques for classifying vehicle maneuvers fail to utilize sensor data, such as GPS data and IMU data, and lack the contextual detail to accurately classify the vehicle maneuver.”).
The combination of references thus far does not teach the remaining limitation that the low frequency CNN model “receives at least one of GPS speed, horizontal accuracy, or altitude.”
Rahmes teaches “receives at least one of GPS speed, horizontal accuracy, or altitude” [[0027] Referring additionally to the flow diagram 50 of FIG. 3, an example aircraft trajectory anomaly detection process is now described. The process starts (Block 51) with providing positional (e.g., GPS) information data as input trajectory data, which includes a state vector composition of sequential latitudes, longitudes, and altitudes as input data into an ensemble of DNNs, at Block 52. At Block 53, the vector may be resampled to normalize timing and velocity samples. At Block 54, normal and abnormal trajectory data are input into DNNs as training data and testing data. At Block 55, the DNNs use convolutional neural network (CNN) or long short-term memory (LSTM) architectures from which a residual neural network is constructed (Block 56). At Block 57, the DNNs learn to distinguish anomalies.]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of the references combine thus far with the teachings of Rahmes by implementing the low frequency model so that it receives altitude. The motivation would have been to analyze a variable that is relevant to detecting anomalies in the trajectory of a vehicle, as suggested by Rahmes (see paragraph quoted above).
As to claim 16, the combination of Nag, Kwon, and Yang teaches the system of claim 15, and the use of “global positioning system (GPS) speeds” [Nag, [0066]: “If the inertial sensor measurements exceed the threshold, a vehicle event can be detected, and a feature extraction process, such as the feature extraction process 218, can be performed on the sensor data 122. The feature extraction process 218 can generate inertial features 225A, GPS speed features 225B, and vehicle class information 225C (e.g., the features 225).”] but does not teach the further limitations of the instant dependent claim.
Simoncini teaches “wherein the high frequency CNN model receives accelerometer magnitude and interpolated global positioning system (GPS) speeds to a matching frequency that is convoluted together,” [[0016]: “In some implementations, the first CNN model may resample the GPS data and the IMU data, via interpolation, so that the GPS data and the IMU data have the same quantity of samples (e.g., that is a multiple θ=3 of the quantity of video data timestamps). The first CNN model may apply a set of convolutional operations, and may utilize a max pooling operation of size θ to align the sampling rates of the GPS data and the IMU data with the timestamps of the video data. The first CNN model may apply the same convolutional operation to process each signal independently to learn filters to be applied to a generic signal and to extract features describing a temporal evolution and preserving individual signal semantic meaning.” Note that aligning the sampling rates of both the GPS and the IMU data with the timestamps of the video data constitutes matching their frequencies. Applying the same convolution operation to the signals and processing them in the same model constitutes convolving them together, especially since the CNN is multilayered (see [0023]). Furthermore, note that “IMU data” refers to an accelerometer. See [0012]: “The IMU data may include data identifying acceleration measurements and angular velocity measurements of the vehicles over time.” In regards to the limitation of GPU speeds. While “speed” is not explicitly stated, this limitation is already taught in the existing combination of references, and Simoncini teaches that the GPU data includes location and time (“The GPS data may include data identifying GPS locations of the vehicles over time.”). Thus, while Simoncini does not explicitly mention “speed,” its teachings are consistent with the use of GPS data that is already processed in the form of speed data.]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of the references combined thus far with the teachings of Simoncini so as to arrive at the above-discussed features of the instant claim. The motivation for doing so would have been to prepare data in a manner that enables utilization of both GPS and IMU data to classify vehicle activity (see Simoncini, [0008]: “current techniques for classifying vehicle maneuvers fail to utilize sensor data, such as GPS data and IMU data, and lack the contextual detail to accurately classify the vehicle maneuver.”).
The combination of references thus far does not teach the remaining limitation that the low frequency CNN model “receives at least one of GPS speed, horizontal accuracy, or altitude.”
Rahmes teaches “receives at least one of GPS speed, horizontal accuracy, or altitude” [[0027] Referring additionally to the flow diagram 50 of FIG. 3, an example aircraft trajectory anomaly detection process is now described. The process starts (Block 51) with providing positional (e.g., GPS) information data as input trajectory data, which includes a state vector composition of sequential latitudes, longitudes, and altitudes as input data into an ensemble of DNNs, at Block 52. At Block 53, the vector may be resampled to normalize timing and velocity samples. At Block 54, normal and abnormal trajectory data are input into DNNs as training data and testing data. At Block 55, the DNNs use convolutional neural network (CNN) or long short-term memory (LSTM) architectures from which a residual neural network is constructed (Block 56). At Block 57, the DNNs learn to distinguish anomalies.]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of the references combine thus far with the teachings of Rahmes by implementing the low frequency model so that it receives altitude. The motivation would have been to analyze a variable that is relevant to detecting anomalies in the trajectory of a vehicle, as suggested by Rahmes (see paragraph quoted above).
5. Claims 10-11 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Nag in view of Kwon, and further in view of Pal et al. (US 2018/0033220 A1) (“Pal”).
As to claim 10, the combination of Nag and Kwon teaches the method of claim 8, further comprising:
determining, according to the set of one or more kinematic variables, that the event occurred within a geographic region indicated by one or more geospatial filters; [Nag, [0040]: “The GPS measurements can also be used by the computing device 115 to estimate a location, speed, or direction of travel of the vehicle 110, for example, when determining whether the vehicle is traveling toward a region in which accidents or collision events frequently occur.” Nag, [0007]: “The method may include identifying a region within which vehicle events are to be classified as non-collision events by executing a clustering technique using a plurality of vehicle events.” Nag, [0091]: “For example, one or more regions (e.g., epsilon distance of cluster points) within which vehicle events are to be classified as non-collision events can be identified by executing a clustering function over the location data of classified non-collision events. Then, when a collision event are detected, and it is determined the collision event has a confidence level below a threshold and is located within the region, notifications corresponding to that collision event can be suppressed, as described herein.” Note that matching the current event with a pre-identified region constitutes the use of a geospatial filter (filtering regions to determine one that matches the currently detected collision event).] and
generating the prediction that the event is the second type of impact […]. [Nag, [0091]: “when a collision event are detected, and it is determined the collision event has a confidence level below a threshold.” Note that below the threshold corresponds to non-collision.]
The combination of references thus far does not teach the limitation that the prediction is “based on the determination that the event occurred within the geographic region.”
Pal teaches “based on the determination that the event occurred within the geographic region.” [[0017]: “The method 100 functions to enable vehicular accident detection based on movement data (e.g., relating to position, velocity, and/or acceleration) and/or supplemental data.” [0024]: “Further, the technology can take advantage of the non-generic sensor data and/or supplemental data (e.g., vehicle sensor data, weather data, traffic data, biosignal sensor data, etc.) to better improve the understanding of correlations between such data and vehicular accident events, leading to an increased understanding of vehicular-related and driver-related variables affecting vehicular accidents.” Here, the supplemental data includes historical traffic accident data and correlating the location of the data with the user. See [0074]: “Traffic data can include any one or more of: accident data (e.g., number of accidents within a predetermined radius of the user, accident frequency, accident rate, types of accidents, frequency of accidents, etc.).”]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of the references combined thus far with the teachings of Pal by implementing the prediction to be “based on the determination that the event occurred within the geographic region.” The motivation would have been to utilize supplemental data that can “improve the understanding of correlations between such data and vehicular accident events” (Pal, [0024]).
As to claim 11, the combination of Nag and Kwon teaches the method of claim 8, further comprising:
converting the set of one or more kinematic variables into features that represent GPS speed [In general, the features are extracted from sensor data. See Nag, [0006]: “The method may include generating a set of features by executing a feature extraction function using at least one of the inertial sensor data or the speed captured during the time period.” Regarding GPS speed, see Nag, [0040]: “The computing device 115 can use signals captured by the GPS receiver and/or motion sensors (e.g., the sensor data 122) to estimate the speed of the vehicle 110. For example, by periodically accessing GPS measurements from the sensor data 122 and measuring the difference between two position measurements over time, the computing device 115 can estimate an average velocity of the vehicle 110.”] and […] accelerometer magnitude properties associated with the movement of the mobile device, [Nag, [0057]: “The plots represents 6-axis time-series signals from accelerometer and gyroscope sensors, for each event.” [0040]: “The sensors 135 may include odometry information, which may indicate patterns or intensity of acceleration performed by a driver in the vehicle.”] wherein the prediction score is determined based on the features, and wherein the machine-learning collision prediction model is trained to learn which of the features contribute to predicting whether the movement is associated with the first type of impact. [Nag, [0061] The historical data 265 can be utilized as training data to train one or more of the artificial intelligence models 230, for example, using a suitable supervised machine-learning algorithm. The historical data 265 can include sets of historical features 225 captured during operation of one or more vehicles…. The sets of historical data 265, with the ground-truth labels, can be used in a supervised learning process to train one or more of the artificial intelligence models 230.”]
The combination of references thus far does not teach the limitation that the features represent “altitude properties.”
Pal teaches “altitude properties” [[0104]: “Block S142 preferably includes extracting a vehicle motion characteristic (e.g., to compare against motion characteristic threshold). Vehicle motion characteristics and motion characteristic thresholds can typify motion characteristic types including any one or more of… position (e.g., altitude, GPS coordinates, direction, location, etc.)”]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of the references combined thus far with the teachings of Pal by including altitude as an element represented by the features. Doing so would have enabled the vehicle’s altitude to be taken into consideration by the model, so as to account for a more complete set of vehicle motion characteristics relevant to detecting vehicle related-events, as suggested by Pal (see part quoted above and [0021]: “accurately distinguish between collision events (e.g., single-vehicle collisions, multi-vehicle collisions, etc.) and non-collision events (e.g., hard braking, rapid acceleration, etc.)”).
As to claim 19, the combination of Nag and Kwon teaches the system of claim 17, wherein the computer-executable instructions further configure the one or more processors to cause the system to:
determine, according to the set of one or more kinematic variables, that the event occurred within a geographic region indicated by one or more geospatial filters; [Nag, [0040]: “The GPS measurements can also be used by the computing device 115 to estimate a location, speed, or direction of travel of the vehicle 110, for example, when determining whether the vehicle is traveling toward a region in which accidents or collision events frequently occur.” Nag, [0007]: “The method may include identifying a region within which vehicle events are to be classified as non-collision events by executing a clustering technique using a plurality of vehicle events.” Nag, [0091]: “For example, one or more regions (e.g., epsilon distance of cluster points) within which vehicle events are to be classified as non-collision events can be identified by executing a clustering function over the location data of classified non-collision events. Then, when a collision event are detected, and it is determined the collision event has a confidence level below a threshold and is located within the region, notifications corresponding to that collision event can be suppressed, as described herein.” Note that matching the current event with a pre-identified region constitutes the use of a geospatial filter (filtering regions to determine one that matches the currently detected collision event).] and
generate the prediction that the event is the second type of impact […] [Nag, [0091]: “when a collision event are detected, and it is determined the collision event has a confidence level below a threshold.” Note that below the threshold corresponds to non-collision.]
The combination of references thus far does not teach the limitation that the prediction is “based on the determination that the event occurred within the geographic region.”
Pal teaches “based on the determination that the event occurred within the geographic region.” [[0017]: “The method 100 functions to enable vehicular accident detection based on movement data (e.g., relating to position, velocity, and/or acceleration) and/or supplemental data.” [0024]: “Further, the technology can take advantage of the non-generic sensor data and/or supplemental data (e.g., vehicle sensor data, weather data, traffic data, biosignal sensor data, etc.) to better improve the understanding of correlations between such data and vehicular accident events, leading to an increased understanding of vehicular-related and driver-related variables affecting vehicular accidents.” Here, the supplemental data includes historical traffic accident data and correlating the location of the data with the user. See [0074]: “Traffic data can include any one or more of: accident data (e.g., number of accidents within a predetermined radius of the user, accident frequency, accident rate, types of accidents, frequency of accidents, etc.).”]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of the references combined thus far with the teachings of Pal by implementing the prediction to be “based on the determination that the event occurred within the geographic region.” The motivation would have been to utilize supplemental data that can “improve the understanding of correlations between such data and vehicular accident events” (Pal, [0024]).
6. Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Nag in view of Yang.
As to claim 20, Nag teaches one or more tangible non-transitory computer-readable storage media storing computer-executable instructions for performing a computer process on a computing system, the computer process comprising: [[0009]: “Another embodiment is directed to a system for detecting a vehicle collision. The system can include at least one processor coupled to a non-transitory memory.” [0095]: “When implemented in software, the functions may be stored as one or more instructions or code on a non-transitory computer-readable or processor-readable storage medium.”]
inputting, in a machine-learning collision prediction model, [The limitation of a “machine-learning collision prediction model” is taught in the form of a set of machine learning models that includes an “initial classifier 304” ([0069]), which is a trained machine learning model (see [0068]: “The initial classifier 304 can be trained prior to performing the operations of the flow diagram 300, for example, using a supervised learning process…” [0068]: “The initial classifier 304 can include any number of layers, such as fully connected layers, convolutional layers, activation layers, softmax layers, or other types of neural network layers.” The limitations of both a first and a third layer are met by the teaching of plural layers.] a set of one or more kinematic variables associated with a trigger event based on movement of a mobile device, wherein the set of one or more kinematic variables are recorded over a duration of time, [[0031]: “In a non-limiting example, the computing device can determine a vehicle event based on inertial sensor data and speed from at least one sensor in a housing inside a cabin of a vehicle and classify the vehicle event as a collision event or a non-collision event based on the inertial sensor data.” [0066]: “If the inertial sensor measurements exceed the threshold, a vehicle event can be detected, and a feature extraction process, such as the feature extraction process 218, can be performed on the sensor data 122. The feature extraction process 218 can generate inertial features 225A, GPS speed features 225B, and vehicle class information 225C (e.g., the features 225).” Here, speed is a kinematic variable, as well as inertial sensor data (see [0039], which teaches accelerometers). The data is recorded. See [0018]: “FIG. 4 shows an example time-series graph showing windows of sensor data before, during, and after a potential collision event, according to an embodiment.”] […]
generating, by the machine-learning collision prediction model, a prediction score, wherein the prediction score is based on the set of one or more kinematic variables and associated with the event; [[0069]: “The initial classifier 304 may be further trained to generate a confidence value, which may indicate the likelihood that the prediction of the collision or the non-collision is accurate.” The output is based on the kinematic variables. See [0066]: “The features 225 can be extracted for a number of time windows 215 in the sensor data 122, as described herein. Upon extracting the features that correspond to the vehicle event, the features can be provided as input to the initial classifier 304.”]
determining a prediction, according to the prediction score, that the event is one of a first type of impact; [[0023]: “The systems and methods described herein provide techniques to both detect and classify various collision events and non-collision events (e.g., that may indicate a false positive potential collision event) …Such approaches cannot detect and classify non-collision events or collision events into sub-categories (e.g., types or classes of non-collision or collision events involving the vehicle).” That is, non-collision/collision is an event type. Note that as defined in this application (see paragraph 6 of the specification), non-collision is considered to be a type of impact. See also [0076]: “The collision classifier 314 can generate a prediction of a collision subclass, which can include a front-end collision, a rear-end (e.g., a back-end) collision, a left collision, a right collision, a low-clearance (e.g., top) collision, a ground clearance collision (e.g., an undercarriage collision), a bird or animal collision, or a vehicle topple event, among others.” This is based on the prediction score. See [0075]: “However, if the confidence value of the predicted collision event is greater than or equal to the threshold, the features 225 (e.g., which may have undergone a transformation process 230) can be provided as input to the collision classifier 314.”] and
outputting the prediction, wherein the outputted prediction is time-oriented. [The generation of the prediction also constitutes outputting. Furthermore, the prediction is also outputted as a notification, as disclosed in [0077]: “The notification may include a type of subclass and in some scenarios, the notification may further include a query and requests a response from the driver.” Furthermore, the prediction is time-oriented because it is a prediction for a certain time window. See, e.g., [0088]: “Determining a vehicle event can include identifying a time period or time window”; [0066]: “The features 225 can be extracted for a number of time windows 215 in the sensor data 122, as described herein.”]
Nag does not teach “wherein the machine-learning collision prediction model includes at least a high frequency convolutional neural network (CNN) model layer and a low frequency CNN layer, wherein the high frequency CNN layer and the low frequency CNN layer extract features for time segments of the duration of time, and the respective features associated with respective time segments are fed into one or more long short term memory (LSTM) model layers.”
Yang teaches “wherein the machine-learning collision prediction model includes at least a high frequency convolutional neural network (CNN) model layer and a low frequency CNN layer, wherein the high frequency CNN layer and the low frequency CNN layer extract features for time segments of the duration of time, and the respective features associated with respective time segments are fed into one or more long short term memory (LSTM) model layers.” [§ II.A, paragraphs 1 and 4: “The overall framework of short-term WPP method based on CEEMDAN-FTC and parallel CNN-LSTM is shown in Fig.1… Total four CNN-LSTMs are established, in which the input feature is high frequency component, low frequency component, trend component, and original wind speed, respectively. The input of one CNN-LSTM is the original wind speed in four height and the input of rest three CNN-LSTMs are the three datasets derived in Step 3. Those four CNN-LSTMs are parallel to each other and are connected by a fully connected layer between the LSTM hidden layer and output layer.” As shown in FIG. 1, each CNN feeds into a respective LSTM. In doing so, the CNN acts as a feature extractor. See § II.C.1 (“In essence, CNN is a mathematical model that transforms the original input into a new feature representation through multilevel data transformation and dimensionality reduction.”). The limitation of “duration of time” is covered by the fact that the CNNs act on time series data (as indicated by the fact that the input has high/low frequencies) and mentioned in § I, paragraph 3 (“analyzing time-series data”).]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Nag with the teachings of Yang by implementing the use of a parallel CNN-LSTMs, so as to arrive at the claimed invention of the instant dependent claim. The motivation for doing so would have been to specialize different CNN-LSTMs models for data of different frequencies for improved prediction accuracy, as suggested by Yang (abstract: “In the second stage, a novel parallel CNN-LSTM neural network architecture is proposed as WPP model, in which the input feature consists both three frequency components derived in the first stage and the original wind speed sequence in different height. The results show that the proposed method is an effective short-term WPP method which improves the prediction accuracy.”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. The following document depicts the state of the art.
Liu et al., “DSDCLA: driving style detection via hybrid CNN-LSTM with multi-level attention fusion,” Applied Intelligence (2023) 53:19237–19254 teaches the use of CNN-LSTM for vehicle applications, and also specifically teach separate CNNs for different modalities.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YAO DAVID HUANG whose telephone number is (571)270-1764. The examiner can normally be reached Monday - Friday 9:00 am - 5:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached at (571) 270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Y.D.H./Examiner, Art Unit 2124
/MIRANDA M HUANG/Supervisory Patent Examiner, Art Unit 2124
Prosecution Insights