DETAILED ACTION
This action is in response the communications filed on 05/15/2026 in which claims 12-13, 17, 19 and 21-22 are amended, and claims 12-22 are pending. The non-final of the previous action is withdrawn, and new grounds of rejection are set forth below. This action is made NONFINAL.
--
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 12-21 are rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claim 12 is unclear what the claim is directed to. The claim starts with “A control device implemented in a vehicle” but contains no further limitations to the “control device”. Instead, the claim recites limitations for a “perception system…comprising an estimation device…”, “the estimation device comprising an online learning module…the learning module comprising: a forward propagation module…a fusion system configured to…and a backpropagation module configured to…”. It is unclear if the claim is attempting to be directed to a “control device” (and if so, what the components are) or the “perception system.”
Claims 13-21 are also rejected due to their dependency on a rejected claim.
112
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
-
Claims 12-22 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more
Step 1: Claims 12-21 recite a device. Claim 22 recites a method. Therefore, claims 12-21 are directed to a machine, and claims 22 are directed to a process.
With respect to claim 12:
2A Prong 1: The claim recites a judicial exception.
determine a fusion output comprising a consolidated object state vector by implementing at least one sensor fusion algorithm based on said predicted output (mathematical concept, in light of specification 0139, 0141, 0155, 0225, 0234)
update the weights associated with the neural network online by determining a loss function representing an error between an improved predicted value of said fusion output and said predicted output by performing a gradient descent backpropagation (mathematical concept - mathematical calculation, in light of specification 0155-0156)
2A Prong 2: The judicial exception is not integrated into a practical application.
the vehicle comprising a perception system using a set of sensors, each sensor providing data (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting; providing data using a set of sensors)
the perception system comprising an estimation device configured to estimate a variable comprising at least one feature in relation to one or more objects detected in an environment of the vehicle, the estimation device comprising an online learning module using a neural network to estimate said variable, the neural network being associated with a set of weights, the learning module comprising (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of using a neural network to estimate the variable)
propagate data from one or more sensors applied at an input of the neural network, so as to provide a predicted output comprising an estimation of said variable (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of using the neural network to propagate data to provide an output)
a forward propagation module configured to… a fusion system configured to… a backpropagation module configured to… (mere instructions to apply an exception, (2) Whether the claim invokes computers - MPEP 2106.05(f); generic computer components; 112(f) limitation is interpreted as the combination of a processor and those modules)
said consolidated object state vector being used for controlling the vehicle; (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; using a vector for controlling the vehicle)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
the vehicle comprising a perception system using a set of sensors, each sensor providing data (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting, and WURC: receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 - MPEP 2106.05(d)(II)(i)); providing data using a set of sensors)
the perception system comprising an estimation device configured to estimate a variable comprising at least one feature in relation to one or more objects detected in an environment of the vehicle, the estimation device comprising an online learning module using a neural network to estimate said variable, the neural network being associated with a set of weights, the learning module comprising (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of using a neural network to estimate the variable)
propagate data from one or more sensors applied at an input of the neural network, so as to provide a predicted output comprising an estimation of said variable (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of using the neural network to propagate data to provide an output)
a forward propagation module configured to… a fusion system configured to… a backpropagation module configured to… (mere instructions to apply an exception, (2) Whether the claim invokes computers - MPEP 2106.05(f); generic computer components; 112(f) limitation is interpreted as the combination of a processor and those modules)
said consolidated object state vector being used for controlling the vehicle; (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; using a vector for controlling the vehicle)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 13:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein said variable is a state vector comprising information in relation to at least one of a position and a movement of an object detected by the perception system. (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; Claim 12 recites “using a neural network to estimate said variable,” which is mere instructions to apply an exception – MPEP 2106.05(f). Specifying the details of the variable does not cause the limitation to integrate the exception into a practical application)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein said variable is a state vector comprising information in relation to at least one of a position and a movement of an object detected by the perception system. (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; Claim 12 recites “using a neural network to estimate said variable,” which is mere instructions to apply an exception – MPEP 2106.05(f). Specifying the details of the variable does not cause the limitation to be significantly more than the judicial exception.)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 14:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein said state vector further comprises information in relation to one or more detected objects (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; Claim 12 recites “using a neural network to estimate said variable” and claim 13 recites “said variable is a state vector” which is mere instructions to apply an exception – MPEP 2106.05(f). Specifying the details of the state variable does not cause the limitation to integrate the exception into a practical application)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein said state vector further comprises information in relation to one or more detected objects (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; Claim 12 recites “using a neural network to estimate said variable” and claim 13 recites “said variable is a state vector” which is mere instructions to apply an exception – MPEP 2106.05(f). Specifying the details of the state variable does not cause the limitation to be significantly more than the judicial exception.)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 15:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein said state vector further comprises trajectory parameters of a target object (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; Claim 12 recites “using a neural network to estimate said variable” and claim 13 recites “said variable is a state vector” which is mere instructions to apply an exception – MPEP 2106.05(f). Specifying the details of the state variable does not cause the limitation to integrate the exception into a practical application)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein said state vector further comprises trajectory parameters of a target object (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; Claim 12 recites “using a neural network to estimate said variable” and claim 13 recites “said variable is a state vector” which is mere instructions to apply an exception – MPEP 2106.05(f). Specifying the details of the state variable does not cause the limitation to be significantly more than the judicial exception.)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 16:
2A Prong 1: the claim recites a judicial exception.
wherein said improved predicted value is determined by applying a Kalman filter (mathematical concept, in light of specification 0171-187)
With respect to claim 17:
2A Prong 2: The judicial exception is not integrated into a practical application.
further comprising a replay buffer configured to store the predicted output and the fusion output. (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting; storing the outputs)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
further comprising a replay buffer configured to store the predicted output and the fusion output. (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting, and WURC: storing and retrieving information in memory, Versata Dev. Group, Inc. - MPEP 2106.05(d)(II)(iv); storing the outputs)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 18:
2A Prong 2: The judicial exception is not integrated into a practical application.
further comprising a recurrent neural network encoder configured to encode and compress the data prior to storage in the replay buffer, and a decoder configured to decode and decompress the data extracted from the replay buffer (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of using a RNN to encode and a decoder to decode)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
further comprising a recurrent neural network encoder configured to encode and compress the data prior to storage in the replay buffer, and a decoder configured to decode and decompress the data extracted from the replay buffer (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of using a RNN to encode and a decoder to decode)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 19:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein the decoder is a recurrent neural network decoder (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; Claim 18 recites “a recurrent neural network encoder… a decoder,” which is mere instructions to apply an exception. Specifying the specific model does not cause the limitation to integrate the exception into a practical application.)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein the decoder is a recurrent neural network decoder (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; Claim 18 recites “a recurrent neural network encoder… a decoder,” which is mere instructions to apply an exception. Specifying the specific model does not cause the limitation to be significantly more than the judicial exception.)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 20:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein the replay buffer is prioritized (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting; Claim 17 recites “a replay buffer configured to store the outputs,” which is insignificant extra-solution activity. Specifying the buffer is prioritized does not cause the limitation to integrate the exception into a practical application.)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein the replay buffer is prioritized (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting, and WURC: storing and retrieving information in memory, Versata Dev. Group, Inc. - MPEP 2106.05(d)(II)(iv); Claim 17 recites “a replay buffer configured to store the outputs,” which is insignificant extra-solution activity. Specifying the buffer is prioritized does not cause the limitation to be significantly more than the judicial exception.)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 21:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein the device is configured to implement a condition for testing input data applied at an input of the neural network such that the input data is deleted from the replay buffer when the loss function between the improved predicted value of the fusion output and said predicted output is lower than a predefined threshold. (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; under BRI, using the device to implement a condition to the input of the neural network)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein the device is configured to implement a condition for testing input data applied at an input of the neural network such that the input data is deleted from the replay buffer when the loss function between the improved predicted value of the fusion output and said predicted output is lower than a predefined threshold. (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; under BRI, using the device to implement a condition to the input of the neural network)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 22:
2A Prong 1: The claim recites a judicial exception.
determining a fusion output comprising a consolidated object state vector by implementing at least one sensor fusion algorithm based on said predicted output (mathematical concept, in light of specification 0139, 0141, 0155, 0225, 0234)
updating the weights associated with the neural network online by determining a loss function representing an error between an improved predicted value of said fusion output and said predicted output by performing a gradient descent backpropagation (mathematical concept - mathematical calculation, in light of specification 0155-0156)
2A Prong 2: The judicial exception is not integrated into a practical application.
the vehicle comprising a perception system using a set of sensors, each sensor providing data (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting; providing data using a set of sensors)
the control method comprising: estimating a variable comprising at least one feature in relation to one or more objects detected in an environment of the vehicle, wherein the estimating implements online learning step a neural network to estimate said variable, the neural network being associated with a set of weights, wherein the online learning comprises (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of using a neural network to estimate the variable)
propagating data from one or more sensors, applied at an input of the neural network, so as to provide a predicted output comprising an estimation of said variable (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of using the neural network to propagate data to provide an output)
said consolidated object state vector being used for controlling the vehicle; (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; using a vector for controlling the vehicle)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
the vehicle comprising a perception system using a set of sensors, each sensor providing data (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting; providing data using a set of sensors, and WURC: receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 - MPEP 2106.05(d)(II)(i))
the control method comprising: estimating a variable comprising at least one feature in relation to one or more objects detected in an environment of the vehicle, wherein the estimating implements online learning step a neural network to estimate said variable, the neural network being associated with a set of weights, wherein the online learning comprises (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of using a neural network to estimate the variable)
propagating data from one or more sensors, applied at an input of the neural network, so as to provide a predicted output comprising an estimation of said variable (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of using the neural network to propagate data to provide an output)
said consolidated object state vector being used for controlling the vehicle; (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; using a vector for controlling the vehicle)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 12-16 and 22 rejected under 35 U.S.C. 103 as being unpatentable over Ju ("Interaction-aware Kalman Neural Networks for Trajectory Prediction" 20201019) in view of Haarnoja ("Backprop KF: Learning Discriminative Deterministic State Estimators" 2016)
In regard to claim 12, Ju teaches: A control device implemented in a vehicle, the vehicle comprising a perception system using a set of sensors, each sensor providing data, (Ju, p. 1793, I. Introduction "Autonomous driving systems can be broadly categorized into three hierarchical subsystems, namely perception/localization, planning and control [1]. The perception subsystem [a perception system] refers to the ability to acquire information from the environment via multiple vehicle sensors like GPS, LiDAR, RADAR, and Camera. [a set of sensors]") the perception system comprising an estimation device configured to estimate a variable comprising at least one feature in relation to one or more objects detected in an environment of the vehicle, (Ju, p. 1794, I. Introduction "In this paper, we propose a specific model for the prediction subsystem [the perception system, an estimation device] called Interaction-aware Kalman Neural Networks (IaKNN). IaKNN is a multi-layer architecture consisting of three layers, namely an interaction layer, a motion layer, and a filter layer."; p. 1794, 2) Data-Driven Approach Trajectory Prediction "Our IaKNN model differs from these models in two aspects... Second, IaKNN employs the Kalman filter for optimizing the state estimation [S^t: estimate a variable, a state], where LSTM neural networks are used for learning the time-varying process and measurement noises that are used in the Kalman model..."; p. 1795, V. Problem Statement "Given the past h-length environmental observations Ot := {ot−h+1, ot−h+2, ⋯ , ot}, we aim to predict the future L-length trajectories of each vehicle. [in relation to future trajectories of each vehicle (objects)]") the estimation device comprising an online learning module using a neural network to estimate said variable, the neural network being associated with a set of weights, the learning module comprising: (Ju, p. 1794, I. Introduction "In this paper, we propose a specific model for the prediction subsystem [the estimation device] called Interaction-aware Kalman Neural Networks (IaKNN). [a neural network] IaKNN is a multi-layer architecture consisting of three layers, namely an interaction layer, a motion layer, and a filter layer."; p. 1794, 2) Data-Driven Approach Trajectory Prediction "Our IaKNN model differs from these models in two aspects... Second, IaKNN employs the Kalman filter for optimizing the state estimation [S^t: estimate said variable], where LSTM neural networks are used for learning..."; p. 1795, IV. Kalman Filter "the filter we used is in some sense an advanced version of KF where the noise covariances are being learned online but not pre-set."; p. 1799, D. Implementation Details "The default number of hidden units in LSTMs in the interaction layer and filter layer is set to 32 and all LSTM weight matrices [a set of weights] are initialized using a uniform distribution over [−0.001, 0.001]. The weight matrices for other layers are set with the Xavier initialization.")
a forward propagation module configured to propagate data from one or more sensors applied at an input of the neural network, so as to provide a predicted output comprising an estimation of said variable; (Ju, p. 1796-1797, VI. Methodology "... In the following, we present three layers of IaKNN, namely the interaction layer, the motion layer and the filter layer... In the interaction layer, we aim to extract the interaction-aware accelerations AS from the past traffic environment observations Ot [data applied at an input of the neural newwork, e.g. a camera image]... In the motion layer, we aim to calculate the interaction-aware trajectories T based on the interaction-aware accelerations AS from the interaction layer... In the filter layer, we establish a model based on the Kalman filter to estimate the dynamic trajectories St [S^t: a predicted output, an estimation of said variable] based on the interaction-aware trajectories Tt used as observations..."; forward propagation through three layers Ot-> AS -> Tt - > St (the interaction layer, the motion layer and the filter layer) to
PNG
media_image1.png
338
992
media_image1.png
Greyscale
provide St)
a fusion system configured to determine a fusion output comprising a consolidated object state vector by implementing at least one sensor fusion algorithm based on said predicted output, said consolidated object state vector being used for controlling the vehicle; (Ju, p. 1797, C. Filter Layer "In the filter layer, we establish a model based on the Kalman filter to estimate the dynamic trajectories St based on the interaction-aware trajectories Tt used as observations... We assume N agents (dynamic obstacles) in the multi-agent system. At timestamp t, the state St and the observation Tt of our Equation 2 and 3 could be written as follows. St:= [St1… StN] [a consolidated object state vector]… and Tt := [Tt1… TtN]... 3) Prediction and Updated Steps, The prediction step of the Kalman filter [sensor fusion algorithm based on St] is defined as, St = F ⋅ S^t-1 + B ⋅ Ut-1, Pt = F ⋅ P^t-1 ⋅ FT + Qt, and the update step is as... S^t = St- + Kt ⋅(Tt−St) [a fusion output]..."; p. 1795, Fig. 2 "... time-varying multi-agent kalman neural networks run over the predicted time horizon L to fuse dynamic trajectory St and interaction-aware trajectory Tt. [fusing St and Tt]"; p. 1793, I. Introduction "The control subsystem refers to the ability to execute the planned actions in trajectories from planning by sending accelerations, brake, and steering messages to the actuator on intelligent vehicles. [controlling the vehicle] "; see Fig. 1 the output of laKNN is used for controlling vehicles)
PNG
media_image2.png
592
644
media_image2.png
Greyscale
a backpropagation module configured to update the weights associated with the neural network online… (Ju, p. 1798, C. Filter Layer "Since our desired filter model is time-varying… Qt := LSTM_Q(S-t0:t) and Rt := LSTM_R(Tt0:t), respectively. Here, we adopted LSTM because the noises are generated sequentially with trajectories and LSTM is known for its capability of capturing sequential dynamics [37]. The concrete Qt and Rt are tractable via backpropagation under the end-to-end trainable architecture.")
Ju does not teach, but Haarnoja teaches: by determining a loss function representing an error between an improved predicted value of said fusion output and said predicted output by performing a gradient descent backpropagation. (Haarnoja, p. 4, 4 Discriminative Deterministic State Estimation "Let L(θ)... be the loss [a loss function representing an error] on an entire sequence with respect to θ... We can compute the gradient of l(θ) with respect to the parameters θ by first recursively computing the gradient of the loss with respect to the filter state st from the back to the front according to the following recursion: dL/d_st-1 = ... d_st/d_st-1... (1) [an improved predicted value of said fusion output (st or zt) and said predicted output (st-1 or zt-1)] ... The parameters θ can be optimized with gradient descent using these gradients. This is an instance of backpropagation through time (BPTT), a well known
PNG
media_image3.png
362
472
media_image3.png
Greyscale
algorithm for training recurrent neural networks")
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Ju to incorporate the teachings of Haarnoja by including a deterministic computation graph trained with gradient descent algorithm. Doing so would significant improvement over both standard generative approaches and regular recurrent neural networks. (Haarnoja, p. 1, Abstract "We present an alternative approach where the parameters of the latent state distribution are directly optimized as a deterministic computation graph, resulting in a simple and effective gradient descent algorithm for training discriminative state estimators. We show that this procedure can be used to train state estimators... Our model can be viewed as a type of recurrent neural network... We evaluate our approach on synthetic tracking task... The results show significant improvement over both standard generative approaches and regular recurrent neural networks.")
In regard to claim 13, Ju teaches: wherein said variable is a state vector comprising information in relation to at least one of a position and a movement of an object detected by the perception system. (Ju, p. 1797, C. Filter Layer "In the filter layer, we establish a model based on the Kalman filter to estimate the dynamic trajectories St [said variable] based on the interaction-aware trajectories Tt used as observations... We assume N agents (dynamic obstacles) [an object detected by the perception system] in the multi-agent system. At timestamp t, the state St and the observation Tt of our Equation 2 and 3 could be written as follows. St:= [St1… StN] [e.g. a state vector]… and Tt := [Tt1… TtN] where the state S_it includes positions [a position] p_ik from GPS and velocities v_ik from the wheel odometer and the observation T_it includes the predicted positions p¯_ik and the predicted velocities v¯_ik, where t+1 ≤ k ≤ t+L. Specifically, we have the following. St_i:= [p_t+1 v+t+1…] [e.g. a state vector]... At timestamp t, the filter layer in an operator formula is written as, Filter{W,b} : {S^t-1, Tt, Ut-1} ⟼ S^t, where S^t is the Posteriori estimation of dynamic trajectory [said variable] from t+1 to t+L.")
In regard to claim 14, Ju teaches: wherein said state vector further comprises information in relation to one or more detected objects. (Ju, p. 1797, C. Filter Layer "In the filter layer, we establish a model based on the Kalman filter to estimate the dynamic trajectories St... We assume N agents (dynamic obstacles) [one or more detected objects] in the multi-agent system. At timestamp t, the state St and the observation Tt of our Equation 2 and 3 could be written as follows. St:= [St1… StN] [said state vector]… and Tt := [Tt1… TtN]... Specifically, we have the following. St_i:= [p_t+1 v+t+1…]")
In regard to claim 15, Ju teaches: wherein said state vector further comprises trajectory parameters of a target object. (Ju, p. 1797, C. Filter Layer "In the filter layer, we establish a model based on the Kalman filter to estimate the dynamic trajectories St... We assume N agents (dynamic obstacles) [one or more detected objects] in the multi-agent system. At timestamp t, the state St and the observation Tt of our Equation 2 and 3 could be written as follows. St:= [St1… StN] [said state vector] … and Tt := [Tt1… TtN] where the state S_it includes positions p_ik from GPS and velocities [trajectory parameters (position and velocity) of a target object] v_ik from the wheel odometer and the observation T_it includes the predicted positions p¯_ik and the predicted velocities v¯_ik, where t+1 ≤ k ≤ t+L. Specifically, we have the following. St_i:= [p_t+1 v+t+1…]")
In regard to claim 16, Ju teaches: wherein said improved predicted value is determined by applying a Kalman filter. (Ju, p. 1797, C. Filter Layer "In the filter layer, we establish a model based on the Kalman filter to estimate the dynamic trajectories St based on the interaction-aware trajectories Tt used as observations... the equation of the linear dynamic model could be written as follows. St [said improved predicted value] = F ⋅ St-1 + B ⋅ Ut-1 + ωt (2), Tt = St + ηt (3)"; a state changes from St-1 to St (or St to St+1), making St (or St+1) an improved version of the state)
In regard to claim 22, Ju teaches: A control method implemented in a vehicle, the vehicle comprising a perception system using a set of sensors, each sensor providing data, the control method comprising: (Ju, p. 1793, I. Introduction "Autonomous driving systems can be broadly categorized into three hierarchical subsystems, namely perception/localization, planning and control [1]. The perception subsystem [a perception system] refers to the ability to acquire information from the environment via multiple vehicle sensors like GPS, LiDAR, RADAR, and Camera. [a set of sensors]") estimating a variable comprising at least one feature in relation to one or more objects detected in an environment of the vehicle, (Ju, p. 1794, I. Introduction "In this paper, we propose a specific model for the prediction subsystem [the perception system, an estimation device] called Interaction-aware Kalman Neural Networks (IaKNN). IaKNN is a multi-layer architecture consisting of three layers, namely an interaction layer, a motion layer, and a filter layer."; p. 1794, 2) Data-Driven Approach Trajectory Prediction "Our IaKNN model differs from these models in two aspects... Second, IaKNN employs the Kalman filter for optimizing the state estimation [S^t: estimate a variable, a state], where LSTM neural networks are used for learning the time-varying process and measurement noises that are used in the Kalman model..."; p. 1795, V. Problem Statement "Given the past h-length environmental observations Ot := {ot−h+1, ot−h+2, ⋯ , ot}, we aim to predict the future L-length trajectories of each vehicle. [in relation to each vehicle (objects detected)]") wherein the estimating implements online learning step a neural network to estimate said variable, the neural network being associated with a set of weights, (Ju, p. 1794, I. Introduction "In this paper, we propose a specific model for the prediction subsystem called Interaction-aware Kalman Neural Networks (IaKNN). [a neural network] IaKNN is a multi-layer architecture consisting of three layers, namely an interaction layer, a motion layer, and a filter layer."; p. 1794, 2) Data-Driven Approach Trajectory Prediction "Our IaKNN model differs from these models in two aspects... Second, IaKNN employs the Kalman filter for optimizing the state estimation [S^t: estimate said variable], where LSTM neural networks are used for learning..."; p. 1795, IV. Kalman Filter "the filter we used is in some sense an advanced version of KF where the noise covariances are being learned online but not pre-set."; p. 1799, D. Implementation Details "The default number of hidden units in LSTMs in the interaction layer and filter layer is set to 32 and all LSTM weight matrices [a set of weights] are initialized using a uniform distribution over [−0.001, 0.001]. The weight matrices for other layers are set with the Xavier initialization.")
wherein the online learning comprises: propagating data from one or more sensors, applied at an input of the neural network, so as to provide a predicted output comprising an estimation of said variable; (Ju, p. 1796-1797, VI. Methodology "... In the following, we present three layers of IaKNN, namely the interaction layer, the motion layer and the filter layer... In the interaction layer, we aim to extract the interaction-aware accelerations AS from the past traffic environment observations Ot [data applied at an input of the neural newwork, e.g. a camera (sensor) image]... In the motion layer, we aim to calculate the interaction-aware trajectories T based on the interaction-aware accelerations AS from the interaction layer... In the filter layer, we establish a model based on the Kalman filter to estimate the dynamic trajectories St [S^t: a predicted output, an estimation of said variable] based on the interaction-aware trajectories Tt used as observations..."; forward propagation through three layers Ot-> AS -> Tt - > St (the interaction layer, the motion layer and the filter layer) to provide St)
determining a fusion output comprising a consolidated object state vector by implementing at least one sensor fusion algorithm based on said predicted output, said consolidated object state vector being used for controlling the vehicle; (Ju, p. 1797, C. Filter Layer "In the filter layer, we establish a model based on the Kalman filter to estimate the dynamic trajectories St based on the interaction-aware trajectories Tt used as observations... We assume N agents (dynamic obstacles) in the multi-agent system. At timestamp t, the state St and the observation Tt of our Equation 2 and 3 could be written as follows. St:= [St1… StN] [a consolidated object state vector]… and Tt := [Tt1… TtN]... 3) Prediction and Updated Steps, The prediction step of the Kalman filter [sensor fusion algorithm based on St] is defined as, St = F ⋅ S^t-1 + B ⋅ Ut-1, Pt = F ⋅ P^t-1 ⋅ FT + Qt, and the update step is as... S^t = St- + Kt ⋅(Tt−St) [a fusion output]..."; p. 1795, Fig. 2 "... time-varying multi-agent kalman neural networks run over the predicted time horizon L to fuse dynamic trajectory St and interaction-aware trajectory Tt. [fusing St and Tt]"; p. 1793, I. Introduction "The control subsystem refers to the ability to execute the planned actions in trajectories from planning by sending accelerations, brake, and steering messages to the actuator on intelligent vehicles. [controlling the vehicle] "; see Fig. 1 the output of laKNN is used for controlling vehicles)
updating the weights associated with the neural network online… (Ju, p. 1798, C. Filter Layer "Since our desired filter model is time-varying… Qt := LSTM_Q(S-t0:t) and Rt := LSTM_R(Tt0:t), respectively. Here, we adopted LSTM because the noises are generated sequentially with trajectories and LSTM is known for its capability of capturing sequential dynamics [37]. The concrete Qt and Rt are tractable via backpropagation under the end-to-end trainable architecture.")
Ju does not teach, but Haarnoja teaches:by determining a loss function representing an error between an improved predicted value of said fusion output and said predicted output by performing a gradient descent backpropagation. (Haarnoja, p. 4, 4 Discriminative Deterministic State Estimation "Let L(θ)... be the loss [a loss function representing an error] on an entire sequence with respect to θ... We can compute the gradient of l(θ) with respect to the parameters θ by first recursively computing the gradient of the loss with respect to the filter state st from the back to the front according to the following recursion: dL/d_st-1 = ... d_st/d_st-1... (1) [an improved predicted value of said fusion output (st or zt) and said predicted output (st-1 or zt-1)] ... The parameters θ can be optimized with gradient descent using these gradients. This is an instance of backpropagation through time (BPTT), a well known algorithm for training recurrent neural networks")
The rationale for combining the teachings of Ju and Haarnoja is the same as set forth in the rejection of claim 12.
Claims 17-19 rejected under 35 U.S.C. 103 as being unpatentable over Ju and Haarnoja as applied to claim 12, and in further view of Andersen ("The Dreaming Variational Autoencoder for Reinforcement Learning Environments" 20181002)
In regard to claim 17, Ju and Haarnoja do not teach, but Andersen teaches: further comprising a replay buffer configured to store the predicted output and the fusion output. (Anderson, p. 5 Algorithm 1 "The Dreaming Variational Autoencoder... Store experience into artificial replay buffer ^D(s^t, at, rt, s^t+1, terminal_t) [store the predicted output and the fusion output]")
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Ju and Haarnoja to incorporate the teachings of Andersen by including Dreaming Variational Autoencoder with LSTM structure. Doing so would better learn longer sequences in continuous state-spaces. (Andersen, p. 4, Fig. 1 "DVAE can also use LSTM to better learn longer sequences in continuous state-spaces.")
In regard to claim 18, Ju and Haarnoja do not teach, but Andersen teaches: further comprising a recurrent neural network encoder configured to encode and compress the data prior to storage in the replay buffer, and a decoder configured to decode and decompress the data extracted from the replay buffer. (Andersen, p. 4, Fig. 1 "DVAE can also use LSTM [recurrent encoder and recurrent decoder] to better learn longer sequences in continuous state-spaces."; p. 12, 6.3 Deep Line Wars Environment Modeling using DVAE "we expand the DVAE algorithm with LSTM to improve the interpretation of animations, illustrated Figure 1."; p. 5 Algorithm 1 "The Dreaming Variational Autoencoder... "; see Fig. 1, the recurrent structure of the encoder Q and the decoder P; see Algorithm 1, line 19 [the data (st) extracted from the replay buffer], line 21 [Q(z|X) encode the data (X including st)], line 22 [P(X|z) decode the data (X including st, extracted from the buffer)], line 23 [prior to storage in the replay buffer])
PNG
media_image4.png
398
426
media_image4.png
Greyscale
PNG
media_image5.png
272
1571
media_image5.png
Greyscale
The rationale for combining the teachings of Ju, Haarnoja and Andersen is the same as set forth in the rejection of claim 18.
In regard to claim 19, Ju and Haarnoja do not teach, but Andersen teaches: wherein the decoder is a recurrent neural network decoder. (Andersen, p. 4, Fig. 1 "DVAE can also use LSTM [recurrent decoder] to better learn longer sequences in continuous state-spaces."; p. 12, 6.3 Deep Line Wars Environment Modeling using DVAE "we expand the DVAE algorithm with LSTM to improve the interpretation of animations, illustrated Figure 1."; see Fig. 1, the recurrent structure of the encoder Q and the decoder P)
The rationale for combining the teachings of Ju, Haarnoja and Andersen is the same as set forth in the rejection of claim 17.
Claim 20 rejected under 35 U.S.C. 103 as being unpatentable over Ju and Haarnoja as applied to claim 17, and in further view of Luciw (US 20190061147 A1)
In regard to claim 20, Ju and Haarnoja do not teach, but Luciw teaches: wherein the replay buffer is prioritized. (Luciw, [0056] "A prioritization method can also be applied to pruning the memory. Instead of preferentially sampling the experiences with the highest priorities from experience memory D")
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Ju and Haarnoja to incorporate the teachings of Luciw by including TD-error as the basis for prioritization. Doing so would increase learning efficiency and eventual performance. (Luciw, [0051] "Using TD-error as the basis for prioritization for Double DQN increases learning efficiency and eventual performance.")
Claim 21 rejected under 35 U.S.C. 103 as being unpatentable over Ju and Haarnoja as applied to claim 17, and in view of Luciw in further view of Liu (TW 201315187 A)
In regard to claim 21, Ju does not teach, but Haarnoja teaches: the loss function between the improved predicted value of the fusion output and said predicted output… (Haarnoja, p. 4, 4 Discriminative Deterministic State Estimation "Let L(θ)... be the loss on an entire sequence with respect to θ... We can compute the gradient of l(θ) with respect to the parameters θ by first recursively computing the gradient of the loss with respect to the filter state st from the back to the front according to the following recursion: dL/d_st-1 = ... d_st/d_st-1... (1) [an improved predicted value of said fusion output (st or zt) and said predicted output (st-1 or zt-1)]... ")
The rationale for combining the teachings of Ju and Haarnoja is the same as set forth in the rejection of claim 12.
Ju and Haarnoja do not teach, but Luciw teaches: wherein the device is configured to implement a condition for testing input data applied at an input of the neural network such that the input data is deleted from the replay buffer when the loss function... (Luciw, [0051] "an approximation of expected learning progress is the temporal difference (TD) error... (6) Using TD-error as the basis for prioritization [the loss function between the value predicted for this input sample and the fusion output] for Double DQN increases learning efficiency and eventual performance. However, other prioritization methods could be used, such as prioritization by dissimilarity."; [0056] "A prioritization method can also be applied to pruning the memory. Instead of preferentially sampling the experiences with the highest priorities from experience memory D, the experiences with the lowest priorities are preferentially removed from experience memory D. Erasing memories is more final than assigning priorities, but can be necessary depending on the application."; experiences include input data st)
The rationale for combining the teachings of Ju, Haarnoja and Luciw is the same as set forth in the rejection of claim 20.
Ju, Haarnoja and Luciw do not teach, but Liu teaches: … is lower than a predefined threshold. (Liu, p. 14 "the content may be removed when the high speed access memory reaches a predetermined fullness level and/or the priority of the published content is prioritized below a threshold value. [a predefined threshold] In an embodiment, when the high speed access memory reaches a predetermined level of overflow (eg, when the memory is full, 95% full, or 90% full), the lowest priority content of the high speed access can be cleared.")
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Ju, Haarnoja and Luciw to incorporate the teachings of Liu by including user-defined policies for removing the content from the memory. Doing so would allow to free up previously unavailable memory according to the policies. (Liu, p. 14 "it may be implemented to remove content from high speed access functions and/or to publish (or delete content from the list) predefined, user-defined… policies."; p. 20 "The request may include an instruction to remove the content if the available storage or memory in the selected SRF 220-2 is insufficient.")
Response to Arguments
Applicant's amendments with respect to the claim objections and 112(b) rejection have been fully considered and are sufficient to overcome the objections. The objections and 112(b) rejection have been withdrawn.
Applicant's arguments with respect to the rejection of the claims under 35 U.S.C. 101 have been fully considered but they are not persuasive:
Argument: (p. 7-8) However, this characterization improperly expands the mental process grouping beyond its permissible scope… Indeed, the invention recited in Claim 12 requires a vehicle-deployed system that processes streaming multi-sensor data, such as LiDAR and camera inputs, in real time, propagating data… As shown in Example 47 discussed in the 2024 Guidance, limitations that cannot be practically performed in the human mind do not recite mental processes. By the same logic, real-time multi-sensor neural network inference and online weight updating in a vehicle perception context are computationally intractable for the human mind and therefore do not constitute mental processes under Step 2A Prong 1.
Response: The non-final of the previous action is withdrawn, and new grounds of rejection are set forth above. Further, it’s unclear which claim with limitations (d)-(f) in Example 47 that the Applicant refers to. Assume the Applicant refers to the limitations (d)-(f) in claim 3 of Example 47, which describes detecting a source address associated with network packets, dropping packets and blocking traffic. Those steps are not aligned with steps of the claimed invention.
Argument: (p. 8-9) Additionally, regarding Step 2A Prong 2, Applicant respectfully submits that the Examiner evaluates each claim element in isolation and concludes that none integrates the exception into a practical application… Claim 12, when read as a whole, recites a vehicle-integrated control device that combines a multi-sensor perception system, an online learning neural network… This closed-loop, real-time architecture constitutes a specific technological improvement to vehicle perception systems, which is analogous to how Claim 2 from Example 46 was found eligible because limitation (d) does not merely link the judicial exceptions to a technical field, but instead adds a meaningful limitation that employs the information provided by the judicial exception to operate equipment… Finally, Applicant respectfully disagrees with the Examiner's categorization of the forward propagation module, the fusion system, and the backpropagation module as "mere instructions to apply an exception" based solely on the generality of their functional description, without accounting for how these elements interact…
Response: At step 2A Prong Two and step 2B, the step of each sensor providing data is insignificant extra-solution activity – MPEP 2106.05(g), and the steps of using a neural network to estimate the variable, using the neural network to forward propagate data to provide an output, and consolidated vector being used for controlling the vehicle are mere instruction to apply an exception – MPEP 2106.05(f). Therefore, as a whole, the additional elements do not provide improvements and are not indicative of integration into a practical application.
Further, claim 2 from Example 46 recites “(d) automatically sending a control signal to the feed dispenser to dispense a therapeutically effective amount of supplemental salt and minerals mixed with feed when the analysis results for the animal indicate that the animal is exhibiting an aberrant behavioral pattern indicative of grass tetany.” This specific step is not aligned with steps of the claimed invention.
Applicant's arguments with respect to the rejection of the claims under 35 U.S.C. 103 have been fully considered but they are moot:
Argument: (p. 10-12) Independent Claim 12 is hereby amended to recite, in part, "a fusion system
configured to determine a fusion output comprising a consolidated object state vector by implementing at least one sensor fusion algorithm based on said predicted output, said consolidated object state vector being used for controlling the vehicle." See, for example, pre-numbered paragraph [0092] on pages 11 and 12 of the original specification. Applicant respectfully submits that the combined teachings of Fang and Gou do not disclose or suggest this limitation.
Response: the arguments do not apply to the references (Ju and Haarnoja) being used in the current rejection.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SU-TING CHUANG whose telephone number is (408)918-7519. The examiner can normally be reached Monday - Thursday 8-5 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached at (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.C./Examiner, Art Unit 2146
/SHAHID K KHAN/Primary Examiner, Art Unit 2146
Prosecution Insights