Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
According to the first part of the analysis, claim 20 does not fall into any one of the four statutory categories (i.e. process, machine, manufacture, or composition of matter). The claim directed a transitory signal is not a machine, manufacture, or composition of matter. The examiner notes that the claim includes a transitory media per spec [0035] [ computer-usable or computer readable medium may include any apparatus that stores, communicates, propagates, or transports the program for use by or in connection with the instruction execution system, apparatus, or device. Therefore, the broadest reasonable interpretation of the claimed computer readable medium includes signals per.
Claim 20 is rejected under 101 as a result of including a not patent-eligible subject matter as noted above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-7, 10-12, 14-16, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over
Travis Watkins et.al. (hereinafter Watkins) US-11983494-B1,
in view of Hassnaa Moustafa et.al (hereinafter Moustafa) US-20220126864-A1.
In regard to claim 1:
A computer-implemented method for training a transformer-based policy network and Generative Adversarial Network (GAN), comprising:
- initializing a transformer-based policy network to model action sequences by encoding temporal dependencies within sensor data, where the transformer-based policy network utilizes multi-head self-attention mechanisms to process sequential sensor inputs, the multi-head self-attention mechanisms being pre-trained on a labeled dataset having sensor data from known low-risk action sequences
[Col 24, lines 24-40]:
Still referring to FIG. 1, in some cases, one or more generative machine learning models may include generative adversarial network (GAN). As used in this disclosure, a “generative adversarial network” is a type of artificial neural network with at least two sub models (e.g., neural networks), a generator, and a discriminator, that compete against each other in a process that ultimately results in the generator learning to generate new data samples, wherein the “generator” is a component of the GAN that learns to create hypothetical data by incorporating feedbacks from the “discriminator” configured to distinguish real data from the hypothetical data. In some cases, generator may learn to make discriminator classify its output as real. In an embodiment, discriminator may include a supervised machine learning model while generator may include an unsupervised machine learning model as described in further detail with reference to FIG. 2.
[Col 44, lines 37-43]:
Still referring to FIG. 4, computing device 408 may build decision tree by following relational identification; for example, relational indication may specify that a first rule module receives an input from at least a second rule module and generates an output to at least a third rule module, and so forth
[Col 44, line 67]; [Col 45, lines 1-5]:
entire data structures may become remote resources to which further iterations or steps of data structures and/or decision trees may transmit data and from which further iterations or steps of generation of data structure receive data, for instance as part of a decision in a given decision tree node.
[Col 46, lines 42-43]:
A resource may be a physical machine e.g. a ride share vehicle or any other asset.
[Col 20, lines 19-29]:
Still referring to FIG. 1, an attention mechanism may include generalized attention self-attention, multi-head attention, additive attention, global attention, and the like. In generalized attention, when a sequence of words or an image is fed to LLM, it may verify each element of the input sequence and compare it against the output sequence. Each iteration may involve the mechanism's encoder capturing the input sequence and comparing it with each element of the decoder's sequence. From the comparison scores, the mechanism may then select the words or parts of the image that it needs to pay attention to
[Col 13, lines 29-40]:
Still referring to FIG. 1, a score assessment 148 is conducted based on first textual data set 112, second textual data set 144, a set of score metadata based on both data sets, all additional user inputs, and any applicable deadlines or statistical information available to processor 104. Score assessment 148 evaluates the relevant aspects of an engagement to determine a plurality of sub-scores. These sub-scores are isolated quantified analyses of the various aspects of any engagement and include measuring the impact of any potential issues or concerns associated with the engagement and prompts user on how to most effectively mitigate the risk or optimize a situation.
[Col 13, lines 59-65]:
Sub-scores may additionally incorporate any other relevant and available data as factors within sub-score generation. Sub-scores generated by score assessment 148 may be based on proprietary data from historic engagements, machine-learning training data from prior engagements, or any other available assessment tool.
[Col 19, lines 40-50]:
A “transformer architecture,” for the purposes of this disclosure is a neural network architecture that uses self-attention and positional encoding. Transformer architecture may be designed to process sequential input data, such as natural language, with applications towards tasks such as translation and text summarization. Transformer architecture may process the entire input all at once. “Positional encoding,” for the purposes of this disclosure, refers to a data processing technique that encodes the location or position of an entity in a sequence.
[Col 19, lines 56-69]:
In some embodiments, position vectors for a plurality of positions in a sequence may be assembled into a position matrix, wherein each row of position matrix may represent a position in the sequence.
[Col 8, lines 20-24]:
As a further example, chatbot may display a list of possible forms to file or situations which would lead user to specific forms or actions, from which user may select each applicable form or situation the user currently has needs or application for.
[Col 9, lines 10-16]:
Training data may also be in the form of user feedback on a completed machine-learning evolution. Where a conversion is successful, user may provide feedback in the form of validating a successful operation, in which case machine-learning processes 128 would either take no action, or promote the associations relied upon for that specific engagement.
[BRI: an encoder to act as a policy network in which the encoder’s final hidden states (or a subset of them) can be used as the policy’s input and the policy network can then decide actions (e.g., next token, control signal) based on the encoder’s contextualized representations. Thus, an LLM encoder can absolutely be a Transformer with self-attention and positional encoding, and in that case, it can serve as a policy network by feeding its contextualized representations into a separate policy head. This is a standard pattern in encoder-only and encoder-decoder architectures
Watkins does not explicitly disclose:
- training a generator within the GAN to produce generated action sequences by feeding noise and latent representations of sensor data, where the generator employs an adapted transformer architecture to enhance diversity and realism of the generated action sequences, which mimic behavior of low-risk action sequences;
- concurrently training a discriminator within the GAN to differentiate between action sequences derived from the labeled dataset and synthetic action sequences produced by the generator;
- employing a feedback loop where assessments by the discriminator are used to adjust parameters for an iterative improvement process configured to produce sequences indistinguishable from real low-risk action sequences;
- and deploying the trained transformer-based policy network and GAN to process incoming unlabeled sensor data for real-time generation of risk scores and identification of low-risk action sequences upon reaching a predetermined threshold for accuracy in distinguishing between real and synthetic action sequences.
However, Moustafa discloses:
- training a generator within the GAN to produce generated action sequences by feeding noise and latent representations of sensor data, where the generator employs an adapted transformer architecture to enhance diversity and realism of the generated action sequences, which mimic behavior of low-risk action sequences;
[0365]:
Generative adversarial networks (GANs) have value but depend heavily on the selection of
adversarial data in a training set.
[0326]:
At 3106, a score is computed for the sensor data that indicates its overall goodness or quality. The score is based on the geolocation and environmental information obtained at 3104. For example, the score may be based on a location score computed from the geolocation and environmental information as described above. In some cases, the score may also be based on additional scoring information associated with the sensor data. For example, the score may be based a noise score, object diversity score, or other scores computed for the sensor data.
[0330]:
In some embodiments, irregular driving patterns can be modeled as a sequence of driving
actions that deviates from the normal behavior expected by the autonomous
vehicle. FIGS. 32 and 33 illustrate two examples of irregular driving patterns, and how an
autonomous vehicle may learn to adapt its behavior in response to observing such
behaviors.
[0376]:
For example, if a control event is a hard braking event, the vehicle behavior model may determine that the braking event is outside a normal threshold for braking and indicates a high probability of fault in the vehicle system
[0252]:
information may be collected for a pull-over request and/or remote valet event by the affected vehicles and/or surrounding sensor devices, and this information may be shared and leveraged to enhance autonomous driving systems
- concurrently training a discriminator within the GAN to differentiate between action sequences derived from the labeled dataset and synthetic action sequences produced by the generator;
[0627]:
A communication system 8900, as shown in FIG. 89, resolves many of the aforementioned issues (and more). In at least one embodiment, a privacy-preserving computer vision system employs a Generative Adversarial Network (GAN) to preserve privacy in computer vision applications while maintaining the utility of the data and minimally affecting computer vision capabilities. GANs are usually comprised of two neural networks, which may be referred to herein as a “generator” (or “generative model”) and a “discriminator” (or “discriminative model”). The generator learns from one (true) dataset and then tries to generate new data that resembles the training dataset. The discriminator tries to discriminate between the new data (produced by the generator) and the true data. The generator's goal is to increase the error rate of the discriminative network (e.g., “fool” the discriminator network) by producing novel synthesized instances that appear to have come from the true data distribution.
- employing a feedback loop where assessments by the discriminator are used to adjust parameters for an iterative improvement process configured to produce sequences indistinguishable from real low-risk action sequences;
[0739]:
As shown in FIG. 112, automated driving may rely on a very fast feedback loop using a logic engine 11202 (which includes perception, fusion planning, driver policy, and decision-making aspects), and Distributed Actuation of the AV 11204 based on the output of such engines. Each of these meta-modules may be dependent on input or processing that is assumed to be trustworthy
0571] At 7620, based on input received from the driver, the autonomous vehicle determines what the situation is with the driver or the reason for the driver initiating the takeover request. If, for example, the situation is identified to be a risk for a driver or passenger (e.g., screaming, unsafe behavior, etc.), then re-planning may need to be considered for the route, and so the autonomous vehicle may modify the autonomous driving mode to pull over to stop at 7622.
[0604]:
System 8500 demonstrates an approach to extract anomaly events on the training and inference phases. Anomaly threshold 8506 is calculated during the training phase, where the network calculates the borderline between learned, unlearned, and anomaly events. In a particular embodiment, the anomaly threshold 8506 is based on a sigmoid function used by one or both of the baseline model 8504 and the SRU model 8502. The anomaly threshold 8506 may be used to adjust parameters of the SRU model 8502 during training.
(BRI: context of AI-driven risk management and operational optimization, sequences indistinguishable from real low-risk action sequences are typically generated using Generative Adversarial Networks (GANs) trained on labeled datasets of known low-risk behaviors
[0442]:
in yet another example, the sensor data alone may be used to generate a context of “over speed limit”. The determined context(s) is often expressed as metadata associated with the raw data.
[0603]:
FIG. 85 depicts a system 8500 for anomaly detection in accordance with certain embodiments. The addition of an anomaly detector may enhance the intelligence of a system to enable reporting of unknown situations (e.g., time-based events) that would not have been detected previously. A new ML model based on an LSTM or GRU architecture (termed Smart Recurrent Unit (SRU) model 8502 herein) may be provided and used in conjunction with a standard LSTM or GRU model (“baseline model” 8504). In various embodiments, the architecture of the SRU model 8502 may be similar to the architecture of the baseline predictor, but may be specially tuned to detect anomalies. In various embodiments, the system 8500 is able to both encode a newly arriving sequence of anomaly data (e.g., encode the sequence as an unknown class) as well as decode a given data representation to an anomaly tag (e.g., over time, identify new anomaly classes and apply labels accordingly). Any suitable data sequence may be recognized as an anomaly by the system 8500. For example, an anomaly may be an unknown detected object or an unknown detected event sequence. In various embodiments, the addition of the SRU model may enhance the system's intelligence to report unknown situations (time-based events) that were not been seen by the system previously (either at training or test phases). The system may be able to encode a new sequence of anomaly data and assign a label to it to create a new class. When the label is generated, any given data representation to this type of anomaly may be decoded.
- and deploying the trained transformer-based policy network and GAN to process incoming unlabeled sensor data for real-time generation of risk scores and identification of low-risk action sequences upon reaching a predetermined threshold for accuracy in distinguishing between real and synthetic action sequences.
[0251]:
in some implementations, machine learning models may be built and trained from this information and such machine learning models may be deployed on and executed by roadside units, cloud-based support systems, remote valet computing systems, or the in-vehicle systems of the autonomous vehicles themselves to provide logic for predictively determining potential remote valet handoffs.
[0189]:
In unsupervised learning, the model may be built from a set of data which contains only inputs and no desired outputs. The unsupervised model may be used to find structure in the data (e.g., grouping or clustering of data points) by discovering patterns in the data.
[0469]:
the synthetic attack data may be used as augmentation to boost the resiliency against the attacks on deep learning algorithms or classical ML algorithms. During training, the adversarial images with their correct labels are incorporated as part of the training set to refine the learning model
[0538]:
When the system determines to handoff to the human driver, the driver is notified of an imminent handoff. In order to do so, the system may engage with the driver in one or more of several possible manners. For example, the system may engage in a verbal manner with the driver. For example, text with correct semantics and syntax may be built by a natural language generation engine and then transformed into synthetic speech audio by a text-to-speech engine to produce a verbal message describing the handoff.
[0437]:
FIG. 50 depicts an example flow for handling data based on categorization in accordance with certain embodiments. At 5002, an instance of one or more objects from data captured by one or more sensors of a vehicle is identified. At 5004, a categorization of the instance is performed by checking the instance against a plurality of categories and assigning at least one category of the plurality of categories to the instance. At 5006, a score is determined based on the categorization of the instance. At 5008, a data handling policy for the instance is selected based at least in part on the score. At 5010, the instance is processed based on the determined data handling policy.
[0499]:
in yet another embodiment, the RSS enforcer 5908 may use policy information (e.g., based on preferences of the driver or based on safety considerations) to determine the acceleration value. For example, the RSS enforcer 5908 may favor longitudinal accelerations over latitudinal accelerations or vice versa. As another example, the RSS enforcer 5908 may favor accelerations that are more comfortable to the driver (e.g., slower braking or smaller steering adjustments may be preferred over hard braking or swerving). In various embodiments, the decision may be based on both safety and comfort, with related metrics calculated from the same set of motion parameters and vehicle characteristics. [BRI: a transformer-based policy network can represent the decision-making component of a verbal engagement system in a smart vehicle, especially when the goal is to balance driver preferences and safety considerations in real time-unsafe implies risk]
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Watkins and Moustafa.
Watkins teaches GAN and discriminator based training for vehicle driver risk assessment.
Moustafa teaches concurrently training a discriminator within the GAN and deploying the transformed-based policy network.
One of ordinary skill would have motivation to combine Watkins and Moustafa that can provide improved safety of the handoff process of the vehicle to a driver (Moustafa [0520])
In regard to claim 2 :
Watkins does not explicitly disclose:
- wherein training includes monitoring vehicles to collect the sensor data where the action sequences include driver actions.
However, Moustafa discloses:
- wherein training includes monitoring vehicles to collect the sensor data where the action sequences include driver actions
[0169]
For instance, compute resources and autonomous driving logic used to facilitate machine learning model training and use of such machine learning models may be provided on the in-vehicle computing systems entirely or partially on both the in-vehicle systems and some external systems
[0286]:
a vehicle may include (as an OEM component or aftermarket component) a monitor to observe a human driver's performance and build a behavioral model for this driver or a group of drivers (e.g., by sharing the monitoring data with a cloud-based aggregator application).
[0330]:
In some embodiments, irregular driving patterns can be modeled as a sequence of driving actions that deviates from the normal behavior expected by the autonomous vehicle. FIGS. 32 and 33 illustrate two examples of irregular driving patterns, and how an autonomous vehicle may learn to adapt its behavior in response to observing such behaviors.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Watkins, and Moustafa.
Watkins teaches GAN and discriminator based training for vehicle driver risk assessment.
Moustafa teaches concurrently training GAN and discriminator and deploying the transformed-based policy network.
One of ordinary skill would have motivation to combine Watkins and Moustafa that can provide improved safety of the handoff process of the vehicle to a driver (Moustafa [0520])
In regard to claim 3 :
Watkins does not explicitly disclose:
- wherein the action sequences include historical driver actions.
However, Moustafa discloses:
- wherein the action sequences include historical driver actions.
[0533] :
Driver historical data 6904 may include any suitable background information that may inform the level of attentiveness of the driver. For example, historical data 6904 may include historical data for a driver including instances of driving under intoxication (DUI), past accidents, instances of potentially dangerous actions taken by a driver
In regard to claim 5 :
Watkins discloses:
- wherein the sensor data includes action sequences and temporal correlations are discerned using an adapted Transformer architecture to capture subtle and long-range dependencies within the action sequences
[Col 22, lines 50-52]:
With continued reference to FIG. 1, LLM may generate an output. In some embodiments, LLM may include multiple sets of transformer architecture
[Col 32, lines 25-32]:
training data may be selected to span a set of likely circumstances or inputs for a machine-learning model and/or process to encounter when deployed. For instance, and without limitation, for each category of input data to a machine-learning process or model that may exist in a range of values in a population of phenomena such as time series data, images, user data, process data, physical data, or the like
[Col 32, lines 33-40]:
select training examples representing each possible value on such a range and/or a representative sample of values on such a range. Selection of a representative sample may include selection of training examples in proportions matching a statistically determined and/or predicted distribution of such values according to relative frequency, such that, for instance,
[Col 31, lines 5-9]:
Multiple categories of data elements may be related in training data 204 according to various correlations; correlations may indicate causative and/or predictive links between categories of data elements, which may be modeled as relationships
[Col 21, lines 50-51]:
With continued reference to FIG. 1, in some embodiments, input to decoder may go through an embedding layer and positional encoding layer in order to obtain positional embeddings. Decoder may include a first multi-headed attention layer, wherein the first multi-headed attention layer may receive positional embeddings.
[Col 29, lines 46-51]:
if a message making up a mathematical representation of file is well-designed and implemented correctly, any alteration of the file may result in a mismatch with the digital signature; the mathematical representation may be produced using an alteration-sensitive, reliably reproducible algorithm, such as a hashing algorithm
[BRI: adapting Transformers for subtle and long-range action dependencies involves strategic attention layering, fine-grained positional encoding, and contextual cross-attention,
ensuring the model can reason over long horizons while detecting minute, context-sensitive change and with a time series range, when modeled with a statistical distribution and analyzed for relative frequencies and correlations, shows persistent, causally linked patterns over long time intervals, it strongly indicates long-range dependencies]
In regard to claim 6:
Watkins does not explicitly disclose:
- where the discriminator utilizes a convolutional neural network (CNN) architecture optimized to identify subtle distinctions between real and generated action sequences.
However, Moustafa discloses:
- where the discriminator utilizes a convolutional neural network (CNN) architecture optimized to identify subtle distinctions between real and generated action sequences.
[1724]:
the first neural network is a generative model, and where the GAN includes a second neural network that is a discriminative model.
[0678]:
At 10006, a discriminator of the configured GAN model receives the disguised image from the generator. At 10008, the discriminator performs convolutional neural network operations on the disguised image to classify the disguised image as real or fake.
[0846]:
the parameters of the sensor motion and other required parameters may be measured (e.g., using other sensors) or may be estimated from the data itself. To summarize, the warping action may account for many types of geometric differences between sensor data streams, and may result in spatial and temporal alignment (or registration) of the data into a normalized configuration.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Watkins, and Moustafa.
Watkins teaches GAN and discriminator based training for vehicle driver risk assessment.
Moustafa teaches concurrently training GAN and discriminator and deploying the transformed-based policy network.
One of ordinary skill would have motivation to combine Watkins and Moustafa that can provide improved safety of the handoff process of the vehicle to a driver (Moustafa [0520])
In regard to claim 7:
Watkins discloses:
- further comprising utilizing a performance prediction neural network to assign risk scores to individual actions and identify low-risk action sequences, where the neural network is based on a Transformer architecture that performs analysis of actions in a context of their temporal sequence.
[Col 13, lines 33-35]:
Score assessment 148 evaluates the relevant aspects of an engagement to determine a plurality of sub-scores.
[Col 35, lines 41-51]:
scoring function may, for instance, seek to maximize the probability that a given input and/or combination of elements inputs is associated with a given output to minimize the probability that a given input is not associated with a given output. Scoring function may be expressed as a risk function representing an “expected loss” of an algorithm relating inputs to outputs, where loss is computed as an error function representing a degree to which a prediction generated by the relation is incorrect when compared to a given input-output pair provided in training data 204.
[Col 38, lines 12-19]:
A naïve Bayes algorithm may be generated by first transforming training data into a frequency table. Processor may then calculate a likelihood table by calculating probabilities of different data entries and classification labels. Processor may utilize a naïve Bayes equation to calculate a posterior probability for each class. A class containing the highest posterior probability is the outcome of prediction.
disclosure, will be aware of various ways in which steps, sequences of steps, processing tasks, and/or data may be subdivided, shared, or otherwise dealt with using iteration, recursion, and/or parallel processing.
In regard to claim 10:
Watkins discloses:
- A system for training a transformer-based policy network and Generative Adversarial Network (GAN), comprising: a hardware processor; and a memory that stores a computer program which, when executed by the hardware processor, causes the hardware processor to: [Col 22, lines 63-64]:[Col 39, lines 32-67]; [Col 40, lines 1-6]:
[Col 19, lines 60-64]:
With continued reference to FIG. 1, LLM and/or transformer architecture may include an attention mechanism. An “attention mechanism,” as used herein, is a part of a neural architecture that enables a system to dynamically quantify the relevant features of the input data.
- initialize a transformer-based policy network to model action sequences by encoding temporal dependencies within sensor data, where the transformer-based policy network utilizes multi-head self-attention mechanisms to process sequential sensor inputs, the multi-head self-attention mechanisms being pre-trained on a labeled dataset having sensor data from known low-risk action sequences;
[Col 24, lines 24-40]:
Still referring to FIG. 1, in some cases, one or more generative machine learning models may include generative adversarial network (GAN). As used in this disclosure, a “generative adversarial network” is a type of artificial neural network with at least two sub models (e.g., neural networks), a generator, and a discriminator, that compete against each other in a process that ultimately results in the generator learning to generate new data samples, wherein the “generator” is a component of the GAN that learns to create hypothetical data by incorporating feedbacks from the “discriminator” configured to distinguish real data from the hypothetical data. In some cases, generator may learn to make discriminator classify its output as real. In an embodiment, discriminator may include a supervised machine learning model while generator may include an unsupervised machine learning model as described in further detail with reference to FIG. 2.
[Col 44, lines 37-43]:
Still referring to FIG. 4, computing device 408 may build decision tree by following relational identification; for example, relational indication may specify that a first rule module receives an input from at least a second rule module and generates an output to at least a third rule module, and so forth
[Col 44, line 67]; [Col 45, lines 1-5]:
entire data structures may become remote resources to which further iterations or steps of data structures and/or decision trees may transmit data and from which further iterations or steps of generation of data structure receive data, for instance as part of a decision in a given decision tree node.
[Col 46, lines 42-43]:
A resource may be a physical machine e.g. a ride share vehicle or any other asset.
[Col 20, lines 19-29]:
Still referring to FIG. 1, an attention mechanism may include generalized attention self-attention, multi-head attention, additive attention, global attention, and the like. In generalized attention, when a sequence of words or an image is fed to LLM, it may verify each element of the input sequence and compare it against the output sequence. Each iteration may involve the mechanism's encoder capturing the input sequence and comparing it with each element of the decoder's sequence. From the comparison scores, the mechanism may then select the words or parts of the image that it needs to pay attention to
[Col 13, lines 29-40]:
Still referring to FIG. 1, a score assessment 148 is conducted based on first textual data set 112, second textual data set 144, a set of score metadata based on both data sets, all additional user inputs, and any applicable deadlines or statistical information available to processor 104. Score assessment 148 evaluates the relevant aspects of an engagement to determine a plurality of sub-scores. These sub-scores are isolated quantified analyses of the various aspects of any engagement and include measuring the impact of any potential issues or concerns associated with the engagement and prompts user on how to most effectively mitigate the risk or optimize a situation.
[Col 13, lines 59-65]:
Sub-scores may additionally incorporate any other relevant and available data as factors within sub-score generation. Sub-scores generated by score assessment 148 may be based on proprietary data from historic engagements, machine-learning training data from prior engagements, or any other available assessment tool.
[Col 19, lines 40-50]:
A “transformer architecture,” for the purposes of this disclosure is a neural network architecture that uses self-attention and positional encoding. Transformer architecture may be designed to process sequential input data, such as natural language, with applications towards tasks such as translation and text summarization. Transformer architecture may process the entire input all at once. “Positional encoding,” for the purposes of this disclosure, refers to a data processing technique that encodes the location or position of an entity in a sequence.
[Col 19, lines 56-69]:
In some embodiments, position vectors for a plurality of positions in a sequence may be assembled into a position matrix, wherein each row of position matrix may represent a position in the sequence.
[Col 8, lines 20-24]:
As a further example, chatbot may display a list of possible forms to file or situations which would lead user to specific forms or actions, from which user may select each applicable form or situation the user currently has needs or application for.
[Col 9, lines 10-16]:
Training data may also be in the form of user feedback on a completed machine-learning evolution. Where a conversion is successful, user may provide feedback in the form of validating a successful operation, in which case machine-learning processes 128 would either take no action, or promote the associations relied upon for that specific engagement.
[BRI: an encoder to act as a policy network in which the encoder’s final hidden states (or a subset of them) can be used as the policy’s input and the policy network can then decide actions (e.g., next token, control signal) based on the encoder’s contextualized representations. Thus, an LLM encoder can absolutely be a Transformer with self-attention and positional encoding, and in that case, it can serve as a policy network by feeding its contextualized representations into a separate policy head. This is a standard pattern in encoder-only and encoder-decoder architectures
Watkins does not explicitly disclose:
- train a generator within the GAN to produce generated action sequences by feeding noise and latent representations of sensor data, where the generator employs an adapted transformer architecture to enhance diversity and realism of the generated action sequences, which mimic behavior of low-risk action sequences;
- concurrently train a discriminator within the GAN to differentiate between action sequences derived from the labeled dataset and synthetic action sequences produced by the generator; utilize a feedback loop where assessments by the discriminator are used to adjust parameters for an iterative improvement process configured to produce sequences indistinguishable from real low-risk action sequences;
- and deploying the trained transformer-based policy network and GAN to process incoming unlabeled sensor data for real-time generation of risk scores and identification of low-risk action sequences upon reaching a predetermined threshold for accuracy in distinguishing between real and synthetic action sequences.
However, Moustafa discloses:
- train a generator within the GAN to produce generated action sequences by feeding noise and latent representations of sensor data, where the generator employs an adapted transformer architecture to enhance diversity and realism of the generated action sequences, which mimic behavior of low-risk action sequences;
[0365]:
Generative adversarial networks (GANs) have value but depend heavily on the
selection of adversarial data in a training set.
[0326]:
At 3106, a score is computed for the sensor data that indicates its overall goodness or quality. The score is based on the geolocation and environmental information obtained at 3104. For example, the score may be based on a location score computed from the geolocation and environmental information as described above. In some cases, the score may also be based on additional scoring information associated with the sensor data. For example, the score may be based a noise score, object diversity score, or other scores computed for the sensor data.
[0330]:
In some embodiments, irregular driving patterns can be modeled as a sequence of driving
actions that deviates from the normal behavior expected by the autonomous
vehicle. FIGS. 32 and 33 illustrate two examples of irregular driving patterns, and how an
autonomous vehicle may learn to adapt its behavior in response to observing such
behaviors.
[0376]:
For example, if a control event is a hard braking event, the vehicle behavior model may determine that the braking event is outside a normal threshold for braking and indicates a high probability of fault in the vehicle system
[0252]:
information may be collected for a pull-over request and/or remote valet event by the affected vehicles and/or surrounding sensor devices, and this information may be shared and leveraged to enhance autonomous driving systems
- concurrently train a discriminator within the GAN to differentiate between action sequences derived from the labeled dataset and synthetic action sequences produced by the generator;
[0627]:
A communication system 8900, as shown in FIG. 89, resolves many of the aforementioned issues (and more). In at least one embodiment, a privacy-preserving computer vision system employs a Generative Adversarial Network (GAN) to preserve privacy in computer vision applications while maintaining the utility of the data and minimally affecting computer vision capabilities. GANs are usually comprised of two neural networks, which may be referred to herein as a “generator” (or “generative model”) and a “discriminator” (or “discriminative model”). The generator learns from one (true) dataset and then tries to generate new data that resembles the training dataset. The discriminator tries to discriminate between the new data (produced by the generator) and the true data. The generator's goal is to increase the error rate of the discriminative network (e.g., “fool” the discriminator network) by producing novel synthesized instances that appear to have come from the true data distribution.
- utilize a feedback loop where assessments by the discriminator are used to adjust parameters for an iterative improvement process configured to produce sequences indistinguishable from real low-risk action sequences;
[0739]:
As shown in FIG. 112, automated driving may rely on a very fast feedback loop using a logic engine 11202 (which includes perception, fusion planning, driver policy, and decision-making aspects), and Distributed Actuation of the AV 11204 based on the output of such engines. Each of these meta-modules may be dependent on input or processing that is assumed to be trustworthy
[0571]:
At 7620, based on input received from the driver, the autonomous vehicle determines what the situation is with the driver or the reason for the driver initiating the takeover request. If, for example, the situation is identified to be a risk for a driver or passenger (e.g., screaming, unsafe behavior, etc.), then re-planning may need to be considered for the route, and so the autonomous vehicle may modify the autonomous driving mode to pull over to stop at 7622.
[0604]:
System 8500 demonstrates an approach to extract anomaly events on the training and inference phases. Anomaly threshold 8506 is calculated during the training phase, where the network calculates the borderline between learned, unlearned, and anomaly events. In a particular embodiment, the anomaly threshold 8506 is based on a sigmoid function used by one or both of the baseline model 8504 and the SRU model 8502. The anomaly threshold 8506 may be used to adjust parameters of the SRU model 8502 during training.
(BRI: context of AI-driven risk management and operational optimization, sequences indistinguishable from real low-risk action sequences are typically generated using Generative Adversarial Networks (GANs) trained on labeled datasets of known low-risk behaviors
[0442]:
in yet another example, the sensor data alone may be used to generate a context of “over speed limit”. The determined context(s) is often expressed as metadata associated with the raw data.
[0603]:
FIG. 85 depicts a system 8500 for anomaly detection in accordance with certain embodiments. The addition of an anomaly detector may enhance the intelligence of a system to enable reporting of unknown situations (e.g., time-based events) that would not have been detected previously. A new ML model based on an LSTM or GRU architecture (termed Smart Recurrent Unit (SRU) model 8502 herein) may be provided and used in conjunction with a standard LSTM or GRU model (“baseline model” 8504). In various embodiments, the architecture of the SRU model 8502 may be similar to the architecture of the baseline predictor, but may be specially tuned to detect anomalies. In various embodiments, the system 8500 is able to both encode a newly arriving sequence of anomaly data (e.g., encode the sequence as an unknown class) as well as decode a given data representation to an anomaly tag (e.g., over time, identify new anomaly classes and apply labels accordingly). Any suitable data sequence may be recognized as an anomaly by the system 8500. For example, an anomaly may be an unknown detected object or an unknown detected event sequence. In various embodiments, the addition of the SRU model may enhance the system's intelligence to report unknown situations (time-based events) that were not been seen by the system previously (either at training or test phases). The system may be able to encode a new sequence of anomaly data and assign a label to it to create a new class. When the label is generated, any given data representation to this type of anomaly may be decoded.
- and deploying the trained transformer-based policy network and GAN to process incoming unlabeled sensor data for real-time generation of risk scores and identification of low-risk action sequences upon reaching a predetermined threshold for accuracy in distinguishing between real and synthetic action sequences.
[0251]:
in some implementations, machine learning models may be built and trained from this information and such machine learning models may be deployed on and executed by roadside units, cloud-based support systems, remote valet computing systems, or the in-vehicle systems of the autonomous vehicles themselves to provide logic for predictively determining potential remote valet handoffs.
[0189]:
In unsupervised learning, the model may be built from a set of data which contains only inputs and no desired outputs. The unsupervised model may be used to find structure in the data (e.g., grouping or clustering of data points) by discovering patterns in the data.
[0469]:
the synthetic attack data may be used as augmentation to boost the resiliency against the attacks on deep learning algorithms or classical ML algorithms. During training, the adversarial images with their correct labels are incorporated as part of the training set to refine the learning model
[0538]:
When the system determines to handoff to the human driver, the driver is notified of an imminent handoff. In order to do so, the system may engage with the driver in one or more of several possible manners. For example, the system may engage in a verbal manner with the driver. For example, text with correct semantics and syntax may be built by a natural language generation engine and then transformed into synthetic speech audio by a text-to-speech engine to produce a verbal message describing the handoff.
[0437]:
FIG. 50 depicts an example flow for handling data based on categorization in accordance with certain embodiments. At 5002, an instance of one or more objects from data captured by one or more sensors of a vehicle is identified. At 5004, a categorization of the instance is performed by checking the instance against a plurality of categories and assigning at least one category of the plurality of categories to the instance. At 5006, a score is determined based on the categorization of the instance. At 5008, a data handling policy for the instance is selected based at least in part on the score. At 5010, the instance is processed based on the determined data handling policy.
[0499]:
in yet another embodiment, the RSS enforcer 5908 may use policy information (e.g., based on preferences of the driver or based on safety considerations) to determine the acceleration value. For example, the RSS enforcer 5908 may favor longitudinal accelerations over latitudinal accelerations or vice versa. As another example, the RSS enforcer 5908 may favor accelerations that are more comfortable to the driver (e.g., slower braking or smaller steering adjustments may be preferred over hard braking or swerving). In various embodiments, the decision may be based on both safety and comfort, with related metrics calculated from the same set of motion parameters and vehicle characteristics. [BRI: a transformer-based policy network can represent the decision-making component of a verbal engagement system in a smart vehicle, especially when the goal is to balance driver preferences and safety considerations in real time-unsafe implies risk]
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Watkins and Moustafa.
Watkins teaches GAN and discriminator based training for vehicle driver risk assessment.
Moustafa teaches concurrently training GAN and discriminator and deploying the transformed-based policy network.
One of ordinary skill would have motivation to combine Watkins and Moustafa that can provide improved safety of the handoff process of the vehicle to a driver (Moustafa [0520])
In regard to claim 11 :
Watkins does not explicitly disclose:
- wherein training includes monitoring vehicles to collect the sensor data where the action sequences include driver actions.
However, Tran discloses:
- wherein training includes monitoring vehicles to collect the sensor data where the action sequences include driver actions
[0169]
For instance, compute resources and autonomous driving logic used to facilitate machine learning model training and use of such machine learning models may be provided on the in-vehicle computing systems entirely or partially on both the in-vehicle systems and some external systems
[0286]:
a vehicle may include (as an OEM component or aftermarket component) a monitor to observe a human driver's performance and build a behavioral model for this driver or a group of drivers (e.g., by sharing the monitoring data with a cloud-based aggregator application).
[0330]:
In some embodiments, irregular driving patterns can be modeled as a sequence of driving actions that deviates from the normal behavior expected by the autonomous vehicle. FIGS. 32 and 33 illustrate two examples of irregular driving patterns, and how an autonomous vehicle may learn to adapt its behavior in response to observing such behaviors.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Watkins, and Moustafa.
Watkins teaches GAN and discriminator based training for vehicle driver risk assessment.
Moustafa teaches concurrently training GAN and discriminator and deploying the transformed-based policy network.
One of ordinary skill would have motivation to combine Watkins and Moustafa that can provide improved safety of the handoff process of the vehicle to a driver (Moustafa [0520])
In regard to claim 12 :
Watkins does not explicitly disclose:
- wherein the action sequences include historical driver actions.
However, Moustafa discloses:
- wherein the action sequences include historical driver actions.
[0533] :
Driver historical data 6904 may include any suitable background information that may inform the level of attentiveness of the driver. For example, historical data 6904 may include historical data for a driver including instances of driving under intoxication (DUI), past accidents, instances of potentially dangerous actions taken by a driver
obstacle.
In regard to claim 14 :
Watkins discloses:
- wherein the sensor data includes action sequences and temporal correlations are discerned using an adapted Transformer architecture to capture subtle and long-range dependencies within the action sequences
[Col 22, lines 50-52]:
With continued reference to FIG. 1, LLM may generate an output. In some embodiments, LLM may include multiple sets of transformer architecture
[Col 32, lines 25-32]:
training data may be selected to span a set of likely circumstances or inputs for a machine-learning model and/or process to encounter when deployed. For instance, and without limitation, for each category of input data to a machine-learning process or model that may exist in a range of values in a population of phenomena such as time series data, images, user data, process data, physical data, or the like
[Col 32, lines 33-40]:
select training examples representing each possible value on such a range and/or a representative sample of values on such a range. Selection of a representative sample may include selection of training examples in proportions matching a statistically determined and/or predicted distribution of such values according to relative frequency, such that, for instance,
[Col 31, lines 5-9]:
Multiple categories of data elements may be related in training data 204 according to various correlations; correlations may indicate causative and/or predictive links between categories of data elements, which may be modeled as relationships
[Col 21, lines 50-51]:
With continued reference to FIG. 1, in some embodiments, input to decoder may go through an embedding layer and positional encoding layer in order to obtain positional embeddings. Decoder may include a first multi-headed attention layer, wherein the first multi-headed attention layer may receive positional embeddings.
[Col 29, lines 46-51]:
if a message making up a mathematical representation of file is well-designed and implemented correctly, any alteration of the file may result in a mismatch with the digital signature; the mathematical representation may be produced using an alteration-sensitive, reliably reproducible algorithm, such as a hashing algorithm
[BRI: adapting Transformers for subtle and long-range action dependencies involves strategic attention layering, fine-grained positional encoding, and contextual cross-attention,
ensuring the model can reason over long horizons while detecting minute, context-sensitive change and with a time series range, when modeled with a statistical distribution and analyzed for relative frequencies and correlations, shows persistent, causally linked patterns over long time intervals, it strongly indicates long-range dependencies]
In regard to claim 15:
Watkins does not explicitly disclose:
- where the discriminator utilizes a convolutional neural network (CNN) architecture optimized to identify subtle distinctions between real and generated action sequences.
However, Moustafa discloses:
- where the discriminator utilizes a convolutional neural network (CNN) architecture optimized to identify subtle distinctions between real and generated action sequences.
[1724]:
the first neural network is a generative model, and where the GAN includes a second neural network that is a discriminative model.
[0678]:
At 10006, a discriminator of the configured GAN model receives the disguised image from the generator. At 10008, the discriminator performs convolutional neural network operations on the disguised image to classify the disguised image as real or fake.
[0846]:
the parameters of the sensor motion and other required parameters may be measured (e.g., using other sensors) or may be estimated from the data itself. To summarize, the warping action may account for many types of geometric differences between sensor data streams, and may result in spatial and temporal alignment (or registration) of the data into a normalized configuration.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Watkins, and Moustafa.
Watkins teaches GAN and discriminator based training for vehicle driver risk assessment.
Moustafa teaches concurrently training GAN and discriminator and deploying the transformed-based policy network.
One of ordinary skill would have motivation to combine Watkins and Moustafa that can provide improved safety of the handoff process of the vehicle to a driver (Moustafa [0520])
In regard to claim 16:
Watkins discloses:
- wherein the computer program causes the hardware processor to utilize a performance prediction neural network to assign risk scores to individual actions and identify low-risk action sequences, where the neural network is based on a Transformer architecture that performs analysis of actions in a context of their temporal sequence.
[Col 30, lines 39-47]:
A “machine-learning process,” as used in this disclosure, is a process that automatedly uses training data 204 to generate an algorithm instantiated in hardware or software logic, data structures, and/or functions that will be performed by a computing device/module to produce outputs 208 given data provided as inputs 212; this is in contrast to a non-machine-learning software program where the commands to be executed are determined in advance by a user and written in a programming language.
[Col 13, lines 33-35]:
Score assessment 148 evaluates the relevant aspects of an engagement to determine a plurality of sub-scores.
[Col 35, lines 41-51]:
scoring function may, for instance, seek to maximize the probability that a given input and/or combination of elements inputs is associated with a given output to minimize the probability that a given input is not associated with a given output. Scoring function may be expressed as a risk function representing an “expected loss” of an algorithm relating inputs to outputs, where loss is computed as an error function representing a degree to which a prediction generated by the relation is incorrect when compared to a given input-output pair provided in training data 204.
[Col 38, lines 12-19]:
A naïve Bayes algorithm may be generated by first transforming training data into a frequency table. Processor may then calculate a likelihood table by calculating probabilities of different data entries and classification labels. Processor may utilize a naïve Bayes equation to calculate a posterior probability for each class. A class containing the highest posterior probability is the outcome of prediction.
disclosure, will be aware of various ways in which steps, sequences of steps, processing tasks, and/or data may be subdivided, shared, or otherwise dealt with using iteration, recursion, and/or parallel processing.
In regard to claim 19:
Watkins discloses:
- wherein the transformer-based policy network employs a self-attention mechanism that enables simultaneous processing of multiple trajectories for modelling complex driver behaviors over time.
[Col 36, lines 40-53]:
A computing device, processor, and/or module may perform any step, sequence of steps, or algorithm in parallel, such as simultaneously and/or substantially simultaneously performing a step two or more times using two or more parallel threads, processor cores, or the like; division of tasks between parallel threads and/or processes may be performed according to any protocol suitable for division of tasks between iterations. Persons skilled in the art, upon reviewing the entirety of this disclosure, will be aware of various ways in which steps, sequences of steps, processing tasks, and/or data may be subdivided, shared, or otherwise dealt with using iteration, recursion, and/or parallel processing.
In regard to claim 20:
Watkins discloses:
- A computer-readable medium storing instructions that, when executed by a processor, perform a method for training a transformer-based policy network and Generative Adversarial Network (GAN), comprising:
[Col 22, lines 63-64]:[Col 39, lines 32-67]; [Col 40, lines 1-6]:
[Col 53, lines 13-14]:
a machine-readable storage medium does not include transitory forms of signal transmission.
[Col 19, lines 60-64]:
With continued reference to FIG. 1, LLM and/or transformer architecture may include an attention mechanism. An “attention mechanism,” as used herein, is a part of a neural architecture that enables a system to dynamically quantify the relevant features of the input data.
- initializing a transformer-based policy network to model action sequences by encoding temporal dependencies within sensor data, where the transformer-based policy network utilizes multi-head self-attention mechanisms to process sequential sensor inputs, the multi-head self-attention mechanisms being pre-trained on a labeled dataset having sensor data from known low-risk action sequences;
[Col 24, lines 24-40]:
Still referring to FIG. 1, in some cases, one or more generative machine learning models may include generative adversarial network (GAN). As used in this disclosure, a “generative adversarial network” is a type of artificial neural network with at least two sub models (e.g., neural networks), a generator, and a discriminator, that compete against each other in a process that ultimately results in the generator learning to generate new data samples, wherein the “generator” is a component of the GAN that learns to create hypothetical data by incorporating feedbacks from the “discriminator” configured to distinguish real data from the hypothetical data. In some cases, generator may learn to make discriminator classify its output as real. In an embodiment, discriminator may include a supervised machine learning model while generator may include an unsupervised machine learning model as described in further detail with reference to FIG. 2.
[Col 44, lines 37-43]:
Still referring to FIG. 4, computing device 408 may build decision tree by following relational identification; for example, relational indication may specify that a first rule module receives an input from at least a second rule module and generates an output to at least a third rule module, and so forth
[Col 44, line 67]; [Col 45, lines 1-5]:
entire data structures may become remote resources to which further iterations or steps of data structures and/or decision trees may transmit data and from which further iterations or steps of generation of data structure receive data, for instance as part of a decision in a given decision tree node.
[Col 46, lines 42-43]:
A resource may be a physical machine e.g. a ride share vehicle or any other asset.
[Col 20, lines 19-29]:
Still referring to FIG. 1, an attention mechanism may include generalized attention self-attention, multi-head attention, additive attention, global attention, and the like. In generalized attention, when a sequence of words or an image is fed to LLM, it may verify each element of the input sequence and compare it against the output sequence. Each iteration may involve the mechanism's encoder capturing the input sequence and comparing it with each element of the decoder's sequence. From the comparison scores, the mechanism may then select the words or parts of the image that it needs to pay attention to
[Col 13, lines 29-40]:
Still referring to FIG. 1, a score assessment 148 is conducted based on first textual data set 112, second textual data set 144, a set of score metadata based on both data sets, all additional user inputs, and any applicable deadlines or statistical information available to processor 104. Score assessment 148 evaluates the relevant aspects of an engagement to determine a plurality of sub-scores. These sub-scores are isolated quantified analyses of the various aspects of any engagement and include measuring the impact of any potential issues or concerns associated with the engagement and prompts user on how to most effectively mitigate the risk or optimize a situation.
[Col 13, lines 59-65]:
Sub-scores may additionally incorporate any other relevant and available data as factors within sub-score generation. Sub-scores generated by score assessment 148 may be based on proprietary data from historic engagements, machine-learning training data from prior engagements, or any other available assessment tool.
[Col 19, lines 40-50]:
A “transformer architecture,” for the purposes of this disclosure is a neural network architecture that uses self-attention and positional encoding. Transformer architecture may be designed to process sequential input data, such as natural language, with applications towards tasks such as translation and text summarization. Transformer architecture may process the entire input all at once. “Positional encoding,” for the purposes of this disclosure, refers to a data processing technique that encodes the location or position of an entity in a sequence.
[Col 19, lines 56-69]:
In some embodiments, position vectors for a plurality of positions in a sequence may be assembled into a position matrix, wherein each row of position matrix may represent a position in the sequence.
[Col 8, lines 20-24]:
As a further example, chatbot may display a list of possible forms to file or situations which would lead user to specific forms or actions, from which user may select each applicable form or situation the user currently has needs or application for.
[Col 9, lines 10-16]:
Training data may also be in the form of user feedback on a completed machine-learning evolution. Where a conversion is successful, user may provide feedback in the form of validating a successful operation, in which case machine-learning processes 128 would either take no action, or promote the associations relied upon for that specific engagement.
[BRI: an encoder to act as a policy network in which the encoder’s final hidden states (or a subset of them) can be used as the policy’s input and the policy network can then decide actions (e.g., next token, control signal) based on the encoder’s contextualized representations. Thus, an LLM encoder can absolutely be a Transformer with self-attention and positional encoding, and in that case, it can serve as a policy network by feeding its contextualized representations into a separate policy head. This is a standard pattern in encoder-only and encoder-decoder architectures
Watkins does not explicitly disclose:
- train a generator within the GAN to produce generated action sequences by feeding noise and latent representations of sensor data, where the generator employs an adapted transformer architecture to enhance diversity and realism of the generated action sequences, which mimic behavior of low-risk action sequences;
- concurrently training a discriminator within the GAN to differentiate between action sequences derived from the labeled dataset and synthetic action sequences produced by the generator; employing a feedback loop where assessments by the discriminator are used to adjust parameters for an iterative improvement process configured to produce sequences indistinguishable from real low-risk action sequences;
- and deploying the trained transformer-based policy network and GAN to process incoming unlabeled sensor data for real-time generation of risk scores and identification of low-risk action sequences upon reaching a predetermined threshold for accuracy in distinguishing between real and synthetic action sequences.
However, Moustafa discloses:
- train a generator within the GAN to produce generated action sequences by feeding noise and latent representations of sensor data, where the generator employs an adapted transformer architecture to enhance diversity and realism of the generated action sequences, which mimic behavior of low-risk action sequences;
[0365]:
Generative adversarial networks (GANs) have value but depend heavily on the
selection of adversarial data in a training set.
[0326]:
At 3106, a score is computed for the sensor data that indicates its overall goodness or quality. The score is based on the geolocation and environmental information obtained at 3104. For example, the score may be based on a location score computed from the geolocation and environmental information as described above. In some cases, the score may also be based on additional scoring information associated with the sensor data. For example, the score may be based a noise score, object diversity score, or other scores computed for the sensor data.
[0330]:
In some embodiments, irregular driving patterns can be modeled as a sequence of driving
actions that deviates from the normal behavior expected by the autonomous
vehicle. FIGS. 32 and 33 illustrate two examples of irregular driving patterns, and how an
autonomous vehicle may learn to adapt its behavior in response to observing such
behaviors.
[0376]:
For example, if a control event is a hard braking event, the vehicle behavior model may determine that the braking event is outside a normal threshold for braking and indicates a high probability of fault in the vehicle system
[0252]:
information may be collected for a pull-over request and/or remote valet event by the affected vehicles and/or surrounding sensor devices, and this information may be shared and leveraged to enhance autonomous driving systems
- concurrently training a discriminator within the GAN to differentiate between action sequences derived from the labeled dataset and synthetic action sequences produced by the generator;
[0627]:
A communication system 8900, as shown in FIG. 89, resolves many of the aforementioned issues (and more). In at least one embodiment, a privacy-preserving computer vision system employs a Generative Adversarial Network (GAN) to preserve privacy in computer vision applications while maintaining the utility of the data and minimally affecting computer vision capabilities. GANs are usually comprised of two neural networks, which may be referred to herein as a “generator” (or “generative model”) and a “discriminator” (or “discriminative model”). The generator learns from one (true) dataset and then tries to generate new data that resembles the training dataset. The discriminator tries to discriminate between the new data (produced by the generator) and the true data. The generator's goal is to increase the error rate of the discriminative network (e.g., “fool” the discriminator network) by producing novel synthesized instances that appear to have come from the true data distribution.
- employing a feedback loop where assessments by the discriminator are used to adjust parameters for an iterative improvement process configured to produce sequences indistinguishable from real low-risk action sequences;
[0739]:
As shown in FIG. 112, automated driving may rely on a very fast feedback loop using a logic engine 11202 (which includes perception, fusion planning, driver policy, and decision-making aspects), and Distributed Actuation of the AV 11204 based on the output of such engines. Each of these meta-modules may be dependent on input or processing that is assumed to be trustworthy
[0571]:
At 7620, based on input received from the driver, the autonomous vehicle determines what the situation is with the driver or the reason for the driver initiating the takeover request. If, for example, the situation is identified to be a risk for a driver or passenger (e.g., screaming, unsafe behavior, etc.), then re-planning may need to be considered for the route, and so the autonomous vehicle may modify the autonomous driving mode to pull over to stop at 7622.
[0604]:
System 8500 demonstrates an approach to extract anomaly events on the training and inference phases. Anomaly threshold 8506 is calculated during the training phase, where the network calculates the borderline between learned, unlearned, and anomaly events. In a particular embodiment, the anomaly threshold 8506 is based on a sigmoid function used by one or both of the baseline model 8504 and the SRU model 8502. The anomaly threshold 8506 may be used to adjust parameters of the SRU model 8502 during training.
(BRI: context of AI-driven risk management and operational optimization, sequences indistinguishable from real low-risk action sequences are typically generated using Generative Adversarial Networks (GANs) trained on labeled datasets of known low-risk behaviors
[0442]:
in yet another example, the sensor data alone may be used to generate a context of “over speed limit”. The determined context(s) is often expressed as metadata associated with the raw data.
[0603]:
FIG. 85 depicts a system 8500 for anomaly detection in accordance with certain embodiments. The addition of an anomaly detector may enhance the intelligence of a system to enable reporting of unknown situations (e.g., time-based events) that would not have been detected previously. A new ML model based on an LSTM or GRU architecture (termed Smart Recurrent Unit (SRU) model 8502 herein) may be provided and used in conjunction with a standard LSTM or GRU model (“baseline model” 8504). In various embodiments, the architecture of the SRU model 8502 may be similar to the architecture of the baseline predictor, but may be specially tuned to detect anomalies. In various embodiments, the system 8500 is able to both encode a newly arriving sequence of anomaly data (e.g., encode the sequence as an unknown class) as well as decode a given data representation to an anomaly tag (e.g., over time, identify new anomaly classes and apply labels accordingly). Any suitable data sequence may be recognized as an anomaly by the system 8500. For example, an anomaly may be an unknown detected object or an unknown detected event sequence. In various embodiments, the addition of the SRU model may enhance the system's intelligence to report unknown situations (time-based events) that were not been seen by the system previously (either at training or test phases). The system may be able to encode a new sequence of anomaly data and assign a label to it to create a new class. When the label is generated, any given data representation to this type of anomaly may be decoded.
- and deploying the trained transformer-based policy network and GAN to process incoming unlabeled sensor data for real-time generation of risk scores and identification of low-risk action sequences upon reaching a predetermined threshold for accuracy in distinguishing between real and synthetic action sequences.
[0251]:
in some implementations, machine learning models may be built and trained from this information and such machine learning models may be deployed on and executed by roadside units, cloud-based support systems, remote valet computing systems, or the in-vehicle systems of the autonomous vehicles themselves to provide logic for predictively determining potential remote valet handoffs.
[0189]:
In unsupervised learning, the model may be built from a set of data which contains only inputs and no desired outputs. The unsupervised model may be used to find structure in the data (e.g., grouping or clustering of data points) by discovering patterns in the data.
[0469]:
the synthetic attack data may be used as augmentation to boost the resiliency against the attacks on deep learning algorithms or classical ML algorithms. During training, the adversarial images with their correct labels are incorporated as part of the training set to refine the learning model
[0538]:
When the system determines to handoff to the human driver, the driver is notified of an imminent handoff. In order to do so, the system may engage with the driver in one or more of several possible manners. For example, the system may engage in a verbal manner with the driver. For example, text with correct semantics and syntax may be built by a natural language generation engine and then transformed into synthetic speech audio by a text-to-speech engine to produce a verbal message describing the handoff.
[0437]:
FIG. 50 depicts an example flow for handling data based on categorization in accordance with certain embodiments. At 5002, an instance of one or more objects from data captured by one or more sensors of a vehicle is identified. At 5004, a categorization of the instance is performed by checking the instance against a plurality of categories and assigning at least one category of the plurality of categories to the instance. At 5006, a score is determined based on the categorization of the instance. At 5008, a data handling policy for the instance is selected based at least in part on the score. At 5010, the instance is processed based on the determined data handling policy.
[0499]:
in yet another embodiment, the RSS enforcer 5908 may use policy information (e.g., based on preferences of the driver or based on safety considerations) to determine the acceleration value. For example, the RSS enforcer 5908 may favor longitudinal accelerations over latitudinal accelerations or vice versa. As another example, the RSS enforcer 5908 may favor accelerations that are more comfortable to the driver (e.g., slower braking or smaller steering adjustments may be preferred over hard braking or swerving). In various embodiments, the decision may be based on both safety and comfort, with related metrics calculated from the same set of motion parameters and vehicle characteristics. [BRI: a transformer-based policy network can represent the decision-making component of a verbal engagement system in a smart vehicle, especially when the goal is to balance driver preferences and safety considerations in real time-unsafe implies risk]
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Watkins, and Moustafa.
Watkins teaches GAN and discriminator based training for vehicle driver risk assessment.
Moustafa teaches concurrently training GAN and discriminator and deploying the transformed-based policy network.
One of ordinary skill would have motivation to combine Watkins and Moustafa that can provide improved safety of the handoff process of the vehicle to a driver (Moustafa [0520])
Claims 4 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over
Travis Watkins et.al. (hereinafter Watkins) US-11983494-B1,
in view of Hassnaa Moustafa et.al(hereinafter Moustafa) US-20220126864-A1,
further in view of Angelo DALLI (hereinafter DALLI)
US-20230259771-A1.
In regard to claim 4 :
Watkins and Moustafa do not explicitly disclose:
- further comprising: cleaning the sensor data by computing statistical measures on the sensor data;
However, DALLI discloses:
- further comprising: cleaning the sensor data by computing statistical measures on the sensor data;
[0167]:
an exemplary embodiment, a combination of (i.) a suitable correlation analysis method such as the Pearson product-moment correlation coefficient (PPMCC); or (ii.) a suitable variance analysis method such as ANOVA may be used with a suitable machine learning method such as gradient-descent or mutual information-based methods to provide a practical moderator discovery implementation.
[0427]:
an AutoXAI model search may be used to generate XAI models that may process sequence data. Sequence data may include a number of data points which contain feature data in various sequential formats including, but not limited to: 2D data, 3D data, transactional data, sensor data, image data, hyper-spectral data, natural language text, video data, audio data, haptic data, LIDAR data, RADAR data, SONAR data, and the like.
[0159]:
Applying a combination of standardization, data cleansing, data transforms, data profiling, data matching, data linking, data conformity checks, data accuracy checks, data precision checks, data bias checks, and data interpolation methods to the data available to the EIGS [and its components
[0432]:
In another exemplary embodiment, an AutoXAI system may be implemented within a camera processing system to automatically generate, insert and blend accurately positioned, framed, scaled, lit, and rendered synthetic images of people and objects within a combination of actual imagery and computer-generated imagery
[0229]:
In an exemplary embodiment, an exemplary AutoXAI system may be incorporated within an Explanation and Interpretation Generation System (EIGS),
[0180]:
The associations and assumptions component 1515 may contain information pertaining to observations about data accessible to the model 904 and/or the EIGS, that may determine a combination of: (i.) statistical associations between sets of data variables, either directly or via some recursive process; (ii.) conditional probabilities between data variables that may or may not need causal information; (iii.) inferences and associations obtained from data using conditional expectation or similar methods; (iv.) answers to conditional probability sentences of the form P(y|x)=p, where the probability of an event Y=y, given that X=x was observed, is equal to p, or a similar equivalent for the logic system being used in the EIGS in an exemplary embodiment; (v.) answers to exemplary scenario analysis of the form “What is X?”, “How would observing X change the current beliefs in Y?” and similar types of questions.
- and filtering the sensor data to remove unrelated data by employing a Pearson correlation coefficient computation.
[0167]:
an exemplary embodiment, a combination of (i.) a suitable correlation analysis method such as the Pearson product-moment correlation coefficient (PPMCC); or (ii.) a suitable variance analysis method such as ANOVA may be used with a suitable machine learning method such as gradient-descent or mutual information-based methods to provide a practical moderator discovery implementation.
[0427]:
an AutoXAI model search may be used to generate XAI models that may process sequence data. Sequence data may include a number of data points which contain feature data in various sequential formats including, but not limited to: 2D data, 3D data, transactional data, sensor data, image data, hyper-spectral data, natural language text, video data, audio data, haptic data, LIDAR data, RADAR data, SONAR data, and the like.
[0159]:
Applying a combination of standardization, data cleansing, data transforms, data profiling, data matching, data linking, data conformity checks, data accuracy checks, data precision checks, data bias checks, and data interpolation methods to the data available to the EIGS and its components [BRI: sensor data cleansing includes filtering as a core part of the process]
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Watkins, Moustafa, and DALLI.
Watkins teaches GAN and discriminator based training for vehicle driver risk assessment.
Moustafa teaches concurrently training GAN and discriminator and deploying the transformed-based policy network.
Tran teaches training a generator to produce generated action sequences presenting a low risk action sequence.
DALLI teaches the cleaning of the sensor data and Pearson correlation coefficient.
One of ordinary skill would have motivation to combine Watkins, Moustafa, and DALLI that
provides improved resulting quality of the explainable machine learning models output during the iterative process of an AutoXAI system (DALLI [0308]).
In regard to claim 13 :
Watkins and Moustafa do not explicitly disclose:
- further comprising: cleaning the sensor data by computing statistical measures on the sensor data;
However, DALLI discloses:
- further comprising: cleaning the sensor data by computing statistical measures on the sensor data;
[0167]:
an exemplary embodiment, a combination of (i.) a suitable correlation analysis method such as the Pearson product-moment correlation coefficient (PPMCC); or (ii.) a suitable variance analysis method such as ANOVA may be used with a suitable machine learning method such as gradient-descent or mutual information-based methods to provide a practical moderator discovery implementation.
[0427]:
an AutoXAI model search may be used to generate XAI models that may process sequence data. Sequence data may include a number of data points which contain feature data in various sequential formats including, but not limited to: 2D data, 3D data, transactional data, sensor data, image data, hyper-spectral data, natural language text, video data, audio data, haptic data, LIDAR data, RADAR data, SONAR data, and the like.
[0159]:
Applying a combination of standardization, data cleansing, data transforms, data profiling, data matching, data linking, data conformity checks, data accuracy checks, data precision checks, data bias checks, and data interpolation methods to the data available to the EIGS [and its components
[0432]:
In another exemplary embodiment, an AutoXAI system may be implemented within a camera processing system to automatically generate, insert and blend accurately positioned, framed, scaled, lit, and rendered synthetic images of people and objects within a combination of actual imagery and computer-generated imagery
[0229]:
In an exemplary embodiment, an exemplary AutoXAI system may be incorporated within an Explanation and Interpretation Generation System (EIGS),
[0180]:
The associations and assumptions component 1515 may contain information pertaining to observations about data accessible to the model 904 and/or the EIGS, that may determine a combination of: (i.) statistical associations between sets of data variables, either directly or via some recursive process; (ii.) conditional probabilities between data variables that may or may not need causal information; (iii.) inferences and associations obtained from data using conditional expectation or similar methods; (iv.) answers to conditional probability sentences of the form P(y|x)=p, where the probability of an event Y=y, given that X=x was observed, is equal to p, or a similar equivalent for the logic system being used in the EIGS in an exemplary embodiment; (v.) answers to exemplary scenario analysis of the form “What is X?”, “How would observing X change the current beliefs in Y?” and similar types of questions.
- and filtering the sensor data to remove unrelated data by employing a Pearson correlation coefficient computation.
[0167]:
an exemplary embodiment, a combination of (i.) a suitable correlation analysis method such as the Pearson product-moment correlation coefficient (PPMCC); or (ii.) a suitable variance analysis method such as ANOVA may be used with a suitable machine learning method such as gradient-descent or mutual information-based methods to provide a practical moderator discovery implementation.
[0427]:
an AutoXAI model search may be used to generate XAI models that may process sequence data. Sequence data may include a number of data points which contain feature data in various sequential formats including, but not limited to: 2D data, 3D data, transactional data, sensor data, image data, hyper-spectral data, natural language text, video data, audio data, haptic data, LIDAR data, RADAR data, SONAR data, and the like.
[0159]:
Applying a combination of standardization, data cleansing, data transforms, data profiling, data matching, data linking, data conformity checks, data accuracy checks, data precision checks, data bias checks, and data interpolation methods to the data available to the EIGS and its components [BRI: sensor data cleansing includes filtering as a core part of the process]
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Watkins, Tran, and DALLI.
Watkins teaches GAN and discriminator based training for vehicle driver risk assessment.
Moustafa teaches concurrently training GAN and discriminator and deploying the transformed-based policy network.
DALLI teaches the cleaning of the sensor data and Pearson correlation coefficient.
One of ordinary skill would have motivation to combine Watkins, Moustafa, and DALLI that
provides improved resulting quality of the explainable machine learning models output during the iterative process of an AutoXAI system (DALLI [0308]).
Claims 8-9 and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over
Travis Watkins et.al. (hereinafter Watkins) US-11983494-B1,
in view of Hassnaa Moustafa et.al (hereinafter Moustafa) US-20220126864-A1,
further in view of Ha Q Tran et.al(hereinafter Tran) US-10928830-B1.
In regard to claim 8:
Watkins and Moustfa do not explicitly disclose:
- further comprising estimating an insurance premium for drivers in accordance with the risk scores.
However, Tran discloses:
- further comprising estimating an insurance premium for drivers in accordance with the risk scores.
[Col 8, lines 19-23]:
FIG. 15 is a diagram illustrates generally, an overview of a reasonableness determination system that may allow drivers to obtain action recommendations based on the driver behavior parameters
[Col 8, lines 27-39]:
FIG. 17 is a flow chart illustrates generally, a method for selectively providing insurance information to a service provider
FIG. 18 is a diagram illustrates generally, an exemplary system that customizes insurance rates to correspond to behavior driver
FIG. 19 is a diagram illustrates generally an insurance rate adjustment component that further includes an analyzer component
FIG. 20 illustrates generally, a method for customizing insurance rates of a driver,
FIG. 21 illustrates generally, a method for presenting information related to a real-time insurance rate
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Watkin, Moustafa and Tran.
Watkins teaches GAN and discriminator based training for vehicle driver risk assessment.
Moustafa teaches concurrently training GAN and discriminator and deploying the transformed-based policy network.
Tran teaches insurance premium.
One of ordinary skill would have motivation to combine Watkins, Moustafa and Tran that detects vehicles to improve performance and inadvertently compromise the computers' decision-making ability (Tran [Col 7, lines 17-19]).
In regard to claim 9:
Watkins and Moustfa do not explicitly disclose:
- further comprising employing the risk scores to directly adjust insurance policy terms for potential low-risk drivers.
However, Tran discloses:
- further comprising employing the risk scores to directly adjust insurance policy terms for potential low-risk drivers.
[Col 8, lines 34-39]:
FIG. 19 is a diagram illustrates generally an insurance rate adjustment component that further includes an analyzer component, according to embodiments as disclosed herein;
FIG. 20 illustrates generally, a method for customizing insurance rates of a driver,
[Col 61, lines 21-35]:
in an embodiment, insurance rate can be calculated in real-time and as such can more accurately reflect appropriate coverage for a situation of a driver. A plurality of different factors can influence a likelihood of the driver being involved in an accident, having a vehicle stolen, and the like. For example, if the driver is travelling through bad weather, then risk can be higher and a rate can be increased in real-time as weather conditions change-conversely, if there is relatively little traffic surrounding the driver's vehicle, then the rate can be lowered. An algorithm or complex model can be used to calculate the insurance rates and can be disclosed to the driver through the display. In an embodiment, the rate adjustment component 804 can be configured to evaluate the insurance rate information against current vehicle operation by the driver
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Watkins, Moustafa and Tran.
Watkins teaches GAN and discriminator based training for vehicle driver risk assessment.
Moustafa teaches concurrently training GAN and discriminator and deploying the transformed-based policy network.
Tran teaches insurance premium.
One of ordinary skill would have motivation to combine Watkins, Moustafa and Tran that detects vehicles to improve performance and inadvertently compromise the computers' decision-making ability (Tran [Col 7, lines 17-19]).
In regard to claim 17:
Watkins and Moustfa do not explicitly disclose:
- further comprising estimating an insurance premium for drivers in accordance with the risk scores.
However, Tran discloses:
- further comprising estimating an insurance premium for drivers in accordance with the risk scores.
[Col 8, lines 19-23]:
FIG. 15 is a diagram illustrates generally, an overview of a reasonableness determination system that may allow drivers to obtain action recommendations based on the driver behavior parameters
[Col 8, lines 27-39]:
FIG. 17 is a flow chart illustrates generally, a method for selectively providing insurance information to a service provider
FIG. 18 is a diagram illustrates generally, an exemplary system that customizes insurance rates to correspond to behavior driver
FIG. 19 is a diagram illustrates generally an insurance rate adjustment component that further includes an analyzer component
FIG. 20 illustrates generally, a method for customizing insurance rates of a driver,
FIG. 21 illustrates generally, a method for presenting information related to a real-time insurance rate
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Watkin, Moustafa and Tran.
Watkins teaches GAN and discriminator based training for vehicle driver risk assessment.
Moustafa teaches concurrently training GAN and discriminator and deploying the transformed-based policy network.
Tran teaches insurance premium.
One of ordinary skill would have motivation to combine Watkins, Moustafa and Tran that detects vehicles to improve performance and inadvertently compromise the computers' decision-making ability (Tran [Col 7, lines 17-19]).
In regard to claim 18:
Watkins and Moustfa do not explicitly disclose:
- further comprising employing the risk scores to directly adjust insurance policy terms for potential low-risk drivers.
However, Tran discloses:
- further comprising employing the risk scores to directly adjust insurance policy terms for potential low-risk drivers.
[Col 8, lines 34-39]:
FIG. 19 is a diagram illustrates generally an insurance rate adjustment component that further includes an analyzer component, according to embodiments as disclosed herein;
FIG. 20 illustrates generally, a method for customizing insurance rates of a driver,
[Col 61, lines 21-35]:
in an embodiment, insurance rate can be calculated in real-time and as such can more accurately reflect appropriate coverage for a situation of a driver. A plurality of different factors can influence a likelihood of the driver being involved in an accident, having a vehicle stolen, and the like. For example, if the driver is travelling through bad weather, then risk can be higher and a rate can be increased in real-time as weather conditions change-conversely, if there is relatively little traffic surrounding the driver's vehicle, then the rate can be lowered. An algorithm or complex model can be used to calculate the insurance rates and can be disclosed to the driver through the display. In an embodiment, the rate adjustment component 804 can be configured to evaluate the insurance rate information against current vehicle operation by the driver
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Watkins, Moustafa and Tran.
Watkins teaches GAN and discriminator based training for vehicle driver risk assessment.
Moustafa teaches concurrently training GAN and discriminator and deploying the transformed-based policy network.
Tran teaches insurance premium.
One of ordinary skill would have motivation to combine Watkins, Moustafa and Tran that detects vehicles to improve performance and inadvertently compromise the computers' decision-making ability (Tran [Col 7, lines 17-19]).
Conclusion
Any inquiry concerning this communication or earlier communications from the
examiner should be directed to TIRUMALE KRISHNASWAMY RAMESH whose telephone number is (571)272-4605. The examiner can normally be reached by phone.
Examiner interviews are available via telephone, in-person, and video conferencing
using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li B Zhen can be reached on phone (571-272-3768). The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be
obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit:
https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for
information about filing in DOCX format. For additional questions, contact the Electronic
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO
Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TIRUMALE K RAMESH/Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights