DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Amendments
This Office Action is in response to the amendment filed on 06/30/2026.
Claims 1, 8, 13, and 19 have been amended.
No claims have been cancelled.
Claims 20 and 21 have been added.
Claim 12 is indicated as being currently amended in the remarks and amended claims document but claim 12 contains no changes. For purposes of examination, claim 12 will be treated as a previously presented claim.
The objections and rejections from the prior correspondence that are not restated herein are withdrawn.
Response to Arguments
Applicant's arguments filed on 06/30/2026 have been fully considered.
Applicant's arguments regarding the 35 U.S.C. 103 rejections of the previous office action have been fully considered but are not persuasive. Applicant argues that the generation and classification models of Nakata do not have the same or similar architectures as recited in amended claim 1, and that none of the cited references, including Nakata, nor any combination of the cited references, teach or suggest at least the features of "initially training a first artificial neural network with first input data and first pseudo data, wherein the first pseudo data is or was generated by a second artificial neural network in a virgin state, wherein the first and second artificial neural networks have the same or similar architectures" recited with the other features in the ordered combination of claim 1.
Examiner respectfully disagrees. The combination of HASSAN and NAKATA teaches the limitation indicated above. Specifically, HASSAN teaches:
initially training a first artificial neural network with first input data and first pseudo data, […] (HASSAN [0031] teaches: "At block 320, a pre-trained deep neural network model is trained (i.e., training) using the plurality of real roadway scene images (i.e., with first input data) and the plurality of augmented roadway scene images (i.e., and first pseudo data) to generate a newly-trained deep neural network model. For example, the deep neural network model 116 stored in the memory 106 is trained to detect traffic lights in image data using the plurality of real roadway scene images and the plurality of augmented roadway scene images. In some implementations, the pre-trained deep neural network model is an untrained deep neural network model (i.e., a first artificial neural network).")
HASSAN is not relied upon for teaching, but NAKATA teaches:
[…] wherein the first pseudo data is or was generated by a second artificial neural network in a virgin state, wherein the first and second artificial neural networks have the same or similar architectures; (NAKATA [0030] teaches: "The generation unit 101 initializes a generation model that generates pseudo data (Step S101). For example, the generation model is a neural network model with three layers. [...] In the initialization, the weight of each node of the generation model is set randomly, for example." NAKATA [0032] teaches: “The classification model is a neural network with three layers, for example. Similarly to the generation model, the classification model can be constructed with a technique similar to machine learning using a neural network (i.e., wherein the first and second artificial neural networks have the same or similar architectures).” Examiner's note: Applicant defines "virgin state" as a neural network with its parameters and weights set to random values, per paragraph [0072] of the instant application. Under broadest reasonable interpretation, a second artificial neural network in a virgin state can be interpreted as the generation model being initialized so that the weight of each node is set randomly.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of HASSAN and NAKATA before them, to include NAKATA's generation model that generates pseudo data in HASSAN’s detection method. One would have been motivated to make such a combination in order to generate pseudo data to increase the precision of the classification model (NAKATA [0051]).
Applicant further argues that new claims 20 and 21 recite that it is a same first artificial neural network that generates the first pseudo data while in the virgin state, that is trained by the first pseudo data, and that generates the second pseudo data, and that such a solution is not known from any of the cited prior art individually or in any combination.
Examiner respectfully disagrees. The newly introduced reference of THALER [0044] and [0061] teaches that a self-learning, multilayer, auto-associative neural network begins to generate servo patterns (i.e., first pseudo data) in an initialized state (i.e., while in a virgin state). Further, since independent claims 1 and 13 are rejected, the dependent claims 2-12 and 14-19 respectively are also rejected, as shown in the 35 U.S.C. 103 rejections below.
Claim Objections
Claims 20 and 21 are objected to because the claims recite "the first artificial neural network, or another artificial neural network" and "the trained first or further artificial neural network". It appears that "or further artificial neural network" is referring to "or another artificial neural network". For purposes of examination, the limitations will be construed as "the first artificial neural network, ora further artificial neural network".
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2-3, 7, 9-10, and 21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding Claim 2:
The claim recites "generating the first pseudo data using the first artificial neural network while in the virgin state". However, claim 1 was amended to only recite "wherein the first pseudo data is or was generated by a second artificial neural network in a virgin state". Removing the embodiment where the first pseudo data is generated by the first artificial neural network creates ambiguity as to which neural network is generating the first pseudo data. For purposes of examination, and to keep claim 2 consistent with other amendments in claim 1, the limitation of claim 2 will be construed as "generating the first pseudo data using the second artificial neural network while in the virgin state."
Regarding Claim 3:
The claim recites "wherein the second pseudo data is generated by the first artificial neural network” but claim 1 recites "generating second pseudo data". This creates ambiguity as to which artificial neural network is actually being used to generated the first pseudo data. For purposes of examination, the limitation of claim 3 will be construed as "wherein the second pseudo data is generated by the second artificial neural network".
Regarding Claim 7:
The claim recites "wherein generating the first pseudo data comprises: a) injecting a first random sample into the first or second artificial neural network". However, claim 1 was amended so that the first pseudo data is generated using the second artificial neural network. Since generating the first pseudo data is a process performed by the second artificial neural network, the limitation of claim 7 stating that generating the first pseudo data comprises injecting a first random sample into the first artificial neural network creates ambiguity as to which artificial neural network is used for generating the first pseudo data. For purposes of examination, the limitation of claim 7 will be construed as "wherein generating the first pseudo data comprises: a) injecting a first random sample into the second artificial neural network".
Regarding Claims 9 and 10:
The dependent claims inherit the deficiencies of their respective parent claims and are likewise rejected.
Regarding Claim 21:
The claim lacks antecedent basis for "the first pseudo data". For purposes of examination, the limitation will be construed as "a first pseudo data".
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4 and 7-10 are rejected under 35 U.S.C. 103 as being unpatentable over HASSAN (US 20210303886 A1) in view of ANS ("Neural networks with a self-refreshing memory: knowledge transfer in sequential learning tasks without catastrophic forgetting"), AGUIRRE (US 20190135300 A1), and NAKATA (US 20200090036 A1), hereafter HASSAN, ANS, AGUIRRE, and NAKATA, respectively.
Regarding Claim 1:
HASSAN teaches:
A method comprising: training an artificial neural network by: initially training a first artificial neural network with first input data and first pseudo data, […] (HASSAN [0031] teaches: "At block 320, a pre-trained deep neural network model is trained (i.e., training) using the plurality of real roadway scene images (i.e., with first input data) and the plurality of augmented roadway scene images (i.e., and first pseudo data) to generate a newly-trained deep neural network model. For example, the deep neural network model 116 stored in the memory 106 is trained to detect traffic lights in image data using the plurality of real roadway scene images and the plurality of augmented roadway scene images. In some implementations, the pre-trained deep neural network model is an untrained deep neural network model (i.e., a first artificial neural network).")
HASSAN is not relied upon for teaching:
wherein the first pseudo data is or was generated by a second artificial neural network in a virgin state, wherein the first and second artificial neural networks have the same or similar architectures;
generating second pseudo data using the second artificial neural network following at least partially transferring knowledge from the first artificial neural network to the second artificial neural network; and
training the first artificial neural network, or a further artificial neural network, with the second pseudo data and second input data; and
using the trained first or further artificial neural network in a hardware system to control one or more actuators.
However, ANS teaches: generating second pseudo data using the second artificial neural network following at least partially transferring knowledge from the first artificial neural network to the second artificial neural network; (ANS [pg. 4, section 2.1 A dual-network architecture] teaches: “Stage (I) is in fact intended to ‘transport’ the previously learned information from NET 1 to NET 2 (i.e., following at least partially transferring knowledge from the first artificial neural network to the second artificial neural network). A second processing procedure, denoted stage (II) (right part of figure 1), is intended to allow NET 1 to learn new external items. During this stage, NET 2 is ‘bombarded’ by the random generator and NET 1 becomes receptive again to external events it can then learn. A new population of external items must be trained concurrently with pseudoitems originating continuously from NET 2 (i.e., generating second pseudo data using the second artificial neural network), these pseudoitems being generated exactly as in NET 1 during stage (I).”)
training the first artificial neural network, or a further artificial neural network, with the second pseudo data and second input data; (ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "After a fixed number of reinjections (which is a simulation parameter, denoted R) within the NET 1 auto-associative part, the current generated input and output activities are, respectively, transmitted to the NET 2 input and output layers for training (i.e., training [...] a further artificial neural network). [...] A new population of external items (i.e., second input data) must be trained concurrently with pseudoitems originating continuously (i.e., second pseudo data) from NET 2, these pseudoitems being generated exactly as in NET 1 during stage (I). Subsequently, stages (I) and (II) are supposed to work alternatively for each new occurring population of ‘actual’ items.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of HASSAN and ANS before them, to include ANS’ reverberating architecture with self-refreshing memory in HASSAN’s detection method. One would have been motivated to make such a combination in order to overcome catastrophic forgetting in sequential learning tasks (ANS [pg. 4, section 2.1 A dual-network architecture]).
HASSAN in view of ANS is not relied upon for teaching:
wherein the first pseudo data is or was generated by a second artificial neural network in a virgin state, wherein the first and second artificial neural networks have the same or similar architectures;
using the trained first or further artificial neural network in a hardware system to control one or more actuators.
However, AGUIRRE teaches: using the trained first or further artificial neural network in a hardware system to control one or more actuators. (AGUIRRE [0026] teaches: "[…] collect sensor data, process the sensor data, and analyze the sensor data as disclosed herein to detect anomalies in the sensor data and control operations of the autonomous vehicle 100." AGUIRRE [0087] teaches: "FIG. 12 is a block diagram of an example processing platform 1200 structured to execute the instructions of FIG. 11 to implement the example autonomous driving apparatus 300 of FIG. 3 and/or the example anomaly detection apparatus 306 of FIGS. 3 and 4 to perform unsupervised multimodal anomaly detection for the example autonomous vehicle 100 (FIGS. 1-5). The processor platform 1200 can be, for example, a server, a computer, a self-learning machine (e.g., a neural network) (i.e., trained neural network), or any other type of computing device." Examiner's note: under BRI, "one […] actuator" can be interpreted as the autonomous vehicle, which is controlled by using the outputs of the collected and processed sensor data. A person having ordinary skill in the art could configure ANS’ NET 1 neural network to control AGUIRRE’s autonomous vehicle (i.e., actuator).”)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of HASSAN, ANS, and AGUIRRE before them, to include AGUIRRE’s sensors that collects data to train auto-encoders and control the operations of an autonomous vehicle in HASSAN and ANS’ detection method. One would have been motivated to make such a combination in order to address sensor malfunction over time in autonomous vehicles and making the autonomous vehicle computer sufficiently unaffected by changes in sensor operation, thereby increasing the flexibility and robustness of the autonomous vehicle computer (AGUIRRE [0096]).
HASSAN in view of ANS and AGUIRRE is not relied upon for teaching, but NAKATA teaches: wherein the first pseudo data is or was generated by a second artificial neural network in a virgin state, wherein the first and second artificial neural networks have the same or similar architectures; (NAKATA [0030] teaches: "The generation unit 101 initializes a generation model that generates pseudo data (Step S101). For example, the generation model is a neural network model with three layers. [...] In the initialization, the weight of each node of the generation model is set randomly, for example." NAKATA [0032] teaches: “The classification model is a neural network with three layers, for example. Similarly to the generation model, the classification model can be constructed with a technique similar to machine learning using a neural network (i.e., wherein the first and second artificial neural networks have the same or similar architectures).” Examiner's note: Applicant defines "virgin state" as a neural network with its parameters and weights set to random values, per paragraph [0072] of the instant application. Under broadest reasonable interpretation, a second artificial neural network in a virgin state can be interpreted as the generation model neural network being initialized so that the weight of each node is set randomly.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of HASSAN, ANS, AGUIRRE, and NAKATA before them, to include NAKATA's generation model that generates pseudo data in HASSAN, ANS, and AGUIRRE’s detection method. One would have been motivated to make such a combination in order to generate pseudo data to increase the precision of the classification model (NAKATA [0051]).
Regarding Claim 2:
HASSAN in view of ANS, AGUIRRE, and NAKATA teaches the elements of claim 1 as outlined above. NAKATA further teaches:
prior to initially training the first artificial neural network: generating the first pseudo data using the first artificial neural network while in the virgin state; (NAKATA [0022] teaches: “The generation unit 101 generates pseudo data used for learning of a classification model (i.e., prior to initially training the first artificial neural network).” NAKATA [0030] teaches: "The generation unit 101 initializes a generation model that generates pseudo data (Step S101) (i.e., generating the first pseudo data using the second artificial neural network). For example, the generation model is a neural network model with three layers. [...] In the initialization, the weight of each node of the generation model is set randomly (i.e., while in the virgin state), for example." Examiner's note: Applicant defines "virgin state" as a neural network with its parameters and weights set to random values, per paragraph [0072] of the instant application. Further, as stated above in the 112(b) rejections, “generating the first pseudo data using the first artificial neural network” is being construed as “generating the first pseudo data using the second artificial neural network.” Under BRI, the second artificial neural network in a virgin state can be interpreted as the generation model neural network being initialized so that the weight of each node is set randomly.)
and storing the first pseudo data to a memory. (NAKATA [0020] teaches: “The storage 121 stores therein various types of data used in various types of processing performed by the classification device 100. For example, the storage 121 stores therein training data, pseudo data, parameters of a generation model, parameters of a classification model, and the like.”)
Regarding Claim 3:
HASSAN in view of ANS, AGUIRRE, and NAKATA teaches the elements of claim 2 as outlined above. ANS further teaches:
wherein the second pseudo data is generated by the first artificial neural network […] (ANS [pg. 4, section 2.1 A dual-network architecture] teaches: “Stage (I) is in fact intended to ‘transport’ the previously learned information from NET 1 to NET 2. A second processing procedure, denoted stage (II) (right part of figure 1), is intended to allow NET 1 to learn new external items. During this stage, NET 2 is ‘bombarded’ by the random generator and NET 1 becomes receptive again to external events it can then learn. A new population of external items must be trained concurrently with pseudoitems originating continuously from NET 2 (i.e., the second pseudo data is generated by the second artificial neural network), these pseudoitems being generated exactly as in NET 1 during stage (I).” Examiner’s note: As stated above in the 112(b) rejections, “second pseudo data is generated by the first artificial neural network” is being construed as “second pseudo data is generated by the second artificial neural network.”)
[…] training the first artificial neural network with the second pseudo data and the second input data. (ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "A new population of external items (i.e., with second input data) must be trained concurrently with pseudoitems originating continuously (i.e., second pseudo data) from NET 2, these pseudoitems being generated exactly as in NET 1 during stage (I).” ANS [pg. 4, Figure 1] teaches: “Stage (II): the NET 1 network is learning (i.e., training the first artificial neural network) external items along with pseudoitems generated by the reverberating process in NET 2 (learning with self-refreshing of old information).”)
NAKATA further teaches: and stored to the memory prior to training […] (NAKATA [0030] teaches: "The generation unit 101 initializes a generation model that generates pseudo data (Step S101).” NAKATA [0020] teaches: “The storage 121 stores therein various types of data used in various types of processing performed by the classification device 100. For example, the storage 121 stores therein training data, pseudo data, parameters of a generation model, parameters of a classification model, and the like.”)
Regarding Claim 4:
HASSAN in view of ANS, AGUIRRE, and NAKATA teaches the elements of claim 1 as outlined above. ANS further teaches:
prior to generating the second pseudo data, at least partially transferring knowledge held by the first artificial neural network to the second artificial neural network, […] (ANS [pg. 3, section 2. Reverberating networks with a self-refreshing memory: sequential learning without catastrophic forgetting] teaches: "A simple explanation of the basic functioning of the architecture, shown in figure 1, would be to consider an initial state in which NET 1 has already learned completely a given set of external associative items and NET 2 is still "empty" (i.e. with random connection weights). [...] Stage (I) is in fact intended to ‘transport’ the previously learned information from NET 1 to NET 2.")
[...] wherein the second pseudo data is generated using the second artificial neural network, wherein the training of the first artificial neural network with the second pseudo data and second input data is performed at least partially in parallel with the generation of pseudo data by the second artificial neural network. (ANS [pg. 1, Abstract] teaches: "We show that transfer is more efficient when two related tasks are sequentially learned than when they are learned concurrently.” ANS [pg. 8, section 2.2. Simulations] teaches: "In stage (I), NET 2 is trained (i.e., training of the first artificial neural network) on pseudoitems (i.e., with the second pseudo data) generated by NET 1 from random input seeds (i.e., wherein the second pseudo data is generated using the second artificial neural network)." ANS [pg. 2, section 1. Introduction] teaches: "Once the learning of the first set A of associative pairs is completed and before the learning of the second set B starts, the network is stimulated by random input patterns, each generating a corresponding output pattern. These input–output pairs are successively stored in a pseudopopulation which is then considered as having captured something reflecting the set A structure. During the learning of the second set B (i.e., second input data), the network is concurrently trained (i.e., performed at least partially in parallel) on the input–output pairs previously stored in the pseudopopulation (i.e., with generation of pseudo data by the second artificial neural network).")
Regarding Claim 7:
HASSAN in view of ANS, AGUIRRE, and NAKATA teaches the elements of claim 1 as outlined above. ANS further teaches:
wherein generating the first pseudo data comprises: a) injecting a first random sample into the first or second artificial neural network, (ANS [pg. 7, section 2.2 Simulations] teaches: "The noise generator produces binary inputs, though random real-valued inputs should be suitable as well." ANS [pg. 4, Figure 1] teaches Stage (I) with NET 1 receiving the binary inputs or random real-valued inputs (i.e., injecting a first random sample into the second artificial neural network) from the Noise Generator. Figure 1 also shows Stage (II), where NET 2 receives inputs from the Noise Generator for generating input activity exactly as for NET 1 during Stage (I).)
wherein the first or second artificial neural network is configured to implement at least an auto-associative function for replicating input samples at one or more of its outputs, at least some of the replicated input samples present at the outputs forming the first pseudo data. (ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "After a fixed number of reinjections (which is a simulation parameter, denoted R) within the NET 1 auto-associative part, the current generated input and output activities are, respectively, transmitted to the NET 2 input and output layers for training. [...] For each of the successive random input seeds, the corresponding pseudoitems generated by reverberation within NET 1 are trained in NET 2." ANS [pg. 6-7, section 2.2 Simulations] teaches: "For each of the two simulation conditions, we start from an initial state of the system in which NET 1 has already completely learn the first set of 916 items (Dec-Add or Max-Op population) and NET 2 is still empty." Examiner's note: ANS’ Figure 1 explicitly teaches NET 1 and NET 2 having an auto-associative part (i.e., to implement at least an auto-associative function) which generates pseudoitems by the reinjecting process. Therefore, under broadest reasonable interpretation, for replicating input samples at one or more of its outputs can be interpreted as the reinjecting process that generates pseudoitems (i.e., at least some of the replicated input samples present at the outputs forming the first pseudo data) based on the Noise Generator inputs to NET 1 or NET 2, as described in ANS’ Figure 1.)
Regarding Claim 8:
HASSAN in view of ANS, AGUIRRE, and NAKATA teaches the elements of claim 1 as outlined above. ANS further teaches:
wherein generating the second pseudo data comprises: a) injecting a second sample into the first or second artificial neural network, (ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "This first resulting input layer activity is then reinjected in the hidden layer, which creates a new output and input activity. This second input activity is reinjected in the hidden layer, hence recreating a third input–output activity, and so on. This back and forth flow of activity between the hidden and input layers is termed a ‘reverberating’ process.")
wherein the first or second artificial neural network is configured to implement at least an auto-associative function for replicating input samples at one or more of its outputs, at least some of the replicated input samples present at the outputs forming the second pseudo data, (ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "After a fixed number of reinjections (which is a simulation parameter, denoted R) within the NET 1 auto-associative part, the current generated input and output activities are, respectively, transmitted to the NET 2 input and output layers for training. [...] For each of the successive random input seeds, the corresponding pseudoitems generated by reverberation within NET 1 are trained in NET 2." ANS [pg. 6-7, section 2.2 Simulations] teaches: "For each of the two simulation conditions, we start from an initial state of the system in which NET 1 has already completely learn the first set of 916 items (Dec-Add or Max-Op population) and NET 2 is still empty." Examiner's note: ANS’ Figure 1 explicitly teaches NET 1 and NET 2 having an auto-associative part (i.e., to implement at least an auto-associative function) which generates pseudoitems by the reinjecting process. Therefore, under broadest reasonable interpretation, for replicating input samples at one or more of its outputs can be interpreted as the reinjecting process that generates pseudoitems (i.e., at least some of the replicated input samples present at the outputs forming the second pseudo data) based on the Noise Generator inputs to NET 1 or NET 2, as described in ANS’ Figure 1.)
wherein the second sample is a random sample or a real sample. (ANS [pg. 7, section 2.2 Simulations] teaches: "The noise generator produces binary inputs, though random real-valued inputs should be suitable as well.")
Regarding Claim 9:
HASSAN in view of ANS, AGUIRRE, and NAKATA teaches the elements of claim 7 as outlined above. ANS further teaches:
wherein generating the first and/or second pseudo data further comprises: b) reinjecting a pseudo sample, generated based on the replicated sample present at the one or more outputs of the first or second artificial neural network, into the first or second artificial neural network in order to generate a new replicated sample at the one or more outputs; and c) repeating b) one or more times to generate a plurality of reinjected pseudo samples; (ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "This first resulting input layer activity is then reinjected in the hidden layer, which creates a new output and input activity. This second input activity is reinjected in the hidden layer, hence recreating a third input–output activity, and so on. This back and forth flow of activity between the hidden and input layers is termed a ‘reverberating’ process.")
wherein the first and/or second pseudo data comprises at least two of said reinjected pseudo samples originating from the same first or second sample and corresponding output values generated by the first or second artificial neural network. (ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "This first resulting input layer activity is then reinjected in the hidden layer, which creates a new output and input activity. This second input activity is reinjected in the hidden layer, hence recreating a third input–output activity, and so on. This back and forth flow of activity between the hidden and input layers is termed a ‘reverberating’ process. After a fixed number of reinjections (which is a simulation parameter, denoted R) within the NET 1 auto-associative part, the current generated input and output activities are, respectively, transmitted to the NET 2 input and output layers for training. The current NET 1 output plays the role of a pseudo heteroassociative target for NET 2 and the current NET 1 input plays the role of both a pseudo input and a pseudo auto-associative target for NET 2." Furthermore, ANS [pg. 4, Figure 1] teaches the back and forth flow of activity for both NET 1 and NET 2, where both networks can reinject the auto-associative target inputs (i.e., reinjected pseudo samples originating from the same first or second sample) and output hetero-associative targets (i.e., corresponding output values generated by the first or second artificial neural network).)
Regarding Claim 10:
HASSAN in view of ANS, AGUIRRE, and NAKATA teaches the elements of claim 9 as outlined above. ANS further teaches:
wherein the first and/or second artificial neural network implements a learning function, […], (ANS [pg. 3-4, section 2.1 A dual-network architecture] teaches: "When NET 1 (i.e., the first artificial neural network) is presented with a given external input–target pair to learn (i.e., implements a learning function), the error function to be minimized in the learning algorithm is based not only on the error between the computed output and the output target, but also on the error between the computed activation from hidden units to input units and the external input pattern. [...] The current NET 1 output plays the role of a pseudo heteroassociative target for NET 2 […].” ANS [pg. 4, Figure 1] teaches the back and forth flow of activity for both NET 1 and NET 2, where both networks can reinject the auto-associative target inputs and output hetero-associative targets.)
and wherein the corresponding output values of the first pseudo data comprise pseudo […] generated by the learning function based on the reinjected pseudo samples. (ANS [pg. 4, Figure 1] teaches NET 1 learning pseudoitems generated by NET 2 and outputting a pseudo heteroassociative target (i.e., pseudo targets generated by the learning function). This process is continuous for learning new populations, and therefore the pseudoitem outputs of NET 2 are reinjected into NET 1 for producing new output and input activity, based on the reinjected pseudoitems from NET 2.”)
NAKATA further teaches: which is for example a classification function (NAKATA [0015] teaches: "(S3) A classification model is learned to accurately classify the training data and the pseudo data into an existing class and a pseudo class. (S4) The classification model is learned to accurately classify the training data into a class of each supervised label.")
[…] pseudo labels […] (NAKATA [0015] teaches: "(S3) A classification model is learned to accurately classify the training data and the pseudo data into an existing class and a pseudo class. (S4) The classification model is learned to accurately classify the training data into a class of each supervised label.")
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over HASSAN in view of ANS, AGUIRRE, and NAKATA, as applied to claim 1 above, and further in view of MAJUMDAR (US 20170039469 A1), hereafter MAJUMDAR.
Regarding Claim 5:
HASSAN in view of ANS, AGUIRRE, and NAKATA teaches the elements of claim 1 as outlined above. ANS further teaches:
if the one or more third input data samples do not correspond to a […] that is already known, generating third pseudo data using the first artificial neural network, or using the second artificial neural network following at least partially transferring knowledge again from the first artificial neural network to the second artificial neural network, [...] (ANS [pg. 2, section 1. Introduction] teaches: "Catastrophic interference can be eliminated in sequential learning by using a rehearsal mechanism: the old information previously learned by a network is continually refreshed (i.e. retrained) during the learning of new information." ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "This first resulting input layer activity is then reinjected in the hidden layer, which creates a new output (i.e., generating third pseudo data) and input activity. This second input activity is reinjected in the hidden layer, hence recreating a third input–output activity, and so on. This back and forth flow of activity between the hidden and input layers is termed a ‘reverberating’ process." ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "After a fixed number of reinjections (which is a simulation parameter, denoted R) within the NET 1 (i.e., using the first artificial neural network) auto-associative part, the current generated input and output activities are, respectively, transmitted to the NET 2 input and output layers for training. [...] For each of the successive random input seeds, the corresponding pseudoitems generated by reverberation within NET 1 are trained in NET 2." ANS [pg. 4, Figure 1] teaches: “Stage (I): the NET 2 network is learning pseudoitems generated by the reverberating process in NET 1 (transport of NET 1 memory towards NET 2) (i.e., following at least partially transferring knowledge […] from the first artificial neural network to the second artificial neural network). Stage (II): the NET 1 network is learning external items along with pseudoitems generated by the reverberating process in NET 2 (learning with self-refreshing of old information).” ANS [pg. 4, section 2.1 A dual-network architecture] teaches: “Subsequently, stages (I) and (II) are supposed to work alternatively for each new occurring population of ‘actual’ items.” ANS [pg. 1, Abstract] teaches: “With a self-refreshing memory network knowledge can be saved for a long time and therefore reused in subsequent acquisitions.” Examiner’s note: ANS teaches a method for acquiring new knowledge in a way that each time a new population with items that do not belong to a learnt population are to be learned, the learning is done with self-refreshing memory for transferring knowledge from NET 1 to NET 2 (i.e., transferring knowledge again). Furthermore, ANS’ Figure 1 Stage (II) teaches NET 2 generating auto-associative targets (i.e., pseudo data) after Stage (I), where the transport of NET 1 memory towards NET 2 occurs.)
[…] and training the first artificial neural network with the third pseudo data and third input data samples. (ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "After a fixed number of reinjections (which is a simulation parameter, denoted R) within the NET 1 auto-associative part, the current generated input and output activities are, respectively, transmitted to the NET 2 input and output layers for training. [...] A new population of external items must be trained concurrently with pseudoitems originating continuously from NET 2, these pseudoitems being generated exactly as in NET 1 during stage (I). Subsequently, stages (I) and (II) are supposed to work alternatively for each new occurring population of ‘actual’ items (i.e., third input data samples).")
HASSAN in view of ANS, AGUIRRE, and NAKATA is not relied upon for teaching, but MAJUMDAR teaches: detecting, using a novelty detector, whether one or more third input data samples correspond to a class that is already known to the first artificial neural network; (MAJUMDAR [0003] teaches: "Certain aspects of the present disclosure generally relate to machine learning and, more particularly, to improving systems and methods for detecting unknown classes and initializing classifiers for unknown classes." MAJUMDAR [0038] teaches: "Furthermore, a second classifier (i.e., using a novelty detector) may be specified to categorize a sample as known or unknown based on the classification scores." MAJUMDAR [0069] teaches: "Still, in some cases, such as during the test of a network (i.e., the first artificial neural network), a sample (i.e., third input data samples) may not belong to any of the classes that were used to train a multi-class classifier (i.e., first artificial neural network). Thus, it may be desirable to generate a framework to initialize the multi-class classifier to label an object as belonging to a known class or belonging to an unknown class. The multi-class classifier may be referred to as the classifier." MAJUMDAR [0071] teaches: "Additionally, in some cases there may only be one class for a classifier. For example, the classifier may be specified to determine whether an object is a dog. In this example, when an image is presented to the classifier, the object is classified either as a dog or not a dog. Still, it may be desirable to create a framework to initialize a classifier to recognize an input as belonging to the one known class or to assign an unknown label to the input (i.e., correspond to a class that is already known)." Examiner's note: Paragraphs [0136]-[0138] of the specification describe using a novelty detector for implementing incremental learning after training and during the inference phase (e.g., testing phase) of the ANN. Under broadest reasonable interpretation, the one or more third input data samples are input data fed to the artificial neural network during the inference phase. MAJUMDAR [0069] explicitly teaches initializing a classifier (i.e., novelty detector) for labeling the input data as known or unknown during the test of a network.)
[…] class that is already known […] (MAJUMDAR [0069] teaches: "Still, in some cases, such as during the test of a network, a sample may not belong to any of the classes that were used to train a multi-class classifier. Thus, it may be desirable to generate a framework to initialize the multi-class classifier to label an object as belonging to a known class or belonging to an unknown class.”)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of HASSAN, ANS, AGUIRRE, NAKATA, and MAJUMDAR before them, to include MAJUMDAR’s method of detecting unknown classes using classifier to label an object as belonging to a known or unknown class in HASSAN, ANS, AGUIRRE, and NAKATA’s detection method. One would have been motivated to make such a combination so that when a machine learning model receives a sample during a test phase, we can detect if the sample belongs to a known or unknown class (MAJUMDAR [0069]).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over HASSAN in view of ANS, AGUIRRE, and NAKATA, as applied to claim 1 above, and further in view of MAJUMDAR and DERR (US 20200007249 A1), hereafter DERR.
Regarding Claim 6:
HASSAN in view of ANS, AGUIRRE, and NAKATA teaches the elements of claim 1 as outlined above. HASSAN in view of ANS, AGUIRRE, and NAKATA is not relied upon for teaching, but MAJUMDAR teaches:
detecting, by a controller, whether one or more third input data samples correspond to a new distribution not already learnt by the first artificial neural network; (MAJUMDAR [0003] teaches: "Certain aspects of the present disclosure generally relate to machine learning and, more particularly, to improving systems and methods for detecting unknown classes and initializing classifiers for unknown classes." MAJUMDAR [0041] teaches: “In an aspect of the present disclosure, the instructions loaded into the general-purpose processor 102 (i.e., controller) may comprise code for generating a first classifier for first classes. In one configuration, the output has a dimension of at least two. The instructions loaded into the general-purpose processor 102 may also comprise code for designing a second classifier to receive the output of the first classifier to decide whether the input data belongs to the first classes or at least one second class.” MAJUMDAR [0038] teaches: "Furthermore, a second classifier may be specified to categorize a sample as known or unknown based on the classification scores." MAJUMDAR [0069] teaches: "Still, in some cases, such as during the test of a network, a sample (i.e., third input data samples) may not belong to any of the classes (i.e., correspond to a new distribution) that were used to train a multi-class classifier (i.e., first artificial neural network). Thus, it may be desirable to generate a framework to initialize the multi-class classifier to label an object as belonging to a known class or belonging to an unknown class. The multi-class classifier may be referred to as the classifier." MAJUMDAR [0071] teaches: "Additionally, in some cases there may only be one class for a classifier. For example, the classifier may be specified to determine whether an object is a dog. In this example, when an image is presented to the classifier, the object is classified either as a dog or not a dog. Still, it may be desirable to create a framework to initialize a classifier to recognize an input as belonging to the one known class or to assign an unknown label to the input.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of HASSAN, ANS, AGUIRRE, NAKATA, and MAJUMDAR before them, to include MAJUMDAR’s processor for detecting unknown classes using classifier to label an object as belonging to a known or unknown class in HASSAN, ANS, AGUIRRE, and NAKATA’s detection method. One would have been motivated to make such a combination so that when a machine learning model receives a sample during a test phase, we can detect if the sample belongs to a known or unknown class (MAJUMDAR [0069]).
However, HASSAN in view of ANS, AGUIRRE, NAKATA, and MAJUMDAR is not relied upon for teaching, but DERR teaches: if the one or more third input data samples do correspond to the new distribution, creating a new system for learning the one or more third input data samples, the new system comprising at least a further first artificial neural network. (DERR [0186] teaches: "At block 1306, known signals are classified and unknown signals detected using RF measurements 1304 from RF measurement source 1302. Results 1308 including known signal classes and unknown signal classes are reported to users 1310. At block 1312, unknown signal characterization is performed on the detected unknown signals from block 1306 using incremental learning techniques to train new classifiers to classify unknown signals." Examiner's note: Paragraph [0136] of the specification states: "Furthermore, after training and during the inference phase of the ANN, it may be desirable to permit incremental learning, such that new classes of data samples can continue to be learn and handled by the system." DERR triggers the incremental learning for training new classifiers to classify unknown signals. A person having ordinary skill in the art would recognize that incremental learning is conceptually equivalent to ANS' sequential learning.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of HASSAN, ANS, AGUIRRE, NAKATA, MAJUMDAR, and DERR before them, to include DERR’s training of new classifiers when unknown signals (i.e., input data) is received in HASSAN, ANS, AGUIRRE, NAKATA, and MAJUMDAR’s detection method. One would have been motivated to make such a combination in order to incrementally build learned models and deploying classifiers for improving the accuracy of a deployed monitoring system over time (DERR [0183]).
Claims 11 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over HASSAN in view of ANS, AGUIRRE, and NAKATA, as applied to claim 1 above, and further in view of MAJUMDAR and DELGADO (US 20210089895 A1), hereafter DELGADO.
Regarding Claim 11:
HASSAN in view of ANS, AGUIRRE, and NAKATA teaches the elements of claim 1 as outlined above. AGUIRRE further teaches:
providing, by one or more sensors, one or more third input data samples to a novelty detector; (AGUIRRE [0041] teaches: "The anomaly detection apparatus 306 is provided with the example anomaly detector 412 (i.e., novelty detector) to detect anomalies represented in collected sensor data (i.e., one or more third input data samples). For example, the anomaly detector 412 can detect an anomaly in the probabilistic deviation estimation generated by the extractive deviation distribution analyzer 410." AGUIRRE [0026] teaches: “The example autonomous driving apparatus 300 represents operational components that execute as part of operating the autonomous vehicle 100 to collect sensor data, process the sensor data, and analyze the sensor data as disclosed herein to detect anomalies in the sensor data and control operations of the autonomous vehicle 100.”)
However, HASSAN in view of ANS, AGUIRRE, and NAKATA is not relied upon for teaching, but MAJUMDAR teaches: detecting whether the one or more third input data samples correspond to a class that is already known to the first artificial neural network, and if so, […] (MAJUMDAR [0003] teaches: "Certain aspects of the present disclosure generally relate to machine learning and, more particularly, to improving systems and methods for detecting unknown classes and initializing classifiers for unknown classes." MAJUMDAR [0038] teaches: "Furthermore, a second classifier may be specified to categorize a sample as known or unknown based on the classification scores." MAJUMDAR [0069] teaches: "Still, in some cases, such as during the test of a network (i.e., the first artificial neural network), a sample (i.e., third input data samples) may not belong to any of the classes that were used to train a multi-class classifier (i.e., first artificial neural network). Thus, it may be desirable to generate a framework to initialize the multi-class classifier to label an object as belonging to a known class or belonging to an unknown class. The multi-class classifier may be referred to as the classifier." MAJUMDAR [0071] teaches: "Additionally, in some cases there may only be one class for a classifier. For example, the classifier may be specified to determine whether an object is a dog. In this example, when an image is presented to the classifier, the object is classified either as a dog or not a dog. Still, it may be desirable to create a framework to initialize a classifier to recognize an input as belonging to the one known class or to assign an unknown label to the input (i.e., correspond to a class that is already known)." Examiner's note: Paragraphs [0136]-[0138] of the specification describe using a novelty detector for implementing incremental learning after training and during the inference phase (e.g., testing phase) of the ANN. Under broadest reasonable interpretation, the one or more third input data samples are input data fed to the artificial neural network during the inference phase. MAJUMDAR [0069] explicitly teaches initializing a classifier (i.e., novelty detector) for labeling the input data as known or unknown during the test of a network.)
However, HASSAN in view of ANS, AGUIRRE, NAKATA, and MAJUMDAR is not relied upon for teaching, but DELGADO teaches: […] providing the third input data samples to an inference module comprising the trained first or further artificial neural network in order to generate a predicted label for controlling the one or more actuators. (DELGADO [0009] teaches: “Furthermore, an indication of confidence in the reliability of the models results (predictions) is desirable.” DELGADO [0010] teaches: “For a counterfactual generation device, the obtained explanations and measures of reliability of the class predictions from a neural network (i.e., inference module) may further be used for the control of devices, such as the vehicle control for automated driving (i.e., for controlling the one or more actuators).” DELGADO [0100] teaches: “In a neural network designed for classification such as neural network 300, the output layer 303 receives values from at least one of the preceding hidden layers, e.g., from hidden layer 302b.” DELGADO [0101] teaches: “In the following, class predictions may also be referred to as predictions, predicted class labels or predicted classification labels (i.e., predicted label).” DELGADO [0104] teaches: “In order for the neural network 300 to be able to classify (i.e., in order to generate a predicted label) input sensor data (i.e., providing the third input data samples to an inference module), in particular image data, it is first trained (i.e., comprising the trained first or further artificial neural network) accordingly based on input training (image) data.”)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of HASSAN, ANS, AGUIRRE, NAKATA, MAJUMDAR, and DELGADO before them, to include DELGADO’s generating prediction labels for controlling an autonomous vehicle in HASSAN, ANS, AGUIRRE, NAKATA, and MAJUMDAR’s detection method. One would have been motivated to make such a combination in order to find meaningful (understandable and/or plausible for a human being) of the class prediction and/or to identify the root causes of the classification and/or to show to a user of the method which significant and meaningful changes are necessary for the neural network to change the classification score of a given input sensor data sample (DELGADO [0014]).
Regarding Claim 12:
HASSAN in view of ANS, AGUIRRE, NAKATA, MAJUMDAR, and DELGADO teaches the elements of claim 11 as outlined above. MAJUMDAR further teaches:
if the one or more third input data samples do not correspond to a class that is already known to the first artificial neural network: (MAJUMDAR [0069] teaches: "Still, in some cases, such as during the test of a network (i.e., the first artificial neural network), a sample (i.e., third input data samples) may not belong to any of the classes that were used to train a multi-class classifier (i.e., first artificial neural network). Thus, it may be desirable to generate a framework to initialize the multi-class classifier to label an object as belonging to a known class or belonging to an unknown class (i.e., do not correspond to a class that is already known).
ANS further teaches: generating third pseudo data using the first artificial neural network, or using the second artificial neural network following at least partially transferring knowledge again from the first artificial neural network to the second artificial neural network; (ANS [pg. 2, section 1. Introduction] teaches: "Catastrophic interference can be eliminated in sequential learning by using a rehearsal mechanism: the old information previously learned by a network is continually refreshed (i.e. retrained) during the learning of new information." ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "This first resulting input layer activity is then reinjected in the hidden layer, which creates a new output (i.e., generating third pseudo data) and input activity. This second input activity is reinjected in the hidden layer, hence recreating a third input–output activity, and so on. This back and forth flow of activity between the hidden and input layers is termed a ‘reverberating’ process." ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "After a fixed number of reinjections (which is a simulation parameter, denoted R) within the NET 1 (i.e., using the first artificial neural network) auto-associative part, the current generated input and output activities are, respectively, transmitted to the NET 2 input and output layers for training. [...] For each of the successive random input seeds, the corresponding pseudoitems generated by reverberation within NET 1 are trained in NET 2." ANS [pg. 4, Figure 1] teaches: “Stage (I): the NET 2 network is learning pseudoitems generated by the reverberating process in NET 1 (transport of NET 1 memory towards NET 2) (i.e., following at least partially transferring knowledge […] from the first artificial neural network to the second artificial neural network). Stage (II): the NET 1 network is learning external items along with pseudoitems generated by the reverberating process in NET 2 (learning with self-refreshing of old information).” ANS [pg. 4, section 2.1 A dual-network architecture] teaches: “Subsequently, stages (I) and (II) are supposed to work alternatively for each new occurring population of ‘actual’ items.” ANS [pg. 1, Abstract] teaches: “With a self-refreshing memory network knowledge can be saved for a long time and therefore reused in subsequent acquisitions.” Examiner’s note: ANS teaches a method for acquiring new knowledge in a way that each time a new population with items that do not belong to a learnt population are to be learned, the learning is done with self-refreshing memory for transferring knowledge from NET 1 to NET 2 (i.e., transferring knowledge again). Furthermore, ANS’ Figure 1 Stage (II) teaches NET 2 generating auto-associative targets (i.e., pseudo data) after Stage (I), where the transport of NET 1 memory towards NET 2 occurs.)
training the first artificial neural network with the third pseudo data and third input data samples. (ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "After a fixed number of reinjections (which is a simulation parameter, denoted R) within the NET 1 auto-associative part, the current generated input and output activities are, respectively, transmitted to the NET 2 input and output layers for training. [...] A new population of external items must be trained (i.e., training) concurrently with pseudoitems originating continuously from NET 2, these pseudoitems being generated exactly as in NET 1 during stage (I). Subsequently, stages (I) and (II) are supposed to work alternatively for each new occurring population of ‘actual’ items (i.e., third input data samples).")
Claims 13 and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over HASSAN in view of NAKATA, AGUIRRE, RAGHAVAN (US 20200302339 A1), hereafter RAGHAVAN, and ANS.
Regarding Claim 13:
HASSAN teaches:
A system comprising: a first artificial neural network; (HASSAN [0031] teaches: “In some implementations, the pre-trained deep neural network model is an untrained deep neural network model (i.e., a first artificial neural network).")
initially training the first artificial neural network with first input data and with the first pseudo data; (HASSAN [0031] teaches: "At block 320, a pre-trained deep neural network model is trained (i.e., training) using the plurality of real roadway scene images (i.e., with first input data) and the plurality of augmented roadway scene images (i.e., and first pseudo data) to generate a newly-trained deep neural network model. For example, the deep neural network model 116 stored in the memory 106 is trained to detect traffic lights in image data using the plurality of real roadway scene images and the plurality of augmented roadway scene images. In some implementations, the pre-trained deep neural network model is an untrained deep neural network model (i.e., a first artificial neural network).")
HASSAN is not relied upon for teaching:
a second artificial neural network in a virgin state and configured to generate first pseudo data, wherein the first and second artificial neural networks have the same or similar architectures;
one or more actuators; and
one or more circuits or processors configured to generate old memories for use in training the first artificial neural network by:
generating second pseudo data using the second artificial neural network following at least partially transferring knowledge from the first artificial neural network to the second artificial neural network;
training the first artificial neural network, or a further artificial neural network, with the second pseudo data and second input data; and
controlling the one or more actuators using the trained first or further artificial neural network.
However, NAKATA teaches: a second artificial neural network in a virgin state and configured to generate first pseudo data, […]; (NAKATA [0030] teaches: "The generation unit 101 initializes a generation model that generates pseudo data (i.e., and configured to generate first pseudo data) (Step S101). For example, the generation model is a neural network model with three layers. [...] In the initialization, the weight of each node of the generation model is set randomly, for example." Examiner's note: Applicant defines "virgin state" as a neural network with its parameters and weights set to random values, per paragraph [0072] of the instant application. Under broadest reasonable interpretation, a second artificial neural network in a virgin state can be interpreted as the generation model neural network being initialized so that the weight of each node is set randomly.)
[…] wherein the first and second artificial neural networks have the same or similar architectures; (NAKATA [0032] teaches: “The classification model is a neural network with three layers, for example. Similarly to the generation model, the classification model can be constructed with a technique similar to machine learning using a neural network (i.e., wherein the first and second artificial neural networks have the same or similar architectures).”)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of HASSAN and NAKATA before them, to include NAKATA's generation model that generates pseudo data in HASSAN’s detection method. One would have been motivated to make such a combination in order to generate pseudo data to increase the precision of the classification model (NAKATA [0051]).
HASSAN in view of NAKATA is not relied upon for teaching:
one or more actuators; and
one or more circuits or processors configured to generate old memories for use in training the first artificial neural network by:
generating second pseudo data using the second artificial neural network following at least partially transferring knowledge from the first artificial neural network to the second artificial neural network;
training the first artificial neural network, or a further artificial neural network, with the second pseudo data and second input data; and
controlling the one or more actuators using the trained first or further artificial neural network.
However, AGUIRRE teaches: one or more actuators; (AGUIRRE [0026] teaches: “The example autonomous driving apparatus 300 represents operational components that execute as part of operating the autonomous vehicle 100 to collect sensor data, process the sensor data, and analyze the sensor data as disclosed herein to detect anomalies in the sensor data and control operations of the autonomous vehicle 100 (i.e., one or more actuators).”)
controlling the one or more actuators using the trained first or further artificial neural network. (AGUIRRE [0026] teaches: "[…] collect sensor data, process the sensor data, and analyze the sensor data as disclosed herein to detect anomalies in the sensor data and control operations of the autonomous vehicle 100." AGUIRRE [0087] teaches: "FIG. 12 is a block diagram of an example processing platform 1200 structured to execute the instructions of FIG. 11 to implement the example autonomous driving apparatus 300 of FIG. 3 and/or the example anomaly detection apparatus 306 of FIGS. 3 and 4 to perform unsupervised multimodal anomaly detection for the example autonomous vehicle 100 (FIGS. 1-5). The processor platform 1200 can be, for example, a server, a computer, a self-learning machine (e.g., a neural network) (i.e., trained neural network), or any other type of computing device." Examiner's note: under BRI, "one […] actuator" can be interpreted as the autonomous vehicle, which is controlled by using the outputs of the collected and processed sensor data. A person having ordinary skill in the art could configure ANS’ NET 1 neural network to control AGUIRRE’s autonomous vehicle (i.e., actuator).)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of HASSAN, NAKATA, and AGUIRRE before them, to include AGUIRRE’s sensors that collects data to train auto-encoders and control the operations of an autonomous vehicle in HASSAN and NAKATA’s detection method. One would have been motivated to make such a combination in order to address sensor malfunction over time in autonomous vehicles and making the autonomous vehicle computer sufficiently unaffected by changes in sensor operation, thereby increasing the flexibility and robustness of the autonomous vehicle computer (AGUIRRE [0096]).
HASSAN in view of NAKATA and AGUIRRE is not relied upon for teaching:
one or more circuits or processors configured to generate old memories for use in training the first artificial neural network by:
generating second pseudo data using the second artificial neural network following at least partially transferring knowledge from the first artificial neural network to the second artificial neural network;
training the first artificial neural network, or a further artificial neural network, with the second pseudo data and second input data; and
However, RAGHAVAN teaches: one or more circuits or processors configured to generate old memories for use in training the first artificial neural network (RAGHAVAN [0009] teaches: “Furthermore, the techniques disclosed herein may enable a machine learning system to perform scalable, lifelong learning of solutions for new tasks the machine learning system has not previously been trained to solve, while reducing the occurrence of catastrophic forgetting (e.g., forgetting solutions to old tasks as a result of learning solutions to new tasks).” RAGHAVAN [0010] teaches: “[…] generative memory is capable of generating the auxiliary data from old tasks for use in training the machine learning model to obtain labels for new tasks for which the machine learning model has not previously been trained.” RAGHAVAN [0043] teaches: “Computation engine 230 includes machine learning system 102, observational module 118 and generative memory 104. Each of machine learning system 102, observational module 118, and generative memory 104 may represent software executable by processing circuitry 206 and stored on storage device 208, or a combination of hardware and software. Such processing circuitry 206 may include any one or more of a microprocessor, a controller, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or equivalent discrete or integrated logic circuitry.”)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of HASSAN, NAKATA, AGUIRRE, and RAGHAVAN before them, to include RAGHAVAN’s generative memory in HASSAN, NAKATA, and AGUIRRE’s detection method. One would have been motivated to make such a combination in order to obtain labels for new tasks for which the machine learning model has not previously learned to reduce the occurrence of catastrophic forgetting (RAGHAVAN [0009]-[0010]).
HASSAN in view of NAKATA, AGUIRRE, and RAGHAVAN is not relied upon for teaching, but ANS teaches: generating second pseudo data using the second artificial neural network following at least partially transferring knowledge from the first artificial neural network to the second artificial neural network; (ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "This first resulting input layer activity is then reinjected in the hidden layer, which creates a new output (i.e., generating second pseudo data) and input activity. This second input activity is reinjected in the hidden layer, hence recreating a third input–output activity, and so on. This back and forth flow of activity between the hidden and input layers is termed a ‘reverberating’ process." ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "After a fixed number of reinjections (which is a simulation parameter, denoted R) within the NET 1 (i.e., using the second artificial neural network) auto-associative part, the current generated input and output activities are, respectively, transmitted to the NET 2 input and output layers for training. [...] For each of the successive random input seeds, the corresponding pseudoitems generated by reverberation within NET 1 are trained in NET 2.")
training the first artificial neural network, or a further artificial neural network, with the second pseudo data and second input data; (ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "After a fixed number of reinjections (which is a simulation parameter, denoted R) within the NET 1 auto-associative part, the current generated input and output activities are, respectively, transmitted to the NET 2 input and output layers for training (i.e., training [...] a further artificial neural network). [...] A new population of external items (i.e., second input data) must be trained concurrently with pseudoitems originating continuously (i.e., second pseudo data) from NET 2, these pseudoitems being generated exactly as in NET 1 during stage (I). Subsequently, stages (I) and (II) are supposed to work alternatively for each new occurring population of ‘actual’ items.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of HASSAN, NAKATA, AGUIRRE, RAGHAVAN, and ANS before them, to include ANS’ reverberating architecture with self-refreshing memory in HASSAN, NAKATA, AGUIRRE, and RAGHAVAN’s detection method. One would have been motivated to make such a combination in order to overcome catastrophic forgetting in sequential learning tasks (ANS [pg. 4, section 2.1 A dual-network architecture]).
Regarding Claim 18:
HASSAN in view of NAKATA, AGUIRRE, RAGHAVAN, and ANS teaches the elements of claim 13 as outlined above. Additionally, the claim recites similar limitations as corresponding claim 7 and is rejected for similar reasons as claim 7 using similar teachings and rationale. NAKATA further teaches:
wherein the one or more circuits or processors is configured to generate the first pseudo data […] (NAKATA [0027] teaches: “For example, the units (the generation unit 101, the learning unit 102, the classification unit 103, and the output control unit 104) described above are implemented by a singular processor or plural processors (i.e., one or more […] processors).” NAKATA [0030] teaches: "The generation unit 101 initializes a generation model that generates pseudo data (i.e., configured to generate the first pseudo data) (Step S101).”)
Regarding Claim 19:
HASSAN in view of NAKATA, AGUIRRE, RAGHAVAN, and ANS teaches the elements of claim 13 as outlined above. NAKATA further teaches:
wherein the one or more circuits or processors is configured to generate the […] pseudo data […] (NAKATA [0027] teaches: “For example, the units (the generation unit 101, the learning unit 102, the classification unit 103, and the output control unit 104) described above are implemented by a singular processor or plural processors (i.e., one or more […] processors).” NAKATA [0030] teaches: "The generation unit 101 initializes a generation model that generates pseudo data (i.e., configured to generate […] pseudo data) (Step S101).”)
ANS further teaches: […] generate the second pseudo data by: a) injecting a second sample into the first or second artificial neural network, wherein the first or second artificial neural network is configured to implement at least an auto-associative function for replicating input samples at one or more of its outputs, at least some of the replicated input samples present at the outputs forming the second pseudo data, wherein the second sample is a random sample or a real sample. (ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "This first resulting input layer activity is then reinjected in the hidden layer, which creates a new output and input activity. This second input activity is reinjected in the hidden layer, hence recreating a third input–output activity, and so on. This back and forth flow of activity between the hidden and input layers is termed a ‘reverberating’ process." Examiner’s note: A person of ordinary skill in the art would recognize that ANS teaches generating pseudo auto-associative targets and that NAKATA teaches a generation unit for generating pseudo data, and therefore ANS’ pseudo auto-associative targets (i.e., pseudo data) can be generated using NAKATA’s generation unit.)
Claims 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over HASSAN in view of NAKATA, AGUIRRE, RAGHAVAN, and ANS, as applied to claim 13 above, and further in view of MAJUMDAR.
Regarding Claim 14:
HASSAN in view of NAKATA, AGUIRRE, RAGHAVAN, and ANS teaches the elements of claim 13 as outlined above. ANS further teaches:
generate third pseudo data using the first artificial neural network, or using the second artificial neural network following at least partially transferring knowledge again from the first artificial neural network to the second artificial neural network; (ANS [pg. 2, section 1. Introduction] teaches: "Catastrophic interference can be eliminated in sequential learning by using a rehearsal mechanism: the old information previously learned by a network is continually refreshed (i.e. retrained) during the learning of new information." ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "This first resulting input layer activity is then reinjected in the hidden layer, which creates a new output (i.e., generating third pseudo data) and input activity. This second input activity is reinjected in the hidden layer, hence recreating a third input–output activity, and so on. This back and forth flow of activity between the hidden and input layers is termed a ‘reverberating’ process." ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "After a fixed number of reinjections (which is a simulation parameter, denoted R) within the NET 1 (i.e., using the first artificial neural network) auto-associative part, the current generated input and output activities are, respectively, transmitted to the NET 2 input and output layers for training. [...] For each of the successive random input seeds, the corresponding pseudoitems generated by reverberation within NET 1 are trained in NET 2." ANS [pg. 4, Figure 1] teaches: “Stage (I): the NET 2 network is learning pseudoitems generated by the reverberating process in NET 1 (transport of NET 1 memory towards NET 2) (i.e., following at least partially transferring knowledge […] from the first artificial neural network to the second artificial neural network). Stage (II): the NET 1 network is learning external items along with pseudoitems generated by the reverberating process in NET 2 (learning with self-refreshing of old information).” ANS [pg. 4, section 2.1 A dual-network architecture] teaches: “Subsequently, stages (I) and (II) are supposed to work alternatively for each new occurring population of ‘actual’ items.” ANS [pg. 1, Abstract] teaches: “With a self-refreshing memory network knowledge can be saved for a long time and therefore reused in subsequent acquisitions.” Examiner’s note: ANS teaches a method for acquiring new knowledge in a way that each time a new population with items that do not belong to a learnt population are to be learned, the learning is done with self-refreshing memory for transferring knowledge from NET 1 to NET 2 (i.e., transferring knowledge again). Furthermore, ANS’ Figure 1 Stage (II) teaches NET 2 generating auto-associative targets (i.e., pseudo data) after Stage (I), where the transport of NET 1 memory towards NET 2 occurs.)
training the first artificial neural network with the third pseudo data and third input data samples. (ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "After a fixed number of reinjections (which is a simulation parameter, denoted R) within the NET 1 auto-associative part, the current generated input and output activities are, respectively, transmitted to the NET 2 input and output layers for training. [...] A new population of external items must be trained concurrently with pseudoitems originating continuously from NET 2, these pseudoitems being generated exactly as in NET 1 during stage (I). Subsequently, stages (I) and (II) are supposed to work alternatively for each new occurring population of ‘actual’ items (i.e., third input data samples).")
However, HASSAN in view of NAKATA, AGUIRRE, RAGHAVAN, and ANS is not relied upon for teaching, but MAJUMDAR teaches: further comprising a novelty detector configured to: detect whether one or more third input data samples correspond to a class that is already known to the first artificial neural network; (MAJUMDAR [0003] teaches: "Certain aspects of the present disclosure generally relate to machine learning and, more particularly, to improving systems and methods for detecting unknown classes and initializing classifiers for unknown classes." MAJUMDAR [0038] teaches: "Furthermore, a second classifier (i.e., a novelty detector) may be specified to categorize a sample as known or unknown (i.e., configured to detect) based on the classification scores." MAJUMDAR [0069] teaches: "Still, in some cases, such as during the test of a network (i.e., the first artificial neural network), a sample (i.e., third input data samples) may not belong to any of the classes that were used to train a multi-class classifier (i.e., first artificial neural network). Thus, it may be desirable to generate a framework to initialize the multi-class classifier to label an object as belonging to a known class or belonging to an unknown class. The multi-class classifier may be referred to as the classifier." MAJUMDAR [0071] teaches: "Additionally, in some cases there may only be one class for a classifier. For example, the classifier may be specified to determine whether an object is a dog. In this example, when an image is presented to the classifier, the object is classified either as a dog or not a dog. Still, it may be desirable to create a framework to initialize a classifier to recognize an input as belonging to the one known class or to assign an unknown label to the input (i.e., correspond to a class that is already known)." Examiner's note: Paragraphs [0136]-[0138] of the specification describe using a novelty detector for implementing incremental learning after training and during the inference phase (e.g., testing phase) of the ANN. Under broadest reasonable interpretation, the one or more third input data samples are input data fed to the artificial neural network during the inference phase. MAJUMDAR [0069] explicitly teaches initializing a classifier (i.e., novelty detector) for labeling the input data as known or unknown during the test of a network.)
wherein, if the one or more third input data samples do not correspond to a class that is already known, the one or more circuits or processors is further configured to: (MAJUMDAR [0038] teaches: "Furthermore, a second classifier may be specified to categorize a sample as known or unknown (i.e., configured to detect) based on the classification scores." MAJUMDAR [0069] teaches: "Still, in some cases, such as during the test of a network, a sample (i.e., the one or more third input data samples) may not belong to any of the classes that were used to train a multi-class classifier (i.e., first artificial neural network). Thus, it may be desirable to generate a framework to initialize the multi-class classifier to label an object as belonging to a known class or belonging to an unknown class.” Examiner’s note: MAJUMDAR explicitly teaches generating a framework during the test of a network to label objects in cases where a class does not belong to any of the classes used during training (i.e., wherein, if the one or more third input data samples do not correspond to a class that is already known).)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of HASSAN, NAKATA, AGUIRRE, RAGHAVAN, ANS, and MAJUMDAR before them, to include MAJUMDAR’s method of detecting unknown classes using classifier to label an object as belonging to a known or unknown class in HASSAN, NAKATA, AGUIRRE, RAGHAVAN, and ANS’ detection method. One would have been motivated to make such a combination so that when a machine learning model receives a sample during a test phase, we can detect if the sample belongs to a known or unknown class (MAJUMDAR [0069]).
Regarding Claim 15:
HASSAN in view of NAKATA, AGUIRRE, RAGHAVAN, ANS, and MAJUMDAR teaches the elements of claim 14 as outlined above. AGUIRRE further teaches:
further comprising one or more sensors configured to provide the one or more third input data samples. (AGUIRRE [0041] teaches: "The anomaly detection apparatus 306 is provided with the example anomaly detector 412 to detect anomalies represented in collected sensor data (i.e., one or more third input data samples). For example, the anomaly detector 412 can detect an anomaly in the probabilistic deviation estimation generated by the extractive deviation distribution analyzer 410." AGUIRRE [0026] teaches: “The example autonomous driving apparatus 300 represents operational components that execute as part of operating the autonomous vehicle 100 to collect sensor data, process the sensor data, and analyze the sensor data as disclosed herein to detect anomalies in the sensor data and control operations of the autonomous vehicle 100.” AGUIRRE [0044] teaches: “To process input sensor data from multiple ones of the sensors (i.e., one or more sensors configured to provide the one or more third input data samples) 202, 204, 206, 304, multiple ones of the auto-encoder 600 can be employed as described below in connection with FIG. 7.”)
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over HASSAN in view of NAKATA, AGUIRRE, RAGHAVAN, ANS, and MAJUMDAR, as applied to claim 14 above, and further in view of DELGADO.
Regarding Claim 16:
HASSAN in view of NAKATA, AGUIRRE, RAGHAVAN, ANS, and MAJUMDAR teaches the elements of claim 14 as outlined above. Additionally, the claim recites similar limitations as corresponding claim 11 and is rejected for similar reasons as claim 11 using similar teachings and rationale. Additionally, DELGADO teaches:
[…] the one or more circuits or processors is further configured to provide the third input data samples to an inference module comprising the trained first or further artificial neural network in order to generate a predicted label for controlling the one or more actuators. (DELGADO [0180] teaches: “The method of FIG. 6 may be performed by one or more processors. The term “processor” can be understood as any type of entity that allows the processing of data or signals. For example, the data or signals may be treated according to at least one (i.e., one or more than one) specific function performed by the processor.” DELGADO [0058] teaches: “FIG. 6 shows a flow diagram illustrating an exemplary method of the present invention for generating a counterfactual data sample using a neural network.” DELGADO [0010] teaches: “For a counterfactual generation device, the obtained explanations and measures of reliability of the class predictions from a neural network (i.e., inference module) may further be used for the control of devices, such as the vehicle control for automated driving (i.e., for controlling the one or more actuators).”)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of HASSAN, NAKATA, AGUIRRE, RAGHAVAN, ANS, MAJUMDAR, and DELGADO before them, to include DELGADO’s generating prediction labels for controlling an autonomous vehicle in HASSAN, NAKATA, AGUIRRE, RAGHAVAN, ANS, and MAJUMDAR’s detection method. One would have been motivated to make such a combination in order to find meaningful (understandable and/or plausible for a human being) of the class prediction and/or to identify the root causes of the classification and/or to show to a user of the method which significant and meaningful changes are necessary for the neural network to change the classification score of a given input sensor data sample (DELGADO [0014]).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over HASSAN in view of NAKATA, AGUIRRE, RAGHAVAN, and ANS, as applied to claim 13 above, and further in view of MAJUMDAR and DERR.
Regarding Claim 17:
HASSAN in view of NAKATA, AGUIRRE, RAGHAVAN, and ANS teaches the elements of claim 13 as outlined above. HASSAN in view of NAKATA, AGUIRRE, RAGHAVAN, and ANS is not relied upon for teaching, but MAJUMDAR teaches:
further comprising a controller configured to: detect whether one or more third input data samples correspond to a new distribution not already learnt by the first artificial neural network; (MAJUMDAR [0003] teaches: "Certain aspects of the present disclosure generally relate to machine learning and, more particularly, to improving systems and methods for detecting unknown classes and initializing classifiers for unknown classes." MAJUMDAR [0041] teaches: “In an aspect of the present disclosure, the instructions loaded into the general-purpose processor 102 (i.e., controller) may comprise code for generating a first classifier for first classes. In one configuration, the output has a dimension of at least two. The instructions loaded into the general-purpose processor 102 may also comprise code for designing a second classifier to receive the output of the first classifier to decide whether the input data belongs to the first classes or at least one second class.” MAJUMDAR [0038] teaches: "Furthermore, a second classifier may be specified to categorize a sample as known or unknown based on the classification scores." MAJUMDAR [0069] teaches: "Still, in some cases, such as during the test of a network, a sample (i.e., third input data samples) may not belong to any of the classes (i.e., correspond to a new distribution not already learnt) that were used to train a multi-class classifier (i.e., first artificial neural network). Thus, it may be desirable to generate a framework to initialize the multi-class classifier to label an object as belonging to a known class or belonging to an unknown class. The multi-class classifier may be referred to as the classifier." MAJUMDAR [0071] teaches: "Additionally, in some cases there may only be one class for a classifier. For example, the classifier may be specified to determine whether an object is a dog. In this example, when an image is presented to the classifier, the object is classified either as a dog or not a dog. Still, it may be desirable to create a framework to initialize a classifier to recognize an input as belonging to the one known class or to assign an unknown label to the input.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of HASSAN, NAKATA, AGUIRRE, RAGHAVAN, ANS, and MAJUMDAR before them, to include MAJUMDAR’s processor for detecting unknown classes using classifier to label an object as belonging to a known or unknown class in HASSAN, NAKATA, AGUIRRE, RAGHAVAN, and ANS’ detection method. One would have been motivated to make such a combination so that when a machine learning model receives a sample during a test phase, we can detect if the sample belongs to a known or unknown class (MAJUMDAR [0069]).
However, HASSAN in view of NAKATA, AGUIRRE, RAGHAVAN, ANS, and MAJUMDAR is not relied upon for teaching, but DERR teaches: if the one or more third input data samples correspond to the new distribution, to create a new system for learning the one or more third input data samples, the new system comprising at least a further first artificial neural network. (DERR [0186] teaches: "At block 1306, known signals are classified and unknown signals detected using RF measurements 1304 from RF measurement source 1302. Results 1308 including known signal classes and unknown signal classes are reported to users 1310. At block 1312, unknown signal characterization is performed on the detected unknown signals from block 1306 using incremental learning techniques to train new classifiers to classify unknown signals." Examiner's note: Paragraph [0136] of the specification states: "Furthermore, after training and during the inference phase of the ANN, it may be desirable to permit incremental learning, such that new classes of data samples can continue to be learn and handled by the system." DERR triggers the incremental learning for training new classifiers to classify unknown signals. A person having ordinary skill in the art would recognize that incremental learning is conceptually equivalent to ANS' sequential learning.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of HASSAN, NAKATA, AGUIRRE, RAGHAVAN, ANS, MAJUMDAR, and DERR before them, to include DERR’s training of new classifiers when unknown signals (i.e., input data) is received in HASSAN, NAKATA, AGUIRRE, RAGHAVAN, ANS, and MAJUMDAR’s detection method. One would have been motivated to make such a combination in order to incrementally build learned models and deploying classifiers for improving the accuracy of a deployed monitoring system over time (DERR [0183]).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over THALER (US 20070011119 A1), hereafter THALER, in view of HASSAN and ANS.
Regarding Claim 20:
THALER teaches:
A method comprising: training an artificial neural network by: - initially training a first artificial neural network with […] first pseudo data, wherein the first pseudo data was generated by the first artificial neural network while in a virgin state; (THALER [0006] teaches: “One aspect of the invention generally pertains to a neural-network based system capable of bootstrapping itself to progressively higher levels of adaptability and creativity upon the introduction of the system to a specific application domain wherein it engages in successive cycles of experimentation and learning from the results of such experimentation.” THALER [0044] teaches: “For example, if the perturbed network illustrated in FIG. 1 is generating servo settings (i.e., wherein the first pseudo data was generated by the first artificial neural network) for a hexapod robot, there the perturbed net is called the "Leg Motion Imagination Engine" (i.e., first artificial neural network), which begins in an initialized state characterized by random connection weights (i.e., while in a virgin state). The Imagination Engine and critic network launch into an experimental cycle in which noise is injected into the Imagination Engine so as to create a pattern of servo settings for a virtual robot, see FIG. 2. If forward motion is detected by onboard sensors (i.e., sonar), which are virtual in this case, the servo pattern is learned by the Imagination Engine (i.e., initially training a first artificial neural network) that takes the form of an auto-associative network.”)
- generating second pseudo data using the first artificial neural network; (THALER [0061] teaches: “In this application (illustrated in FIGS. 11-14), a self-learning, multilayer, auto-associative neural network is used to selectively capture those pulse width modulation sequences of a 12 degree of freedom hexapod robot that lead to forward motion. […] Having discovered servo sequences that lead to forward behavior, the weights of this network may be degraded via perturbations to generate important alternative behaviors such as backward, turning, and sidle motions (i.e., generating second pseudo data using).” Examiner’s note: The auto-associative network in THALER [0061] refers to the “Leg Motion Imagination Engine” (i.e., the first artificial neural network), which learns the servo patterns when forward motion is detected. This same network can generate alternative motions for the auto-associative network to learn.)
using the trained first or further artificial neural network in a hardware system to control one or more actuators. (THALER [0065] teaches: “Once trained in this way, we may later operate the robot in this newly invented locomotive mode.”)
THALER is not relied upon for teaching:
- initially training a first artificial neural network with first input data […]
- training the first artificial neural network, or another artificial neural network, with the second pseudo data and second input data; and
However, HASSAN teaches: initially training a first artificial neural network with first input data […] (HASSAN [0031] teaches: "At block 320, a pre-trained deep neural network model is trained (i.e., training) using the plurality of real roadway scene images (i.e., with first input data) and the plurality of augmented roadway scene images to generate a newly-trained deep neural network model. For example, the deep neural network model 116 stored in the memory 106 is trained to detect traffic lights in image data using the plurality of real roadway scene images and the plurality of augmented roadway scene images. In some implementations, the pre-trained deep neural network model is an untrained deep neural network model (i.e., a first artificial neural network).”)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of THALER and HASSAN before them, to include HASSAN’s real roadway scene images for training a neural network in THALER’s autonomous bootstrapping system. One would have been motivated to make such a combination because deep neural networks require large amounts of labeled training data to avoid overfitting (HASSAN [0003]).
THALER in view of HASSAN is not relied upon for teaching, but ANS teaches: - training the first artificial neural network, or another artificial neural network, with the second pseudo data and second input data; and (ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "After a fixed number of reinjections (which is a simulation parameter, denoted R) within the NET 1 auto-associative part, the current generated input and output activities are, respectively, transmitted to the NET 2 input and output layers for training (i.e., training [...] another artificial neural network). [...] A new population of external items (i.e., second input data) must be trained concurrently with pseudoitems originating continuously (i.e., second pseudo data) from NET 2, these pseudoitems being generated exactly as in NET 1 during stage (I). Subsequently, stages (I) and (II) are supposed to work alternatively for each new occurring population of ‘actual’ items.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of THALER, HASSAN, and ANS before them, to include ANS’ reverberating architecture with self-refreshing memory in THALER and HASSAN’s autonomous bootstrapping system. One would have been motivated to make such a combination in order to overcome catastrophic forgetting in sequential learning tasks (ANS [pg. 4, section 2.1 A dual-network architecture]).
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over THALER in view of RAGHAVAN, HASSAN, and ANS.
Regarding Claim 21:
THALER teaches:
A system comprising: a first artificial neural network; a memory storing the first pseudo data generated by the first artificial neural network while in a virgin state; (THALER [0006] teaches: “One aspect of the invention generally pertains to a neural-network based system (i.e., a system) capable of bootstrapping itself to progressively higher levels of adaptability and creativity upon the introduction of the system to a specific application domain wherein it engages in successive cycles of experimentation and learning from the results of such experimentation.” THALER [0044] teaches: “For example, if the perturbed network illustrated in FIG. 1 is generating servo settings (i.e., the first pseudo data generated by the first artificial neural network) for a hexapod robot, there the perturbed net is called the "Leg Motion Imagination Engine" (i.e., first artificial neural network), which begins in an initialized state characterized by random connection weights (i.e., while in a virgin state). The Imagination Engine and critic network launch into an experimental cycle in which noise is injected into the Imagination Engine so as to create a pattern of servo settings for a virtual robot, see FIG. 2. If forward motion is detected by onboard sensors (i.e., sonar), which are virtual in this case, the servo pattern is learned by the Imagination Engine (i.e., a first artificial neural network) that takes the form of an auto-associative network.” THALER [0089] teaches: “It will be obvious to those skilled the art that the present invention can be used for many purposes and applications other than the non-limiting examples described herein. Furthermore, those of skill in the art will recognize that the systems described herein may be implemented through software simulated neural networks or hardware-based network structures.” Examiner’s note: One skilled in the art would recognize that for the self-learning auto-associative network to learn the servo patterns, they must be stored before they can be used for learning. Under BRI, a memory storing the first pseudo data can be interpreted as the memory component of the hardware used to implement THALER’s autonomous bootstrapping system.)
one or more actuators; and (THALER [0065] teaches: “Once trained in this way, we may later operate the robot in this newly invented locomotive mode.”)
- initially training the first artificial neural network […] with the first pseudo data; (THALER [0044] teaches: “If forward motion is detected by onboard sensors (i.e., sonar), which are virtual in this case, the servo pattern (i.e., with the first pseudo data) is learned by the Imagination Engine (i.e., initially training the first artificial neural network) that takes the form of an auto-associative network.”)
- generating second pseudo data using the first artificial neural network; (THALER [0061] teaches: “In this application (illustrated in FIGS. 11-14), a self-learning, multilayer, auto-associative neural network is used to selectively capture those pulse width modulation sequences of a 12 degree of freedom hexapod robot that lead to forward motion. […] Having discovered servo sequences that lead to forward behavior, the weights of this network may be degraded via perturbations to generate important alternative behaviors such as backward, turning, and sidle motions (i.e., generating second pseudo data using).” Examiner’s note: The auto-associative network in THALER [0061] refers to the “Leg Motion Imagination Engine” (i.e., the first artificial neural network), which learns the servo patterns when forward motion is detected. This same network can generate alternative motions for the auto-associative network to learn.)
controlling the one or more actuators using the trained first or further artificial neural network. (THALER [0065] teaches: “Once trained in this way, we may later operate the robot in this newly invented locomotive mode.”)
THALER is not relied upon for teaching:
one or more circuits or processors configured to generate old memories for use in training the first artificial neural network by:
- initially training the first artificial neural network […]
- training the first artificial neural network, or another artificial neural network, with the second pseudo data and second input data; and
However, RAGHAVAN teaches: one or more circuits or processors configured to generate old memories for use in training the first artificial neural network by: (RAGHAVAN [0009] teaches: “Furthermore, the techniques disclosed herein may enable a machine learning system to perform scalable, lifelong learning of solutions for new tasks the machine learning system has not previously been trained to solve, while reducing the occurrence of catastrophic forgetting (e.g., forgetting solutions to old tasks as a result of learning solutions to new tasks).” RAGHAVAN [0010] teaches: “[…] generative memory is capable of generating the auxiliary data from old tasks for use in training the machine learning model to obtain labels for new tasks for which the machine learning model has not previously been trained.” RAGHAVAN [0043] teaches: “Computation engine 230 includes machine learning system 102, observational module 118 and generative memory 104. Each of machine learning system 102, observational module 118, and generative memory 104 may represent software executable by processing circuitry 206 and stored on storage device 208, or a combination of hardware and software. Such processing circuitry 206 may include any one or more of a microprocessor, a controller, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or equivalent discrete or integrated logic circuitry.”)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of THALER and RAGHAVAN before them, to include RAGHAVAN’s generative memory in THALER’s autonomous bootstrapping system. One would have been motivated to make such a combination in order to obtain labels for new tasks for which the machine learning model has not previously learned to reduce the occurrence of catastrophic forgetting (RAGHAVAN [0009]-[0010]).
THALER in view of RAGHAVAN is not relied upon for teaching:
- initially training the first artificial neural network […]
- training the first artificial neural network, or another artificial neural network, with the second pseudo data and second input data; and
However, HASSAN teaches: - initially training the first artificial neural network […] (HASSAN [0031] teaches: "At block 320, a pre-trained deep neural network model is trained (i.e., training) using the plurality of real roadway scene images (i.e., with first input data) and the plurality of augmented roadway scene images to generate a newly-trained deep neural network model. For example, the deep neural network model 116 stored in the memory 106 is trained to detect traffic lights in image data using the plurality of real roadway scene images and the plurality of augmented roadway scene images. In some implementations, the pre-trained deep neural network model is an untrained deep neural network model (i.e., a first artificial neural network).”)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of THALER, RAGHAVAN, and HASSAN before them, to include HASSAN’s real roadway scene images for training a neural network in THALER and RAGHAVAN’s autonomous bootstrapping system. One would have been motivated to make such a combination in order to avoid overfitting by requiring large amounts of labeled training data for training deep neural networks (HASSAN [0003]).
THALER in view of RAGHAVAN and HASSAN is not relied upon for teaching, but ANS teaches: - training the first artificial neural network, or another artificial neural network, with the second pseudo data and second input data; and (ANS [pg. 4, section 2.1 A dual-network architecture] teaches: "After a fixed number of reinjections (which is a simulation parameter, denoted R) within the NET 1 auto-associative part, the current generated input and output activities are, respectively, transmitted to the NET 2 input and output layers for training (i.e., training [...] another artificial neural network). [...] A new population of external items (i.e., second input data) must be trained concurrently with pseudoitems originating continuously (i.e., second pseudo data) from NET 2, these pseudoitems being generated exactly as in NET 1 during stage (I). Subsequently, stages (I) and (II) are supposed to work alternatively for each new occurring population of ‘actual’ items.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of THALER, RAGHAVAN, HASSAN, and ANS before them, to include ANS’ reverberating architecture with self-refreshing memory in THALER, RAGHAVAN, and HASSAN’s autonomous bootstrapping system. One would have been motivated to make such a combination in order to overcome catastrophic forgetting in sequential learning tasks (ANS [pg. 4, section 2.1 A dual-network architecture]).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Alvaro S Laham Bauzo whose telephone number is (571) 272-5650. The examiner can normally be reached Mon-Fri 7:30 AM - 11:00 AM | 1:00 PM - 5:30 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed, can be reached on (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.S.L./Examiner, Art Unit 2146
/USMAAN SAEED/Supervisory Patent Examiner, Art Unit 2146
Prosecution Insights