DETAILED ACTION
This final action is in response to the amendment and remarks filed on 07/01/2025 for application 17/555,577.
Claims 1-8 and 10-24 have been amended. Claims 9 and 25 are cancelled. Claims 26-27 are newly added claims. Claims 1-8, 10-24, and 26-27 are pending in the case. Claims 1 and 22 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) filed 06/25/2025 and 10/06/2025 have been fully considered by the examiner.
Response to Amendment
The amendment filed 07/01/2025 has been entered.
Applicant’s amendment to the claims with respect to resolving claim objections has been considered, and overcomes the objections set forth in the office action mailed 04/14/2025. Consequently, the previous objections have been withdrawn; applicant is directed towards newly raised objections to amended claims 1, 13, 19, and 22 set forth below.
Applicant’s amendment to the claims, particularly the claim limitations that were previously treated in accordance with 35 U.S.C. 112(f) in the office action mailed 04/14/2025, results in them no longer invoking an interpretation under 35 U.S.C. 112(f). Consequently, the interpretations have been withdrawn.
Applicant’s amendment to the claims with respect to resolving indefiniteness rejections under 35 U.S.C. 112(b) has been considered, and overcomes the 112(b) rejections set forth in the office action mailed 04/07/2025. Consequently, the rejections have been withdrawn.
Applicant’s amendment to the claims with respect to resolving non-statutory subject matter rejections and non-eligible subject matter rejections under 35 U.S.C. 101 has been considered, and overcomes the 101 rejections set forth in the office action mailed 04/07/2025. Consequently, the rejections have been withdrawn.
Claim Objections
Claims 1, 13, 19 and 22 are objected to because of the following informalities:
In claims 1 and 22, “the hardware component is selected from the group” should read “the hardware component is selected from a group” to establish proper antecedent basis.
In claim 1, “the learning component is trained with the maximum likelihood learning algorithm” should read “the learning component is trained with a maximum likelihood learning algorithm” to establish proper antecedent basis.
In claim 22, “training the learning component…with the maximum likelihood learning algorithm” should read “training the learning component…with a maximum likelihood learning algorithm” to establish proper antecedent basis.
In claim 13, “wherein each first neuron is connected to a corresponding parrot neuron, wherein the corresponding parrot neurons are connected to the output neurons, and wherein the corresponding parrot neurons are connected to an inhibiting neuron” should read “wherein each first neuron is connected to a corresponding parrot neuron, thereby forming a plurality of corresponding parrot neurons, wherein the plurality of corresponding parrot neurons are connected to the output neurons, and wherein the plurality of corresponding parrot neurons are connected to an inhibiting neuron” or be likewise amended to establish proper antecedent basis for a plurality of “corresponding parrot neurons”.
In claim 19, “with at least source configured for providing raw data” should read “with at least one source configured for providing raw data” to correct an apparent typographical error.
Appropriate corrections are required.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-8, 10-17, 19, and 27 of the instant application are provisionally rejected on the grounds of nonstatutory double patenting as being unpatentable over claims 10-24 of co-pending application 17/563,480 in view of Jiang et al., (“Encoding Temporal Information for Time-Aware Link Prediction”, published 2016), hereinafter Jiang.
Claims 18 and 20 of the instant application are provisionally rejected on the grounds of nonstatutory double patenting as being unpatentable over claim 10 of co-pending application 17/563,480 in view of Jiang, further in view of Kammerer et al., (“Process-Driven and Flow-Based Processing of Industrial Sensor Data”, published 2020), hereinafter Kammerer.
Claims 22, 24, and 26 of the instant application are provisionally rejected on the grounds of nonstatutory double patenting as being unpatentable over claim 25 of co-pending application 17/563,480, further in view of Pecevski and Maass (“Learning Probabilistic Inference through Spike-Timing Dependent Plasticity”, published 25 Mar 2016), hereinafter Pecevski-Maass.
Claim 23 of the instant application is provisionally rejected on the grounds of nonstatutory double patenting as being unpatentable over claim 25 of co-pending application 17/563,480 in view of Pecevski-Maass, further in view of Kammerer (“Process-Driven and Flow-Based Processing of Industrial Sensor Data”, published 2020).
Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of the instant application are largely covered in scope by the claims of the co-pending application, such that a person of ordinary skill in the art would deem the claims of the instant application to be an obvious variation.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
The table below shows similarities and differences between the instant application and the co-pending application; the left side contains claims 1-8, 10-20, 22-24, and 26-27 of the instant application, while the right side contains portions of claims 1 and 10-25 of the co-pending application 17/563,480. Differences that amount to more than minor variations in language and/or punctuation are highlighted and further discussed below.
Instant Application (17/555,577)
Co-pending Application (17/563,480)
Claim 1. A neuromorphic hardware for processing a knowledge graph
comprising observed triple statements,
wherein the neuromorphic hardware comprises a hardware component, a learning component, and a control component,
wherein each triple statement in the knowledge graph is structured according to s – p – o, wherein s and o are different entities and p is a relation between s and o and is directed from s to o,
wherein the hardware component is selected from the group consisting of an application specific integrated circuit, a field-programmable gate array, a wafer-scale integration, a hardware with mixed-mode VLSI neurons, a neural processing unit, or a mixed-signal neuromorphic processor,
wherein the learning component comprises an input layer and an output layer,
wherein the input layer comprises a plurality of node embedding populations, wherein each node embedding population comprises a plurality of neurons and represents an entity contained in the observed triple statements,
wherein the output layer comprises output neurons representing a likelihood for each triple statement
generated in a sampling mode of the learning component, and
wherein the control component is configured for sequentially switching the learning component: (i) into a data-driven learning mode in which the learning component is trained with the maximum likelihood learning algorithm minimizing energy in a probabilistic, sampling-based model, using only the observed triple statements which are assigned low energy values, wherein the probabilistic, sampling-based model is derived from an energy function, and wherein the observed triple statements have minimal energy,
(ii) into the sampling mode, and (iii) into a model-driven learning mode configured for training the component with the maximum likelihood learning algorithm using only the generated triple statements, with the learning component learning to assign high energy values to the generated triple statements.
(Claim 2, depending from claim 1)
wherein the control component is configured to:
present inputs to the learning component by selectively activating subject and object populations among the node embedding populations,
set hyperparameters of the learning component, in particular a factor (η) that modulates learning updates of the learning component,
read output of the learning component, and
use output of the learning component as feedback to the learning component.
(Claim 3, depending from claim 1)
wherein the output layer has one output neuron for each relation type of the knowledge graph.
(Claim 4, depending from claim 3)
wherein the output neurons are stochastic dendritic output neurons, storing embeddings of relations that are given between a subject and an object in the observed triple statements in each dendrite of a plurality of dendrites of each output neuron, summing all dendrites of each output neuron into a final score, wherein the final score of each output neuron is transformed into a probability using an activation function.
(Claim 5 , depending from claim 4)
wherein an output of the activation function is a prediction of the likelihood of a triple statement or a transition probability.
(Claim 6, depending from claim 4)
wherein learning updates for the embeddings of relations are computed directly in the dendrites of the stochastic, dendritic output neurons.
(Claim 7, depending from claim 4)
wherein learning updates for one node embedding population of the plurality of node embedding populations are computed using static feedback connections from each dendrite of each output neuron to respective neurons of the one node embedding population.
(Claim 8, depending from claim 1)
wherein in the sampling mode, by sampling from an activation function, a binary output signals to the control component whether a triple statement is accepted.
(Claim 10, depending from claim 1)
wherein the plurality of node embedding populations comprise a first node embedding population and a second node embedding population,
wherein the first node embedding population comprises first neurons representing a first entity contained in the observed triple statements, and wherein the first node embedding population is characterized by first spike times of the first neurons during a recurring time interval,
wherein the second node embedding population comprises second neurons representing a second entity contained in the observed triple statements, and wherein the second node embedding population is characterized by second spike times of the second neurons during the recurring time interval, and
wherein a relation between the first entity and the second entity is represented as the differences between the first spike times and the second spike times.
(Claim 11, depending from claim 10)
wherein the differences between the first spike times and the second spike times consider an order of the first spike times in relation to the second spike times, or wherein the differences are absolute values.
(Claim 12, depending from claim 10)
wherein the relation is stored in one of the output neurons, and wherein the relation is given by vector components that are stored in dendrites of the output neurons.
(Claim 13, depending from claim 10)
wherein the first neurons are connected to a monitoring neuron,
wherein each first neuron is connected to a corresponding parrot neuron,
wherein the corresponding parrot neurons are connected to the output neurons, and wherein the corresponding parrot neurons are connected to an inhibiting neuron.
(Claim 14, depending from claim 10)
wherein the first neurons and the second neurons are spiking neurons that are non-leaky integrate-and-fire neurons or current-based leaky integrate-and-fire neurons.
(Claim 15, depending from claim 10)
wherein each of the first neurons and each of the second neurons spikes only once during the recurring time interval, or wherein only a first spike during the recurring time interval is counted.
(Claim 16, depending from claim 1)
wherein each node embedding population is connected to an inhibiting neuron and is selectable by inhibition of the inhibiting neuron.
(Claim 17, depending from claim 1)
An industrial device, comprising the neuromorphic hardware
(Claim 18, depending from claim 17)
wherein the industrial device is a field device, an edge device, a sensor device, an industrial controller, a PLC controller, an industrial PC implementing a SCADA system, a network hub, an industrial ethernet switch, or an industrial gateway connecting an automation system to cloud computing resources.
(Claim 19, depending from claim 17)
with at least source configured for providing raw data, with an ETL component, configured for converting the raw data into the observed triple statements, using mapping rules, with a triple store, storing the observed triple statements, and wherein the learning component is configured for performing an inference in an inference mode, wherein the at least one source is at least one sensor, at least one data source, or both the at least one sensor and the at least one data source.
(Claim 20, depending from claim 19)
with a statement handler, configured for triggering an automated action based on inference of the learning component.
Claim 22. A method for training a learning component to learn inference on a knowledge graph
comprising observed triple statements, said method comprising:
switching, by a control component, the learning component into a data-driven learning mode,
wherein a neuromorphic hardware for processing the knowledge graph comprises a hardware component, the learning component, and the control component,
wherein each triple statement in the knowledge graph is structured according to s - p - o, wherein s and o different entities and p is a relation between s and o and is directed from s to o,
wherein the hardware component is selected from the group consisting of an application specific integrated circuit, a field-programmable gate array, a wafer-scale integration, a hardware with mixed-mode VLSI neurons, a neural processing unit, or a mixed-signal neuromorphic processor,
wherein the learning component comprises an input layer and an output layer,
wherein the input layer comprises a plurality of node embedding populations, wherein each node embedding population comprises a plurality of neurons and represents an entity contained in the observed triple statements,
wherein the output layer comprises output neurons representing a likelihood for each triple statement generated in a sampling mode of the learning component;
training the learning component, which is in the data-driven learning mode, with the maximum likelihood learning algorithm minimizing energy in the probabilistic, sampling-based model, using only the observed triple statements which are assigned low energy values, wherein the probabilistic, sampling-based model is derived from an energy function, and wherein the observed triple statements have minimal energy;
switching, by the control component, the learning component from the data-driven learning mode into the sampling mode;
generating, with the learning component being in the sampling mode, triple statements;
switching, by the control component, the learning component from the sampling mode into a model-driven learning mode; and
training the learning component, which is in the model-driven learning mode, with the maximum likelihood learning algorithm using only the generated triple statements, with the learning component learning to assign high energy values to the generated triple statements.
(Claim 23, depending from claim 22)
wherein the knowledge graph is an industrial knowledge graph describing parts of an industrial system,
with nodes of the knowledge graph representing physical objects, wherein the physical objects include sensors, industrial controllers, robots, drives, manufactured objects, and either tools or both the tools and elements of a bill of materials, and
with nodes of the knowledge graph representing abstract entities including sensor measurements, sensor attributes, configurations of the physical objects or skills of the physical objects, production schedules and plans.
(Claim 24, depending from claim 22)
A non-transitory computer-readable storage media having instructions stored thereon, said instructions executable by one or more processors of a computer system, wherein execution of the instructions causes the one or more processors to perform the method
(Claim 26, depending from claim 22)
wherein the data-driven learning mode includes a positive learning phase,
wherein the sampling mode includes a free-running phase, and
wherein the model-driven learning mode includes a negative learning phase.
(Claim 27, depending from claim 1)
wherein the data-driven learning mode includes a positive learning phase, wherein the sampling mode includes a free-running phase, and wherein the model-driven learning mode includes a negative learning phase.
Claim 1. An industrial device for building or for both building and processing a knowledge graph comprising:
at least one sensor and/or at least one data source that provides raw data;
an ETL component that converts the raw data into triple statements, using mapping rules;
a triple store, storing the triple statements as a dynamically changing knowledge graph;
a learning component that processes the triple statements in a learning mode and performs an inference in an inference mode; and
a control component that sequentially switches the learning component: (i) into a data-driven learning mode in which the learning component is trained using observed triple statements, (ii) into a sampling mode of the learning component, wherein triple statements are generated in the sampling mode, and (iii) into a model-driven learning mode that trains the learning component using the generated triple statements.
wherein the learning component or both the learning component and the control component are implemented as: (i) neuromorphic hardware comprising an application specific integrated circuit, a field-programmable gate array, or a wafer-scale integration, or (ii) a hardware with mixed-mode VLSI neurons, a neuromorphic processor, a neural processing unit, or a mixed-signal neuromorphic processor.
(repeated from claim 1)
a learning component; and
a control component
wherein the learning component or both the learning component and the control component are implemented as: (i) neuromorphic hardware
(Examiner Note: As per a broadest reasonable interpretation, triple statements are inherently “structured according to s – p – o, wherein s and o are different entities and p is a relation between s and o and is directed from s to o")
(repeated from claim 1)
neuromorphic hardware comprising an application specific integrated circuit, a field-programmable gate array, or a wafer-scale integration, or (ii) a hardware with mixed-mode VLSI neurons, a neuromorphic processor, a neural processing unit, or a mixed-signal neuromorphic processor.
(Claim 10, depending from claim 1)
wherein the learning component includes:
an input layer comprising a plurality of node embedding populations of neurons, wherein each node embedding population comprises a plurality of neurons and represents an entity contained in the triple statements; and
an output layer, containing output neurons that represent a likelihood for each possible triple statement.
(repeated from claim 1)
a sampling mode, wherein triple statements are generated in the sampling mode
(repeated from claim 1)
a control component that sequentially switches the learning component: (i) into a data-driven learning mode in which the learning component is trained using observed triple statements,
(ii) into a sampling mode of the learning component, wherein triple statements are generated in the sampling mode, and (iii) into a model-driven learning mode that trains the learning component using the generated triple statements.
(Claim 11, depending from claim 10)
wherein the control component is configured to
present inputs to the learning component by selectively activating subject and object populations among the node embedding populations;
set hyperparameters of the learning component, in particular a factor (η) that modulates learning updates of the learning component;
read output of the learning component; and
use output of the learning component as feedback to the learning component.
(Claim 12, depending from claim 10)
wherein the output layer has one output neuron for each relation type of the knowledge graph.
(Claim 13, depending from claim 12)
wherein the output neurons are stochastic dendritic output neurons, storing embeddings of relations that are given between a subject and an object in the observed triple statements in each dendrite of a plurality of dendrites of each output neuron, summing all dendrites of each output neuron into a final score, wherein the final score of each output neuron is transformed into a probability using an activation function.
(Claim 14, depending from claim 13)
wherein depending on a current mode of the learning component, an output of the activation function is a prediction of the likelihood of a triple statement or a transition probability, wherein the current mode is the data-driven learning mode, the sampling mode, or the model-driven learning mode.
(Claim 15, depending from claim 13)
wherein learning updates for the embeddings of relations are computed directly in the dendrites of the stochastic, dendritic output neurons.
(Claim 16, depending from claim 13)
wherein learning updates for one node embedding population of the plurality of node embedding populations are computed using static feedback connections from each dendrite of each output neuron to respective neurons of the one node embedding population.
(Claim 17, depending from claim 10)
wherein in the sampling mode, by sampling from an activation function, a binary output signals to the control component whether a triple statement is accepted.
(Claim 18, depending from claim 10)
wherein the plurality of node embedding population comprises a first node embedding population and a second node embedding population,
wherein the first node embedding population comprises first neurons and represents a first node in the knowledge graph, wherein the first node is characterized by sequentially ordered first spike times of the first neurons during a recurring time interval, wherein the second node embedding population comprises second neurons and represents a second node in the knowledge graph, wherein the second node is characterized by sequentially ordered second spike times of the second neurons during the recurring time interval, and
wherein a relation between the first node and the second node is represented as the differences between the first spike times and the second spike times.
(Claim 19, depending from claim 18)
wherein the differences between the first spike times and the second spike times consider an order of the first spike times in relation to the second spike times, or
wherein the differences are absolute values.
(Claim 20, depending from claim 18)
wherein the relation is stored in one of the output neurons; and
wherein the relation is given by vector components that are stored in dendrites of the output neuron.
(Claim 21, depending from claim 18)
wherein the first neurons are connected to a monitoring neuron;
wherein each first neuron is connected to a corresponding parrot neuron;
wherein the corresponding parrot neurons are connected to the output neurons; and
wherein the corresponding parrot neurons are connected to an inhibiting neuron.
(Claim 22, depending from claim 18)
wherein the first neurons and the second neurons are spiking neurons, non-leaky integrate-and-fire neurons or current-based leaky integrate- and-fire neurons.
(Claim 23, depending from claim 18)
wherein each of the first neurons and each of the second neurons only spikes once during the recurring time interval, or
wherein only a first spike during the recurring time interval is counted.
(Claim 24, depending from claim 10)
wherein each node embedding population is connected to an inhibiting neuron, and therefore selectable by inhibition of the inhibiting neuron.
(repeated from claim 1)
An industrial device comprising:
a learning component; and
a control component
wherein the learning component or both the learning component and the control component are implemented as: (i) neuromorphic hardware
(claim 10, as above)
(repeated from claim 1)
at least one sensor and/or at least one data source that provides raw data;
an ETL component that converts the raw data into triple statements, using mapping rules;
a triple store, storing the triple statements as a dynamically changing knowledge graph;
a learning component that processes the triple statements in a learning mode and performs an inference in an inference mode; and
(claim 10, as above)
Claim 25. A method for building or for both building and processing a knowledge graph by an industrial device, the method comprising:
providing, by at least one sensor, at least one data source raw data;
converting, by an ETL component, the raw data into triple statements, using mapping rules;
storing, by a triple store, the triple statements as a dynamically changing knowledge graph;
processing, by a learning component, the triple statements in a learning mode;
providing a control component that sequentially switches the learning component: (i) into a data-driven learning mode in which the learning component is trained with a maximum likelihood learning algorithm minimizing energy in a probabilistic, sampling-based model, using only observed triple statements which are assigned low energy values, wherein the probabilistic, sampling-based model is derived from an energy function, and wherein the observed triple statements have minimal energy, (ii) into a sampling mode of the learning component, wherein triple statements are generated in the sampling mode, and (iii) into a model-driven learning mode that trains the learning component with the maximum likelihood learning algorithm using only the generated triple statements, with the learning component learning to assign high energy values to the generated triple statements;
switching, by the control component, operation of the learning component from the learning mode to an inference mode; and
performing, by the control component, an inference in the inference mode,
wherein the learning component or both the learning component and the control component are implemented as: (i) neuromorphic hardware comprising an application specific integrated circuit, a field-programmable gate array, or a wafer-scale integration, or (ii) a hardware with mixed-mode VLSI neurons, a neuromorphic processor, a neural processing unit, or a mixed-signal neuromorphic processor.
(repeated from claim 25)
wherein the learning component or both the learning component and the control component are implemented as: (i) neuromorphic hardware
(Examiner Note: as per a broadest reasonable interpretation, triple statements are inherently “structured according to s – p – o, wherein s and o are different entities and p is a relation between s and o and is directed from s to o")
(repeated from claim 25)
(i) neuromorphic hardware comprising an application specific integrated circuit, a field-programmable gate array, or a wafer-scale integration, or (ii) a hardware with mixed-mode VLSI neurons, a neuromorphic processor, a neural processing unit, or a mixed-signal neuromorphic processor
(repeated from claim 25)
a sampling mode of the learning component, wherein triple statements are generated in the sampling mode,
(repeated from claim 25)
a data-driven learning mode in which the learning component is trained with a maximum likelihood learning algorithm minimizing energy in a probabilistic, sampling-based model, using only observed triple statements which are assigned low energy values, wherein the probabilistic, sampling-based model is derived from an energy function, and wherein the observed triple statements have minimal energy
(repeated from claim 25)
providing a control component that sequentially switches the learning component: (ii) into a sampling mode of the learning component, wherein triple statements are generated in the sampling mode,
(iii) into a model-driven learning mode that trains the learning component with the maximum likelihood learning algorithm using only the generated triple statements, with the learning component learning to assign high energy values to the generated triple statements;
(repeated from claim 25)
method for building or for both building and processing a knowledge graph by an industrial device,
(claim 25, as above)
(Examiner Note: Merely implementing the claimed method via a generic computer system is an obvious variation)
(repeated from claim 25)
(i) into a data-driven learning mode in which the learning component is trained with a maximum likelihood learning algorithm minimizing energy in a probabilistic, sampling-based model, using only observed triple statements which are assigned low energy values, wherein the probabilistic, sampling-based model is derived from an energy function, and wherein the observed triple statements have minimal energy,
(Examiner Note: As per a broadest reasonable interpretation, the functions of the claimed data-driven learning mode recite a positive learning phase, as energy-based models inherently train (i.e., learning phase) to assign low energy values to high probability (i.e., positive) configurations))
(ii) into a sampling mode of the learning component, wherein triple statements are generated in the sampling mode, and
(Examiner Note: As per a broadest reasonable interpretation, the functions of the claimed sampling mode inherently recite a free-running phase, i.e., generating new data points based on the learned distribution)
(iii) into a model-driven learning mode that trains the learning component with the maximum likelihood learning algorithm using only the generated triple statements, with the learning component learning to assign high energy values to the generated triple statements;
(Examiner Note: As per a broadest reasonable interpretation, the functions of the model-driven learning mode recite a negative learning phase, as energy-based models inherently train (i.e., learning phase) to assign high energy values to low probability (i.e., negative) configurations)
(claim 10, as above)
Examiner Note: As per a broadest reasonable interpretation, the functions of the claimed sampling mode inherently recite a free-running phase, i.e., generating new data points based on the learned distribution)
Claims 10-24 of the co-pending application do not explicitly teach a data-driven learning mode in which the learning component is trained with the maximum likelihood learning algorithm minimizing energy in a probabilistic, sampling-based model, using only the observed triple statements which are assigned low energy values, wherein the probabilistic, sampling-based model is derived from an energy function, and wherein the observed triple statements have minimal energy, or a model-driven learning mode configured for training the component with the maximum likelihood learning algorithm using only the generated triple statements, with the learning component learning to assign high energy values to the generated triple statements, or wherein the data-driven learning mode includes a positive learning phase, wherein the sampling mode includes a free-running phase, and wherein the model-driven learning mode includes a negative learning phase, as recited in claims 1-8, 10-20, and 27 of the co-pending application.
In the same field of endeavor, Jiang teaches a method of performing inference on a knowledge graph comprising triple statements (“Knowledge bases (KBs) such as Freebase (Bollacker et al., 2008) and YAGO (Fabian et al., 2007) play a pivotal role in many NLP related applications. KBs consist of facts in the form of triplets (ei, r, ej), indicating that head entity ei and tail entity ej is linked by relation r. Although KBs are large, they are far from complete. Link prediction is to predict relations between entities based on existing triplets, which can alleviate the incompleteness of current KBs…This paper mainly focuses on incorporating the temporal order information and proposes a time-aware link prediction model” [Jiang Introduction page 1]) comprising a data-driven learning mode in which the learning component is trained with the maximum likelihood learning algorithm minimizing energy in a probabilistic, sampling-based model, using only the observed triple statements which are assigned low energy values, wherein the probabilistic, sampling-based model is derived from an energy function, and wherein the observed triple statements have minimal energy, (“Triple classification aims to judge whether an unseen triple is correct or not…To create labeled data for classification, for each triple in the test and validation sets, we construct a corresponding negative triple…During triple classification a triple is predicted as positive if the score is below a relation-specific threshold; otherwise as negative” [Jiang page 4 Triple Classification] ; “…we make use of the triple set Δ and follow the same strategy adopted in previous methods such as TransE [equation 2] For each candidate triple, it requires positive triples to have lower scores than negative triples” [Jiang Time-Aware KB Embedding page 2]; “The optimization is to minimize the joint score function, [equation 3] where x+ is the positive triple (quad), x- is corresponding the negative triple” [Jiang Time-Aware KB Embedding page 2]; Triples in test and validation sets are based on real (i.e., observed) data and are therefore positive, unlike negative triples which are constructed (i.e. generated); the predictive model is derived from a function [see equation 2] wherein positive triples have lower scores (i.e., minimal energies) compared to negative triples. Jiang further discloses minimizing a joint score (i.e., energy) function that includes f(x+) as a parameter, wherein f(x+) represents the energy of a positive (i.e., observed) triple. Energy of a triple is equivalent to calculating a dissimilarity measure within the triple statement via an L1 norm [see equation 2]; The joint score algorithm therefore minimizes the energy, or dissimilarity, of entities within a positive triple, which is equivalent to maximizing their likelihood (i.e., correctness) within a triple classification [Triple Classification page 4] task),
a model-driven learning mode configured for training the component with the maximum likelihood learning algorithm using only the generated triple statements, with the learning component learning to assign high energy values to the generated triple statements, (“The optimization is to minimize the joint score function, [equation 3] where x+ is the positive triple (quad), x- is corresponding the negative triple” [Jiang Time-Aware KB Embedding page 2]; Jiang discloses minimizing a joint score (i.e., energy) function that includes -f(x-) as a parameter, wherein f(x-) represents the energy of a negative (i.e., generated) triple. Energy of a triple is equivalent to calculating a dissimilarity measure within the triple statement via an L1 norm [see equation 2]; The joint score algorithm therefore minimizes the negative energy, functionally equivalent to maximizing the energy, of the negative triples, and assigns higher energy values to negative triples than positive triples), and
wherein the data-driven learning mode includes a positive learning phase, and wherein the model-driven learning mode includes a negative learning phase, ([Jiang Time-Aware KB Embedding page 2] as detailed above; The data-driven learning mode comprises training on the positive triples (i.e., positive learning), and the model-driven learning mode comprises training on the negative triples (i.e., negative learning)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Jiang into the co-pending application because they are both directed towards performing inference on a knowledge graph comprising triple statements. Incorporating the teachings of Jiang would improve the co-pending application by improving accuracy in modeling temporal dynamics (“To make more accurate predictions, it is non-trivial for existing KB embedding methods to incorporate temporal order information. This paper mainly focuses on incorporating the temporal order information and proposes a time-aware link prediction model" [Jiang Introduction page 1]; "In this paper, we propose a general time-aware KB embedding, which incorporates creation time of entities and imposes temporal order constraints on the geometric structure of the embedding space and enforce it to be temporally consistent and accurate" [Jiang Conclusion page 4]).
Claim 10 of the co-pending application does not explicitly teach wherein the industrial device is a field device, an edge device, a sensor device, an industrial controller, a PLC controller, an industrial PC implementing a SCADA system, a network hub, an industrial ethernet switch, or an industrial gateway connecting an automation system to cloud computing resources, or with a statement handler, configured for triggering an automated action based on inference of the learning component, as recited in claims 18 and 20 of the instant application.
In the same field of endeavor, Kammerer teaches a system of inference on a knowledge graph wherein the industrial device is a PLC controller, (“In industrial machines, sensors and actors are typically controlled by a programmable logic controller (see Figure 7a)” [Kammerer page 12]; [Kammerer Figure 7 Schematic overview of context-aware process execution framework on page 13]; "Execution contexts, in turn, can be mapped to a context graph, which is a direct acyclic graph and represents the logical structure of a cyber-physical system. Therefore, each node in a context graph has predefined context types and can be used as a basis for the concept called context-aware process family, which is introduced in the following" [Kammerer page 13]) with a statement handler, configured for triggering an automated action based on inference of the learning component ([Kammerer see Figure 7 on page 13; see Context-aware Process Execution]; "Context-aware process execution (CaPE) enables the management of context-aware processes. It supports the modeling of process variants at design time and the automated, controlled adaption of processes at runtime" [Kammerer page 13]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teaching of Kammerer into the co-pending application because they are both directed towards implementing an industrial device that processes data. By disclosing an incorporation of the PLC controller within a larger framework of collecting and processing sensor data ([Kammerer Figure 7 on page 13]; “Typically, a PLC is connected to other information systems, such as industrial PCs (IPCs), equipped with human–machine interfaces (HMI) to configure and to control the PLC execution” [Kammerer page 8]), incorporating the teachings of Kammerer would enable the co-pending application to effectively collect, process, and utilize massive amounts of data ("Besides the discussed sensor differences, sensor data are typically delivered from sensor subsystems (e.g., a programmable logic controller, PLC) and continuously streamed to subsequent processing components, which must then cope with massive amounts of data…To tackle the aforementioned challenges, a sensor processing pipeline (SPP) is proposed, which provides solutions for capturing, processing, storing, and visualizing raw sensor data in a continuous processing pipeline" [Kammerer pages 1-2]).
Claim 25 of the co-pending application does not explicitly teach wherein the learning component comprises an input layer and an output layer, wherein the input layer comprises a plurality of node embedding populations, wherein each node embedding population comprises a plurality of neurons and represents an entity contained in the observed triple statements, and wherein the output layer comprises output neurons representing a likelihood for each triple statement, as recited in claims 22, 24, and 26 of the instant application.
In the same field of endeavor, Pecevski-Maass teaches a system of performing probabilistic inference on triple statements of a Bayesian network using neuromorphic hardware (“The learning model that is presented in this article ties in to this second approach, and shows that stochastic networks of neurons are able to automatically absorb the relevant statistical information from examples that they receive. As a result, we have now one first complete theory for the emergence of probabilistic inference in networks of spiking neurons through learning… We first show how an extension of an ubiquitous network motif of cortical microcircuits, interconnected populations of pyramidal cells with lateral inhibition (Winner- Take-All (WTA) circuits; Douglas and Martin, 2004; Nessler et al., 2013), gives rise to the basic building block for absorbing probabilistic information from examples” [Pecevski-Maass page 2 Introduction]; “We show that the underlying distribution p* can be learnt (approximately) from examples for this visual perception task, and that the network N which learns this approximation learns simultaneously to deal with the explaining away effect as an emergent phenomenon. The structure of the neural network N suitable for learning this target probability distribution p* is given in Figure 6C. It consists of four interconnected learning modules, where the connections between the learning modules reflect the dependencies between the RVs in the Bayesian network in Figure 6B” [Pecevski-Maass page 12]; [see Figure 6 Description of the perceptual explaining away example on page 10]) wherein the learning component comprises an input layer and an output layer, (“However, although the architecture of N will obviously have to depend on the number of random variables of p*, we show that it suffices to assume that it consists of recursive interconnections of different copies of a simple generic network motif, to which we refer as a stochastic association module. This network motif is a three-layer feedforward network of excitatory spiking neurons with lateral inhibition on the hidden layer (see Fig. 2). We show that this simple microcircuit motif can be viewed as an atomic learning module, that extracts via STDP and intrinsic plasticity from examples probabilistic associations between input variables x and output variable z that are encoded through population coding on its input and output layer” [Pecevski-Maass page 3 Results]) wherein the input layer comprises a plurality of node embedding populations, wherein each node embedding population comprises a plurality of neurons and represents an entity contained in the observed triple statements, (see Figure 2 including input neurons layer comprising populations of neurons xi – “Figure 2. Structure of a stochastic association module that is able to learn probabilistic associations between multinomial variables x = (x1,…xl) and z through STDP. Populations of neurons xi (i = 1, . . . , I) on the first layer encode the values of input variables xi.” [Pecevski-Maass page 4]; Each input variable xi (which can be drawn from RVs comprising nodes of a Bayesian network, as explained above) is encoded by a respective population of neurons (i.e., node population)) wherein the output layer comprises output neurons representing a likelihood for each triple statement (see Figure 2 including output neurons layer– “The population of neurons on the third layer encodes the value of z… STDP applied to the weights wim, j l of synaptic connections from the first layer to the neurons α on the hidden layer enables the network to approximate for any network input x through the firing probability of neurons on the third layer the distribution of values z that were associated with x in previously processed examples <x, z>” [Pecevski-Maass page 4]; “Note that in general the same input x will occur in combination with different values z(1), z(2), . . . of z in the training examples, and the goal of learning is to learn for each value z(i) the probability that it occurs for input x” [Pecevski-Maass page 4 A network module for learning stochastic associations]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Pecevski-Maass into the co-pending application because they are both directed towards performing probabilistic inference on triple statements of a Bayesian network using neuromorphic hardware. Incorporating the teachings of Pecevski-Maass would enable the recognized improved computational efficiencies (e.g., requiring a smaller number of hidden neurons) (“However, there the number of hidden neurons αk for a random variable yk was required to be exponentially large in the number of variables in the Markov blanket of yk. In contrast, in the learning approach of this article, one can employ in principle any number, also a very small number, of hidden neurons in αk” [Pecevski-Maass page 11])
Claim 25 of the co-pending application does not explicitly teach describing parts of an industrial system, with nodes of the knowledge graph representing physical objects, wherein the physical objects include sensors, industrial controllers, robots, drives, manufactured objects, and either tools or both the tools and elements of a bill of materials, and with nodes of the knowledge graph representing abstract entities including sensor measurements, sensor attributes, configurations of the physical objects or skills of the physical objects, production schedules and plans, as recited in claim 23 of the instant application.
In the same field of endeavor, Kammerer teaches a system of inference on a knowledge graph describing parts of an industrial system, (“In industrial machines, sensors and actors are typically controlled by a programmable logic controller (see Figure 7a)” [Kammerer page 12]; [Kammerer Figure 7 Schematic overview of context-aware process execution framework on page 13]; "Execution contexts, in turn, can be mapped to a context graph, which is a direct acyclic graph and represents the logical structure of a cyber-physical system. Therefore, each node in a context graph has predefined context types and can be used as a basis for the concept called context-aware process family, which is introduced in the following" [Kammerer page 13]) with nodes of the knowledge graph representing physical objects, wherein the physical objects include sensors, industrial controllers, robots, drives, manufactured objects, and either tools or both the tools and elements of a bill of materials, ([Kammerer page 13]; A cyber-physical system [see Figure 4 Information flow processing schema on page 10 and Figure 7 Schematic overview of context-aware process execution framework on page 13] comprises physical objects including sensors) and with nodes of the knowledge graph representing abstract entities including sensor measurements, sensor attributes, configurations of the physical objects or skills of the physical objects, production schedules and plans ([Kammerer page 13]; A cyber-physical system [see Figure 4 Information flow processing schema on page 10 and Figure 7 Schematic overview of context-aware process execution framework on page 13] comprises abstract entities including sensor data/measurements).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Kammerer into the co-pending application because they are both directed towards performing inference on a knowledge graph. Incorporating the teachings of Kammerer would expand the scope of the co-pending application to industrial applications such as predictive maintenance (“For machine manufacturing companies, besides the production of high quality and reliable machines, requirements have emerged to maintain machine-related aspects through digital services. The development of such services in the field of the Industrial Internet of Things (IIoT) is dealing with solutions such as effective condition monitoring and predictive maintenance” [Kammerer Abstract]).
Claims 1, 10-11, 22-24, and 26-27 of the instant application are provisionally rejected on the grounds of nonstatutory double patenting as being unpatentable over claim 26 of co-pending application 17/570,113 in view of Pecevski-Maass (“Learning Probabilistic Inference through Spike-Timing Dependent Plasticity”, published 25 Mar 2016).
Claim 23 of the instant application is provisionally rejected on the grounds of nonstatutory double patenting as being unpatentable over claim 26 of co-pending application 17/570,113 in view of Pecevski-Maass, further in view of Kammerer (“Process-Driven and Flow-Based Processing of Industrial Sensor Data”, published 2020).
Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of the instant application are largely covered in scope by the claims of the co-pending application, such that a person of ordinary skill in the art would deem the claims of the instant application to be an obvious variation.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
The table below shows similarities and differences between the instant application and the co-pending application; the left side contains claims 1, 10-11, 22-24, and 26-27 of the instant application, while the right side contains portions of claims 1 and 24-26 of the co-pending application 17/570,113. Differences that amount to more than minor variations in language and/or punctuation are highlighted and further discussed below.
Instant Application (17/555,577)
Co-pending Application (17/570,113)
Claim 1. A neuromorphic hardware for processing a knowledge graph comprising observed triple statements,
wherein the neuromorphic hardware comprises a hardware component, a learning component, and a control component,
wherein each triple statement in the knowledge graph is structured according to s - p - o, wherein s and o are different entities and p is a relation between s and o and is directed from s to o,
wherein the hardware component is selected from the group consisting of an application specific integrated circuit, a field-programmable gate array, a wafer-scale integration, a hardware with mixed-mode VLSI neurons, a neural processing unit, or a mixed-signal neuromorphic processor,
wherein the learning component comprises an input layer and an output layer,
wherein the input layer comprises a plurality of node embedding populations, wherein each node embedding population comprises a plurality of neurons and represents an entity contained in the observed triple statements,
wherein the output layer comprises output neurons representing a likelihood for each triple statement generated in a sampling mode of the learning component, and
wherein the control component is configured for sequentially switching the learning component: (i) into a data-driven learning mode in which the learning component is trained with the maximum likelihood learning algorithm minimizing energy in a probabilistic, sampling-based model, using only the observed triple statements which are assigned low energy values, wherein the probabilistic, sampling-based model is derived from an energy function, and wherein the observed triple statements have minimal energy,
ii) into the sampling mode, and
(iii) into a model-driven learning mode configured for training the component with the maximum likelihood learning algorithm using only the generated triple statements, with the learning component learning to assign high energy values to the generated triple statements.
(Claim 10, depending from claim 1)
wherein the plurality of node embedding populations comprise a first node embedding population and a second node embedding population,
wherein the first node embedding population comprises first neurons representing a first entity contained in the observed triple statements, and wherein the first node embedding population is characterized by first spike times of the first neurons during a recurring time interval,
wherein the second node embedding population comprises second neurons representing a second entity contained in the observed triple statements, and wherein the second node embedding population is characterized by second spike times of the second neurons during the recurring time interval, and
wherein a relation between the first entity and the second entity is represented as the differences between the first spike times and the second spike times.
(Claim 11, depending from claim 10)
wherein the differences between the first spike times and the second spike times consider an order of the first spike times in relation to the second spike times, or wherein the differences are absolute values.
Claim 22. A method for training a learning component to learn inference on a knowledge graph comprising observed triple statements, said method comprising:
switching, by a control component, the learning component into a data-driven learning mode,
wherein a neuromorphic hardware for processing the knowledge graph comprises a hardware component, the learning component, and the control component,
wherein each triple statement in the knowledge graph is structured according to s - p - o, wherein s and o different entities and p is a relation between s and o and is directed from s to o,
wherein the hardware component is selected from the group consisting of an application specific integrated circuit, a field-programmable gate array, a wafer-scale integration, a hardware with mixed-mode VLSI neurons, a neural processing unit, or a mixed-signal neuromorphic processor,
wherein the learning component comprises an input layer and an output layer,
wherein the input layer comprises a plurality of node embedding populations, wherein each node embedding population comprises a plurality of neurons and represents an entity contained in the observed triple statements,
wherein the output layer comprises output neurons representing a likelihood for each triple statement generated in a sampling mode of the learning component;
training the learning component, which is in the data-driven learning mode, with the maximum likelihood learning algorithm minimizing energy in the probabilistic, sampling-based model, using only the observed triple statements which are assigned low energy values, wherein the probabilistic, sampling-based model is derived from an energy function, and wherein the observed triple statements have minimal energy;
switching, by the control component, the learning component from the data-driven learning mode into the sampling mode;
generating, with the learning component being in the sampling mode, triple statements;
switching, by the control component, the learning component from the sampling mode into a model-driven learning mode; and
training the learning component, which is in the model-driven learning mode, with the maximum likelihood learning algorithm using only the generated triple statements, with the learning component learning to assign high energy values to the generated triple statements.
(Claim 23, depending from claim 22)
wherein the knowledge graph is an industrial knowledge graph describing parts of an industrial system,
with nodes of the knowledge graph representing physical objects, wherein the physical objects include sensors, industrial controllers, robots, drives, manufactured objects, and either tools or both the tools and elements of a bill of materials, and
with nodes of the knowledge graph representing abstract entities including sensor measurements, sensor attributes, configurations of the physical objects or skills of the physical objects, production schedules and plans.
(Claim 24, depending from claim 22)
A non-transitory computer-readable storage media having instructions stored thereon, said instructions executable by one or more processors of a computer system, wherein execution of the instructions causes the one or more processors to perform the method
(Claim 26, depending from claim 22)
wherein the data-driven learning mode includes a positive learning phase,
wherein the sampling mode includes a free-running phase, and
wherein the model-driven learning mode includes a negative learning phase.
(Claim 27, depending from claim 1)
wherein the data-driven learning mode includes a positive learning phase,
wherein the sampling mode includes a free-running phase, and
wherein the model-driven learning mode includes a negative learning phase.
Claim 1. Neuromorphic hardware for storing or for both storing and processing a knowledge graph that comprises observed triple statements, said neuromorphic hardware comprising a hardware component, a learning component, and a control component,
wherein the learning component comprises an input layer,
wherein the input layer comprises a first node embedding population and a second node embedding population,
wherein the first node embedding population comprises N first neurons and represents a first node n1 in the knowledge graph, wherein the first node n1 is characterized by N sequentially ordered first spike times tii, ..., tiN of the N first neurons during a recurring time interval, wherein N is at least 2,
wherein the second node embedding population comprises N second neurons and represents a second node n2 in the knowledge graph, wherein the second node n2 is characterized by N sequentially ordered second spike times t21, ..., t2N of the N second neurons during the recurring time interval, and
wherein each triple statement Sp of P triple statements has a rank θ1.2.p with respect to each relation p of P relations, wherein P is at least 2, wherein each relation p connects the first node n1 and the second node n2 to each other in accordance with a representation as n1 - p - n2 of the triple statement Sp, wherein for p = 1, ..., P: θ1.2.p = ∑Nn=1Δn, wherein Δn is either (Δtn-rp) or II Δtn – rp II, wherein Δtn=t1n - t2n. and wherein rp is a specified spike time difference associated with relation p.
(Claim 24, depending from claim 1)
wherein the control component sequentially switches the learning component: (i) into a data-driven learning mode, (ii) into a sampling mode of the learning component, and (iii) into a model-driven learning mode.
(Claim 25, depending from claim 24)
wherein the control component switches the learning component into the data-driven learning mode in which the learning component is trained with a maximum likelihood learning algorithm minimizing energy in a probabilistic, sampling-based model, using only observed triple statements,
wherein the control component switches the learning component into the sampling mode of the learning component, wherein triple statements are generated in the sampling mode, and
wherein the control component switches the learning component into the model-driven learning mode that trains the learning component with the maximum likelihood learning algorithm using only the generated triple statements.
(Claim 26, depending from claim 25)
wherein during said training the learning component in the data-driven learning mode: the observed triple statements are assigned low energy values, the probabilistic, sampling-based model is derived from an energy function, and the observed triple statements have minimal energy, and
wherein during said training the learning component in the model-driven learning mode, the learning component learns to assign high energy values to the generated triple statements.
(Examiner Note: As per a broadest reasonable interpretation, triple statements are inherently “structured according to s – p – o, wherein s and o are different entities and p is a relation between s and o and is directed from s to o")
(repeated from claim 1)
wherein the learning component comprises an input layer,
wherein the input layer comprises a first node embedding population and a second node embedding population,
wherein the first node embedding population comprises N first neurons and represents a first node n1 in the knowledge graph,
wherein the second node embedding population comprises N second neurons and represents a second node n2 in the knowledge graph
wherein each relation p connects the first node n1 and the second node n2 to each other in accordance with a representation as n1 - p - n2 of the triple statement Sp
(repeated from claim 25)
wherein the control component switches the learning component into the sampling mode of the learning component, wherein triple statements are generated in the sampling mode
(repeated from claim 24)
wherein the control component sequentially switches the learning component: (i) into a data-driven learning mode,
(repeated from claim 25)
into the data-driven learning mode in which the learning component is trained with a maximum likelihood learning algorithm minimizing energy in a probabilistic, sampling-based model, using only observed triple statements,
(repeated from claim 26)
wherein during said training the learning component in the data-driven learning mode: the observed triple statements are assigned low energy values, the probabilistic, sampling-based model is derived from an energy function, and the observed triple statements have minimal energy,
(repeated from claim 24)
(ii) into a sampling mode of the learning component,
(repeated from claim 24)
(iii) into a model-driven learning mode.
(repeated from claim 25)
into the model-driven learning mode that trains the learning component with the maximum likelihood learning algorithm using only the generated triple statements.
(repeated from claim 26)
wherein during said training the learning component in the model-driven learning mode, the learning component learns to assign high energy values to the generated triple statements.
(repeated from claim 1)
wherein the input layer comprises a first node embedding population and a second node embedding population,
wherein the first node embedding population comprises N first neurons and represents a first node n1 in the knowledge graph, wherein the first node n1 is characterized by N sequentially ordered first spike times tii, ..., tiN of the N first neurons during a recurring time interval, wherein N is at least 2,
wherein the second node embedding population comprises N second neurons and represents a second node n2 in the knowledge graph, wherein the second node n2 is characterized by N sequentially ordered second spike times t21, ..., t2N of the N second neurons during the recurring time interval, and
wherein each relation p connects the first node n1 and the second node n2 to each other in accordance with a representation as n1 - p - n2 of the triple statement Sp, wherein for p = 1, ..., P: θ1.2.p = ∑Nn=1Δn, wherein Δn is either (Δtn-rp) or II Δtn – rp II, wherein Δtn=t1n - t2n. and wherein rp is a specified spike time difference associated with relation p.
(repeated from claim 1)
wherein Δn is either (Δtn-rp) or II Δtn – rp II, wherein Δtn=t1n - t2n. and wherein rp is a specified spike time difference associated with relation p.
(repeated from claim 1)
(Examiner Note: Merely claiming the method implemented via the claimed system is an obvious variation)
Neuromorphic hardware for storing or for both storing and processing a knowledge graph that comprises observed triple statements,
a control component that sequentially switches the learning component: (i) into a data-driven learning mode
said neuromorphic hardware comprising a hardware component, a learning component, and a control component,
(Examiner Note: As per a broadest reasonable interpretation, triple statements are inherently “structured according to s – p – o, wherein s and o are different entities and p is a relation between s and o and is directed from s to o")
(repeated from claim 1)
wherein the learning component comprises an input layer,
wherein the input layer comprises a first node embedding population and a second node embedding population,
wherein the first node embedding population comprises N first neurons and represents a first node n1 in the knowledge graph,
wherein the second node embedding population comprises N second neurons and represents a second node n2 in the knowledge graph
wherein each relation p connects the first node n1 and the second node n2 to each other in accordance with a representation as n1 - p - n2 of the triple statement Sp
(repeated from claim 25)
wherein the control component switches the learning component into the sampling mode of the learning component, wherein triple statements are generated in the sampling mode
(repeated from claim 24)
wherein the control component sequentially switches the learning component: (i) into a data-driven learning mode,
(repeated from claim 25)
into the data-driven learning mode in which the learning component is trained with a maximum likelihood learning algorithm minimizing energy in a probabilistic, sampling-based model, using only observed triple statements,
(repeated from claim 26)
wherein during said training the learning component in the data-driven learning mode: the observed triple statements are assigned low energy values, the probabilistic, sampling-based model is derived from an energy function, and the observed triple statements have minimal energy,
(repeated from claim 24)
a control component that sequentially switches the learning component: (ii) into a sampling mode of the learning component, wherein triple statements are generated in the sampling mode,
(repeated from claim 24)
a control component that sequentially switches the learning component: (iii) into a model-driven learning mode that trains the learning component using the generated triple statements.
(repeated from claim 25)
into the model-driven learning mode that trains the learning component with the maximum likelihood learning algorithm using only the generated triple statements.
(repeated from claim 26)
wherein during said training the learning component in the model-driven learning mode, the learning component learns to assign high energy values to the generated triple statements
(claim 26, as above)
(claim 26, as above)
(Examiner Note: Merely implementing the claimed system via a generic computer-readable storage medium is an obvious variation)
(repeated from claim 25)
(i) into a data-driven learning mode in which the learning component is trained with a maximum likelihood learning algorithm minimizing energy in a probabilistic, sampling-based model, using only observed triple statements which are assigned low energy values, wherein the probabilistic, sampling-based model is derived from an energy function, and wherein the observed triple statements have minimal energy,
(Examiner Note: As per a broadest reasonable interpretation, the functions of the claimed data-driven learning mode recite a positive learning phase, as energy-based models inherently train (i.e., learning phase) to assign low energy values to high probability (i.e., positive) configurations))
(ii) into a sampling mode of the learning component, wherein triple statements are generated in the sampling mode, and
(Examiner Note: As per a broadest reasonable interpretation, the functions of the claimed sampling mode inherently recite a free-running phase, i.e., generating new data points based on the learned distribution)
(iii) into a model-driven learning mode that trains the learning component with the maximum likelihood learning algorithm using only the generated triple statements, with the learning component learning to assign high energy values to the generated triple statements;
(Examiner Note: As per a broadest reasonable interpretation, the functions of the model-driven learning mode recite a negative learning phase, as energy-based models inherently train (i.e., learning phase) to assign high energy values to low probability (i.e., negative) configurations)
(repeated from claim 25)
(i) into a data-driven learning mode in which the learning component is trained with a maximum likelihood learning algorithm minimizing energy in a probabilistic, sampling-based model, using only observed triple statements which are assigned low energy values, wherein the probabilistic, sampling-based model is derived from an energy function, and wherein the observed triple statements have minimal energy,
(Examiner Note: As per a broadest reasonable interpretation, the functions of the claimed data-driven learning mode recite a positive learning phase, as energy-based models inherently train (i.e., learning phase) to assign low energy values to high probability (i.e., positive) configurations))
(ii) into a sampling mode of the learning component, wherein triple statements are generated in the sampling mode, and
(Examiner Note: As per a broadest reasonable interpretation, the functions of the claimed sampling mode inherently recite a free-running phase, i.e., generating new data points based on the learned distribution)
(iii) into a model-driven learning mode that trains the learning component with the maximum likelihood learning algorithm using only the generated triple statements, with the learning component learning to assign high energy values to the generated triple statements;
(Examiner Note: As per a broadest reasonable interpretation, the functions of the model-driven learning mode recite a negative learning phase, as energy-based models inherently train (i.e., learning phase) to assign high energy values to low probability (i.e., negative) configurations)
Claim 26 of the co-pending application does not explicitly teach wherein the hardware component is selected from the group consisting of an application specific integrated circuit, a field-programmable gate array, a wafer-scale integration, a hardware with mixed-mode VLSI neurons, a neural processing unit, or a mixed-signal neuromorphic processor, and an output layer, wherein the output layer comprises output neurons representing a likelihood for each triple statement, as recited in claims 1, 10-11, 22-24, and 26-27 of the instant application.
In the same field of endeavor, Pecevski-Maass teaches a system of performing probabilistic inference on triple statements of a Bayesian network using neuromorphic hardware (“The learning model that is presented in this article ties in to this second approach, and shows that stochastic networks of neurons are able to automatically absorb the relevant statistical information from examples that they receive. As a result, we have now one first complete theory for the emergence of probabilistic inference in networks of spiking neurons through learning… We first show how an extension of an ubiquitous network motif of cortical microcircuits, interconnected populations of pyramidal cells with lateral inhibition (Winner- Take-All (WTA) circuits; Douglas and Martin, 2004; Nessler et al., 2013), gives rise to the basic building block for absorbing probabilistic information from examples” [Pecevski-Maass page 2 Introduction]; “We show that the underlying distribution p* can be learnt (approximately) from examples for this visual perception task, and that the network N which learns this approximation learns simultaneously to deal with the explaining away effect as an emergent phenomenon. The structure of the neural network N suitable for learning this target probability distribution p* is given in Figure 6C. It consists of four interconnected learning modules, where the connections between the learning modules reflect the dependencies between the RVs in the Bayesian network in Figure 6B” [Pecevski-Maass page 12]; [see Figure 6 Description of the perceptual explaining away example on page 10]) wherein the hardware component is selected from the group consisting of an application specific integrated circuit, a field-programmable gate array, a wafer-scale integration, a hardware with mixed-mode VLSI neurons, a neural processing unit, or a mixed-signal neuromorphic processor, (“Altogether, our computer simulations and our theoretical analyses demonstrate that networks of spiking neurons can emulate probabilistic inference for general Bayesian networks. Hence we propose to view probabilistic inference in graphical models as a generic computational paradigm, that can help us to understand the computational organization of networks of neurons in the brain, and in particular the computational role of precisely structured cortical microcircuit motifs" [Pecevski page 5 Introduction]; The underlying neuromorphic hardware is a precise cortical microcircuit structure (i.e., application specific integrated circuit)), wherein the learning component comprises an output layer, (“However, although the architecture of N will obviously have to depend on the number of random variables of p*, we show that it suffices to assume that it consists of recursive interconnections of different copies of a simple generic network motif, to which we refer as a stochastic association module. This network motif is a three-layer feedforward network of excitatory spiking neurons with lateral inhibition on the hidden layer (see Fig. 2). We show that this simple microcircuit motif can be viewed as an atomic learning module, that extracts via STDP and intrinsic plasticity from examples probabilistic associations between input variables x and output variable z that are encoded through population coding on its input and output layer” [Pecevski-Maass page 3 Results]) wherein the output layer comprises output neurons representing a likelihood for each triple statement (see Figure 2 including output neurons layer– “The population of neurons on the third layer encodes the value of z… STDP applied to the weights wim, j l of synaptic connections from the first layer to the neurons α on the hidden layer enables the network to approximate for any network input x through the firing probability of neurons on the third layer the distribution of values z that were associated with x in previously processed examples <x, z>” [Pecevski-Maass page 4]; “Note that in general the same input x will occur in combination with different values z(1), z(2), . . . of z in the training examples, and the goal of learning is to learn for each value z(i) the probability that it occurs for input x” [Pecevski-Maass page 4 A network module for learning stochastic associations]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Pecevski-Maass into the co-pending application because they are both directed towards performing probabilistic inference on triple statements of a Bayesian network using neuromorphic hardware. Incorporating the teachings of Pecevski-Maass would enable the recognized improved computational efficiencies (e.g., requiring a smaller number of hidden neurons) (“However, there the number of hidden neurons αk for a random variable yk was required to be exponentially large in the number of variables in the Markov blanket of yk. In contrast, in the learning approach of this article, one can employ in principle any number, also a very small number, of hidden neurons in αk” [Pecevski-Maass page 11]).
Claim 26 of the co-pending application does not explicitly teach wherein the knowledge graph is an industrial knowledge graph describing parts of an industrial system, with nodes of the knowledge graph representing physical objects, wherein the physical objects include sensors, industrial controllers, robots, drives, manufactured objects, and either tools or both the tools and elements of a bill of materials, and with nodes of the knowledge graph representing abstract entities including sensor measurements, sensor attributes, configurations of the physical objects or skills of the physical objects, production schedules and plans, as recited in claim 23 of the instant application.
In the same field of endeavor, Kammerer teaches a system of inference on an industrial knowledge graph describing parts of an industrial system, (“In industrial machines, sensors and actors are typically controlled by a programmable logic controller (see Figure 7a)” [Kammerer page 12]; [Kammerer Figure 7 Schematic overview of context-aware process execution framework on page 13]; "Execution contexts, in turn, can be mapped to a context graph, which is a direct acyclic graph and represents the logical structure of a cyber-physical system. Therefore, each node in a context graph has predefined context types and can be used as a basis for the concept called context-aware process family, which is introduced in the following" [Kammerer page 13]) with nodes of the knowledge graph representing physical objects, wherein the physical objects include sensors, industrial controllers, robots, drives, manufactured objects, and either tools or both the tools and elements of a bill of materials, ([Kammerer page 13]; A cyber-physical system [see Figure 4 Information flow processing schema on page 10 and Figure 7 Schematic overview of context-aware process execution framework on page 13] comprises physical objects including sensors) and with nodes of the knowledge graph representing abstract entities including sensor measurements, sensor attributes, configurations of the physical objects or skills of the physical objects, production schedules and plans ([Kammerer page 13]; A cyber-physical system [see Figure 4 Information flow processing schema on page 10 and Figure 7 Schematic overview of context-aware process execution framework on page 13] comprises abstract entities including sensor data/measurements).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Kammerer into the co-pending application because they are both directed towards performing inference on a knowledge graph. Incorporating the teachings of Kammerer would expand the scope of the co-pending application to industrial applications such as predictive maintenance (“For machine manufacturing companies, besides the production of high quality and reliable machines, requirements have emerged to maintain machine-related aspects through digital services. The development of such services in the field of the Industrial Internet of Things (IIoT) is dealing with solutions such as effective condition monitoring and predictive maintenance” [Kammerer Abstract]).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 8-12, 14-16, 22, and 26-27 are rejected under 35 U.S.C. 103 as being unpatentable over Pecevski et al., (“Probabilistic Inference in General Graphical Models through Sampling in Stochastic Networks of Spiking Neurons”, published 2011), hereinafter Pecevski, in view of Pecevski-Maass (“Learning Probabilistic Inference through Spike-Timing Dependent Plasticity”, published 25 Mar 2016) and Jiang (“Encoding Temporal Information for Time-Aware Link Prediction”, published 2016).
Regarding claim 1, Pecevski teaches A neuromorphic hardware for processing a knowledge graph represented by observed triple statements ("We will focus in this article on Bayesian networks, a common type of graphical model for probability distributions...A Bayesian network is a directed graph (without directed cycles), whose nodes represent RVs z1, . . . ,zK. Its graph structure indicates that p(z1, . . . ,zK) admits a factorization of the form [equation 7] where pa(zk) is the set of all (direct) parents of the node indexed by zk...We show that the complexity of the resulting network of spiking neurons for carrying out probabilistic inference for p can be bounded in terms of the graph complexity of the Bayesian network that gives rise to the factorization (7)…Altogether, our computer simulations and our theoretical analyses demonstrate that networks of spiking neurons can emulate probabilistic inference for general Bayesian networks. Hence we propose to view probabilistic inference in graphical models as a generic computational paradigm, that can help us to understand the computational organization of networks of neurons in the brain, and in particular the computational role of precisely structured cortical microcircuit motifs" [Pecevski page 5 Introduction]; Pecevski discloses a spiking neural network system implemented on a cortical microcircuit structure (i.e. neuromorphic hardware) that performs probabilistic inference on (i.e., learns from) a Bayesian network (i.e., knowledge graph). A Bayesian network takes the form of a directed acyclic graph, wherein nodes representing variables (RVs) are connected to other nodes in a parent-child (i.e. subject-object) relationship. In light of the instant specification [page 2 lines 1-5], two connected nodes of a Bayesian network, representing probabilistic relations between variables, are thereby interpretable as a triple, or triple statement)
wherein the neuromorphic hardware comprises a hardware component, a learning component, and a control component, ("We demonstrate in computer simulations that the precisely structured neuronal microcircuits enable networks of spiking neurons to solve through their inherent stochastic dynamics a variety of complex probabilistic inference tasks" [Pecevski page 2 Author Summary]; The structured neuronal microcircuits (i.e., neuromorphic hardware) are inherently a hardware component, and further enable (i.e., control) a spiking neural network framework (i.e. learning component) capable of performing a variety of probabilistic inference tasks)
wherein each triple statement in the knowledge graph is structured according to s - p - o, wherein s and o are different entities and p is a relation between s and o and is directed from s to o, ([Pecevski page 5 Introduction] as detailed above; As explained above, two nodes (i.e., entities) of a Bayesian network connected by a directed edge are interpretable as a triple statement of a knowledge graph – e.g., structured as n1 – edge – n2, wherein the edge is directed from node n1 to node n2)
wherein the hardware component is selected from the group consisting of an application specific integrated circuit, a field-programmable gate array, a wafer-scale integration, a hardware with mixed-mode VLSI neurons, a neural processing unit, or a mixed-signal neuromorphic processor, (“Altogether, our computer simulations and our theoretical analyses demonstrate that networks of spiking neurons can emulate probabilistic inference for general Bayesian networks. Hence we propose to view probabilistic inference in graphical models as a generic computational paradigm, that can help us to understand the computational organization of networks of neurons in the brain, and in particular the computational role of precisely structured cortical microcircuit motifs" [Pecevski page 5 Introduction]; The underlying neuromorphic hardware is a precise cortical microcircuit structure (i.e., application specific integrated circuit))
However, Pecevski does not explicitly teach wherein a learning component comprises an input layer and an output layer, wherein the input layer comprises a plurality of node populations, wherein each node population comprises a plurality of neurons and represents an entity contained in the observed triple statements, wherein the output layer comprises output neurons representing a likelihood for each triple statement generated in a sampling mode of the learning component, and wherein a control component is configured for switching the learning component: into a learning mode and into the sampling mode.
In the same field of endeavor, Pecevski-Maass teaches a system of performing probabilistic inference on triple statements of a Bayesian network using neuromorphic hardware (“The learning model that is presented in this article ties in to this second approach, and shows that stochastic networks of neurons are able to automatically absorb the relevant statistical information from examples that they receive. As a result, we have now one first complete theory for the emergence of probabilistic inference in networks of spiking neurons through learning… We first show how an extension of an ubiquitous network motif of cortical microcircuits, interconnected populations of pyramidal cells with lateral inhibition (Winner- Take-All (WTA) circuits; Douglas and Martin, 2004; Nessler et al., 2013), gives rise to the basic building block for absorbing probabilistic information from examples” [Pecevski-Maass page 2 Introduction]; “We show that the underlying distribution p* can be learnt (approximately) from examples for this visual perception task, and that the network N which learns this approximation learns simultaneously to deal with the explaining away effect as an emergent phenomenon. The structure of the neural network N suitable for learning this target probability distribution p* is given in Figure 6C. It consists of four interconnected learning modules, where the connections between the learning modules reflect the dependencies between the RVs in the Bayesian network in Figure 6B” [Pecevski-Maass page 12]; [see Figure 6 Description of the perceptual explaining away example on page 10]) wherein a learning component comprises an input layer and an output layer (“However, although the architecture of N will obviously have to depend on the number of random variables of p*, we show that it suffices to assume that it consists of recursive interconnections of different copies of a simple generic network motif, to which we refer as a stochastic association module. This network motif is a three-layer feedforward network of excitatory spiking neurons with lateral inhibition on the hidden layer (see Fig. 2). We show that this simple microcircuit motif can be viewed as an atomic learning module, that extracts via STDP and intrinsic plasticity from examples probabilistic associations between input variables x and output variable z that are encoded through population coding on its input and output layer” [Pecevski-Maass page 3 Results])
wherein the input layer comprises a plurality of node populations, wherein each node population comprises a plurality of neurons and represents an entity contained in the observed triple statements, (see Figure 2 including input neurons layer comprising populations of neurons xi – “Figure 2. Structure of a stochastic association module that is able to learn probabilistic associations between multinomial variables x = (x1,…xl) and z through STDP. Populations of neurons xi (i = 1, . . . , I) on the first layer encode the values of input variables xi.” [Pecevski-Maass page 4]; Each input variable xi (which can be drawn from RVs comprising nodes of a Bayesian network, as explained above) is encoded by a respective population of neurons (i.e., node population))
wherein the output layer comprises output neurons representing a likelihood for each triple statement generated in a sampling mode of the learning component, (see Figure 2 including output neurons layer– “The population of neurons on the third layer encodes the value of z… STDP applied to the weights wim, j l of synaptic connections from the first layer to the neurons α on the hidden layer enables the network to approximate for any network input x through the firing probability of neurons on the third layer the distribution of values z that were associated with x in previously processed examples <x, z>” [Pecevski-Maass page 4]; “Note that in general the same input x will occur in combination with different values z(1), z(2), . . . of z in the training examples, and the goal of learning is to learn for each value z(i) the probability that it occurs for input x” [Pecevski-Maass page 4 A network module for learning stochastic associations]; “The other approach for emulating probabilistic inference in networks of spiking neurons is based on the assumption that a network of neurons can “embody” a distribution p in such a way that it can generate samples from p. Probabilistic inference for p can then be performed through simple operations on these samples” [Pecevski-Maass page 2 Introduction]; “We provide in this article a proof of principle that these parameters of p do not have to be programmed into the network: they can be learnt by a network of spiking neurons via simple local plasticity rules from examples y˜ that are generated by p” [Pecevski-Maass page 2 Results]) and
wherein a control component is configured for switching the learning component: into a learning mode and into the sampling mode (“We first show how an extension of an ubiquitous network motif of cortical microcircuits, interconnected populations of pyramidal cells with lateral inhibition (Winner- Take-All (WTA) circuits; Nessler et al., 2013), gives rise to the basic building block for absorbing probabilistic information from examples. The output neurons of an array of WTA-circuits form the hidden layer of a three-layer feedforward learning module, to which we refer as a “stochastic association module.”” [Pecevski-Maass page 2 Introduction]; “We want to determine under what conditions a network module is able to create autonomously from exposure to these examples an internal model p(x, z) for p*(x, z), that approximates p* when the number of examples grows…We present in Materials and Methods a rigorous proof that after learning the distribution of output values z for a given network input x approximates in this stochastic association module the conditional distribution pzx of the joint distribution px, z from which the values of z and x are drawn in the training examples” [Pecevski-Maass pages 4-5 A network model for learning stochastic associations]; “Like other generative models for unsupervised learning, our model also aims at extracting underlying structure in the training examples (Hinton et al., 1995), so that it can even generate fake examples that share the discovered underlying structure (Fig. 7)” [Pecevski-Maass pages 4-5 A network model for learning stochastic associations]; The underlying cortical microcircuits (i.e., control component) enable the spiking neural network framework (i.e., learning component), which follows a typical generative model learning procedure of first learning an underlying distribution based on training data (i.e., learning mode) and then generating examples based on the learned distribution (i.e., sampling mode))
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein a learning component comprises an input layer and an output layer, wherein the input layer comprises a plurality of node populations, wherein each node population comprises a plurality of neurons and represents an entity contained in the observed triple statements, wherein the output layer comprises output neurons representing a likelihood for each triple statement generated in a sampling mode of the learning component, and wherein a control component is configured for switching the learning component: into a learning mode and into the sampling mode as taught by Pecevski-Maass into Pecevski because they are both directed towards performing probabilistic inference on triple statements of a Bayesian network using neuromorphic hardware. Pecevski-Maass explicitly discloses similarity to the framework taught in Pecevski, and expands upon a similar base structure (“The structure of the network N in Figure 5 is very similar to the structure of a constructed network of spiking neurons that directly mimics a representation of p* by a Bayesian network according to Pecevski et al. (2011)” [Pecevski-Maass page 11]). Incorporating the teachings of Pecevski-Maass would thereby enable the recognized improved computational efficiencies (e.g., requiring a smaller number of hidden neurons) (“However, there the number of hidden neurons αk for a random variable yk was required to be exponentially large in the number of variables in the Markov blanket of yk. In contrast, in the learning approach of this article, one can employ in principle any number, also a very small number, of hidden neurons in αk” [Pecevski-Maass page 11])
However, the combination does not explicitly teach nodes and edges of the knowledge graph being represented by vector embeddings (such that the node populations of Pecevski-Maass would then represent node embedding populations), or explicitly teach sequentially switching from (i) a data-driven learning mode in which [a] learning component is trained with the maximum likelihood learning algorithm minimizing energy in a probabilistic, sampling-based model, using only observed triple statements which are assigned low energy values, wherein the probabilistic, sampling-based model is derived from an energy function, and wherein the observed triple statements have minimal energy, ii) into [a] sampling mode for generating triple statements, and (iii) into a model-driven learning mode configured for training the component with the maximum likelihood learning algorithm using only the generated triple statements, with the learning component learning to assign high energy values to the generated triple statements.
In the same field of endeavor, Jiang teaches a method of performing inference on a knowledge graph comprising triple statements, (“Knowledge bases (KBs) such as Freebase (Bollacker et al., 2008) and YAGO (Fabian et al., 2007) play a pivotal role in many NLP related applications. KBs consist of facts in the form of triplets (ei, r, ej), indicating that head entity ei and tail entity ej is linked by relation r. Although KBs are large, they are far from complete. Link prediction is to predict relations between entities based on existing triplets, which can alleviate the incompleteness of current KBs…This paper mainly focuses on incorporating the temporal order information and proposes a time-aware link prediction model” [Jiang Introduction page 1]) wherein nodes and edges of the knowledge graph (i.e., entities and relations of a triple statement) are represented by vector embeddings ("Recently a promising approach for this task called knowledge base embedding (Nickel et al., 2011; Bordes et al., 2011; Socher et al., 2013) aims to embed entities and relations into a continuous vector space while preserving certain information of the KB graph. TransE (Bordes et al., 2013) is a typical model considering relation vector as translating operations between head and tail vector" [Jiang Introduction page 1]; "To make the embedding space compatible with the observed triples, we make use of the triple set △ and follow the same strategy adopted in previous methods such as TransE" [Jiang page 2 Time-Aware KB Embedding]), further comprising:
sequentially switching from (i) a data-driven learning mode in which [a] learning component is trained with the maximum likelihood learning algorithm minimizing energy in a probabilistic, sampling-based model, using only observed triple statements which are assigned low energy values, wherein the probabilistic, sampling-based model is derived from an energy function, and wherein the observed triple statements have minimal energy, (“Triple classification aims to judge whether an unseen triple is correct or not…To create labeled data for classification, for each triple in the test and validation sets, we construct a corresponding negative triple…During triple classification a triple is predicted as positive if the score is below a relation-specific threshold; otherwise as negative” [Jiang page 4 Triple Classification] ; “…we make use of the triple set Δ and follow the same strategy adopted in previous methods such as TransE [equation 2] For each candidate triple, it requires positive triples to have lower scores than negative triples” [Jiang Time-Aware KB Embedding page 2]; “The optimization is to minimize the joint score function, [equation 3] where x+ is the positive triple (quad), x- is corresponding the negative triple” [Jiang Time-Aware KB Embedding page 2]; Triples in test and validation sets are based on real (i.e., observed) data and are therefore positive, unlike negative triples which are constructed (i.e. generated); the predictive model is derived from a function [see equation 2] wherein positive triples have lower scores (i.e., minimal energies) compared to negative triples. Jiang further discloses minimizing a joint score (i.e., energy) function that includes f(x+) as a parameter, wherein f(x+) represents the energy of a positive (i.e., observed) triple. Energy of a triple is equivalent to calculating a dissimilarity measure within the triple statement via an L1 norm [see equation 2]; The joint score algorithm therefore minimizes the energy, or dissimilarity, of entities within a positive triple, which is equivalent to maximizing their likelihood (i.e., correctness) within a triple classification [Triple Classification page 4] task),
ii) into [a] sampling mode for generating triple statements, (“To create labeled data for classification, for each triple in the test and validation sets, we construct a corresponding negative triple by randomly corrupting the entities” [Jiang Triple Classification page 4]; Negative triples are artificially constructed (i.e. generated). The model generates a corresponding negative triple for each positive triple, and therefore alternates (i.e., sequentially switches) between two modes of training: using only the positive (i.e., observed) triples versus using only the negative (i.e., generated) triples), and
(iii) into a model-driven learning mode configured for training the component with the maximum likelihood learning algorithm using only the generated triple statements, with the learning component learning to assign high energy values to the generated triple statements (“The optimization is to minimize the joint score function, [equation 3] where x+ is the positive triple (quad), x- is corresponding the negative triple” [Jiang Time-Aware KB Embedding page 2]; Jiang discloses minimizing a joint score (i.e., energy) function that includes -f(x-) as a parameter, wherein f(x-) represents the energy of a negative (i.e., generated) triple. Energy of a triple is equivalent to calculating a dissimilarity measure within the triple statement via an L1 norm [see equation 2]; The joint score algorithm therefore minimizes the negative energy, functionally equivalent to maximizing the energy, of the negative triples, and assigns higher energy values to negative triples than positive triples).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated nodes and edges of the knowledge graph being represented by vector embeddings, and sequentially switching from (i) a data-driven learning mode in which [a] learning component is trained with the maximum likelihood learning algorithm minimizing energy in a probabilistic, sampling-based model, using only observed triple statements which are assigned low energy values, wherein the probabilistic, sampling-based model is derived from an energy function, and wherein the observed triple statements have minimal energy, ii) into [a] sampling mode for generating triple statements, and (iii) into a model-driven learning mode configured for training the component with the maximum likelihood learning algorithm using only the generated triple statements, with the learning component learning to assign high energy values to the generated triple statements as taught by Jiang into the combination because Pecevski, Pecevski-Maass, and Jiang are all directed towards performing inference on a knowledge graph comprising triple statements.
TransE, which forms the base upon which the method of Jiang expands [Jiang Introduction page 1] is known in the art as a method of modelling large-scale, multi-relational datasets/knowledge bases [see Bordes (cited in IDS) Introduction pages 1-2]; it is further known that multi-relational data can be modeled by learning/operating on vector representations (i.e., embeddings) of entities and relationships via "Bayesian clustering frameworks or energy-based frameworks" [Bordes Introduction page 2], wherein TransE is an energy-based framework. Pecevski further indicates that the disclosed system of probabilistic inference can be applied to a variety of graphical models, including both energy-based models (e.g., a Boltzmann machine) ("The results of [21] and numerous related results suggest that the brain is able to carry out probabilistic inference for more complex distributions than the 2nd order Boltzmann distribution" [Pecevski Introduction page 4]) and Bayesian models ("We will focus in this article on Bayesian networks, a common type of graphical model for probability distributions. But our results can also be applied for other types of graphical models" [Pecevski Introduction page 5]). A person of ordinary skill would have thereby been able to combine the teachings of the probabilistic inference systems of Pecevski and Pecevski-Maass and the Trans-E energy-based model of Jiang to arrive at the claimed invention with a reasonable expectation of success.
Jiang further discloses an improvement over TransE by incorporating an awareness of time and temporal order into the embedding space ("Traditional KB embedding models such as TransE often confuse relations such as wasBornIn and diedIn when predicting (person,?,location) because TransE learns only from time-unknown facts and cannot distinguish relations with similar semantic meaning. To make more accurate predictions, it is non-trivial for existing KB embedding methods to incorporate temporal order information. This paper mainly focuses on incorporating the temporal order information and proposes a time-aware link prediction model" [Jiang Introduction page 1]). Given that the spiking neural network system of Pecevski is heavily interrelated with temporal dynamics, including temporal order of neuron spikes, by nature ("But it is in conflict with basic features of networks of spiking neurons, where each action potential (spike) of a neuron triggers inherent temporal processes in the neuron itself (e.g. refractory processes), and postsynaptic potentials of specific durations in other neurons to which it is synaptically connected" [Pecevski Introduction page 3]), one of ordinary skill would recognize potential for the system of Jiang to improve the system of Pecevski by improving accuracy in modelling temporal dynamics ("In this paper, we propose a general time-aware KB embedding, which incorporates creation time of entities and imposes temporal order constraints on the geometric structure of the embedding space and enforce it to be temporally consistent and accurate" [Jiang Conclusion page 4]).
Regarding claim 2, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 1, and Pecevski-Maass further teaches present inputs to the learning component by selectively activating subject and object populations among the node embedding populations (“The network learns from presented examples y˜0, y˜1, ..., y˜n, . . . drawn from the target probability distribution p y. In the examples the RV yk assumes an integer value drawn from the set 1, 2, . . . , Myk . An example is presented to the network in form of injected currents in the neurons. These injected currents are assumed to originate from external neurons. The neurons in k are driven by the injected currents such that their firing reflects correctly the values of the RVs yk in the current example y˜n. More precisely, for y˜ k n l the neuron kl receives strong positive current and fires with a high firing rate, whereas the other neurons in the population k receive a strong negative current, which prevents them from firing” [Pecevski-Maass page 26 Theoretical properties of networks of recursively interconnected basic learning modules])
set hyperparameters of the learning component, in particular a factor (η) that modulates learning updates of the learning component, (We use a simple STDP rule, which has the advantage of being theoretically tractable. Let w be the weight of the synapse at the connection from some presynaptic neuron vpre to a postsynaptic neuron vpost. At each postsynaptic spike of neuron vpost at time t this weight undergoes an update: w [Wingdings font/0xDF] w + ηΔw, where η is the learning rate” [Pecevski-Maass page 3 Results])
read output of the learning component, ([Pecevski-Maass page 3 Results] as detailed above; The postsynaptic spikes of neurons are processed (i.e., read)) and
use output of the learning component as feedback to the learning component ([Pecevski-Maass page 3 Results] as detailed above; The postsynaptic spikes are used as feedback to update hyperparameters)
Regarding claim 3, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 1, and Pecevski-Maass further teaches wherein the output layer has one output neuron for each relation type of the knowledge graph (see Figure 3A including output neurons ζ1 and ζ2 – “Figure 3. Learning results for example 1. A, Structure of the learning module. There are two subpopulations 1, 2 of hidden neurons that both receive inputs from the two populations on layer 1 that encode the input variables x1 and x2. Each subpopulation of hidden neurons projects to a different neuron in the population coding of the variable z on layer 3…D, Left, The target probabilities p*(z|x) (grey bars) compared with the learned firing probabilities of the output neurons ζ1 and ζ2 that represent p(z = 1|x;θ) and p(z=2|x;θ) respectively” [Pecevski-Maass page 7]; The output neurons correspond to various conditional probabilities for the input variable with respect to different output values (i.e., relation types))
Regarding claim 4, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 3, and Pecevski-Maass further teaches wherein the output neurons may be stochastic dendritic output neurons as taught in Pecevski (“However if one considers more complex neuron models with nonlinear dendritic processing, they can in principle also represent multimodal distributions (Pecevski et al., 2011, their Figs. 4 and 5; Legenstein and Maass, 2011). Hence, with such more complex neuron models a more shallow learning network could potentially be used as a learning module in our architecture” [Pecevski-Maass page 5]). Pecevski further teaches stochastic dendritic neurons storing embeddings of relations that are given between a subject and an object in the observed triple statements in each dendrite of a plurality of dendrites of each neuron ("We show in this article that the neural sampling method of [1] can be extended to any probability distribution p over binary RVs, in particular to distributions with higher order dependencies among RVs, by using auxiliary spiking neurons in N that do not directly represent RVs zk, or by using nonlinear computational processes in multi-compartment neuron models. As one can expect, the number of required auxiliary neurons or dendritic branches increases with the complexity of the probability distribution p for which the resulting network of spiking neurons has to carry out probabilistic inference" [Pecevski page 4]), summing all dendrites of each neuron into a final score ("The amplitude of the dendritic spikes from the dendritic branch dc,2 v of the principal neuron n2 should be equal to the parameter wc,2 v from (13). The index c identifies the two factors that depend on z2. The membrane voltage at the soma of the principal neuron n2 is then equal to the sum of the contributions from the dendritic spikes of the active dendritic branches" [Pecevski page 10]), wherein the final score of each neuron is transformed into a probability using an activation function ("The firing probability of the neuron model is then p(t) = f(u(t))g(t-t), where t is the time of the last firing of the neuron before time t" [Pecevski Neuron Models section Point neuron model on page 18]; "The membrane potential at the soma of the neuron is a sum of the active and passive contributions from all branches [equation 22, u(t)]. The firing probability in this neuron model and its refractory mechanism are the same as for the point neuron model described above" [Pecevski Neuron Models section Multi-compartment neuron model pages 18-19]).
Regarding claim 5, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 3, and Pecevski further teaches wherein an output of the activation function is a prediction of the likelihood of a triple statement or a transition probability ("The firing probability of the neuron model is then p(t) = f(u(t))g(t-t), where t is the time of the last firing of the neuron before time t" [Pecevski Neuron Models section Point neuron model on page 18]; Firing probability measures probability of neuron firing during a transition between spikes).
Regarding claim 6, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 4, and Pecevski further teaches wherein learning updates for the embeddings of relations are computed directly in the dendrites of the stochastic, dendritic output neurons (“As one can expect, the number of required auxiliary neurons or dendritic branches increases with the complexity of the probability distribution p for which the resulting network of spiking neurons has to carry out probabilistic inference" [Pecevski page 4]; Probabilistic inference tasks are performed within the dendritic components of the spiking neurons)
Regarding claim 8, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 4, and Pecevski further teaches wherein in the sampling mode, by sampling from an activation function, a binary output signals to the control component whether a triple statement is accepted ("A key step for interpreting the firing activity of networks of neurons as sampling from a probability distribution (as proposed in [3]) in a rigorous manner is to define a formal relationship between spikes and samples. As in [1] we relate the firing activity in a network N of K spiking neurons n1, . . . ,nK to sampling from a distribution p(z1, . . . ,zK) over binary variables z1, . . . ,zK by setting zk(t) = 1 if and only if neuron nk has fired within the preceding time interval (t-τ, t) of length τ,…The constant τ models the average length of the effect of a spike on the firing probability of other neurons or of the same neuron" [Pecevski page 3]).
Regarding claim 10, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 1, and Pecevski-Maass further teaches wherein the plurality of node embedding populations comprise a first node embedding population and a second node embedding population, wherein the first node embedding population comprises first neurons representing a first entity contained in the observed triple statements, and wherein the second node embedding population comprises second neurons representing a second entity contained in the observed triple statements (see Figure 2 including input neurons (e.g., including first neurons) and output neurons (e.g., including second neurons) layer [Pecevski-Maass page 4]), as further detailed in claim 1 above. Pecevski further teaches wherein the first node embedding population is characterized by first spike times of the first neurons during a recurring time interval, ("...in this approach the spike times, rather than the firing rate, of the neuron nk carry relevant information as they define the value of the RV zk at a particular moment in time t according to (2)” [Pecevski Introduction page 3]), and wherein the second node embedding population is characterized by second spike times of the second neurons during the recurring time interval, ("...in this approach the spike times, rather than the firing rate, of the neuron nk carry relevant information as they define the value of the RV zk at a particular moment in time t according to (2)” [Pecevski Introduction page 3]) and wherein a relation between the first entity and the second entity is represented as the differences between the first spike times and the second spike times. ("In this spike-time based coding scheme, the relative timing of spikes (which neuron fires simultaneously with whom) receives a direct functional interpretation since it determines the correlation between the corresponding RVs. The NCC requires that for each RV zk the firing probability density rk(t) of its corresponding neuron nk at time t satisfies, if the neuron is not in a refractory period, [equation 3] where z\k denotes the current value of all other RVs, i.e., all zi with i = k" [Pecevski Introduction page 3]; In a spike-time based coding scheme, for each RV, the relative timing of spikes determines the correlations (i.e., differences) between RVs).
Regarding claim 11, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 1, and Pecevski further teaches wherein the differences between the first spike times and the second spike times consider an order of the first spike times in relation to the second spike times, or wherein the differences are absolute values (“Thus if the principal nk neuron is modelled as a point neuron, we have [equation 8] where bk is the bias of neuron (which regulates its excitability), Wki is the strength of the synaptic connection from neuron vi to vk, and zi (t) approximates the time course of the postsynaptic potential in neuron vk caused by a firing of neuron vi” [Pecevski Results page 5-6]; Order of the spike times affects differences, as the firing of first neuron vi affects the postsynaptic potential of second neuron vk).
Regarding claim 12, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 10, and Pecevski further teaches wherein the relation is stored in one of the output neurons, and wherein the relation is given by vector components that are stored in dendrites of the output neurons ("Our models postulate that knowledge is encoded in the brain in the form of probability distributions p, that are not required to be of the restricted form of 2nd order Boltzmann distributions (5). Furthermore they postulate that these distributions are encoded through synaptic weights and neuronal excitabilities, and possibly also through the strength of dendritic branches. Finally, our approach postulates that these learnt and stored probability distributions p are activated through the inherent stochastic dynamics of networks of spiking neurons, using nonlinear features of network motifs and neurons to represent higher order dependencies between RVs" [Pecevski Experimentally Testable Predictions of our Models page 11]; "The alternative model that only uses dendritic computation (Implementation 5) will have groups of dendritic branches corresponding to the different factors. The number of auxiliary neurons that connect to nk in Implementation 4 (and the corresponding number of dendritic branches in Implementation 5) is equal to the sum of the exponents of the sizes of factors that depend on zk: [summation]. where D(zc\k) denotes the number of RVs in the vector zc\ k." [Pecevski page 9])
Regarding claim 14, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 10, and Pecevski further teaches wherein the first neurons and the second neurons are spiking neurons that are non-leaky integrate-and-fire neurons or current-based leaky integrate-and-fire neurons ("We consider in this article two types of models for spiking neurons (see Methods for details):...stochastic leaky integrate –and –fire neurons with absolute and relative refractory periods, formalized in the spike–response framework of [16] (as in [1])," [Pecevski page 3]; "The evidence about known RVs in the neural network was introduced by injected constant current in the corresponding principal neurons of amplitude Az~50 if the value of the RV is 1 and A{~{30 if the value of the RV is 0." [Pecevski page 23]).
Regarding claim 15, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 10, and Pecevski further teaches wherein each of the first neurons and each of the second neurons spikes only once during the recurring time interval, or wherein only a first spike during the recurring time interval is counted (The NCC requires that for each RV zk the firing probability density rk(t) of its corresponding neuron nk at time t satisfies, if the neuron is not in a refractory period, [equation 3]…In the simpler version of this neuron model one assumes that it has an absolute refractory period of length t, and that the instantaneous firing probability pk(t) satisfies outside of its refractory period..." [Pecevski page 3]; The neuron will only fire once within an absolute refractory period (recurring time interval)).
Regarding claim 16, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 1, and Pecevski further teaches wherein each node embedding population is connected to an inhibiting neuron and is selectable by inhibition of the inhibiting neuron (“This distribution requires knowledge about when exactly a neuron nk with zk(t)~1 had fired. Therefore auxiliary RVs f1, . . . ,fK with multinomial or analog values were introduced in [1], that keep track of when exactly in the preceding time interval of length t a neuron nk had fired" [Pecevski page 3]; "The inputs connect to the auxiliary neuron akv either with a direct strong excitatory connection, or through an inhibitory interneuron ikv that connects to the auxiliary neuron. The inhibitory interneuron ikv fires whenever any of the principal neurons of the RVs zBk that connect to it fires" [Pecevski page 20]).
Regarding claim 22, it is a method claim that largely corresponds to the apparatus of claim 1. Consequently, claim 22 is rejected for the same reasons as claim 1 above.
Regarding claim 26, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 22, and Pecevski-Mass further teaches wherein the sampling mode includes a free-running phase ([Pecevski-Maass pages 4-5 A network model for learning stochastic associations] as detailed in claim 1 above; The sampling mode generates examples based on the learned distribution (i.e., is free running)). Jiang further teaches wherein the data-driven learning mode includes a positive learning phase, and wherein the model-driven learning mode includes a negative learning phase ([Jiang Time-Aware KB Embedding page 2] as detailed in claim 1 above; The data-driven learning mode comprises training on the positive triples (i.e., positive learning), and the model-driven learning mode comprises training on the negative triples (i.e., negative learning)).
Regarding claim 27, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 1, and Pecevski-Mass further teaches wherein the sampling mode includes a free-running phase ([Pecevski-Maass pages 4-5 A network model for learning stochastic associations] as detailed in claim 1 above; The sampling mode generates examples based on the learned distribution (i.e., is free running)).. Jiang further teaches wherein the data-driven learning mode includes a positive learning phase, and wherein the model-driven learning mode includes a negative learning phase ([Jiang Time-Aware KB Embedding page 2] as detailed in claim 1 above; The data-driven learning mode comprises training on the positive triples (i.e., positive learning), and the model-driven learning mode comprises training on the negative triples (i.e., negative learning)).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Pecevski, Pecevski-Maass, and Jiang, as applied to claim 1 above, further in view of Hamilton et al., (“Spike-Based Primitives for Graph Algorithms”, available arXiv 2019), hereinafter Hamilton.
Regarding claim 7, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 4, and Pecevski further teaches wherein learning updates for one node embedding population of the plurality of node embedding populations are computed using feedback connections from each dendrite of each output neuron to respective neurons of the one node embedding population ([Pecevski page 4] as detailed in claim 4 above).
However, the combination does not explicitly teach static feedback connections.
In the same field of endeavor, Hamilton teaches a method of performing inference on directed graphs using a spiking neural network system ("A directed graph D(V;E) is defined by a vertex set V (D) and a set of directed arcs E(D)" [Hamilton Graph definitions page 2]; "We employ spiking neuron systems (SNS) S(n1,sk) similar to those described in [18]: they are defined by a set of nonlinear, deterministic neurons n1 and a set of synapses sk" [Hamilton Spiking neuron system definitions page 3]; "For all of the algorithms presented in this paper, the SNS are constructed from a given graph (G or D) via direct mapping. Definition 2.1 (Direct Mapping). A graph (G or D) is directly mapped to an SNS by defining a neuron ni ∈ n for each vi ∈ V and a synapse or pair of synapses for each edge e ∈ E. Directed arcs are mapped to directed synapses" [Hamilton page 3]) that discloses static feedback connections between neurons ("We present algorithms throughout this paper that can be implemented with either static or plastic synapses, and require that plastic synapses can realize a form of spike timing dependent plasticity (STDP)" [Hamilton page 3]; "Theorem 3.4 (Iterative subgraph extraction). If an SNS with static synapses is defined by directly mapping G to S(N;W), then it is possible to set the neuron parameters l’ for ni ∈ {n’} such that applying a stimulus to any neuron in the neuron subset {n’} will only generate a spike response from the remaining neurons in {n’}" [Hamilton page 7]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated static feedback connections as taught by Hamilton into the combination because both Pecevski and Hamilton are directed towards performing inference on directed graphs using a spiking neural network system. Incorporating the teachings of Hamilton would expand the system of the combination to be hardware agnostic (i.e., applicable on a variety of neuromorphic hardware with a variety of capabilities, such as not having synaptic connections) ("This paper introduces mathematical proofs of spike-based primitives for graphical algorithms as well as examples of hardware agnostic implementations" [Hamilton page 1]; "It is worth noting that the relative threshold values, synapse weights, delays and refractory periods will be dependent on individual hardware capabilities, but we focus on presenting our routines with general guidelines showing how spiking neurons can extract and implement each routine." [Hamilton page 2]).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Pecevski, Pecevski-Maass, and Jiang, as applied to claim 10 above, further in view of Pfeil (“Exploring the potential of brain-inspired computing”, available heiDOK 2015).
Regarding claim 13, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 10, and Pecevski further teaches wherein the first neurons are connected to a monitoring neuron, (““This distribution requires knowledge about when exactly a neuron nk with zk(t)~1 had fired. Therefore auxiliary RVs f1, . . . ,fK with multinomial or analog values were introduced in [1], that keep track of when exactly in the preceding time interval of length t a neuron nk had fired" [Pecevski page 3]),
wherein the neurons are connected to output neurons, (“Thus if the principal nk neuron is modelled as a point neuron, we have [equation 8] where bk is the bias of neuron (which regulates its excitability), Wki is the strength of the synaptic connection from neuron vi to vk, and zi (t) approximates the time course of the postsynaptic potential in neuron vk caused by a firing of neuron vi” [Pecevski Results page 5-6]) and
wherein the neurons are connected to an inhibiting neuron ("The inputs connect to the auxiliary neuron akv either with a direct strong excitatory connection, or through an inhibitory interneuron ikv that connects to the auxiliary neuron. The inhibitory interneuron ikv fires whenever any of the principal neurons of the RVs zBk that connect to it fires" [Pecevski page 20]).
However, the combination does not explicitly teach neurons including corresponding parrot neurons wherein each first neuron is connected to a corresponding parrot neuron.
In the same field of endeavor, Pfeil teaches a spiking neural networks framework hosted on neuromorphic hardware ("Having a control software that abstracts hardware greatly simplifies modeling on the neuromorphic hardware system. However, modelers are already struggling with multiple incompatible interfaces to software simulators. That is why our neuromorphic hardware system supports PyNN, a widely used application programming interface (API) that strives for a coherent user interface, allowing portability of neural network models between different software simulation frameworks (e.g., NEST or NEURON) and hardware systems (e.g., the Spikey system)" [Pfeil page 23]; [Pfeil section The Spikey neuromorphic system on pages 19-22]) that discloses parrot neurons wherein each neuron is connected to a corresponding parrot neuron ("All simulations involving synapses are simulated with NEST. Spike trains are applied to built-in parrot neurons, that simply repeat their input, in order to control pre- and postsynaptic spike trains to interconnecting synapses." [Pfeil page 68])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated parrot neurons wherein each neuron is connected to a corresponding parrot neuron as taught by Pfeil into the combination because both Pecevski and Pfeil are directed towards a spiking neural networks framework hosted on neuromorphic hardware. Incorporating the teachings of Pfeil would make it easier to precisely control pre- and post- synaptic spike times ([Pfeil page 68]) within the framework (“Fig. 8 also suggests that different neurons may have drastically different firing rates, where a few neurons fire a lot, and many others fire rarely. This is a consequence both of different marginal probabilities for different RVs, but also of the quite different computational role and dynamics of neurons that represent RVs (‘‘principal neurons’’), and auxiliary neurons that support the realization of the NCC, and which are only activated by a very specific activation patterns of other presynaptic neurons” [Pecevski page 17]).
Claims 17-18 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Pecevski, Pecevski-Maass, and Jiang, as applied to claim 1 above, further in view of Fan et al., (“On Applications of Spiking Neural P Systems”, published 2020), hereinafter Fan.
Regarding claim 17, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 1.
However, the combination does not explicitly teach An industrial device, comprising the neuromorphic hardware.
In the same field of endeavor, Fan teaches a spiking neural networks framework hosted on neuromorphic hardware (“Over the years, spiking neural P systems (SNPS) have grown into a popular model in membrane computing because of their diverse range of applications. In this paper, we give a comprehensive summary of applications of SNPS and its variants, especially highlighting power systems fault diagnoses with fuzzy reasoning SNPS. We also study the structure and workings of these models, their comparisons along with their advantages and disadvantages. We also study the implementation of these models in hardware” [Fan Abstract]) that discloses An industrial device ("Many variants of SNPS have been introduced by incorporating features from the biological neurons...These models have also been used in solving problems related to real life applications such as...programming for logic controllers [8], etc." [Fan page 2]; "The main contributions of this work are as follows: (1) Listing a majority of the SNPS models used for solving problems...Additionally, their use in solving computationally hard problems, the construction of u-fluidic biochip design and programming for PLC (programmable logic controller);" [Fan pages 2-3]; In light of the specification [instant specification page 7 lines 6-13] and the described “real life applications”, PLCs are industrial devices) comprising the neuromorphic hardware ("In Figure 6, we presented a comparison of all SNPS models performing arithmetic and logical operations and also their hardware implementation [Figure 6]" [Fan page 18]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated an industrial device with neuromorphic hardware as taught by Fan into the combination because both Pecevski and Fan are directed towards a spiking neural networks framework hosted on neuromorphic hardware. Incorporating the teachings of Fan would further expand the scope of application of the system of Pecevski as well as its potential implementations in hardware (“SNNs are hardware friendly and energy efficient...The main motivation to prepare this survey is as follows: (3) Study implementations of these models in hardware; (4) Introduce some new ideas to expand the scope of SNPS models" [Fan page 2]).
Regarding claim 18, the combination of Pecevski, Pecevski-Maass, Jiang, and Fan teaches the limitations of parent claim 17 and wherein the industrial device is a field device, an edge device, a sensor device, an industrial controller, a PLC controller, an industrial PC implementing a SCADA system, a network hub, an industrial ethernet switch, or an industrial gateway connecting an automation system to cloud computing resources ([Fan page 2] as detailed in claim 17 above).
Regarding claim 24, the combination of Pecevski, Pecevski-Maass, and Jiang teaches the limitations of parent claim 22. Fan further teaches A non-transitory computer-readable storage media having instructions stored thereon, said instructions executable by one or more processors of a computer system, wherein execution of the instructions causes the one or more processors to perform the method ("One of the major motivations for studying arithmetic and logical operations using SNPS has been the designing of the arithmetic logic unit of CPU under the framework of SNPS which can be useful in constructing novel digital circuits/chips" [Fan page 16]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated A computer-readable storage media having stored thereon: instructions executable by one or more processors of a computer system, wherein execution of the instructions causes the computer system to perform the method as taught by Fan into the combination because both Pecevski and Fan are directed towards a spiking neural networks framework hosted on neuromorphic hardware. Incorporating the teachings of Fan would further enable the system of Pecevski to reap the benefits of improved efficiency of SNPS models (“Along with the advancement of technologies, it has become imperative to construct efficient hardware which can perform complex tasks. SNPS models have some very useful features, such as parallel and distributive architecture, non-determinism, etc., and these features help the models to perform millions of computations very efficiently with minimum time and space resources. These models also can perform basic arithmetic and logic operations which make SNPS an important candidate for designing of CPU” [Fan page 16]).
Claims 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Pecevski, Pecevski-Maass, Jiang, and Fan, as applied to claim 17 above, further in view of Kammerer et al., (“Process-Driven and Flow-Based Processing of Industrial Sensor Data”, published 2020), hereinafter Kammerer.
Regarding claim 19, the combination of Pecevski, Pecevski-Maass, Jiang, and Fan teaches the limitations of parent claim 17, and Pecevski further teaches a triple store, storing the observed triple statements ("Finally, our approach postulates that these learnt and stored probability distributions p are activated through the inherent stochastic dynamics of networks of spiking neurons, using nonlinear features of network motifs and neurons to represent higher order dependencies between RVs" [Pecevski Experimentally Testable Predictions of our Models page 11]) and wherein the learning component is configured for performing an inference in an inference mode (“We show that the complexity of the resulting network of spiking neurons for carrying out probabilistic inference for p can be bounded in terms of the graph complexity of the Bayesian network that gives rise to the factorization (7)." [Pecevski Introduction page 5]).
Fan further teaches application of a spiking neural network to an industrial device (e.g., a PLC controller) [Fan pages 2-3], as detailed in claim 17 above. However, the combination does not explicitly teach a PLC controller with at least source configured for providing raw data, wherein the at least one source is at least one sensor, at least one data source, or both the at least one sensor and the at least one data source and with an ETL component, configured for converting the raw data into the observed triple statements, using mapping rules.
In the same field of endeavor, Kammerer teaches an industrial device, including a PLC controller (“PLC is an example of a complex real-time system, as its output results must be produced in response to input conditions within a limited time period; otherwise, unintended operation may be the result” [Kammerer page 8]) with at least source configured for providing raw data, wherein the at least one source is at least one sensor, at least one data source, or both the at least one sensor and the at least one data source and with an ETL component, configured for converting the raw data into the observed triple statements, using mapping rules ([Kammerer see Figure 7 on page 13; see Sensor Data Management including Sensor Data Acquisition and Processing. See component PLC (labeled “a”) in Sensor Data Capturing; see Context Management including Context Evaluation and Context Mapping]; "In order to manage the execution context of a cyber-physical system, events from a sensor data management component are received by an event processing agent (see Figure 7k). Events are continuously evaluated by executing queries that are stored in an event query repository (see Figure 7l). Each query can include certain context patterns to map events to entities in a context graph (see Figure 7m). Execution contexts, in turn, can be mapped to a context graph, which is a direct acyclic graph and represents the logical structure of a cyber-physical system. Therefore, each node in a context graph has predefined context types and can be used as a basis for the concept called context-aware process family, which is introduced in the following." [Kammerer page 13]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated an industrial device with at least source configured for providing raw data, wherein the at least one source is at least one sensor, at least one data source, or both the at least one sensor and the at least one data source and with an ETL component, configured for converting the raw data into the observed triple statements, using mapping rules as taught by Kammerer into the combination because Fan and Kammerer are both directed towards implementing an industrial device that processes data, and both Pecevski and Kammerer are directed towards performing inference on a knowledge graph. Given that Fan already teaches applicability of a spiking neural networks framework to programming for logic controllers (PLC controller) [Fan pages 2-3], Kammerer would further develop the system of Fan by disclosing an incorporation of the PLC controller within a larger framework of collecting and processing sensor data ([Kammerer Figure 7 on page 13]; “Typically, a PLC is connected to other information systems, such as industrial PCs (IPCs), equipped with human–machine interfaces (HMI) to configure and to control the PLC execution” [Kammerer page 8]). Therefore, incorporating the teachings of Kammerer would enable the system of Fan to effectively collect, process, and utilize massive amounts of data ("Besides the discussed sensor differences, sensor data are typically delivered from sensor subsystems (e.g., a programmable logic controller, PLC) and continuously streamed to subsequent processing components, which must then cope with massive amounts of data…To tackle the aforementioned challenges, a sensor processing pipeline (SPP) is proposed, which provides solutions for capturing, processing, storing, and visualizing raw sensor data in a continuous processing pipeline" [Kammerer pages 1-2]).
Regarding claim 20, the combination of Pecevski, Pecevski-Maass, Jiang, Fan, and Kammerer teaches the limitations of parent claim 19, and Kammerer further teaches the industrial device with a statement handler, configured for triggering an automated action based on inference of the learning component ([Kammerer see Figure 7 on page 13; see Context-aware Process Execution]; "Context-aware process execution (CaPE) enables the management of context-aware processes. It supports the modeling of process variants at design time and the automated, controlled adaption of processes at runtime" [Kammerer page 13]).
Claim 21 is rejected as being unpatentable over Pecevski, Pecevski-Maass and Jiang, as applied to claim 1 above, further in view of Yu et al. (“Traffic Scheduling based on Spiking Neural Network in Hybrid E/O Switching Intra-Datacenter Networks”, published 2020), hereinafter Yu.
Regarding claim 21, the combination of Pecevski, Pecevski-Maass and Jiang teaches the limitations of parent claim 1.
However, the combination does not explicitly teach A server, with the neuromorphic hardware.
In the same field of endeavor, Yu teaches an application of a spiking neural networks framework for predictive inference (“However, the low accuracy of existing deep learning-based prediction approaches, which cannot fully extract the features of burst traffic, directly restricts the efficiency of traffic scheduling. In view of this, this study considers the spiking neural networks that can predict high burstiness and heterogeneous traffic to further improve the efficiency of traffic scheduling. We first propose a supervised spiking neural network (s-SNN) framework for high accuracy traffic prediction in HS-IDCNs” [Yu Abstract]) that discloses a server, with the neuromorphic hardware (“With the overwhelming growth of emerging cloud applications, such as VR/AR and streaming video, the hybrid E/O switching intra-datacenter networks (HS-IDCNs) have been built to interconnect massive servers and to cope with the rich traffic types…with the development of computer technologies, SNNs can be trained in neuromorphic hardware, such as IBM’s TrueNorth chip [7] and Intel’s Loihi processor [8]… To the best of our knowledge, no study has discussed the use of this biology-based neural network for traffic prediction in HS-IDCNs” [Yu Introduction page 1]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated a server, with the neuromorphic hardware as taught by Yu into the combination because both Pecevski and Yu are directed towards applications of a spiking neural networks framework for predictive inference. Incorporating the teachings of Yu would further expand the scope of the probabilistic inference system of Pecevski to cloud computing applications, particularly for improving efficiency of datacenter networks (e.g., predicting path blocking probability) (“The simulation results demonstrate that the s-SNN framework can significantly improve the traffic prediction accuracy, and the proposed TP-TS algorithm can improve the resource utilization and decrease blocking probability of HS-IDCNs” [Yu Introduction page 1]).
Claim 23 is rejected as being unpatentable over Pecevski, Pecevski-Maass and Jiang, as applied to claim 22 above, further in view of Kammerer (“Process-Driven and Flow-Based Processing of Industrial Sensor Data”, published 2020).
Regarding claim 23, the combination of Pecevski, Pecevski-Maass and Jiang teaches the limitations of parent claim 22.
However, the combination does not explicitly teach wherein the knowledge graph is an industrial knowledge graph describing parts of an industrial system, with nodes of the knowledge graph representing physical objects, wherein the physical objects include sensors, industrial controllers, robots, drives, manufactured objects, and either tools or both the tools and elements of a bill of materials, and with nodes of the knowledge graph representing abstract entities including sensor measurements, sensor attributes, configurations of the physical objects or skills of the physical objects, production schedules and plans.
In the same field of endeavor, Kammerer teaches a system of inference on a knowledge graph wherein the knowledge graph is an industrial knowledge graph describing parts of an industrial system, (“In industrial machines, sensors and actors are typically controlled by a programmable logic controller (see Figure 7a)” [Kammerer page 12]; [Kammerer Figure 7 Schematic overview of context-aware process execution framework on page 13]; "Execution contexts, in turn, can be mapped to a context graph, which is a direct acyclic graph and represents the logical structure of a cyber-physical system. Therefore, each node in a context graph has predefined context types and can be used as a basis for the concept called context-aware process family, which is introduced in the following" [Kammerer page 13])
with nodes of the knowledge graph representing physical objects, wherein the physical objects include sensors, industrial controllers, robots, drives, manufactured objects, and either tools or both the tools and elements of a bill of materials ([Kammerer page 13]; A cyber-physical system [see Figure 4 Information flow processing schema on page 10 and Figure 7 Schematic overview of context-aware process execution framework on page 13] comprises physical objects including sensors).
and with nodes of the knowledge graph representing abstract entities including sensor measurements, sensor attributes, configurations of the physical objects or skills of the physical objects, production schedules and plans ([Kammerer page 13]; A cyber-physical system [see Figure 4 Information flow processing schema on page 10 and Figure 7 Schematic overview of context-aware process execution framework on page 13] comprises abstract entities including sensor data/measurements).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein the knowledge graph is an industrial knowledge graph describing parts of an industrial system, with nodes of the knowledge graph representing physical objects, wherein the physical objects include sensors, industrial controllers, robots, drives, manufactured objects, and either tools or both the tools and elements of a bill of materials, and with nodes of the knowledge graph representing abstract entities including sensor measurements, sensor attributes, configurations of the physical objects or skills of the physical objects, production schedules and plans as taught by Kammerer into the combination because both Pecevski and Kammerer are directed towards performing inference on a knowledge graph. Incorporating the teachings of Kammerer would expand the scope of the probabilistic inference system of Pecevski to industrial applications such as predictive maintenance (“For machine manufacturing companies, besides the production of high quality and reliable machines, requirements have emerged to maintain machine-related aspects through digital services. The development of such services in the field of the Industrial Internet of Things (IIoT) is dealing with solutions such as effective condition monitoring and predictive maintenance” [Kammerer Abstract]).
Response to Arguments
The remarks filed 07/01/2025 have been fully considered.
Applicant’s remarks [Remarks pages 15-16] traversing the provisional nonstatutory double patenting rejections set forth in the office action mailed 04/14/2025 have been considered, but are moot because the amendments made to both the instant application (17/555,577) and the co-pending applications at issue (17/563,480 and 17/570,113) have necessitated a new grounds of provisional nonstatutory double patenting rejection as set forth above.
Applicant's remarks [Remarks pages 21-36] traversing the obviousness rejections under 35 U.S.C. 103 set forth in the office action mailed 04/14/2025, with respect to claims 1-8 and 10-24 as amended and newly added claims 26-27, have been considered.
Although a new grounds of rejection has been applied, the examiner has determined a response necessary for certain portions of the remarks [Remarks pages 22-23, pages 23-24], particularly with respect to discussing the broadest reasonable interpretation of claim language.
The remaining remarks, while having been considered, are moot because the new grounds of rejection set forth above does not rely on the reference(s) applied in the prior rejection of record for the subject matter being specifically challenged in applicant' s argument.
Applicant’s argument [Remarks pages 22-23] appears to implicitly suggest that a probabilistic relation between connected nodes in a Bayesian graph is not a triple statement.
The examiner respectfully notes that the discussed probabilistic relations, in this instance, are in reference to conditional probabilities (e.g., likelihood of T given A) (see ASIA Bayesian network in Figure 7A [Pecevski page 12] below), which are represented by a connected pair of nodes, not individual nodes.
PNG
media_image1.png
435
822
media_image1.png
Greyscale
The examiner further notes that under broadest reasonable interpretation in light of the instant specification [page 2 lines 1-5], two nodes of a Bayesian network connected by an edge, representing probabilistic relations between variables, are interpretable as a triple, or triple statement. It is known in the art (see Fukushige et al., “Representing Probabilistic Relations in RDF”, cited in Conclusion) that probabilistic relations between variables are representable in a triple statement (e.g., RDF triple) format.
Applicant’s argument [Remarks pages 23-24] appears to implicitly suggest that “sequentially switching the learning component” into a data-driven learning mode, into a sampling mode, and into a model-driven learning mode recites an order of switching the learning component into each mode followed one after another without interruption, i.e., a consecutive order.
In response, the examiner respectfully notes that per Cambridge Dictionary, “sequential” is defined as “following a particular order”, and not necessarily a consecutive order. Applicant’s argument thereby appears to follow a narrower interpretation that is not recited in the claim language.
Applicant has not presented further substantive arguments with respect to the dependent claims [Remarks pages 25-36]. As such, claims 1-8, 10-24, and 26-27 stand rejected under 35 U.S.C. 103.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Fukushige et al., (“Representing Probabilistic Relations in RDF”, posted online 03/07/2005) discloses a vocabulary for representing probabilistic relations in an RDF triple format, and an algorithm for transforming a set of probabilistic relations in an RDF graph to a Bayesian network.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VIJAY M BALAKRISHNAN whose telephone number is (571) 272-0455. The examiner can normally be reached 10am-5pm EST Mon-Thurs.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JENNIFER WELCH can be reached on (571) 272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/V.M.B./
Examiner, Art Unit 2143
/JENNIFER N WELCH/Supervisory Patent Examiner, Art Unit 2143
Prosecution Insights