DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d) based on an application filed in FEDERAL REPUBLIC OF GERMANY on August 3rd, 2022. The certified copy has been filed in parent Application No. 18/363,512, filed on August 1st, 2023. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 6, 8, 15 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Every dependent claim is/are subject to the same rejection(s) as the independent claim from which it depends.
Regarding claim 6, 8:
The limitations use “and/or” which render the claim(s) indefinite. The use of “and/or” render the claim indefinite because the scope of the claim(s) is unascertainable, therefore, indefinite. The use of “or” can include all of the limitations but and/or does not make it clear as to what limitation(s) are needed.
Regarding claim 15:
The claim is indefinite because it is unclear whether claim 15 is directed to an apparatus/system or a method. The limitation is interpreted as a system.
Appropriate fixes are required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-15 rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., an abstract idea) without significantly more.
Regarding claim 1 and analogous claim 14 and 15:
Step 1 (whether a claim is to a statutory category):
Yes, the claim is within the four statutory categories (a process, machine, manufacture or composition of matter). Claim 1 recites a method, therefore, falls within a process category. Claim 14 recites non-transitory machine-readable storage medium, therefore, falls within a manufacture category. Claim 15 recites a system, therefore, falls within a machine category.
Step 2A Prong 1 (whether a claim is directed to a judicial exception):
Yes, “selecting, from the generic knowledge graph, a subgraph relating to a context for solving the specified task;” describes a mental process (i.e., evaluation, judgement) wherein evaluating information and selecting information (see MPEP 2106.04(a)(2)(III)). And, “ascertaining, for each training example, a respective feature map using the feature extractor of the neural network;” describes a mental process (i.e., observation) wherein obtaining information (see MPEP 2106.04(a)(2)(III)). And, “ascertaining, from each respective training example in connection with the respective target output, a representation of the subgraph in a space of the respective feature maps;” describes a mental process (i.e., observation, evaluation) wherein organizing and evaluating information (see MPEP 2106.04(a)(2)(III)). And, “evaluating an output from each respective feature map with regard to the specified task;” describes a mental process (i.e., evaluation) wherein evaluating an output (see MPEP 2106.04(a)(2)(III)). And, “assessing, using a specified cost function, to what extent the respective feature maps are similar to the representation of the subgraph;” describes a mental process (i.e., evaluation, judgement) wherein evaluating and comparing information based on cost function and involves calculating and comparing values recites a mathematical concept, as it involves mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP § 2106.04(a)(2), (I)). And, “optimizing parameters that characterize the behavior of the neural network, with a goal that the assessment by the cost function is expected to improve during further processing of training examples; and” involves mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP § 2106.04(a)(2), (I)).
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, “providing training examples labeled with respective target outputs with respect to a specified task;” describes an additional element as “apply it”, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(f)). And, “providing a generic knowledge graph whose nodes represent entities and whose edges represent relationships between the entities;” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.04(d), 2106.05(g). And, “adjusting the evaluation of the feature maps such that the output for each training example corresponds as well as possible to the respective target output for the respective training example” describes an additional element as “apply it”, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(f)).
Step 2B (Inventive concept):
No, it describes an additional element as “apply it”, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(f)).
Regarding claim 2:
Further modifies the abstract idea of claim 1.
Step 2A Prong 1 (whether a claim is directed to a judicial exception):
Yes, “assessing, using a cost function, how well the output of the task head for each training example corresponds to the target output for the respective training example; and” describes a mental process (i.e., evaluation, judgement) wherein evaluating and comparing information based on cost function and involves calculating and comparing values recites a mathematical concept, as it involves mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP § 2106.04(a)(2), (I)). And, “optimizing parameters of the task head with regard to the assessment by the cost function” involves mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP § 2106.04(a)(2), (I)).
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, “wherein the neural network additionally includes a task head configured to evaluate the respective feature maps with regard to the specified task, and wherein the adjustment of the evaluation of the feature maps includes:” describes an additional element as “apply it”, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(f)).
Step 2B (Inventive concept):
No, it does not add significantly more since the intended practical application is well-understood, routine, and conventional and stated at a generic level (i.e., “apply it”, see MPEP 2106.05(f)).
Regarding claim 3:
Further modifies the abstract idea of claim 1.
Step 2A Prong 1 (whether a claim is directed to a judicial exception):
Yes, “wherein the similarity between each respective feature map and the representation of the subgraph is set in relation to the similarity between the representation of the subgraph and feature maps ascertained for other training examples with other respective target outputs” describes a mental process (i.e., evaluation, judgement) wherein evaluating and comparing information and involves calculating and comparing values recites a mathematical concept, as it involves mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP § 2106.04(a)(2), (I)).
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, the claim does not recite additional elements that integrate the judicial exception into a practical application with the words "apply it" (or an equivalent), such as mere instructions to implement an abstract idea (see MPEP 2106.05(f)).
Step 2B (Inventive concept):
No, the claim does not add significantly more since the intended practical application is well-understood, routine, and conventional and stated at a generic level (i.e., “apply it”, see MPEP 2106.05(f)).
Regarding claim 4:
Further modifies the abstract idea of claim 1.
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, “wherein the respective target outputs include classification scores with respect to one or more classes of a specified classification of the measurement data” describes additional elements that integrate the judicial exception into a practical application with the words "apply it" (or an equivalent), such as mere (i.e., selecting a particular data source or type of data to be manipulated) to implement an abstract idea on a computer (see MPEP 2106.05(f)).
Step 2B (Inventive concept):
No, it does not add significantly more since the intended practical application is well-understood, routine, and conventional and stated at a generic level (i.e. “apply it”, see MPEP 2106.05(f)).
Regarding claim 5:
Further modifies the abstract idea of claim 4.
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, “wherein classes of the specified classification represent types of objects whose presence in an area monitored during recording of the measurement data is indicated by the measurement data” describes additional elements that integrate the judicial exception into a practical application with the words "apply it" (or an equivalent), such as mere (i.e., selecting a particular data source or type of data to be manipulated) to implement an abstract idea on a computer (see MPEP 2106.05(f)).
Step 2B (Inventive concept):
No, it does not add significantly more since the intended practical application is well-understood, routine, and conventional and stated at a generic level (i.e. “apply it”, see MPEP 2106.05(f)).
Regarding claim 6:
Further modifies the abstract idea of claim 5.
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, “wherein other vehicles, and/or traffic signs, and/or roadway markings, and/or traffic obstructions and/or other traffic-relevant objects in the vicinity of a vehicle are selected as types of objects” describes additional elements that integrate the judicial exception into a practical application with the words "apply it" (or an equivalent), such as mere (i.e., selecting a particular data source or type of data to be manipulated) to implement an abstract idea on a computer (see MPEP 2106.05(f)).
Step 2B (Inventive concept):
No, it does not add significantly more since the intended practical application is well-understood, routine, and conventional and stated at a generic level (i.e. “apply it”, see MPEP 2106.05(f)).
Regarding claim 7:
Further modifies the abstract idea of claim 4.
Step 2A Prong 1 (whether a claim is directed to a judicial exception):
Yes, “wherein the evaluating of the respective feature maps includes assigning the respective feature maps to classes using a Gaussian process, and the adjustment of the evaluation includes:” involves calculating and comparing values recites a mathematical concept, as it involves mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP § 2106.04(a)(2), (I)). And, “defining decision limits between classes in the space of the respective feature maps based on the respective target outputs” describes a mental process (i.e., evaluation, judgement) wherein evaluating an output and defining a decision (see MPEP 2106.04(a)(2)(III)). And, “ascertaining respective feature maps for all training examples; and” describes a mental process (i.e., evaluation,) wherein evaluating feature maps based on the data (see MPEP 2106.04(a)(2)(III)).
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, the claim does not recite additional elements that integrate the judicial exception into a practical application with the words "apply it" (or an equivalent), such as mere instructions to implement an abstract idea (see MPEP 2106.05(f)).
Step 2B (Inventive concept):
No, it does not add significantly more since the intended practical application is well-understood, routine, and conventional and stated at a generic level (i.e. “apply it”, see MPEP 2106.05(f)).
Regarding claim 8:
Further modifies the abstract idea of claim 1.
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, “wherein images, audio signals, and/or time series of measured values, and/or radar data, and/or lidar data are selected as measurement data” describes additional elements that integrate the judicial exception into a practical application with the words "apply it" (or an equivalent), such as mere (i.e., selecting a particular data source or type of data to be manipulated) to implement an abstract idea on a computer (see MPEP 2106.05(f)).
Step 2B (Inventive concept):
No, it does not add significantly more since the intended practical application is well-understood, routine, and conventional and stated at a generic level (i.e. “apply it”, see MPEP 2106.05(f)).
Regarding claim 9:
Further modifies the abstract idea of claim 1.
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, “wherein the subgraph relates to a visual or taxonomic or functional context” describes additional elements that integrate the judicial exception into a practical application with the words "apply it" (or an equivalent), such as mere (i.e., selecting a particular data source or type of data to be manipulated) to implement an abstract idea on a computer (see MPEP 2106.05(f)).
Step 2B (Inventive concept):
No, it describes an additional element as “apply it”, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(f)).
Regarding claim 10:
Further modifies the abstract idea of claim 1.
Step 2A Prong 1 (whether a claim is directed to a judicial exception):
Yes, “wherein the selection of the subgraph is also included in the optimization with respect to the assessment by the cost function” describes a mental process (i.e., evaluation) wherein selecting information (see MPEP 2106.04(a)(2)(III)) and involves calculating and comparing values recites a mathematical concept, as it involves mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP § 2106.04(a)(2), (I)).
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, the claim does not recite additional elements that integrate the judicial exception into a practical application with the words "apply it" (or an equivalent), such as mere instructions to implement an abstract idea (see MPEP 2106.05(f)).
Step 2B (Inventive concept):
No, the claim does not add significantly more since the intended practical application is well-understood, routine, and conventional and stated at a generic level (i.e., “apply it”, see MPEP 2106.05(f)).
Regarding claim 11:
Further modifies the abstract idea of claim 1.
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, “wherein the representation of the subgraph in the space of the respective feature maps is retrieved from a pre-calculated lookup table based on each training example and the respective target output” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(d), 2106.05(g).).
Step 2B (Inventive concept):
No, the limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). Furthermore the additional element is well‐understood, routine, and conventional as taught by activity is supported under Berkheimer Option 2 (iv.) Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93; See MPEP § 2106.05(d).
Regarding claim 12:
Further modifies the abstract idea of claim 1.
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, “wherein a further machine learning model configured to generate the representation of the subgraph in the space of the feature maps is trained together with the neural network” describes additional elements that integrate the judicial exception into a practical application with the words "apply it" (or an equivalent), such as mere (i.e., selecting a particular data source or type of data to be manipulated) to implement an abstract idea on a computer (see MPEP 2106.05(f)).
Step 2B (Inventive concept):
No, the claim does not add significantly more since the intended practical application is well-understood, routine, and conventional and stated at a generic level (i.e., “apply it”, see MPEP 2106.05(f)).
Regarding claim 13:
Further modifies the abstract idea of claim 1.
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, “measurement data are supplied to the trained neural network so that the trained neural network generates outputs;” describes additional elements that integrate the judicial exception into a practical application with the words "apply it" (or an equivalent), such as mere (i.e., selecting a particular data source or type of data to be manipulated) to implement an abstract idea on a computer (see MPEP 2106.05(f)). And, “a control signal is formed from outputs of the neural network; and” describes additional elements that integrate the judicial exception into a practical application with the words "apply it" (or an equivalent), to implement an abstract idea on a computer (see MPEP 2106.05(f)). And, “a vehicle and/or a driver assistance system and/or a quality control system and/or an area monitoring system and/or a medical imaging system, is controlled using the control signal” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
Step 2B (Inventive concept):
No, the claim does not add significantly more since the intended practical application is well-understood, routine, and conventional and stated at a generic level (i.e., “apply it”, see MPEP 2106.05(f)). Further, the limitation “a vehicle and/or a driver assistance system and/or a quality control system and/or an area monitoring system and/or a medical imaging system, is controlled using the control signal” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 9-10, 12, 14-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Monka et al., Non-Patent Literature (“ConTraKG: Contrastive-based Transfer Learning for Visual Object Recognition using Knowledge Graphs”) in view of Yang et al., Non-Patent Literature (“Knowledge Graph Contrastive Learning for Recommendation”).
Regarding Claim 1 and analogous claim 14, 15:
Monka teaches:
A method for training a neural network for evaluating measurement data, wherein the neural network includes a feature extractor configured to generate feature maps from the measurement data, the method comprising the following steps: (Introduction, para 1, “Deep learning as a machine learning technique is heavily used to successfully solve several computer vision tasks (i.e., wherein training a neural network for evaluating). Its main power is the ability to find complex underlying patterns in the given set(s) of images (i.e., wherein neural network includes a feature extractor under the broadest reasonable interpretation (BRI) is interpreted as find complex patterns). A common method for extracting these patterns is to train a deep neural network (DNN)”...Page 3, para 1, “approach to combine prior domain-invariant knowledge captured by a knowledge graph (KG) with standard deep learning architectures”)
providing training examples labeled with respective target outputs with respect to a specified task; (Page 6, para 3, “Given a source domain DS with input data XS, a corresponding source task TS with labels YS, as well as a target domain DT with input data XT and a target task TT with labels YT,”)
providing a generic knowledge graph whose nodes represent entities and whose edges represent relationships between the entities; (Section 3, para 1, “where a knowledge graph is a graph of data aiming to accumulate and convey real-world knowledge, where entities are represented by nodes and relationships between entities are represented by edges.”)
ascertaining, for each training example, a respective feature map using the feature extractor of the neural network; (Page 6, para 1, “An encoder network E(·) [feature extractor] is part of the DNN and maps images x (i.e., wherein images is interpreted as training example) to a visual embedding (space) hvis = E(x) ∈ RDE, where the activations of the final pooling layer and thus the representation layer have a dimensionality of DE =I (i.e., wherein visual embedding under the broadest reasonable interpretation (BRI) is interpreted as feature map), where i depends on the encoder network that is used.”)
ascertaining, from each respective training example in connection with the respective target output, a representation of the subgraph in a space of the respective feature maps; (Section 3, para 2, “A knowledge graph embedding hKG is learned by a knowledge graph embedding method KGE(·) using entities and relations encoded in the KG.”…Page 8, para 1, “the DNN learns the projection z and the visual domain invariant embedding hinv of input samples with positive index xp to map the corresponding class label ya [target output] in the knowledge graph embedding vector”)
evaluating an output from each respective feature map with regard to the specified task; (Page 9, para 4, “For inference, we add a randomly initialized linear fully connected layer to the trained encoder network so that it is enabled to classify input data. The size of the output vector depends on the number of classes of the task TT if performed on DT and TS if performed on DS”…Page 6, para 3, “Given a source domain DS with input data XS, a corresponding source task TS with labels YS, as well as a target domain DT with input data XT and a target task TT with labels YT,”)
assessing, using a specified cost function, to what extent the respective feature maps are similar to the representation of the subgraph; (Section 4.1, para 1, “To learn a visual domain-invariant embedding space hinv, it is important to define a suitable distance measure between the visual embedding hvis and the knowledge graph embedding hKG. The measurement of similarity is backpropagated into the DNN and minimized through stochastic gradient descent (SGD)”…Page 7, para 1, “The most intuitive way to measure the distance between two vectors is the mean squared error loss (MSE) [specified cost function], also known as L2-loss, used in [26] to align a visual embedding space with a language representation.”)
optimizing parameters that characterize the behavior of the neural network, with a goal that the assessment by the cost function is expected to improve during further processing of training examples; and (Page 7, para 4, “For each anchor a, there can be many positives and negatives and the denominator has a total of 2N−1 terms, containing terms with positive index and terms with negative-index. During training, for any a, the encoder is tuned [optimizing parameters] to maximize the numerator of the log argument, while simultaneously minimizing its denominator. Nya is the total number of images in the minibatch that have the same label ya, as the anchor a, and hence is able to handle arbitrary numbers of positives belonging to the same class. During optimization of the loss function LKG [cost function], the DNN learns the projection z and the visual domain invariant embedding hinv of input samples with positive index xp to map the corresponding class label ya in the knowledge graph embedding vector hKG, a and simultaneously pushes the embedding of the input samples with negative index xn to non-neighboring representations.”)
adjusting the evaluation of the feature maps such that the output for each training example corresponds as well as possible to the respective target output for the respective training example (Section 5.4, para 1, “This linear layer enables the encoder network of the DNN to predict the classes (i.e., wherein adjusting the evaluation of the feature maps) of the dataset and is trained with the standard cross-entropy loss on the full target dataset”)
Monka does not explicitly teach:
selecting, from the generic knowledge graph, a subgraph relating to a context for solving the specified task;
Yang teaches:
selecting, from the generic knowledge graph, a subgraph relating to a context for solving the specified task; (Section 3.2, para 3, “whether the specific knowledge triplet is selected or not during the sampling. By doing so, we can generate knowledge subgraph with different augmented structural views. The objective of our knowledge graph augmentation scheme is to identify items which are less sensitive to structure variation, and more tolerant to the connections with noisy entities.”)
Yang and Monka are both related to the same field of endeavor (i.e., knowledge graphs). In view of the teachings of Yang it would have been obvious for a person of ordinary skill in the art to apply the teachings of Yang to Monka before the effective filing date of the claimed invention in order to improve the relevance of knowledge graphs (Yang, Abstract, “Knowledge Graphs (KGs) have been utilized as useful side information to improve recommendation quality. In those recommender systems, knowledge graph information often contains fruitful facts and inherent semantic relatedness among items. However, the success of such methods relies on the high quality knowledge graphs, and may not learn quality representations with two challenges: i) The long-tail distribution of entities results in sparse supervision signals for KG-enhanced item representation; ii) Real-world knowledge graphs are often noisy and contain topic-irrelevant connections between items and entities.”)
Regarding Claim 2:
Monka, as modified by Yang, teaches the method of claim 1.
Monka further teaches:
wherein the neural network additionally includes a task head configured to evaluate the respective feature maps with regard to the specified task, and (Page 9, para 4, “For inference, we add a randomly initialized linear fully connected layer to the trained encoder network (i.e., wherein under the broadest reasonable interpretation (BRI) task head is interpreted to be the final layer) so that it is enabled to classify input data.”…Section 5.4, para 1, “This linear layer enables the encoder network of the DNN to predict the classes of the dataset (i.e., wherein evaluate feature maps)”)
wherein the adjustment of the evaluation of the feature maps includes: assessing, using a cost function, how well the output of the task head for each training example corresponds to the target output for the respective training example; and (Page 9, para 4, “The size of the output vector depends on the number of classes of the task TT if performed on DT and TS if performed on DS. This linear layer is trained (i.e., wherein the adjustment is interpreted as the training phase) with the default cross-entropy loss [cost function]”)
optimizing parameters of the task head with regard to the assessment by the cost function (Page 9, para 4, “This linear layer is trained with the default cross-entropy loss [cost function]”…Section 4.1, para 1, “To learn a visual domain-invariant embedding space hinv, it is important to define a suitable distance measure between the visual embedding hvis and the knowledge graph embedding hKG. The measurement of similarity is backpropagated into the DNN and minimized through stochastic gradient descent (SGD)”)
The motivation for claim 2 is the same motivation as for claim 1.
Regarding Claim 3:
Monka, as modified by Yang, teaches the method of claim 1.
Monka further teaches:
wherein the similarity between each respective feature map and the representation of the subgraph is set in relation to the similarity between the representation of the subgraph and feature maps ascertained for other training examples with other respective target outputs (Section 4.1, “To learn a visual domain-invariant embedding space hinv, it is important to define a suitable distance measure between the visual embedding hvis and the knowledge graph embedding hKG”...Page 7, para 2, “The contrastive loss is a combination of the softmax function and the hinge-rank loss. It measures the similarity [similarity] of vectors by calculating their cosine distance in the high dimensional space”)
The motivation for claim 3 is the same motivation as for claim 1.
Regarding Claim 4:
Monka, as modified by Yang, teaches the method of claim 1.
Monka further teaches:
wherein the respective target outputs include classification scores with respect to one or more classes of a specified classification of the measurement data (Section 5.4, “This linear layer enables the encoder network of the DNN to predict the classes of the dataset (i.e., wherein the encoder generates classification score)”…Page 9, 4“The size of the output vector depends on the number of classes [one or more classes] of the task TT if performed on DT and TS if performed on DS”…Page 7, para 2, “The contrastive loss is a combination of the softmax function and the hinge rank loss (i.e., wherein under the broadest reasonable interpretation (BRI) classification score is interpreted as the softmax raw outputs scores)”)
The motivation for claim 8 is the same motivation as for claim 1.
Regarding Claim 9:
Monka, as modified by Yang, teaches the method of claim 1.
Monka does not explicitly teach:
wherein the subgraph relates to a visual or taxonomic or functional context
Yang further teaches:
wherein the subgraph relates to a visual or taxonomic or functional context (Section 3.2, para 3, “whether the specific knowledge triplet is selected or not during the sampling. By doing so, we can generate knowledge subgraph with different augmented structural views. The objective of our knowledge graph augmentation scheme is to identify items which are less sensitive to structure variation, and more tolerant to the connections with noisy entities.”)
The motivation for claim 9 is the same motivation as for claim 1.
Regarding Claim 10:
Monka, as modified by Yang, teaches the method of claim 1.
Monka further teaches:
with respect to the assessment by the cost function (Page 9, para 4, “This linear layer is trained with the default cross-entropy loss [cost function]”…Section 4.1, para 1, “To learn a visual domain-invariant embedding space hinv, it is important to define a suitable distance measure between the visual embedding hvis and the knowledge graph embedding hKG. The measurement of similarity is backpropagated into the DNN and minimized through stochastic gradient descent (SGD)”)
Monka does not explicitly teach:
wherein the selection of the subgraph is also included in the optimization
Yang further teaches:
wherein the selection of the subgraph is also included in the optimization (Section 3.2, para 3, “whether the specific knowledge triplet is selected or not during the sampling. By doing so, we can generate knowledge subgraph with different augmented structural views. The objective of our knowledge graph augmentation scheme is to identify items which are less sensitive to structure variation, and more tolerant to the connections with noisy entities.”)
The motivation for claim 10 is the same motivation as for claim 1.
Regarding Claim 12:
Monka, as modified by Yang, teaches the method of claim 1.
Monka further teaches:
wherein a further machine learning model configured to generate the representation of the subgraph in the space of the feature maps is trained together with the neural network. (Page 8, para 1, “During optimization of the loss function LKG, the DNN (i.e., wherein DNN is interpreted as machine learning ) learns the projection z and the visual domain invariant embedding hinv of input samples with positive index xp to map the corresponding class label ya in the knowledge graph embedding vector hKG,a and simultaneously pushes the embedding of the input samples with negative index xn to non-neighboring representations”)
The motivation for claim 12 is the same motivation as for claim 1.
Claim(s) 5-6, 8, 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Monka et al., in view of Yang et al., and further in view of Wickramarachchi et al., Non-Patent Literature (“An Evaluation of Knowledge Graph Embeddings for Autonomous Driving Data: Experience and Practice”).
Regarding Claim 5:
Monka, as modified by Yang, teaches the method of claim 4.
Monka, as modified by Yang, does not explicitly teach:
wherein classes of the specified classification represent types of objects whose presence in an area monitored during recording of the measurement data is indicated by the measurement data
Wickramarachchi teaches:
wherein classes of the specified classification represent types of objects whose presence in an area monitored during recording of the measurement data is indicated by the measurement data (Section 2, para 1, “To annotate the data from the datasets, a scene ontology was used (i.e., wherein under the broadest reasonable interpretation (BRI) an area monitored is interpreted as a scene)”...Abstract, “knowledge graphs (KGs) to manage the vast amount of heterogeneous data generated from vehicular sensors. The various types of equipped sensors include video, LIDAR and RADAR”…Section 2.2, para 1, “In the AD domain, a scene depicts a situation encountered by a vehicle (i.e., wherein an area monitored during recording under the broadest reasonable interpretation (BRI) is interpreted by a scene situation recorded by a vehicle equipped with sensors like video). A few examples may include a vehicle stopped at a traffic light, cruising on the highway, or crashing into another vehicle. The concept of scene acts as the polestar with which all information about the vehicle, and its situation, are integrated (i.e., wherein objects under the broadest reasonable interpretation is interpreted to be street signs, other vehicles, street objects such as poles etc.). More specifically, a scene may include information about time and location, the occurring events, and the participating objects [measurement data]”)
Wickramarachchi and Monka are both related to the same field of endeavor (i.e., knowledge graphs). In view of the teachings of Wickramarachchi it would have been obvious for a person of ordinary skill in the art to apply the teachings of Wickramarachchi to Monka before the effective filing date of the claimed invention in order to improve the embeddings and neural network training with relevance of knowledge graphs (Wickramarachchi, Abstract, “KG in formational detail, algorithms, and datasets– we show that (1) higher levels of informational detail in KGs lead to higher quality embeddings, (2) type and relation semantics are better captured”)
Regarding Claim 6:
Monka, as modified by Yang and Wickramarachchi, teaches the method of claim 5.
Monka, as modified by Yang, does not explicitly teach:
In this limitation the use of and/or makes the claim indefinite, and is interpreted as or.
wherein other vehicles, and/or traffic signs, and/or roadway markings, and/or traffic obstructions and/or other traffic-relevant objects in the vicinity of a vehicle are selected as types of objects. (Page 3, Col 1-2, “In the Base KG, objects and events are explicitly typed to the most specific class possible (i.e., wherein object is interpreted as other vehicles such as car). For example, an object instance representing a car is typed to the Car class. Because the Car class is a sub-class of Vehicle, then the instance is also a type of Vehicle.”)
The motivation for claim 6 is the same motivation as for claim 5.
Regarding Claim 8:
Monka, as modified by Yang, teaches the method of claim 1.
Monka, as modified by Yang, does not explicitly teach:
wherein images, audio signals, and/or time series of measured values, and/or radar data, and/or lidar data are selected as measurement data
Wickramarachchi further teaches:
In this limitation the use of and/or makes the claim indefinite, and is interpreted as or.
wherein images, audio signals, and/or time series of measured values, and/or radar data, and/or lidar data are selected as measurement data (Introduction, para 1, “To meet the increasing data demands of ML algorithms, fleets of vehicles are now deployed in multiple cities around the world and collecting massive amounts of data. These vehicles are equipped with various types of heterogeneous sensors, including but not limited to video, LIDAR, and RADAR. (i.e., wherein the measurement data can be radar data, lidar data etc. )”)
The motivation for claim 6 is the same motivation as for claim 5.
Regarding Claim 13:
Monka, as modified by Yang, teaches the method of claim 1.
Monka further teaches:
measurement data are supplied to the trained neural network so that the trained neural network generates outputs; (Page 6, para 1, “An encoder network E(·) is part of the DNN and maps im ages x to a visual embedding (space) hvis = E(x) ∈ RDE,”…Section 5.4, para 1, “This linear layer enables the encoder network of the DNN to predict the classes of the dataset”)
Monka, as modified by Yang, does not explicitly teach:
a control signal is formed from outputs of the neural network; and a vehicle and/or a driver assistance system and/or a quality control system and/or an area monitoring system and/or a medical imaging system, is controlled using the control signal.
Wickramarachchi further teaches:
a control signal is formed from outputs of the neural network; and (Page 5, para 2, “From this visualization, you can see that instances of events such as stopped car, moving car and parked car (i.e., wherein control signals under the broadest reasonable interpretation (BRI) is interpreted as a signal to control a car like stopped, moving, parked etc.) are clustered around the instances of car. The embeddings represented in this figure are generated from TransE on the “Base KG””)
a vehicle and/or a driver assistance system and/or a quality control system and/or an area monitoring system and/or a medical imaging system, is controlled using the control signal (Introduction, Col 2, para 3, “The generated KGs focus on representing the various scenes, or situations, that an autonomous vehicle encounters on the road. The purpose of creating KGs with varying degrees of detail is to enable an examination of the relationship between KG detail and the quality of derivative embeddings”… (Page 5, para 2, “From this visualization, you can see that instances of events such as stopped car, moving car and parked car (i.e., wherein control signals under the broadest reasonable interpretation (BRI) is interpreted as a signal to control a car like stopped, moving, parked etc.) are clustered around the instances of car. The embeddings represented in this figure are generated from TransE on the “Base KG””)
The motivation for claim 13 is the same motivation as for claim 5.
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Monka et al., in view of Yang et al., and Wickramarachchi et al., and further in view of Jadidi et al., Non-Patent Literature (“Gaussian Processes Semantic Map Representation”).
Regarding Claim 7:
Monka, as modified by Yang, teaches the method of claim 4.
Monka further teaches:
ascertaining respective feature maps for all training examples; and (Page 6, para 1, “An encoder network E(·) is part of the DNN and maps images x (i.e., wherein images is interpreted as training examples) to a visual embedding (space) [feature map] hvis = E(x) ∈ RDE,”)
Monka, as modified by Yang, does not explicitly teach:
wherein the evaluating of the respective feature maps includes assigning the respective feature maps to classes using a Gaussian process, and the adjustment of the evaluation includes:
defining decision limits between classes in the space of the respective feature maps based on the respective target outputs.
Jadidi teaches:
wherein the evaluating of the respective feature maps includes assigning the respective feature maps to classes using a Gaussian process, and the adjustment of the evaluation includes: (Page 3, Problem 1, “(Gaussian processes semantic map). Given a point cloud measurement that is (possibly partially) assigned with noisy semantic class labels, infer a semantic map representation [feature map] of the point cloud as a Gaussian process [using a Gaussian process]”)
defining decision limits between classes in the space of the respective feature maps based on the respective target outputs (Page 4, Col 1, para 6, “the prediction at query points (map points) are performed, we normalize the class probabilities to get p(M = c[j]|z) for the j-th semantic class. One straightforward way to assign hard labels [target outputs] to map points is to find the class with the maximum probability.”)
Jadidi and Monka are both related to the same field of endeavor (i.e., knowledge graphs). In view of the teachings of Jadidi it would have been obvious for a person of ordinary skill in the art to apply the teachings of Jadidi to Monka before the effective filing date of the claimed invention in order to improve the embeddings and neural network training with relevance of knowledge graphs using a Gaussian process (Jadidi , Abstract, “The proposed technique uses Gaussian Processes (GPs) multi-class classification for map inference and is the natural extension of GP occupancy maps from binary to multi-class form. The technique exploits the continuous property of GPs and, as a result, the map can be inferred with any resolution. In addition, the proposed GP Semantic Map (GPSM) learns the structural and semantic correlation from measurements rather than resorting to assumptions, and can flexibly learn the spatial correlation as well as any additional non spatial correlation between map points.”)
Claim(s) 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Monka et al., in view of Yang et al., further in view of Song et al., Non-Patent Literature (“Interactive Visual Pattern Search on Graph Data via Graph Representation Learning”).
Regarding Claim 11:
Monka, as modified by Yang, teaches the method of claim 1.
Monka does not explicitly teach:
wherein the representation of the subgraph in the space of the respective feature maps is retrieved from a pre-calculated lookup table based on each training example and the respective target output
Song teaches:
wherein the representation of the subgraph in the space of the respective feature maps is retrieved from a pre-calculated lookup table based on each training example and the respective target output (Page 2, Col 1, para 2, “In most application scenarios we can precompute and store the vector representations of the query targets for efficient retrieval of the graph matching results (i.e., wherein under the broadest reasonable interpretation (BRI) precomputed is interpreted as pre-calculated lookup table)”…Abstract, “we use graph neural networks (GNNs) to encode a graph as fixed-length latent vector representation, and perform subgraph matching in the latent space.”)
Song and Monka are both related to the same field of endeavor (i.e., knowledge graphs). In view of the teachings of Song it would have been obvious for a person of ordinary skill in the art to apply the teachings of Song to Monka before the effective filing date of the claimed invention in order to improve the embeddings and neural network training with relevance of knowledge graphs using pre-calculated lookup (Song, Page 2, Col 1, para 2, “we can precompute and store the vector representations of the query targets for efficient retrieval of the graph matching results. The visualization interface enables easy search and specification of the graph query patterns. Since the query engine could return a large number of matched graphs, we present the results with different levels-of-details that show the matched graphs in space-efficient,”)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMINA BENOURAIDA whose telephone number is (571)272-4340. The examiner can normally be reached Monday-Friday 8:30am-5pm ET..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J. Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMINA MORENO BENOURAIDA/Examiner, Art Unit 2129 /MICHELLE T BECHTOLD/Supervisory Patent Examiner, Art Unit 2148
Prosecution Insights