CTNF 18/882,563 CTNF 101775 Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. DETAILED ACTION The United States Patent & Trademark Office appreciates the application that is submitted by the inventor/assignee. The United States Patent & Trademark Office reviewed the following application and has made the following comments below. Priority This application claims benefit of priority to U.S. provisional application 63/559,145, filed 02/28/2024 . Information Disclosure Statement The information disclosure statement (IDS) submitted on 12/10/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20 AIA The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action: (a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made. 07-23 AIA The factual inquiries set forth in Graham v. John Deere Co. , 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-21 AIA Claim s 1, 8-11, 17 and 20 are rejected under 35 U.S.C. 103(a) as being unpatentable over Honkala et al. (U.S. Patent Pub. No. 2019/0012581, hereafter referred to as Honkala) in view of Komkov et al. (U.S. Patent Pub No. US 2023/0229897, hereafter referred to as Komkov) . Regarding Claim 1, Honkala teaches a computer-implemented method (Paragraph [0098], Honkala teaches a method implemented with the help of computer program code that resides in a memory and causes the relevant apparatuses to carry out the invention.) for classifying data (Paragraphs [0040], [0044], Honkala teaches classifying input data into one of N possible classes (e.g., classifying if an image contains a cat or a dog). The classification layer classifies the input image into one or more classes according to extracted content. Classes are output from the Convolutional Neural Network (CNN).) , the method comprising: processing data via a trained machine learning model that includes a plurality of layers (Paragraph [0074], Fig. 5, Honkala teaches a pre-trained classification neural network (510) with multiple layers. The pre-trained CNN is used for extracting feature maps from multiple layers. As shown in Fig. 5, the pre-trained CNN receives real samples (520) and generated samples (530).) , PNG media_image1.png 470 763 media_image1.png Greyscale wherein each layer generates one or more corresponding features (Paragraphs [0044], [0074], Fig. 2b, Honkala teaches the convolutional layers (210, 220, 230) perform convolution operations on the image with weights shared spatially. Lower layers (210, 220) extract semantically low-level features, such as edges and textures, from the image. Higher-level layers (230) extract semantically high-level features, such as shapes, objects, scene (e.g., chair, person, room), from the image. The Examiner interprets “each layer” to be each feature extraction layer of the CNN. Lower layers (210, 220) extract low-level features, higher level layers (230) extract high-level features.) ; PNG media_image2.png 251 802 media_image2.png Greyscale generating a first distribution of features based on the one or more corresponding features generated by each layer included in the plurality of layers (Paragraphs [0074], [0063], Fig. 5, Honkala teaches generating distributions (542, 543) of the features/feature maps extracted from intermediate layers (feature extraction layers) of the pre-trained classification network. For each feature map of the pre-trained classification network (510), statistics over all samples are computed, separately for generated samples (542) and real samples (543). For each feature map generated by the feature extraction layers, a distance of distributions is computed. The Examiner interprets “each layer included in the plurality of layers” to mean each feature extraction layer included in the plurality of layers of the CNN.) ; PNG media_image3.png 470 755 media_image3.png Greyscale Honkala does not explicitly disclose determining a first class for the data based on a comparison of the first distribution of features with one or more predefined distributions of features that are associated with one or more classes. Komkov is in the same field of art of processing input data by a neural network to determine a characteristic of the input data such as classification/class. Further, Komkov teaches determining a first class for the data (Paragraph [0088], Komkov teaches determining a characteristic of the first input data. The determining characteristic may be a classification, e.g., a determination of a class among a closed or open number of classes.) based on a comparison of the first distribution of features with one or more predefined distributions of features that are associated with one or more classes (Paragraphs [0079-81], [0077], Fig. 3A, Komkov teaches the neural network may output similarities to some predefined classes. By comparing the input data tensor distribution with representative class tensor distributions, a class with the highest degree of compliance can be obtained. Determination of a classification of the first input data may be based on the determined distance value between distributions.) . PNG media_image4.png 433 317 media_image4.png Greyscale Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Honkala by determining the classification of the data based on a comparison between the input data’s tensor distribution and a representative tensor distribution for a specific class that is taught by Komkov, to make the invention that obtains the class/classification with the highest degree of compliance based on a distance value between the two distributions; thus, one of ordinary skilled in the art would be motivated to combine the references since the tensor distribution of the input data can be used to improve classification and to determine, for the input data, whether the input data diverges from a predetermined distribution which can be helpful for the uncertainty estimation or for the confidence estimation (Komkov, Paragraph [0081]). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. In regards to Claim 8, Honkala in view of Komkov teaches the computer-implemented method of claim 1, further comprising generating the one or more predefined distributions of features when training an untrained machine learning model (Paragraphs [0080-81], [0202], Komkov teaches predetermined distributions (e.g., average of distributions of training-set data or previous production input data the like) or previously formed classes.) in order to produce the trained machine learning model (Paragraphs [0204], [0196-197], [0202], Komkov teaches an architecture of a neural network for a closed-set image recognition. The determining of the characteristic of the input data (e.g., input image) may correspond to determining of a class among a plurality of predetermined classes of data. This may be for a closed-set classification. The determining of the characteristic of the first input data may correspond to determining whether or not the first input data the first input data belongs to one of the predetermined classes of data.). In regards to Claim 9, Honkala in view of Komkov teaches the computer-implemented method of claim 1, wherein the trained machine learning model comprises a classifier neural network (Paragraph [0063], Honkala teaches a pre-trained neural network C can be a classification network such as an Inception model.) . In regards to Claim 10, Honkala in view of Komkov teaches the computer-implemented method of claim 1, wherein the data comprises image data (Abstract, Fig. 2b, Honkala teaches receiving a set of input images (i.e., real images and generated images).) . In regards to Claim 11, Honkala teaches one or more non-transitory computer-readable media storing instructions that, when executed by at least one processor, cause the at least one processor to perform the steps of (Paragraphs [0009-10], Honkala teaches a non-transitory computer readable medium, comprising computer program code configured to, when executed on at least one processor, cause an apparatus or a system to perform a method.) : processing data via a trained machine learning model that includes a plurality of layers (Paragraph [0074], Fig. 5, Honkala teaches a pre-trained classification neural network (510) with multiple layers. The pre-trained CNN is used for extracting feature maps from multiple layers. As shown in Fig. 5, the pre-trained CNN receives real image samples (520) and generated image samples (530).) , wherein each layer generates one or more corresponding features (Paragraphs [0044], [0074], Fig. 2b, Honkala teaches the convolutional layers (210, 220, 230) perform convolution operations on the image with weights shared spatially. Lower layers (210, 220) extract semantically low-level features, such as edges and textures, from the image. Higher-level layers (230) extract semantically high-level features, such as shapes, objects, scene (e.g., chair, person, room), from the image. The Examiner interprets “each layer” to be each feature extraction layer of the CNN. Lower layers (210, 220) extract low-level features, higher level layers (230) extract high-level features.) ; generating a first distribution of features based on the one or more corresponding features generated by each layer included in the plurality of layers (Paragraphs [0074], [0063], Fig. 5, Honkala teaches generating distributions (542, 543) of the features/feature maps extracted from intermediate layers of the pre-trained classification network. For each feature map of the pre-trained classification network (510), statistics over all samples are computed, separately for generated samples (542) and real samples (543). For each feature map, a distance of distributions is computed. The Examiner interprets “each layer included in the plurality of layers” to mean each feature extraction layer included in the plurality of layers of the CNN.) ; Honkala does not explicitly disclose determining a first class for the data based on a comparison of the first distribution of features with one or more predefined distributions of features that are associated with one or more classes. Komkov is in the same field of art of processing input image data using a neural network to determine a characteristic of the input data such as a certain class/classification. Further, Komkov teaches determining a first class for the data (Paragraph [0088], Komkov teaches determining a characteristic of the first input data. The determining characteristic may be a classification, e.g., a determination of a class among a closed or open number of classes.) based on a comparison of the first distribution of features with one or more predefined distributions of features that are associated with one or more classes (Paragraphs [0079], [0081], [0077], Fig. 3A, Komkov teaches comparing the input data tensor distribution with representative class tensor distributions and obtaining a class with the highest degree of compliance. Determination of a classification of the first input data may be based on the determined distance value.) . Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Honkala by determining the classification of the data based on a comparison between the input data’s tensor distribution and a representative tensor distribution for a specific class that is taught by Komkov, to make the invention that obtains the class with the highest degree of compliance based on the distance between the two distributions; thus, one of ordinary skilled in the art would be motivated to combine the references since the tensor distribution of the input data can be used to improve classification and to determine, for the input data, whether the input data diverges from a predetermined distribution which can be helpful for the uncertainty estimation or for the confidence estimation (Komkov, Paragraph [0081]). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. In regards to Claim 17, Honkala in view of Komkov teaches the one or more non-transitory computer-readable media of claim 11, wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform the step of (Paragraph [0009], Honkala teaches a processor which executes computer program code.) generating the one or more predefined distributions of features (Paragraphs [0081], [0202], Komkov teaches predetermined distributions (e.g., average of distributions of training-set data or previous production input data the like).) when training an untrained machine learning model in order to produce the trained machine learning model (Paragraphs [0204], [0196-197], [0202], Komkov teaches an architecture of a neural network for a closed-set image recognition. The determining of the characteristic of the input data (e.g., input image) may correspond to determining of a class among a plurality of predetermined classes of data. This may be for a closed-set classification. The determining of the characteristic of the first input data may correspond to determining whether or not the first input data the first input data belongs to one of the predetermined classes of data.) . In regards to Claim 20, Honkala discloses a system (Paragraph [0008], Honkala teaches an apparatus.) , comprising: one or more memories storing instructions (Paragraph [0008], Fig. 1, Honkala teaches an apparatus comprising memory including program code.) ; and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to (Paragraph [0008], Fig. 1, Honkala teaches an apparatus comprising at least one processor, the memory and the computer program code, configured to, with the at least one processor, cause the apparatus to perform the method.) : process data via a trained machine learning model that includes a plurality of layers (Paragraph [0074], Fig. 5, Honkala teaches a pre-trained classification neural network (510) with multiple layers. The pre-trained CNN is used for extracting feature maps from multiple layers. As shown in Fig. 5, the pre-trained CNN receives real sample images (520) and generated sample images (530).) , wherein each layer generates one or more corresponding features (Paragraphs [0044], [0074], Fig. 2b, Honkala teaches the convolutional layers (210, 220, 230) perform convolution operations on the image with weights shared spatially. Lower layers (210, 220) extract semantically low-level features, such as edges and textures, from the image. Higher-level layers (230) extract semantically high-level features, such as shapes, objects, scene (e.g., chair, person, room), from the image. The Examiner interprets “each layer” to be each feature extraction layer of the CNN. Lower layers (210, 220) extract low-level features, higher level layers (230) extract high-level features.) , generate a first distribution of features based on the one or more corresponding features generated by each layer included in the plurality of layers (Paragraphs [0074], [0063], Fig. 5, Honkala teaches generating distributions (542, 543) of the features/feature maps extracted from intermediate layers of the pre-trained classification network. For each feature map of the pre-trained classification network (510), statistics over all samples are computed, separately for generated samples (542) and real samples (543). The Examiner interprets “each layer included in the plurality of layers” to mean each feature extraction layer included in the plurality of layers of the CNN.) , Honkala does not explicitly disclose determin e (ing) a first class for the data based on a comparison of the first distribution of features with one or more predefined distributions of features that are associated with one or more classes. Komkov is in the same field of art of processing input data by a neural network to determine a characteristic of the input data such as a certain class. Further, Komkov teaches determin e (ing) a first class for the data (Paragraph [0088], Komkov teaches determining a characteristic of the first input data. The determining characteristic may be a classification, e.g., a determination of a class among a closed or open number of classes.) based on a comparison of the first distribution of features with one or more predefined distributions of features that are associated with one or more classes (Paragraphs [0079], [0081], [0077], Komkov teaches comparing the input data tensor distribution with representative class tensor distributions and obtaining a class with the highest degree of compliance. Determination of a classification of the first input data may be based on the determined distance value.) . Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Honkala by determining the classification of the data based on a comparison between the input (image) data’s tensor distribution and a representative tensor distribution for a specific class that is taught by Komkov, to make the invention that obtains the class with the highest degree of compliance based on the distance between the two distributions; thus, one of ordinary skilled in the art would be motivated to combine the references since the tensor distribution of the input data can be used to improve classification and to determine, for the input data, whether the input data diverges from a predetermined distribution which can be helpful for the uncertainty estimation or for the confidence estimation (Komkov, Paragraph [0081]). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention . 07-21 AIA Claim s 2, 4, 12 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over Honkala et al. (U.S. Patent Pub. No. 2019/0012581, hereafter referred to as Honkala) in view of Komkov et al. (U.S. Patent Pub No. US 2023/0229897, hereafter referred to as Komkov) in further view of Bäuerle et al. (NPL “Neural Activation Patterns (NAPs): Visual Explainability of Learned Concepts,” 2022, hereafter referred to as Bäuerle) . Regarding Claim 2, Honkala in view of Komkov teaches the computer-implemented method of claim 1. Honkala in view of Komkov does not explicitly disclose wherein generating the first distribution of features comprises scanning the data to determine an amount of activation of each layer included in the plurality of layers that corresponds to each concept prototype included in a plurality of concept prototypes. Bäuerle is in the same field of art of discovering learned features of a neural network based on analyzing activation values for each layer. Further, Bäuerle teaches wherein generating the first distribution of features comprises scanning the data to determine an amount of activation of each layer included in the plurality of layers (3 Neural Activation Patterns, 3.1 Activation Extraction, Fig. 3, Bäuerle teaches each layer l in a neural network produces an activation vector based on the input it processes.) that corresponds to each concept prototype included in a plurality of concept prototypes ( 3 Neural Activation Patterns, 3.1 Activation Extraction, Fig. 5c, Bäuerle teaches each unit of the activation vector can be thought of as describing a feature the layer tries to represent. The combined activation of different units may represent an interpretable concept a layer has learned. Interpretable concepts can be low-level, e.g., stripes, but low-level concepts can be combined to form higher-level concepts. For example, the features animal, legs, and stripes may form zebra concept. The concept shown in Fig. 5c enables a user to follow images through different layers of the model. The Examiner interprets “interpretable concepts” to be synonymous to “concept prototypes” in light of Applicant’s specification, paragraph [0070], which states “the concept prototypes…for the first layer represent colors, whereas the concept prototypes for the deeper layers… represent objects such as stripes and dots.”) . PNG media_image5.png 560 1406 media_image5.png Greyscale PNG media_image6.png 718 511 media_image6.png Greyscale Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Honkala in view of Komkov by analyzing activation values of all layers of the neural network during an image classification task that is taught by Bäuerle, to make the invention that enables users to investigate how the models understanding of an image type is built throughout successive layers; thus, one of ordinary skilled in the art would be motivated to combine the references since visualization of neural network activations is an important aspect in explainability approaches and by capturing the activation output of each layer, users can understand how different layers of a neural network model process data. Additionally, activation analysis on a layer-level can lead to many different types of insights related to the ML model such as insufficient model performance or bias in the training data (Bäuerle, 8 Conclusion). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. In regards to Claim 4, Honkala in view of Komkov in further view of Bäuerle teaches the computer-implemented method of claim 2, wherein the plurality of concept prototypes includes at least one concept prototype for each layer included in the plurality of layers (Paragraphs [0034], [0038], Honkala teaches each learned layer extracts feature representations from the input data, where features from lower layers represent low-level semantics (i.e., more abstract concepts). The first convolution layer C1 of the CNN consists of extracting 4 feature maps from the first layer (i.e., from the input image). These maps represent low-level features found in the input image, such as edges and corners. The second convolution layer C2 of the CNN, consisting of extracting 6 feature maps from the previous layer, increases the semantic level of extracted features. The third convolution layer C3 may represent more abstract concepts found in images, such as edges and corners, shapes, etc. Under Broadest Reasonable Interpretation (BRI), the Examiner interprets the extracted features such as edges, corners, shapes, etc., to be “concept prototypes” in light of Applicant’s specification, see paragraph [0070], which states, “the concept prototypes for the first layer represent colors, whereas the concept prototypes for the deeper layers…represent objects such as stripes and dots.”) . In regards to Claim 12, Honkala in view of Komkov discloses the one or more non-transitory computer-readable media of claim 11. Honkala in view of Komkov does not explicitly disclose wherein generating the first distribution of features comprises scanning the data to determine an amount of activation of each layer included in the plurality of layers that corresponds to each concept prototype included in a plurality of concept prototypes. Bäuerle is in the same field of art of discovering learned features of a neural network based on analyzing activation values for each layer. Further, Bäuerle teaches wherein generating the first distribution of features comprises scanning the data to determine an amount of activation of each layer included in the plurality of layers (3 Neural Activation Patterns, 3.1 Activation Extraction, Bäuerle teaches each layer l in a neural network produces an activation vector based on the input it processes.) that corresponds to each concept prototype included in a plurality of concept prototypes ( Bäuerle teaches each unit of the activation vector can be thought of as describing a feature the layer tries to represent. The combined activation of different units may represent an interpretable concept a layer has learned. Interpretable concepts can be low-level, e.g., stripes, but low-level concepts can be combined to form higher-level concepts. For example, the features animal, legs, and stripes may form zebra concept. The concept shown in Fig. 5c enables a user to follow images through different layers of the model.) . Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Honkala in view of Komkov by analyzing activation values of all layers of the neural network during an image classification task that is taught by Bäuerle, to make the invention that enables users to investigate how the models understanding of an image type is built throughout successive layers; thus, one of ordinary skilled in the art would be motivated to combine the references since visualization of neural network activations is an important aspect in explainability approaches and by capturing the activation output of each layer, users can understand how different layers of a neural network model process data. Additionally, activation analysis on a layer-level can lead to many different types of insights related to the ML model such as insufficient model performance or bias in the training data (Bäuerle, 8 Conclusion). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. In regards to Claim 14, Honkala in view of Komkov in further view of Bäuerle discloses the one or more non-transitory computer-readable media of claim 12, wherein the plurality of concept prototypes includes at least one concept prototype for each layer included in the plurality of layers (Paragraphs [0034], [0038], Honkala teaches each learned layer extracts feature representations from the input data, where features from lower layers represent low-level semantics (i.e., more abstract concepts). The first convolution layer C1 of the CNN consists of extracting 4 feature maps from the first layer (i.e., from the input image). These maps represent low-level features found in the input image, such as edges and corners. The second convolution layer C2 of the CNN, consisting of extracting 6 feature maps from the previous layer, increases the semantic level of extracted features. The third convolution layer C3 may represent more abstract concepts found in images, such as edges and corners, shapes, etc. Under Broadest Reasonable Interpretation (BRI), the Examiner interprets the extracted features such as edges, corners, shapes, etc., to be “concept prototypes” in light of Applicant’s specification, see paragraph [0070], which states, “the concept prototypes for the first layer represent colors, whereas the concept prototypes for the deeper layers…represent objects such as stripes and dots.”) . 07-21 AIA Claim s 3 and 13 are rejected under 35 U.S.C. 103(a) as being unpatentable over Honkala et al. (U.S. Patent Pub. No. 2019/0012581, hereafter referred to as Honkala) in view of Komkov et al. (U.S. Patent Pub No. US 2023/0229897, hereafter referred to as Komkov) in further view of Bäuerle et al. (NPL “Neural Activation Patterns (NAPs): Visual Explainability of Learned Concepts,” 2022, hereafter referred to as Bäuerle) in further view of Ukai et al. (U.S. Patent Pub. No. 2024/0104914, hereafter referred to as Ukai) . Regarding Claim 3, Honkala in view of Komkov in further view of Bäuerle teaches the computer-implemented method of claim 2. Honkala in view of Komkov in further view of Bäuerle does not explicitly disclose performing one or more operations to generate the plurality of concept prototypes when training a first machine learning model to generate the trained machine learning model. Ukai is in the same field of art of identifying and classifying targets/objects appearing in captured images using a learning model. Further, Ukai teaches performing one or more operations to generate the plurality of concept prototypes when training a first machine learning model to generate the trained machine learning model (Paragraphs [0009], [0091-92], [0199], Ukai teaches generating a plurality of prototype vectors, each being a parameter sequence that is trained as a prototype that indicates a candidate for a specific image feature concept configured by the plurality of channels. The method of producing a learning model includes obtaining a belonged prototype and prototype belongingness in accordance with a plurality of images for learning each of the plurality of classes that are labeled to the plurality of images for learning. The Examiner interprets “one or more operations” are performed to generate the prototype vectors since each prototype vector is a vector that is trained to indicate a specific image feature of a specific image (i.e., the vectors themselves are “trained” to indicate a specific image feature.) The Examiner interprets performing “training” on a vector to be an “operation” under BRI.). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Honkala in view of Komkov in further view of Bäuerle by generating “prototype vectors” that indicate a candidate for a specific image feature concept and subsequently bringing the “prototypes” closer to image features (pixel vectors) at each planar position in a feature map that is taught by Ukai, to make the invention that improves the explainability of the model’s decision-making basis and further improves the explainability of inference results using concepts understandable by humans; thus, one of ordinary skilled in the art would be motivated to combine the references since by making an inference using a learning model obtained by such machine learning, it is possible to show the model’s decision-making basis indicating that an inference target is similar to a partial region at a specific position in an image, thus improving transparency (Ukai, Paragraph [0007]). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. In regards to Claim 13, Honkala in view of Komkov in further view of Bäuerle teaches the one or more non-transitory computer-readable media of claim 12. Honkala in view of Komkov in further view of Bäuerle does not explicitly disclose wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform the step of performing one or more operations to generate the plurality of concept prototypes when training a first machine learning model to generate the trained machine learning model. Ukai is in the same field of art of identifying and classifying targets appearing in captured images using a learning model. Further, Ukai teaches wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform the step of (Paragraph [0060], Ukai teaches a controller configured as a computer system that may include one or more processors (e.g., CPU, GPU). The controller implements various types of processing by causing, for example, the CPU to execute predetermined software programs stored in storage.) performing one or more operations to generate the plurality of concept prototypes when training a first machine learning model to generate the trained machine learning model (Paragraphs [0009], [0091-92], [0199], Ukai teaches generating a plurality of prototype vectors, each being a parameter sequence that is trained as a prototype that indicates a candidate for a specific image feature concept configured by the plurality of channels. The method of producing a learning model includes obtaining a belonged prototype and prototype belongingness in accordance with a plurality of images for learning each of the plurality of classes that are labeled to the plurality of images for learning. The Examiner interprets “one or more operations” are performed to generate the prototype vectors since each prototype vector is a vector that is trained to indicate a specific image feature of a specific image (i.e., the vectors themselves are “trained” to indicate a specific image feature.) The Examiner interprets performing “training” on a vector to be an “operation” under BRI.). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Honkala in view of Komkov in further view of Bäuerle by generating “prototype vectors” that indicate a candidate for a specific image feature concept and bringing the “prototypes” closer to image features (pixel vectors) at each planar position in a feature map that is taught by Ukai, to make the invention that improves explainability of decision-making basis and improving the explainability of inference results using concepts understandable by humans; thus, one of ordinary skilled in the art would be motivated to combine the references since by making an inference using a learning model obtained by such machine learning, it is possible to show the model’s decision-making basis indicating that an inference target is similar to a partial region at a specific position in an image, thus improving transparency (Ukai, Paragraph [0007]). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention . 07-21 AIA Claim s 5 and 15 are rejected under 35 U.S.C. 103(a) as being unpatentable over Honkala et al. (U.S. Patent Pub. No. 2019/0012581, hereafter referred to as Honkala) in view of Komkov et al. (U.S. Patent Pub No. US 2023/0229897, hereafter referred to as Komkov) in further view of Bäuerle et al. (NPL “Neural Activation Patterns (NAPs): Visual Explainability of Learned Concepts,” 2022, hereafter referred to as Bäuerle) in further view of ter Haar Romenij et al. (U.S. Patent No. 10,713,563, hereafter referred to as ter Haar Romenij) . Regarding Claim 5, Honkala in view of Komkov in further view of Bäuerle disclose the computer-implemented method of claim 2. Honkala in view of Komkov in further view of Bäuerle does not explicitly disclose generating the plurality of concept prototypes by performing one or more principal component analysis operations on a plurality of segments of one or more feature maps generated by the plurality of layers. ter Haar Romenij is in the same field of art of performing object classification using a trained CNN. Further, ter Haar Romenij teaches generating the plurality of concept prototypes by performing one or more principal component analysis operations (Col.1, lines 61-67, Col. 2, lines 16-25, ter Haar Romenij teaches the multi-layered deep cascade of neural networks is feed-forward trained by linear principal component analysis (PCA) on patches of training images. The output filters of this PCA are used to convolve the next layer, giving rise to a new set of more complex descriptive features, The process is repeated in increasingly larger contextual layers. For example, the first stage features may represent edges and corners, the second layer features may represent simple shapes, the third layer features may represent more complex shapes, and higher layer features may represent objects of different general types, such as particular kinds of objects. Under BRI, the Examiner interprets extracted features such as edges, corners, simple shapes, complex shapes, etc. to be “concept prototypes” in light of Applicant’s specification, see paragraph [0070], which states, “the concept prototypes … represent different scale meanings in the different layers. Illustratively, the concept prototypes … for the first layer represent colors, whereas the concept prototypes for the deeper layers, … represent objects such as stripes and dots.”) on a plurality of segments of one or more feature maps generated by the plurality of layers (Col. 1, lines 63-67 and Col. 2, lines 1-29, ter Haar Romenij teaches reducing the number of feature vectors by using leading eigenvalues of the principal component analysis (PCA) to select a subset of eigenvectors. Under BRI, the Examiner interprets “eigenvectors” to be “concept prototypes” in light of Applicant’s specification, see paragraphs [0057-58], which states, “the resultant principal direction (i.e., the eigenvector corresponding to the largest eigenvalue… built upon… feature vectors… is referred to herein as the concept prototype.”). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Honkala in view of Komkov in further view of Bäuerle by performing PCA on the feature vectors of the feature maps extracted from the layers of the neural network that is taught by ter Haar Romenij, to make the invention that identifies the feature vectors that represent a majority of the input image while reducing computational complexity; thus, one of ordinary skilled in the art would be motivated to combine the references since PCA, which aims to reduce computational expense by selecting eigenvectors that corresponds to eigenvalues larger than a threshold and represent a majority of the data, is far more accurate and efficient than existing pooling techniques (ter Haar Romenij, Col. 1, lines 61-67 and Col. 2, lines 1-7). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. In regards to Claim 15, Honkala in view of Komkov in further view of Bäuerle disclose the one or more non-transitory computer-readable media of claim 12, wherein the instructions, when executed by the at least one processor further caus e (ing) the at least one processor to perform the step of (Paragraph [0010], Honkala teaches a processor for executing computer program code to cause an apparatus or system to perform a method.) . Honkala in view of Komkov in further view of Bäuerle does not explicitly disclose generating the plurality of concept prototypes by performing one or more principal component analysis operations on a plurality of segments of one or more feature maps generated by the plurality of layers. ter Haar Romenij is in the same field of art of performing object classification using a trained CNN. Further, ter Haar Romenij teaches generating the plurality of concept prototypes by performing one or more principal component analysis operations (Col.1, lines 61-67, Col. 2, lines 16-25, ter Haar Romenij teaches the multi-layered deep cascade of neural networks is feed-forward trained by linear principal component analysis (PCA) on patches of training images. The output filters of this PCA are used to convolve the next layer, giving rise to a new set of more complex descriptive features, The process is repeated in increasingly larger contextual layers. For example, the first stage features may represent edges and corners, the second layer features may represent simple shapes, the third layer features may represent more complex shapes, and higher layer features may represent objects of different general types, such as particular kinds of objects. Under BRI, the Examiner interprets extracted features such as edges, corners, simple shapes, complex shapes, etc. to be “concept prototypes” in light of Applicant’s specification, see paragraph [0070], which states, “the concept prototypes … represent different scale meanings in the different layers. Illustratively, the concept prototypes … for the first layer represent colors, whereas the concept prototypes for the deeper layers, … represent objects such as stripes and dots.”) on a plurality of segments of one or more feature maps generated by the plurality of layers (Col. 1, lines 63-67 and Col. 2, lines 1-29, ter Haar Romenij teaches reducing the number of feature vectors by using leading eigenvalues of the principal component analysis (PCA) to select a subset of eigenvectors. Under BRI, the Examiner interprets “eigenvectors” to be “concept prototypes” in light of Applicant’s specification, see paragraphs [0057-58], which states, “the resultant principal direction (i.e., the eigenvector corresponding to the largest eigenvalue… built upon… feature vectors… is referred to herein as the concept prototype.”). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Honkala in view of Komkov in further view of Bäuerle by performing PCA on the feature vectors of the feature maps extracted from the layers of the neural network that is taught by ter Haar Romenij, to make the invention that identifies the feature vectors that represent a majority of the input image while reducing computational complexity; thus, one of ordinary skilled in the art would be motivated to combine the references since PCA, which aims to reduce computational expense by selecting eigenvectors that corresponds to eigenvalues larger than a threshold and represent a majority of the data, is far more accurate and efficient than existing pooling techniques (ter Haar Romenij, Col. 1, lines 61-67 and Col. 2, lines 1-7). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention . 07-21 AIA Claim s 6, 7, 16, 18 and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable over Honkala et al. (U.S. Patent Pub. No. 2019/0012581, hereafter referred to as Honkala) in view of Komkov et al. (U.S. Patent Pub No. US 2023/0229897, hereafter referred to as Komkov) in further view of Ukai et al. (U.S. Patent Pub. No. 2024,0104914, hereafter referred to as Ukai) . Regarding Claim 6, Honkala in view of Komkov teaches the computer-implemented method of claim 1. Honkala in view of Komkov does not explicitly disclose performing one or more operations to train an untrained machine learning model to generate the trained machine learning model using a loss that is computed based on a plurality of segments of one or more feature maps generated by the untrained machine learning model. Ukai is in the same field of art of identifying and classifying targets appearing in captured images using a learning model. Further, Ukai teaches performing one or more operations to train an untrained machine learning model to generate the trained machine learning model (Paragraphs [0112], [0053], Ukai teaches the learning model is trained to optimize (minimize) an evaluation function such as triplet loss. Accordingly, the learning model is trained such that the similarity of the input image in the input space corresponds to a distance in the feature space. Thus, the positions of distribution of feature vectors in the feature space gradually change with the progress of learning. Through machine learning/training, the trained learning model is generated.) using a loss that is computed based on a plurality of segments of one or more feature maps generated by the untrained machine learning model (Paragraphs [0114], [0130], [0138-139], [0103], Fig. 6, Fig. 9, Ukai teaches the learning model is subjected to machine learning to minimize the whole loss function (L) having three types of evaluation terms (Ltask, Lclst, and Laux). The loss function L may be expressed as a linear sum (linear combination) of the three types of evaluation terms. An evaluation term Lclst is the evaluation term for bringing each prototype vector p closer to the image feature (any pixel vector q) of a partial region of any image. Specifically, the evaluation term Lclst for bringing the prototype vector pk of the belonged prototype PTk for each image i closer to any pixel vector q in the feature map (230) corresponding to the image i. Each prototype vector pk may be brought closer to two or more-pixel vectors q existing in different images. The Examiner interprets “pixel vectors” to be “segments” of feature maps since the pixel vector is a vector that represents an image feature across a plurality of channels at each planar position of each pixel in the feature map. See Fig. 6 below.). PNG media_image7.png 541 778 media_image7.png Greyscale Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Honkala in view of Komkov by optimizing a loss function to train the learning model by bringing each prototype vector p closer to the image feature (pixel vector q) of a partial region of an image that is taught by Ukai, to make the invention that trains each parameter in the learning model (prototype vector p and each parameter relating to the learning model); thus, one of ordinary skilled in the art would be motivated to combine the references since operations based on prototype vectors p, such as extraction of image features and explanation of inference basis become possible in processing such as similar image retrieval processing relating to unclassified images. Additionally, it is possible to improve explainability of inference basis to improve transparency (Ukai, Paragraph [0207]). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. In regards to Claim 7, Honkala in view of Komkov teaches the computer-implemented method of claim 1. Honkala in view of Komkov does not explicitly disclose performing one or more operations to train an untrained machine learning model to generate the trained machine learning model using a loss that increases distances between distributions of features associated with different classes and decreases distances between distributions of features associated with a same class. Ukai is in the same field of art of identifying and classifying targets appearing in captured images using a learning model. Further, Ukai teaches performing one or more operations to train an untrained machine learning model to generate the trained machine learning model (Paragraphs [0112], [0053], Ukai teaches the learning model is trained to optimize (minimize) an evaluation function such as triplet loss. Accordingly, the learning model is trained such that the similarity of the input image in the input space corresponds to a distance in the feature space. Thus, the positions of distribution of feature vectors in the feature space gradually change with the progress of learning. Through machine learning/training, the trained learning model is generated.) using a loss that increases distances between distributions of features associated with different classes and decreases distances between distributions of features associated with a same class (Paragraphs [0117- 121], [0131], [0111-112], Ukai teaches by taking the evaluation term Ltask into consideration, the learning model is trained such that the distribution of the integrated similarity vectors (280) in the feature space becomes closer to the ideal distribution condition. The ideal distribution condition occurs when features corresponding to input images that include subjects of the same class are located at close positions to one another, and feature vectors that correspond to input images relating to different classes are located at distant points from one another. The Examiner interprets the different evaluation terms included in the final loss evaluation function to be losses.) . Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Honkala in view of Komkov by training the machine learning model using a loss term such that the distribution of feature vectors in the feature space become gradually closer to the ideal distribution condition that is taught by Ukai, to make the invention that classifies objects in images accurately; thus, one of ordinary skilled in the art would be motivated to combine the references since prior to learning/training, the distribution of a group of feature vectors based on outputs from the learning model are deviated from the ideal distribution condition. By training the learning model to optimize a loss, the distribution of feature vectors becomes closer to the ideal distribution condition and as a result, can improve the accuracy of inference (Ukai, Paragraphs [0111], [0127]). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. In regards to Claim 16, Honkala in view of Komkov teaches the one or more non-transitory computer-readable media of claim 11, wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform the step of performing one or more operations. Honkala in view of Komkov does not explicitly disclose to train(ing) an untrained machine learning model to generate the trained machine learning model using a first loss that is computed based on a plurality of segments of one or more feature maps generated by the untrained machine learning model and a second loss that increases distances between distributions of features associated with different classes and decreases distances between distributions of features associated with a same class. Ukai is in the same field of art of identifying and classifying targets appearing in captured images using a learning model. Further, Ukai teaches to train(ing) an untrained machine learning model (Paragraphs [0112], [0114], Fig. 6, Ukai teaches the learning model is trained. Specifically, the learning model is subjected to machine learning so as to optimize (minimize) the (whole) evaluation function (loss function) L having three evaluation terms (evaluation functions) Ltask, Lclst, and Laux. The evaluation function may be expressed as a linear sum of the three types of evaluation terms.) to generate the trained machine learning model using a first loss that is computed based on a plurality of segments of one or more feature maps generated by the untrained machine learning model (Paragraphs [0114], [0130], [0138-139], [0103], Fig. 6, Ukai teaches the learning model is subjected to machine learning to minimize the whole loss function (L) having three types of evaluation terms (Ltask, Lclst, and Laux). The loss function L may be expressed as a linear sum (linear combination) of the three types of evaluation terms. An evaluation term Lclst is the evaluation term for bringing each prototype vector p closer to the image feature (any pixel vector q) of a partial region of any image. Specifically, the evaluation term Lclst for bringing the prototype vector pk of the belonged prototype PTk for each image i closer to any pixel vector q in the feature map (230) corresponding to the image i. Each prototype vector pk may be brought closer to two or more-pixel vectors q existing in different images. The Examiner interprets “pixel vectors” to be “segments” of feature maps since the pixel vector is a vector that represents an image feature across a plurality of channels at each planar position of each pixel in the feature map.) and a second loss that increases distances between distributions of features associated with different classes and decreases distances between distributions of features associated with a same class (Paragraphs [0130-131], [0111-112], Ukai teaches by taking the evaluation term Ltask into consideration, the learning model is trained such that the distribution of the integrated similarity vectors (280) in the feature space becomes closer to the ideal distribution condition. The ideal distribution condition occurs when features corresponding to input images that include subjects of the same class are located at close positions to one another, and feature vectors that correspond to input images relating to different classes are located at distant points from one another. The Examiner interprets the different evaluation terms included in the final loss evaluation function to be losses. For example, “Ltask” is being interpreted as a second loss.) . Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Honkala in view of Komkov by minimizing/optimizing a loss function containing multiple terms, in which one of the terms is computed based on a “pixel vector,” which is a region of a feature map and another loss term that brings the distribution of similarity vectors closer to an “ideal distribution” which occurs when images with features of the same class are located at close positions to one another and images with features of different classes are located at distant positions that is taught by Ukai, to make the invention that after training the learning model with these loss terms, is able to identify subjects in input images that correspond to the feature vectors; thus, one of ordinary skilled in the art would be motivated to combine the references since training prototypes to represent a feature that is close to the image feature (pixel vector) of a specific region of a specific image helps improve explainability of the learning model about learning results, particularly, transparency (i.e., the ability to explain with concepts understandable by humans) (Ukai, Paragraph [0284]) . Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. In regards to Claim 18, Honkala in view of Komkov teaches the one or more non-transitory computer-readable media of claim 11, wherein determining the first class comprises: computing a respective distance between the first distribution of features and each predefined distribution included in the one or more predefined distributions (Abstract, Paragraph [0079], Komkov teaches calculating a distance value indicative of a distance between two distributions. The input data tensor distribution may be compared with representative class tensor distributions. A class with the highest degree of compliance can be obtained.) . Honkala in view of Komkov does not explicitly disclose selecting the first class that is associated with a smallest distance included in the respective distances. Ukai is in the same field of art of identifying and classifying targets appearing in captured images using a learning model. Further, Ukai teaches selecting the first class that is associated with a smallest distance included in the respective distances (Paragraphs [0228], [0111], Ukai teaches identifying a feature vector whose distance from the feature vector is smallest in the feature space, as a feature vector of the image that is most similar to the query image. The image processing apparatus recognizes subjects in one input image that corresponds to the identified feature vector.) . Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Honkala in view of Komkov by selecting the class/classification of the input image based on the smallest distance that is taught by Ukai, to make the invention that classifies objects in images based on identifying a minimum distance between the input image distribution and a class distribution; thus, one of ordinary skilled in the art would be motivated to combine the references since the smallest distance is associated with the most similar subjects to the subjects in the input image (Ukai, Paragraph [0228]), enabling the image processing apparatus to extract an image that is most similar to the input images from a plurality of learning images and/or processing for identifying targets (e.g., humans, animals) in an image (Ukai, Paragraph [0054]). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. In regards to Claim 19, Honkala in view of Komkov in further view of Ukai teaches the one or more non-transitory computer-readable media of claim 18, wherein the respective distances are Jensen-Shannon distances (Paragraphs [0077-78], Honkala teaches the Jensen-Shannon distance may be used to compute the distance metric between distributions.) . Pertinent Prior Art 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Rahnama-Moghaddam et al. (U.S. Patent No. 11,227,192) teaches comparing each feature map to the class-conditional distributions constructed from the training dataset. Jones et al. (U.S. Patent Pub. No. 2025/0037446) teaches systems and methods for classifying an input image with a prototypical part neural network including a backbone subnetwork, a prototype subnetwork, and a readout subnetwork to produce an interpretable classification of the input image including one or a combination of a classification result of the input image and an interpretation of the classification result . Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to SYDNEY L BLACKSTEN whose telephone number is (571)272-7120. The examiner can normally be reached 8:30am-4:30pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached at 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /SYDNEY L BLACKSTEN/Examiner, Art Unit 2674 /ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674 Application/Control Number: 18/882,563 Page 2 Art Unit: 2674 Application/Control Number: 18/882,563 Page 3 Art Unit: 2674 Application/Control Number: 18/882,563 Page 4 Art Unit: 2674 Application/Control Number: 18/882,563 Page 5 Art Unit: 2674 Application/Control Number: 18/882,563 Page 6 Art Unit: 2674 Application/Control Number: 18/882,563 Page 7 Art Unit: 2674 Application/Control Number: 18/882,563 Page 8 Art Unit: 2674 Application/Control Number: 18/882,563 Page 9 Art Unit: 2674 Application/Control Number: 18/882,563 Page 11 Art Unit: 2674 Application/Control Number: 18/882,563 Page 12 Art Unit: 2674 Application/Control Number: 18/882,563 Page 13 Art Unit: 2674 Application/Control Number: 18/882,563 Page 15 Art Unit: 2674 Application/Control Number: 18/882,563 Page 16 Art Unit: 2674 Application/Control Number: 18/882,563 Page 18 Art Unit: 2674 Application/Control Number: 18/882,563 Page 19 Art Unit: 2674 Application/Control Number: 18/882,563 Page 20 Art Unit: 2674 Application/Control Number: 18/882,563 Page 21 Art Unit: 2674 Application/Control Number: 18/882,563 Page 22 Art Unit: 2674 Application/Control Number: 18/882,563 Page 23 Art Unit: 2674 Application/Control Number: 18/882,563 Page 24 Art Unit: 2674 Application/Control Number: 18/882,563 Page 25 Art Unit: 2674 Application/Control Number: 18/882,563 Page 26 Art Unit: 2674 Application/Control Number: 18/882,563 Page 27 Art Unit: 2674 Application/Control Number: 18/882,563 Page 28 Art Unit: 2674 Application/Control Number: 18/882,563 Page 30 Art Unit: 2674 Application/Control Number: 18/882,563 Page 31 Art Unit: 2674 Application/Control Number: 18/882,563 Page 32 Art Unit: 2674 Application/Control Number: 18/882,563 Page 33 Art Unit: 2674 Application/Control Number: 18/882,563 Page 34 Art Unit: 2674
Prosecution Insights