Prosecution Insights
Last updated: November 21, 2026
Application No. 18/762,464

DETERMINATION DEVICE, INSPECTION SYSTEM, DETERMINATION METHOD, AND STORAGE MEDIUM

Final Rejection §103
Filed
Jul 02, 2024
Priority
Jan 04, 2022 — JP 2022000059 +1 more
Examiner
BONANSINGA, AARON TIMOTHY
Art Unit
2673
Tech Center
2600 — Communications
Assignee
Kabushiki Kaisha Toshiba
OA Round
2 (Final)
79%
Grant Probability
Favorable
3-4
OA Rounds
8m
Est. Remaining
99%
With Interview

Examiner Intelligence

Grants 79% — above average
79%
Career Allowance Rate
34 granted / 43 resolved
+17.1% vs TC avg
Strong +33% interview lift
Without
With
+33.3%
Interview Lift
resolved cases with interview
Typical timeline
3y 1m
Avg Prosecution
19 currently pending
Career history
73
Total Applications
across all art units

Statute-Specific Performance

§101
5.7%
-34.3% vs TC avg
§103
74.0%
+34.0% vs TC avg
§102
13.0%
-27.0% vs TC avg
§112
7.3%
-32.7% vs TC avg
Black line = Tech Center average estimate • Based on career data from 43 resolved cases

Office Action

§103
DETAILED ACTION Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Priority Receipt is acknowledged of certified copies of papers submitted under 35 U.S.C. 119(a)-(d), which papers have been placed of record in the file. Information Disclosure Statement The information disclosure statement(s) (IDS) submitted on 09/23/2026 have been considered by the Examiner. Examiners Remark(s) The Office did not issue a 35 U.S.C. 101 rejection for claim 10 on the basis of non-statutory transitory forms of signal transmission. Claim 10 states “A storage medium configured to store a program, the program, when executed by a computer, causing the computer to perform the determination method according to claim 9.” The Office understands the “storage medium” as only being directed to a non-transitory storage medium as stated in the specification at Page [24], Line(s) [24-29]: “The processing of the various data described above may be recorded, as a program that can be executed by a computer, in a magnetic disk (a flexible disk, a hard disk, etc.), an optical disk (CD-ROM, CD-R, CD-RW, DVD-ROM, DVD±R, DVD±RW, etc.), semiconductor memory, or another non-transitory computer-readable storage medium.” Response to Arguments Applicant’s arguments, filed 09/08/2026, with respect to claims 1-2 and 5-14, have been fully considered but are moot because the arguments do not apply to the current references and current combinations of references being used in the current rejection. Furthermore, the 35 U.S.C. 112(f) interpretation for claims 1, 4 and 8-9 has been withdrawn. Claim Rejections - 35 USC § 103 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claim 1-2, 5-11 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over KWON et al. (US 20190370661 A1), hereinafter referenced as KWON in view of HEO et al. (US 20210333124 A1), hereinafter referenced as HEO. Regarding claim 1, KWON explicitly teaches a determination device (Fig. 1, #100 called a machine learning device. Paragraph [0028]) configured to determine a suitability of a classification model (Fig. 1, #131 and #171 called a first machine learning classifier and a second machine learning classifier, respectively. Paragraph [0028 and 0039]. In paragraph [0028]-KWON discloses FIG. 1 is a block diagram illustrating a machine learning device 100. Please also see Fig. 15 and read paragraph [00062-0068, and 0128]), the determination device comprising: first processing circuitry (Fig. 1, #120 and #130 called a processor and a neuromorphic processor. Paragraph [0028]-KWON discloses machine learning device 100 includes a bus 110, a processor 120, a neuromorphic processor 130, a random access memory 140, a modem 150, an image database 160, storage 170, and a user interface 180 (wherein neuromorphic processor includes a machine learning classifier). Please also read paragraph [0151]); and a first memory (Fig. 1, #170 called storage. Paragraph [0028]) storing first instructions executable by the first processing circuitry (Fig. 1. Paragraph [0028]-KWON discloses the machine learning device 100 includes a random access memory 140, an image database 160, storage 170. In paragraph [0039]-KWON discloses the storage 170 may store data generated by the processor 120. The storage 170 may store an operating system or firmware code which the processor 120 executes. The storage 170 may store the machine learning classifier 131 (e.g., a first machine learning classifier) generated by the neuromorphic processor 130 as a machine learning classifier 171 (e.g., a second machine learning classifier). The storage 170 may include a nonvolatile memory such as a NAND flash memory, a PRAM, a MRAM, a RRAM, or a FRAM); wherein the classification model comprises a neural network (Fig. 1. Paragraph [0074]-KWON discloses the machine learning classifier 131 is a convolutional neural network (CNN). However, the inventive concept is not limited to CNNs, and may be applied to various other neural networks or machine learning systems); and is configured to output a classification result according to an input of an image (Fig. 1, #IMG called an image. Paragraph [0049]. In paragraph [0032]-KWON discloses the neuromorphic processor 130 may receive images and pieces of class information (e.g., pieces of first class information) of the images from the image database 160. The neuromorphic processor 130 may perform machine learning by using the images. In paragraph [0033]-KWON discloses the neuromorphic processor 130 may generate the pieces of class information of the images by using the machine learning classifier 131. The neuromorphic processor 130 may compare the pieces of class information (e.g., pieces of second class information) generated from the images with the pieces of first class information. Depending on a result of the comparison, the neuromorphic processor 130 may update the machine learning classifier 131. Please also see Fig. 15 and read paragraph [0042, 0053, 0064-0070 and 0111-0114]); wherein the neural network includes an intermediate layer (Fig. 1, #132_1, #132_2, and #132_3, called a first convolutional layer, a second convolutional layer and a third convolutional layer, respectively. Paragraph [0047-0051 and 0058-0061]) configured to output intermediate data (Fig. 1, #DINT1, DINT2, and #DINT3, called intermediate pixel data or first intermediate pixel data, second intermediate pixel data and third intermediate pixel data, respectively. Paragraph [0047-0051]) and an output layer configured to output the classification result (Fig. 1. Paragraph [0041]-KWON discloses the image database 160 may store images of semiconductor devices. In paragraph [0049]-KWON discloses the first convolution layer 132_1 may receive an image IMG from the image database 160. The first convolution layer 132_1 may generate first intermediate pixel data DINT1 by convolving its own kernels with the pixel data of the image IMG. In paragraph [0050]-KWON disclose the second convolution layer 132_2 may generate second intermediate pixel data DINT2 by convolving its own kernels with the first intermediate pixel data DINT1. The third convolution layer 132_3 may generate third intermediate pixel data DINT3 by convolving its own kernels with the second intermediate pixel data DINT2. In paragraph [0051]-KWON discloses data which the first to third convolution layers 132_1 to 132_3 output are referenced as intermediate pixel data. In paragraph [0052]-KWON discloses the classify layer 133 may output second class information ICLS2 by performing an operation including the third intermediate pixel data DINT3 and class parameters. Please also read paragraph [0053 and 0058-0061]); and wherein the first instructions, when executed by the first processing circuitry, cause the first processing circuitry to (Fig. 1. Paragraph [0029]-KWON discloses the processor 120 may control the machine learning device 100. The processor 120 may execute an operating system, firmware, etc. for the purpose of driving the machine learning device 100. In paragraph [0030]-KWON discloses the processor 120 may direct machine learning to the neuromorphic processor 130 and may support machine learning of the neuromorphic processor 130. For example, the processor 120 may allow the neuromorphic processor 130 to communicate with the random access memory 140, the modem 150, the storage 170, and the user interface 180 through the bus 110. In paragraph [0031]-KWON discloses the neuromorphic processor 130 may perform machine learning under control of the processor 120. The neuromorphic processor 130 may include a machine learning classifier 131. Please also read paragraph [0039 and 0151]): acquire the intermediate data (Fig. 1, #DINT1, DINT2, and #DINT3, called intermediate pixel data or first intermediate pixel data, second intermediate pixel data and third intermediate pixel data, respectively. Paragraph [0047-0051]) output from the intermediate layer (Fig. 1, #132_1, #132_2, and #132_3, called a first convolutional layer, a second convolutional layer and a third convolutional layer, respectively. Paragraph [0047-0051 and 0058-0061]) when an input image is input to the classification model (Fig. 1. Paragraph [0041]-KWON discloses the image database 160 may store images of semiconductor devices. In paragraph [0049]-KWON discloses the first convolution layer 132_1 may receive an image IMG from the image database 160. The first convolution layer 132_1 may generate first intermediate pixel data DINT1 by convolving its own kernels with the pixel data of the image IMG. In paragraph [0050]-KWON disclose the second convolution layer 132_2 may generate second intermediate pixel data DINT2 by convolving its own kernels with the first intermediate pixel data DINT1. The third convolution layer 132_3 may generate third intermediate pixel data DINT3 by convolving its own kernels with the second intermediate pixel data DINT2. In paragraph [0051]-KWON discloses data which the first to third convolution layers 132_1 to 132_3 output are referenced as intermediate pixel data. In paragraph [0052]-KWON discloses the classify layer 133 may output second class information ICLS2 by performing an operation including the third intermediate pixel data DINT3 and class parameters. Please also read paragraph [0053 and 0058-0061]); calculate an evaluation value representing a similarity between a plurality of sets of the intermediate data and a plurality of sets of reference data by using a typical value calculated using the plurality of sets of the intermediate data and a reference value calculated using the plurality of sets of reference data (Fig. 1. Paragraph [0052]-KWON discloses the classify layer 133 may output second class information ICLS2 by performing an operation including the third intermediate pixel data DINT3 and class parameters. In paragraph [0063]-KWON discloses the loss layer 134 may receive the second class information ICLS2 from the classify layer 133 and may receive first class information ICLS1 from the image database 160. In the case where the guide map GDM is not provided from the image database 160, the loss layer 134 may compare the second class information ICLS2 and the first class information ICLS1 to perform machine learning. In paragraph [0064]-KWON discloses when the guide map GDM is provided from the image database 160, the loss layer 134 may perform machine learning by comparing the second class information ICLS2, the first class information ICLS1, and the differences D1 to D4 transferred from the first to third activation map generation blocks 136_1 to 136_3 and the class activation map generation block 137. Please also read paragraph [0049-0051, 0069, and 0090-0101]), the plurality of sets of reference data being prepared beforehand (Fig. 1. Paragraph [0041]-KWON discloses the image database 160 may store images of semiconductor devices. Pieces of first class information of the images may include information indicating fault types or normality (i.e., an indication of the absence of faults) of the semiconductor devices. In paragraph [0043]-KWON discloses images stored in the image database 160 may include patterns hindering machine learning. The images which have patterns hindering machine learning or making it difficult to perform learning may be transferred to the neuromorphic processor 130 together with guide maps. The neuromorphic processor 130 may perform machine learning by using the guide maps, thus preventing machine learning from being hindered and supporting learning. That is, the reliability of the machine learning classifier 131 is improved. Please also read paragraph [0049-0052, 0058-0062 and 0111-0114]); and KWON fails to explicitly teach determine the suitability by comparing the evaluation value with a comparison value set based on the plurality of sets of reference data, wherein the plurality of sets of the intermediate data used to calculate the typical value is a prescribed number of sets selected in order from a most recently acquired set of the intermediate data. However, HEO explicitly teach wherein the plurality of sets of the intermediate data used to calculate the typical value (Fig. 5. Paragraph [0008]-HEO discloses the present disclosure provides a method and system for comparing information of a previously stored road-layout map with a currently captured road image to detect and display a region where a change occurs on the corresponding map. In paragraph [0070]-HEO discloses the discriminator 444 included in the domain adapter 440 may be configured to calculate a domain difference between the first intermediate data and the second intermediate data. A difference between an expected value of an output value of the discriminator for first intermediate data and an expected value of an output value of the discriminator for second intermediate data may be calculated as the domain difference (wherein the change detector 480 may calculate a degree of change of the road information on the projected road-layout map through the similarity value calculated by the similarity calculator 464 and assess the change against a threshold, and the similarity determiner 460 may include a metric learning artificial neural network (e.g. triplet loss) to calculate a distance between the features extracted from the captured road image and the road-layout map as a similarity). Further in paragraph [0122]-HEO discloses performance evaluation was basically conducted using the mean average precision score (mAP score). mAPr can represent the performance in terms of image retrieval of the system 180 for detecting changes in road information, and can evaluate how well it can perform a comparison between the captured road images and the road-layout maps belonging to different domains from each other. mAPr may evaluate the minimum number of times that the system 180 for detecting changes in road information can retrieve for the road-layout map corresponding to a query from the entire road layout map dataset when setting the captured road image as a query. mAPr was calculated by averaging the values over the entire test dataset) is a prescribed number of sets selected in order from a most recently acquired set of the intermediate data (Fig. 5. Paragraph [0069]-HEO discloses the domain adapter 440 may be configured to convert the captured road image and the projected road-layout map into first intermediate data and second intermediate data of the same feature space (wherein the encoders pass the current road image and previous road-layout map through a first, second and third structure once, and through a fourth structure nine times to generate first and second intermediate data corresponding to a current road image and a previous road-layout map, respectively, and the intermediate data is then passed to feature extractors through a first structure three times, the data output is combined with data passed through a second structure once and then passed through a second structure three times). Further in paragraph [0111]-HEO discloses the change region detector 502 may receive third intermediate data and fourth intermediate data from the feature extractor of the similarity calculator. Here, the third intermediate data and the fourth intermediate data may correspond to the last intermediate feature tensor among the intermediate feature tensors calculated in the step prior to the global pooling step in the process of extracting the features of the captured road image and the projected road-layout map by the feature extractor. (wherein insntance-wise normalization may also be performed on third and fourth intermediate data). Please also see Fig. 10 and 13-15, and read paragraph [0080-0081, 0122, and 0131-0141]). However, HEO explicitly teaches determining the suitability by comparing the evaluation value with a comparison value set based on the plurality of sets of reference data (Fig. 5. Paragraph [0008]-HEO discloses the present disclosure provides a method and system for comparing information of a previously stored road-layout map with a currently captured road image to detect and display a region where a change occurs on the corresponding map. In paragraph [0070]-HEO discloses the discriminator 444 included in the domain adapter 440 may be configured to calculate a domain difference between the first intermediate data and the second intermediate data. A difference between an expected value of an output value of the discriminator for first intermediate data and an expected value of an output value of the discriminator for second intermediate data may be calculated as the domain difference (wherein a degree of change calculated and assessed against a threshold, and a degree of similarity value is calculated, which may include metric learning (e.g. triplet loss) to calculate a distance between the features of captured road image and the road-layout map). Please also read paragraph [0111-0112 and 0122-0124]), wherein the plurality of sets of the intermediate data used to calculate the typical value is a prescribed number of sets selected in order from a most recently acquired set of the intermediate data (Fig. 5. Paragraph [0069]-HEO discloses the domain adapter 440 may be configured to convert the captured road image and the projected road-layout map into first intermediate data and second intermediate data of the same feature space (wherein the encoders pass the current road image and previous road-layout map through a first, second and third structure once, and through a fourth structure nine times to generate first and second intermediate data corresponding to a current road image and a previous road-layout map, respectively, and the intermediate data is then passed to feature extractors through a first structure three times, the data output is combined with data passed through a second structure once and then passed through a second structure three times). Further in paragraph [0111]-HEO discloses the change region detector 502 may receive third intermediate data and fourth intermediate data from the feature extractor of the similarity calculator (wherein third and fourth intermediate data may be subject to normalization and correspond to the last intermediate feature tensor among the intermediate feature tensors calculated in the step prior). Please also see Fig. 10 and 13-15, and read paragraph [0080-0081, 0122, and 0131-0141]). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KWON of having a determination device, with the teachings of HEO of having determine the suitability by comparing the evaluation value with a comparison value set based on the plurality of sets of reference data, wherein the plurality of sets of the intermediate data used to calculate the typical value is a prescribed number of sets selected in order from a most recently acquired set of the intermediate data. Wherein KWON’s method having determine the suitability by comparing the evaluation value with a comparison value set based on the plurality of sets of reference data, wherein the plurality of sets of the intermediate data used to calculate the typical value is a prescribed number of sets selected in order from a most recently acquired set of the intermediate data. The motivation behind the modification would have been to obtain a method that improves the performance of machine learning models, since both KWON and HEO concern neural networks and intermediate data. Wherein KWON’s provides methods and systems that improve the reliability of machine learning, while HEO’s methods and systems improves the performance of machine learning models and the accuracy of detecting image differences. Please see KWON et al. (US 20190370661 A1), Abstract and Paragraph [0043 and 0127] and HEO et al. (US 20210333124 A1), Abstract and Paragraph [0073 and 0122-0123]. Regarding claim 2, KWON in view of HEO explicitly teaches the determination device according to claim 1, KWON fails to explicitly teach wherein the classification model is trained by metric learning. However, HEO explicitly teaches wherein the classification model is trained by metric learning (Fig. 5. Paragraph [0043]-HEO discloses “domain adaptation” may refer to a technique of transferring from a domain of data that the machine learning model such as the artificial neural network is trained to classify or recognize to another domain or adapting to another domain. Further in paragraph [0073]-HEO discloses a metric learning artificial neural network may be configured to calculate a similarity between the captured road image and the projected road-layout map using a triplet loss function that enforces a similarity between the captured road image and a positive image which is the projected road-layout map corresponding to the captured road image to be larger by a margin than a similarity between the captured road image and a negative image which is the projected road-layout map not corresponding to the captured road image. The margin adapted for the triplet loss function of the metric learning artificial neural network may be adjusted according to the domain difference between the first intermediate data and the second intermediate data calculated by the domain adapter 440). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KWON in view of HEO of having a determination device, with the teachings of HEO of having wherein the classification model is trained by metric learning. Wherein KWON’s method having wherein the classification model is trained by metric learning. The motivation behind the modification would have been to obtain a method that improves the performance of machine learning models, since both KWON and HEO concern neural networks and intermediate data. Wherein KWON’s provides methods and systems that improve the reliability of machine learning, while HEO’s methods and systems improves the performance of machine learning models and the accuracy of detecting image differences. Please see KWON et al. (US 20190370661 A1), Abstract and Paragraph [0043 and 0127] and HEO et al. (US 20210333124 A1), Abstract and Paragraph [0073 and 0122-0123]. Regarding claim 5, KWON in view of HEO explicitly teaches the determination device according to claim 1, KWON further teaches wherein a plurality of the input images (Fig. 1, #IMG called an image. Paragraph [0049]. Please also read paragraph [0053 and 0111-0114]) includes a plurality of first input images classified into a first class by the classification model (Fig. 1, #130 and #131/171 called a neuromorphic processor and a machine learning classifier, respectively. Paragraph [0030-0033 and 0047-0048]. In paragraph [0032]-KWON discloses the neuromorphic processor 130 may receive images and pieces of class information (e.g., pieces of first class information) of the images from the image database 160. The neuromorphic processor 130 may perform machine learning by using the images. In paragraph [0033]-KWON discloses the neuromorphic processor 130 may generate the pieces of class information of the images by using the machine learning classifier 131. The neuromorphic processor 130 may compare the pieces of class information (e.g., pieces of second class information) generated from the images with the pieces of first class information. Depending on a result of the comparison, the neuromorphic processor 130 may update the machine learning classifier 131). Please also read paragraph [0049]), the plurality of sets of intermediate data includes a plurality of sets of first intermediate data (Fig. 1, #DINT1, DINT2, and #DINT3, called intermediate pixel data or first intermediate pixel data, second intermediate pixel data and third intermediate pixel data, respectively. Paragraph [0047-0051 and 0058-0061]) respectively based on inputs of the plurality of first input images to the classification model (Fig. 1. Paragraph [0049]-KWON discloses the first convolution layer 132_1 may receive an image IMG from the image database 160. The image IMG may include pixel data including pixel values respectively corresponding to pixels of a camera. The first convolution layer 132_1 may generate first intermediate pixel data DINT1 by convolving its own kernels with the pixel data of the image IMG. In paragraph [0050]-KWON disclose the second convolution layer 132_2 may generate second intermediate pixel data DINT2 by convolving its own kernels with the first intermediate pixel data DINT1. The third convolution layer 132_3 may generate third intermediate pixel data DINT3 by convolving its own kernels with the second intermediate pixel data DINT2. In paragraph [0051]-KWON discloses data which the first to third convolution layers 132_1 to 132_3 output are referenced as intermediate pixel data. In paragraph [0052]-KWON discloses the classify layer 133 may output second class information ICLS2 by performing an operation including the third intermediate pixel data DINT3 and class parameters. The classify layer 133 may calculate values respectively corresponding to classes which the classify layer 133 generates. The values of the classes may indicate the probability that the image IMG belongs to each class. The classify layer 133 may output information of a class having the highest probability as the second class information ICLS2. Please also read paragraph [0058-0061and 0111-0114]), and the determination of the suitability includes calculating a first evaluation value as the evaluation value by using the plurality of sets of first intermediate data (Fig. 1. Paragraph [0033]-KWON discloses the neuromorphic processor 130 may compare the pieces of class information (e.g., pieces of second class information) generated from the images with the pieces of first class information. Depending on a result of the comparison, the neuromorphic processor 130 may update the machine learning classifier 131). In paragraph [0064]-KWON discloses the loss layer 134 may perform machine learning by comparing the second class information ICLS2, the first class information ICLS1, and the differences D1 to D4 transferred from the first to third activation map generation blocks 136_1 to 136_3 and the class activation map generation block 137. In paragraph [0065]-KWON discloses the loss layer 134 may calculate a difference between the first class information ICLS1 and the second class information ICLS2. Depending on the calculated difference, the loss layer 134 may update weights (e.g., the values of synapses in a neural network) of the first to third convolution layers 132_1 to 132_3 and the classify layer 133. The loss layer 134 may update weights such that the second class information ICLS2 becomes closer to the first class information ICLS1). Regarding claim 6, KWON in view of HEO explicitly teaches the determination device according to claim 5, KWON further teaches wherein the plurality of sets of reference data is acquired from the intermediate layer respectively when a plurality of reference images (Fig. 1, #IMG called an image. Paragraph [0049]. Please also read paragraph [0035 and 0111-0114]) is input to the classification model (Fig. 1, #130 and #131/171 called a neuromorphic processor and a machine learning classifier, respectively. Paragraph [0030-0033 and 0047-0048]. In paragraph [0047]-KWON discloses the neuromorphic processor 130 includes first to third convolution layers 132_1 to 132_3, a classify layer 133, a loss layer 134, a resize block 135, first to third activation map generation blocks 136_1 to 136_3, and a class activation map generation block 137 (wherein the neuromorphic processor includes a machine learning classifier #131 and the first to third convolution layers 132_1 to 132_3 and the classify layer 133 may constitute the machine learning classifier 131). In paragraph [0049]-KWON disclose the first convolution layer 132_1 may receive an image IMG from the image database 160. Please also read paragraph [0053]), the plurality of reference images includes a plurality of first reference images classified into the first class by the classification model (Fig. 1. Paragraph [0032]-KWON discloses the neuromorphic processor 130 may receive images and pieces of class information (e.g., pieces of first class information) of the images from the image database 160. The neuromorphic processor 130 may perform machine learning by using the images. In paragraph [0033]-KWON discloses the neuromorphic processor 130 may generate the pieces of class information of the images by using the machine learning classifier 131. The neuromorphic processor 130 may compare the pieces of class information (e.g., pieces of second class information) generated from the images with the pieces of first class information. Depending on a result of the comparison, the neuromorphic processor 130 may update the machine learning classifier 131). Please also read paragraph [0075-0076 and 0111-0114]]), the plurality of sets of reference data includes a plurality of sets of first reference data respectively based on inputs of the plurality of first reference images to the classification model (Fig. 1. Paragraph [0049]-KWON discloses the first convolution layer 132_1 may generate first intermediate pixel data DINT1 by convolving its own kernels with the pixel data of the image IMG. In paragraph [0050]-KWON discloses the second convolution layer 132_2 may generate second intermediate pixel data DINT2 by convolving its own kernels with the first intermediate pixel data DINT1. The third convolution layer 132_3 may generate third intermediate pixel data DINT3 by convolving its own kernels with the second intermediate pixel data DINT2. In paragraph [0052]-KWON discloses the classify layer 133 may output second class information ICLS2 by performing an operation including the third intermediate pixel data DINT3 and class parameters (wherein a first to third activation map generation block 136_1-3 may generate a first to third activation map from the first to third intermediate pixel data DINT1-3 and generate difference D1-D3 values between intermediate pixel data and guide maps GDM1-3, and a difference value D4 may be generated from a difference between a fourth guide map GDM4 and the class activation map). Please also read paragraph [0058-0063]), and a first comparison value calculated using the plurality of sets of first reference data is set as the comparison value (Fig. 1. Paragraph [0063]-KWON discloses the loss layer 134 may receive the second class information ICLS2 from the classify layer 133 and may receive first class information ICLS1 from the image database 160. In the case where the guide map GDM is not provided from the image database 160, the loss layer 134 may compare the second class information ICLS2 and the first class information ICLS1 to perform machine learning. In paragraph [0064]-KWON discloses when the guide map GDM is provided from the image database 160, the loss layer 134 may perform machine learning by comparing the second class information ICLS2, the first class information ICLS1, and the differences D1 to D4 transferred from the first to third activation map generation blocks 136_1 to 136_3 and the class activation map generation block 137. Please also read paragraph [0041, 0043, 0054-0063, 0111-0114]). Regarding claim 7, KWON in view of HEO explicitly teaches the determination device according to claim 6, KWON further teaches wherein the plurality of input images (Fig. 1. Paragraph [0049]-KWON discloses the first convolution layer 132_1 may receive an image IMG from the image database 160 (wherein the images may be received with Guide maps GDM, which may include information about whether to perform machine learning more preferentially (or dominantly) based on any portion of an image). In paragraph [0053]-KWON discloses the resize block 135 may receive a guide map GDM from the image database 160. The guide map GDM may have the same size as the image IMG. The guide map GDM may include pixel data including weights respectively corresponding to the pixels of the image IMG. Please also read paragraph [0035, 0114, 0122]) further includes a plurality of second input images classified into a second class by the classification model (Fig. 1, #130 and #131/171 called a neuromorphic processor and a machine learning classifier, respectively. Paragraph [0030-0031 and 0047-0048]), the plurality of sets of intermediate data further includes a plurality of sets of second intermediate data respectively based on inputs of the plurality of second input images to the classification model (Fig. 1. Paragraph [0049]-KWON discloses the first convolution layer 132_1 may generate first intermediate pixel data DINT1 by convolving its own kernels with the pixel data of the image IMG. In paragraph [0050]-KWON discloses the second convolution layer 132_2 may generate second intermediate pixel data DINT2 by convolving its own kernels with the first intermediate pixel data DINT1. The third convolution layer 132_3 may generate third intermediate pixel data DINT3 by convolving its own kernels with the second intermediate pixel data DINT2. In paragraph [0052]-KWON discloses the classify layer 133 may output second class information ICLS2 by performing an operation including the third intermediate pixel data DINT3 and class parameters (wherein a first to third activation map generation block 136_1-3 may generate a first to third activation map from the first to third intermediate pixel data DINT1-3 and generate difference D1-D3 values between intermediate pixel data and guide maps GDM1-3, and a difference value D4 may be generated from a difference between a fourth guide map GDM4 and the class activation map). Please also read paragraph [0053, 0058-0063]), the plurality of reference images further includes a plurality of second reference images classified into the second class by the classification model (Fig. 1. Paragraph [0114]-KWON discloses FIG. 11 is a diagram illustrating an example of first images IMG1 and second images IMG2 used for machine learning. The first images IMG1 may have the first class information ICLS1 corresponding to a circle (i.e., first class information), and the second images IMG2 may have the first class information ICLS1 corresponding to a square (i.e., third class information). The neuromorphic processor 130 may perform machine learning such that the machine learning classifier 131 classifies the first images IMG1 as a circle-shaped class (i.e., second class information) and classifies the second images IMG2 as a square-shaped class (i.e., fourth class information). Please also read paragraph [0041, 0043, and 0064-0070]), the plurality of sets of reference data further includes a plurality of sets of second reference data respectively based on inputs of the plurality of second reference images to the classification model (Fig. 1. Paragraph [0111]-KWON discloses the machine learning classifier 131 may be trained such that the image IMG is a class (e.g., an original class) pointed out by the first class information ICLS1. Accordingly, by generating a class activation map corresponding to the first class information ICLS1, whether any portion of the third intermediate pixel data DINT3 is referenced may be determined upon classifying the image IMG as belonging to the original class. In paragraph [0112]-KWON discloses the class activation map generation block 137 may calculate the guide difference D4 between a class activation map of the original class and the fourth guide map GDM4. In paragraph [0113]-KWON discloses the class activation map generation block 137 may calculate 20 class activation maps corresponding to 20 classes. The class activation map generation block 137 may calculate 20 differences D4 by comparing the fourth guide map GDM4 with the 20 class activation maps, respectively. The loss layer 134 may update the class parameters CLSP based on the 20 differences D4. Please also read paragraph [0058-0070]), the determination of the suitability further includes: calculating a second evaluation value (Fig. 1. Paragraph [0052]-KWON discloses the classify layer 133 may output second class information ICLS2 by performing an operation including the third intermediate pixel data DINT3 and class parameters. The classify layer 133 may calculate values respectively corresponding to classes which the classify layer 133 generates. The values of the classes may indicate the probability that the image IMG belongs to each class. The classify layer 133 may output information of a class having the highest probability as the second class information ICLS2. Please also read paragraph [0058-0065 and 0111-0114]) by using the plurality of sets of second intermediate data (Fig. 1. Paragraph [0049]-KWON discloses the first convolution layer 132_1 may generate first intermediate pixel data DINT1 by convolving its own kernels with the pixel data of the image IMG. In paragraph [0050]-KWON discloses the second convolution layer 132_2 may generate second intermediate pixel data DINT2 by convolving its own kernels with the first intermediate pixel data DINT1. The third convolution layer 132_3 may generate third intermediate pixel data DINT3 by convolving its own kernels with the second intermediate pixel data DINT2. In paragraph [0052]-KWON discloses the classify layer 133 may output second class information ICLS2 by performing an operation including the third intermediate pixel data DINT3 and class parameters (wherein a first to third activation map generation block 136_1-3 may generate a first to third activation map from the first to third intermediate pixel data DINT1-3 and generate difference D1-D3 values between intermediate pixel data and guide maps GDM1-3, and a difference value D4 may be generated from a difference between a fourth guide map GDM4 and the class activation map). Please also read paragraph [0058-0063 and 0111-0114]); and determining the suitability by comparing the second evaluation value and a second comparison value, and the second comparison value is calculated using the plurality of sets of second reference data (Fig. 1. Paragraph [0063]-KWON discloses the loss layer 134 may receive the second class information ICLS2 from the classify layer 133 and may receive first class information ICLS1 from the image database 160. In the case where the guide map GDM is not provided from the image database 160, the loss layer 134 may compare the second class information ICLS2 and the first class information ICLS1 to perform machine learning. In paragraph [0064]-KWON discloses when the guide map GDM is provided from the image database 160, the loss layer 134 may perform machine learning by comparing the second class information ICLS2, the first class information ICLS1, and the differences D1 to D4 transferred from the first to third activation map generation blocks 136_1 to 136_3 and the class activation map generation block 137. In paragraph [0065]-KWON discloses the loss layer 134 may calculate a difference between the first class information ICLS1 and the second class information ICLS2. Depending on the calculated difference, the loss layer 134 may update weights (e.g., the values of synapses in a neural network) of the first to third convolution layers 132_1 to 132_3 and the classify layer 133. The loss layer 134 may update weights such that the second class information ICLS2 becomes closer to the first class information ICLS1. Please also read paragraph [0111-0114]). Regarding claim 8, KWON in view of HEO explicitly teaches an inspection system (Fig. 15, #200 called a semiconductor defect classification system. Paragraph [0128]. (wherein the machine learning device includes a bus 110, a processor 120, a neuromorphic processor 130, a random access memory 140, a modem 150, an image database 160, storage 170, and a user interface 180, and the semiconductor defect classification system 200 includes a wafer 210, a manufacture device 220, an automatic defect review device 230, an imaging device 240, image storage 250, a semiconductor defect classification device 260, and a defect image database 270). Please also see Fig. 1 and read paragraph [0028-0032]), comprising: the determination device (Fig. 1, #100 called a machine learning device. Paragraph [0028]. Please also read paragraph [0031-0032]) according to claim 1 (Please see the rejection for claim 1 above); an imaging device (Fig. 15, #240 called an imaging device. Paragraph [0128]. In paragraph [0135]-KWON discloses the imaging device 240 may produce images of locations, which are predicted as a defect is present in semiconductor patterns on the wafer 210, based on the location information LI. The imaging device 240 may include an SEM (Scanning Electron Microscopy) device or an OM (Optical Microscopy) device) configured to acquire the input image (Fig. 1 and 15, #IMG, #OI, #HR, LRI and #RI called an image, an optical image, a high resolution image, a low resolution image and a reference image, respectively. Paragraph [0041 and 0135-0136]. In paragraph [0136]-KWON disclose the imaging device 240 may output a high resolution image HRI, a low resolution image LRI, and a reference image RI, based on the SEM imaging. The imaging device 240 may output an optical image OI based on the OM imaging. Please also see Fig. 1) by imaging an article (Fig. 15, #210 called a wafer. Paragraph [0128]. Please also see Fig. 1 and read paragraph [0035 and 0039]); and an inspection device (Fig. 15, #260 called a defect classification device. Paragraph [0142]-KWON discloses the semiconductor defect classification device 260 may receive the high resolution image HRI, the low resolution image LRI, the reference image RI, and the optical image OI from the image storage 250. The semiconductor defect classification device 260 may receive the first meta information MI1 from the manufacture device 220 and may receive the second meta information M12 from the imaging device 240. Please also see Fig. 1), wherein the inspection device includes: second processing circuitry, and a second memory storing second instructions executable by the second processing circuitry, wherein the second instructions, when executed by the second processing circuitry, cause the second processing circuitry to (Fig. 15. Paragraph [0141]-KWON discloses the image storage 250 may include a volatile memory such as a dynamic random access memory (DRAM) or a static random access memory (SRAM) or a nonvolatile memory such as a flash memory, a magnetic RAM (MRA), a phase-change RAM (PRAM), a ferroelectric RAM (FRAM), or a resistive RAM (RRAM). Further in paragraph [0151]-KWON discloses components according to embodiments of the inventive concept are described by using various blocks. The “block” may be implemented with various hardware devices, such as an integrated circuit, an application specific IC (ASCI), a field programmable gate array (FPGA), and a complex programmable logic device (CPLD), software, such as firmware and applications driven in hardware devices, or a combination of a hardware device and software. Also, “block” may include circuits or intellectual property (IP) implemented with semiconductor devices. Please also see Fig. 1 and read paragraph [0028-0029 and 0039]): input the input image to the classification model, acquire the classification result from the classification model, and output an inspection result of the article visible in the input image according to the classification result (Fig. 15. Paragraph [0143]-KWON discloses the semiconductor defect classification device 260 may classify (or determine), based on machine learning, whether semiconductor patterns of the wafer 210 associated with images have a defect, by using the high resolution image HRI, the low resolution image LRI, the reference image RI, the optical image OI, the first meta information MI1, the second meta information MI2, and third meta information MI3. Please also see Fig. 1). Regarding claim 9, KWON teaches a determination method for determining a suitability of a classification model ((Fig. 1, #131 and #171 called a first machine learning classifier and a second machine learning classifier, respectively. Paragraph [0028 and 0039]. In paragraph [0028]-KWON discloses FIG. 1 is a block diagram illustrating a machine learning device 100. Please also see Fig. 15 and read paragraph [00062-0068, and 0128]) (wherein neuromorphic processor #130 includes a machine learning model classifier #131)), the classification model comprising a neural network (Fig. 1. Paragraph [0074]-KWON discloses the machine learning classifier 131 is a convolutional neural network (CNN). However, the inventive concept is not limited to CNNs, and may be applied to various other neural networks or machine learning systems) and being configured to output a classification result according to an input of an image (Fig. 1, #IMG called an image. Paragraph [0049]. In paragraph [0032]-KWON discloses the neuromorphic processor 130 may receive images and pieces of class information (e.g., pieces of first class information) of the images from the image database 160. The neuromorphic processor 130 may perform machine learning by using the images. In paragraph [0033]-KWON discloses the neuromorphic processor 130 may generate the pieces of class information of the images by using the machine learning classifier 131. The neuromorphic processor 130 may compare the pieces of class information (e.g., pieces of second class information) generated from the images with the pieces of first class information. Depending on a result of the comparison, the neuromorphic processor 130 may update the machine learning classifier 131. Please also see Fig. 15 and read paragraph [0053, 0064-0070 and 0111-0114]), the neural network including an intermediate layer (Fig. 1, #132_1, #132_2, and #132_3, called a first convolutional layer, a second convolutional layer and a third convolutional layer, respectively. Paragraph [0047-0051 and 0058-0061]) configured to output intermediate data (Fig. 1, #DINT1, DINT2, and #DINT3, called intermediate pixel data or first intermediate pixel data, second intermediate pixel data and third intermediate pixel data, respectively. Paragraph [0047-0051]) and an output layer configured to output the classification result (Fig. 3. Paragraph [0047]-KWON discloses referring to FIGS. 1 and 2, the neuromorphic processor 130 includes first to third convolution layers 132_1 to 132_3, a classify layer 133, a loss layer 134, a resize block 135, first to third activation map generation blocks 136_1 to 136_3, and a class activation map generation block 137), the determination method (Fig. 3. Paragraph [0075]-KWON discloses FIG. 3 is a flowchart illustrating an operating method of the neuromorphic processor 130. Please also see Fig. 1-2 and 15 and read paragraph [0064-0070 and 0111-0114]) comprising: the intermediate data output from the intermediate layer when an input image (Fig. 1, #IMG called an image. Paragraph [0049]. In paragraph [0049]-KWON discloses the image IMG may include pixel data including pixel values respectively corresponding to pixels of a camera. Please also read paragraph [0035, 0111-0114 and 0140-0150]) is input to the classification model (Fig. 1, #130 called a neuromorphic processor. Paragraph [0031]. In paragraph [0049]-KWON discloses the first convolution layer 132_1 may receive an image IMG from the image database 160. The first convolution layer 132_1 may generate first intermediate pixel data DINT1 by convolving its own kernels with the pixel data of the image IMG. In paragraph [0050]-KWON discloses the second convolution layer 132_2 may generate second intermediate pixel data DINT2 by convolving its own kernels with the first intermediate pixel data DINT1. The third convolution layer 132_3 may generate third intermediate pixel data DINT3 by convolving its own kernels with the second intermediate pixel data DINT2. Please also read paragraph [0053 and 0058-0062]); calculating an evaluation value representing a similarity between a plurality of sets of the intermediate data and a plurality of sets of reference data by using a typical value calculated using the plurality of sets of the intermediate data and a reference value calculated using the plurality of sets of reference data (Fig. 1. Paragraph [0063]-KWON discloses the loss layer 134 may receive the second class information ICLS2 from the classify layer 133 and may receive first class information ICLS1 from the image database 160. In paragraph [0064]-KWON discloses when the guide map GDM is provided from the image database 160, the loss layer 134 may perform machine learning by comparing the second class information ICLS2, the first class information ICLS1, and the differences D1 to D4 transferred from the first to third activation map generation blocks 136_1 to 136_3 and the class activation map generation block 137. In paragraph [0065]-KWON discloses the loss layer 134 may calculate a difference between the first class information ICLS1 and the second class information ICLS2. Depending on the calculated difference, the loss layer 134 may update weights (e.g., the values of synapses in a neural network) of the first to third convolution layers 132_1 to 132_3 and the classify layer 133. The loss layer 134 may update weights such that the second class information ICLS2 becomes closer to the first class information ICLS1. Please also read paragraph [0041, 0043, 0063-0068 and 0111-0114]), the plurality of sets of reference data being prepared beforehand (Fig. 1. Paragraph [0041]-KWON discloses the image database 160 may store images of semiconductor devices. Pieces of first class information of the images may include information indicating fault types or normality (i.e., an indication of the absence of faults) of the semiconductor devices. In paragraph [0043]-KWON discloses images stored in the image database 160 may include patterns hindering machine learning. The images which have patterns hindering machine learning or making it difficult to perform learning may be transferred to the neuromorphic processor 130 together with guide maps. The neuromorphic processor 130 may perform machine learning by using the guide maps, thus preventing machine learning from being hindered and supporting learning. That is, the reliability of the machine learning classifier 131 is improved. Please also read paragraph [0049-0052, 0058-0062 and 0111-0114]; and KWON fails to explicitly teach determining the suitability by comparing the evaluation value with a comparison value set based on the plurality of sets of reference data, wherein the plurality of sets of the intermediate data used to calculate the typical value is a prescribed number of sets selected in order from a most recently acquired set of the intermediate data. However, HEO explicitly teaches determining the suitability by comparing the evaluation value with a comparison value set based on the plurality of sets of reference data (Fig. 5. Paragraph [0008]-HEO discloses the present disclosure provides a method and system for comparing information of a previously stored road-layout map with a currently captured road image to detect and display a region where a change occurs on the corresponding map. In paragraph [0070]-HEO discloses the discriminator 444 included in the domain adapter 440 may be configured to calculate a domain difference between the first intermediate data and the second intermediate data. A difference between an expected value of an output value of the discriminator for first intermediate data and an expected value of an output value of the discriminator for second intermediate data may be calculated as the domain difference (wherein a degree of change calculated and assessed against a threshold, and a degree of similarity value is calculated, which may include metric learning (e.g. triplet loss) to calculate a distance between the features of captured road image and the road-layout map). Please also read paragraph [0111-0112 and 0122-0124]), wherein the plurality of sets of the intermediate data used to calculate the typical value is a prescribed number of sets selected in order from a most recently acquired set of the intermediate data (Fig. 5. Paragraph [0069]-HEO discloses the domain adapter 440 may be configured to convert the captured road image and the projected road-layout map into first intermediate data and second intermediate data of the same feature space (wherein the encoders pass the current road image and previous road-layout map through a first, second and third structure once, and through a fourth structure nine times to generate first and second intermediate data corresponding to a current road image and a previous road-layout map, respectively, and the intermediate data is then passed to feature extractors through a first structure three times, the data output is combined with data passed through a second structure once and then passed through a second structure three times). Further in paragraph [0111]-HEO discloses the change region detector 502 may receive third intermediate data and fourth intermediate data from the feature extractor of the similarity calculator (wherein third and fourth intermediate data may be subject to normalization and correspond to the last intermediate feature tensor among the intermediate feature tensors calculated in the step prior). Please also see Fig. 10 and 13-15, and read paragraph [0080-0081, 0122, and 0131-0141]). Regarding claim 10, KWON explicitly teaches a storage medium (Fig. 1, #170 called storage. Paragraph [0033]) configured to store a program (Fig. 1. Paragraph [0039]-KWON discloses the storage 170 may store data generated by the processor 120. The storage 170 may store an operating system or firmware code which the processor 120 executes. The storage 170 may store the machine learning classifier 131 (e.g., a first machine learning classifier) generated by the neuromorphic processor 130 as a machine learning classifier 171 (e.g., a second machine learning classifier). The storage 170 may include a nonvolatile memory. Please also see Fig. 15 and read paragraph [0031-0038]), the program, when executed by a computer, causing the computer to perform the determination method KWON fails to explicitly teach the determination method according to claim 9. However, KWON in view of HEO the determination method according to claim 9 (Please see the rejection for claim 9 above). Regarding claim 11, KWON in view of HEO explicitly teaches the determination device according to claim 1, KWON explicitly teaches wherein: each set of the plurality of sets of the intermediate data is output from the intermediate layer when a respective one of a plurality of input images is input to the classification model (Fig. 1. Paragraph [0041]-KWON discloses the image database 160 may store images of semiconductor devices. In paragraph [0049]-KWON discloses the first convolution layer 132_1 may receive an image IMG from the image database 160. The first convolution layer 132_1 may generate first intermediate pixel data DINT1 by convolving its own kernels with the pixel data of the image IMG. In paragraph [0050]-KWON disclose the second convolution layer 132_2 may generate second intermediate pixel data DINT2 by convolving its own kernels with the first intermediate pixel data DINT1. The third convolution layer 132_3 may generate third intermediate pixel data DINT3 by convolving its own kernels with the second intermediate pixel data DINT2. In paragraph [0051]-KWON discloses data which the first to third convolution layers 132_1 to 132_3 output are referenced as intermediate pixel data. In paragraph [0052]-KWON discloses the classify layer 133 may output second class information ICLS2 by performing an operation including the third intermediate pixel data DINT3 and class parameters. Please also read paragraph [0053 and 0058-0061]); and each set of the plurality of sets of reference data is output from the intermediate layer when a respective one of a plurality of reference images is input to the classification model (Fig. 1. Paragraph [0052]-KWON discloses the classify layer 133 may output second class information ICLS2 by performing an operation including the third intermediate pixel data DINT3 and class parameters. In paragraph [0063]-KWON discloses the loss layer 134 may receive the second class information ICLS2 from the classify layer 133 and may receive first class information ICLS1 from the image database 160. In the case where the guide map GDM is not provided from the image database 160, the loss layer 134 may compare the second class information ICLS2 and the first class information ICLS1 to perform machine learning. In paragraph [0064]-KWON discloses when the guide map GDM is provided from the image database 160, the loss layer 134 may perform machine learning by comparing the second class information ICLS2, the first class information ICLS1, and the differences D1 to D4 transferred from the first to third activation map generation blocks 136_1 to 136_3 and the class activation map generation block 137. Please also read paragraph [0041-0042 and 0049-0051, and 0090-0101]). Regarding claim 13, KWON in view of HEO explicitly teaches the determination device according to claim 12, KWON fails to explicitly teach wherein the instructions, when executed by the first processing circuitry, cause the first processing circuitry to: determine that the suitability is in a first state when the evaluation value is less than the comparison value; and determine that the suitability is in a second state indicating degradation of the suitability when the evaluation value is not less than the comparison value. However, HEO explicitly teaches wherein the instructions, when executed by the first processing circuitry, cause the first processing circuitry to (Fig. 3. Paragraph [0057]-HEO discloses FIG. 3 is a block diagram illustrating an internal configuration of a computing device 310 and an information processing system 340. The computing device 310 may include a memory 312, a processor 314, a communication module 316, and an input and output interface 318. The information processing system 340 (e.g., the system 180 for detecting changes in road information) may include a memory 342, a processor 344, a communication module 346, and an input and output interface 348. Further in paragraph [0060]-HEO discloses the processors 314 and 344 may be configured to process instructions of the computer program by performing basic arithmetic, logic, and input and output operations. The instructions may be provided to the processors 314 and 344 from the memories 312 and 342 or the communication modules 316 and 346. For example, the processors 314 and 344 may be configured to execute the received instructions according to program code stored in a recording device such as the memories 312 and 342): determine that the suitability is in a first state when the evaluation value is less than the comparison value (Fig. 5. Paragraph [0073]-HEO discloses the similarity determiner 460 may include a metric learning artificial neural network configured to calculate a distance between the features extracted from the captured road image and the road-layout map (wherein absence/presence of change is based on whether a threshold is above or below a threshold, and the metric learning artificial neural network may be configured to calculate a similarity using a triplet loss function that enforces a similarity between the road image and a positive image (a projected road-layout map corresponding to the captured road image) to be larger by a margin than a similarity between the captured road image and a negative image (a projected road-layout map not corresponding to the captured road image)). The margin adapted for the triplet loss function of the metric learning artificial neural network may be adjusted according to the domain difference between the first intermediate data and the second intermediate data calculated by the domain adapter 440 (wherein in the initial stage of learning where the domain difference between the two input data of the metric learning artificial neural network is large, the margin is decreased and in the training phase in which the domain difference is decreased, the margin is increased). That is, as the distance between the images or the features of the image decreases, it may be determined that the similarity is greater, and as the distance increases, it may be determined that the similarity is smaller. Please also read paragraph [0111-0112 and 0122-0124]); and determine that the suitability is in a second state indicating degradation of the suitability when the evaluation value is not less than the comparison value (Fig. 5. Paragraph [0073]-HEO discloses the similarity determiner 460 may include a metric learning artificial neural network configured to calculate a distance between the features extracted from the captured road image and the road-layout map (wherein absence/presence of change is based on whether a threshold is above or below a threshold, and the metric learning artificial neural network may be configured to calculate a similarity using a triplet loss function that enforces a similarity between the road image and a positive image (a projected road-layout map corresponding to the captured road image) to be larger by a margin than a similarity between the captured road image and a negative image (a projected road-layout map not corresponding to the captured road image)). The margin adapted for the triplet loss function of the metric learning artificial neural network may be adjusted according to the domain difference between the first intermediate data and the second intermediate data calculated by the domain adapter 440 (wherein in the initial stage of learning where the domain difference between the two input data of the metric learning artificial neural network is large, the margin is decreased and in the training phase in which the domain difference is decreased, the margin is increased). That is, as the distance between the images or the features of the image decreases, it may be determined that the similarity is greater, and as the distance increases, it may be determined that the similarity is smaller. Please also read paragraph [0111-0112 and 0122-0124]). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KWON in view of HEO of having a determination device, with the teachings of HEO of having wherein the instructions, when executed by the first processing circuitry, cause the first processing circuitry to: determine that the suitability is in a first state when the evaluation value is less than the comparison value; and determine that the suitability is in a second state indicating degradation of the suitability when the evaluation value is not less than the comparison value. Wherein KWON’s method having wherein the instructions, when executed by the first processing circuitry, cause the first processing circuitry to: determine that the suitability is in a first state when the evaluation value is less than the comparison value; and determine that the suitability is in a second state indicating degradation of the suitability when the evaluation value is not less than the comparison value. The motivation behind the modification would have been to obtain a method that improves the performance of machine learning models, since both KWON and HEO concern neural networks and intermediate data. Wherein KWON’s provides methods and systems that improve the reliability of machine learning, while HEO’s methods and systems improves the performance of machine learning models and the accuracy of detecting image differences. Please see KWON et al. (US 20190370661 A1), Abstract and Paragraph [0043 and 0127] and HEO et al. (US 20210333124 A1), Abstract and Paragraph [0073 and 0122-0123]. Claims 12 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over KWON et al. (US 20190370661 A1), hereinafter referenced as KWON in view of HEO et al. (US 20210333124 A1), hereinafter referenced as HEO and in further view of TRYGG et al. (US 20200074269 A1), hereinafter referenced as TRYGG. Regarding claim 12, KWON in view of HEO explicitly teaches the determination device according to claim 11, KWON fails to explicitly teach wherein: the evaluation value is a distance between the typical value and the reference value; and the comparison value is set based on respective distances, each of the respective distances being between the reference value and a respective set of the plurality of sets of reference data. However, HEO explicitly teaches wherein: the evaluation value is a distance between the typical value and the reference value (Fig. 5. Paragraph [0069]-HEO discloses the domain adapter 440 may be configured to convert the captured road image and the projected road-layout map into first intermediate data and second intermediate data of the same feature space. In paragraph [0070]-HEO discloses the discriminator 444 included in the domain adapter 440 may be configured to receive the first intermediate data and the second intermediate data from each of the encoders 442_1 and 442_2, and calculate a domain difference between the first intermediate data and the second intermediate data. According to an embodiment, a difference between an expected value of an output value of the discriminator for first intermediate data and an expected value of an output value of the discriminator for second intermediate data may be calculated as the domain difference (wherein. Please also see paragraph [0122, 0134 and 0141]); and the comparison value is set based on respective distances, each of the respective distances being between the reference value and a respective set of the plurality of sets of reference data (Fig. 5. Paragraph [0076]-HEO discloses when the similarity value calculated by the similarity calculator 464 is equal to or greater than a preset threshold value, the change detector 480 may determine that there is no change in the current road information as compared with the projected road-layout map. When the calculated similarity value is equal to or less than a preset threshold value, the change detector 480 may determine that the change has occurred in the current road information as compared with the projected road-layout map. Further in paragraph [0095]-HEO discloses the metric learning artificial neural network included in the similarity determiner 460 is configured to calculate a similarity between a captured road image and a projected road-layout map using the triplet loss function that enforces a similarity between the captured road image and a positive image which is the projected road-layout map corresponding to the captured road image to be greater by a margin than a similarity between the captured road image and a negative image which is the projected road-layout map not corresponding to the captured road image. The loss function used for training the metric learning artificial neural network can be expressed as the following equation: Loss(Loss Function)=max (0, (d(x.sub.r, x.sub.p)+m−d(x.sub.r, x.sub.n))) (wherein x.sub.r is a feature value of the captured road image, x.sub.p is a feature value of the positive image, x.sub.n is a feature value of the negative image, and d(x, y) is a distance between x and y. Please also read paragraph [0111-0112 and 0122-0124]). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KWON in view of HEO of having a determination device, with the teachings of HEO of having wherein: the evaluation value is a distance between the typical value and the reference value; and the comparison value is set based on respective distances, each of the respective distances being between the reference value and a respective set of the plurality of sets of reference data. Wherein KWON’s method having wherein: the evaluation value is a distance between the typical value and the reference value; and the comparison value is set based on respective distances, each of the respective distances being between the reference value and a respective set of the plurality of sets of reference data. The motivation behind the modification would have been to obtain a method that improves the performance of machine learning models, since both KWON and HEO concern neural networks and intermediate data. Wherein KWON’s provides methods and systems that improve the reliability of machine learning, while HEO’s methods and systems improves the performance of machine learning models and the accuracy of detecting image differences. Please see KWON et al. (US 20190370661 A1), Abstract and Paragraph [0043 and 0127] and HEO et al. (US 20210333124 A1), Abstract and Paragraph [0073 and 0122-0123]. Although HEO explicitly teaches the comparison value is set based on respective distances, each of the respective distances being between the reference value and a respective set of the plurality of sets of reference data. KWON in view of HEO fail to explicitly teach a distribution of respective distances. However, TRYGG explicitly teaches a distribution of respective distances (Fig. 1. Paragraph [0142]-TRYGG discloses one way to detect outliers during prediction-time when using PLS may be to use Mahalanobis distance. The Mahalanobis distance is a known statistical distance measuring how many standard deviations away an observation is from the origin in a multivariate probability distribution. In paragraph [0145]-TRYGG discloses the method performed by the outlier detection module 104 to detect prediction-time outliers in a deep learning system. When input data is fed through a deep neural network, multiple intermediate representations of the data may exist, where the intermediate representations may be used for prediction (e.g., of a group into which the input data is classified in case the deep neural network is configured to solve a classification problem). One or more of these intermediate representations may be used for detecting outliers during prediction, as well as for performing the prediction). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KWON in view of HEO of having a determination device, with the teachings of TRYGG of having a distribution of respective distances. Wherein KWON’s method having the comparison value is set based on a distribution of respective distances, each of the respective distances being between the reference value and a respective set of the plurality of sets of reference data. The motivation behind the modification would have been to obtain a method that improves the performance of machine learning models, since both KWON and TRYGG concern neural networks and intermediate data. Wherein KWON’s provides methods and systems that improve the reliability of machine learning, while TRYGG’s methods and systems improves the performance and predictions of machine learning models. Please see KWON et al. (US 20190370661 A1), Abstract and Paragraph [0043 and 0127] and TRYGG et al. (US 20200074269 A1), Abstract and Paragraph [0002-0004 and 0178-0181]. Regarding claim 14, KWON in view of HEO explicitly teaches the determination device according to claim 12, KWON fails to explicitly teach wherein the comparison value is set based the respective distances. However, HEO explicitly teaches wherein the comparison value is set based on the respective distances (Fig. 5. Paragraph [0076]-HEO discloses when the similarity value calculated by the similarity calculator 464 is equal to or greater than a preset threshold value, the change detector 480 may determine that there is no change in the current road information as compared with the projected road-layout map. When the calculated similarity value is equal to or less than a preset threshold value, the change detector 480 may determine that the change has occurred in the current road information as compared with the projected road-layout map. Further in paragraph [0095]-HEO discloses the metric learning artificial neural network included in the similarity determiner 460 is configured to calculate a similarity between a captured road image and a projected road-layout map using the triplet loss function that enforces a similarity between the captured road image and a positive image which is the projected road-layout map corresponding to the captured road image to be greater by a margin than a similarity between the captured road image and a negative image which is the projected road-layout map not corresponding to the captured road image. The loss function used for training the metric learning artificial neural network can be expressed as the following equation: Loss(Loss Function)=max (0, (d(x.sub.r, x.sub.p)+m−d(x.sub.r, x.sub.n))) (wherein x.sub.r is a feature value of the captured road image, x.sub.p is a feature value of the positive image, x.sub.n is a feature value of the negative image, and d(x, y) is a distance between x and y. Please also read paragraph [0111-0112 and 0122-0124]). Although HEO explicitly teaches wherein the comparison value is set based on the respective distances. KWON in view of HEO fails to explicitly teaches a standard deviation of the respective distances. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KWON in view of HEO and in further view of TRYGG of having a determination device, with the teachings of TRYGG of having a standard deviation of the respective distances. Wherein KWON’s method having wherein the comparison value is set based on a standard deviation of the respective distances. The motivation behind the modification would have been to obtain a method that improves the performance of machine learning models, since both KWON and TRYGG concern neural networks and intermediate data. Wherein KWON’s provides methods and systems that improve the reliability of machine learning, while TRYGG’s methods and systems improves the performance and predictions of machine learning models. Please see KWON et al. (US 20190370661 A1), Abstract and Paragraph [0043 and 0127] and TRYGG et al. (US 20200074269 A1), Abstract and Paragraph [0002-0004 and 0178-0181]. Conclusion Listed below are the prior arts made of record and not relied upon but are considered pertinent to applicant`s disclosure. PARNABY et al. (US 20230041989 A1)- A method of base calling using at least two base callers is disclosed. The method includes executing at least a first base caller and a second base caller on sensor data generated for sensing cycles in a series of sensing cycles; generating, by the first base caller, first classification information associated with the sensor data, based on executing the first base caller on the sensor data; and generating, by the second base caller, second classification information associated with the sensor data, based on executing the second base caller on the sensor data. In an example, based on the first classification information and the second classification information, a final classification information is generated, where the final classification information includes one or more base calls for the sensor data. ..................................... Please see Para. [0119 and 0152-0154] . Abstract. TAKAFUMI et al. (US 20230316046 A1)- Methods and systems are disclosed for evaluating or training a machine learning module when its corresponding truth data sets are unavailable or unreliable. The methods and systems are configured for evaluating or training a target machine learning module having a first (system) input and a first output, wherein the target module is connected to a second machine learning module having an intermediate input (identical to the first output of the target module) and a second (system) output, by training the second module using received corresponding intermediate and output data sets, generating an evaluation data set using a received system input data set, and evaluating or training the target module using a loss function based on a distance metric between the evaluation data set and a received system output data set corresponding to the system input data set..................................... Please see Fig. 1-2 and 6-7. Abstract. AIZAWA et al. (US 20190362233 A1)- Training neural networks to recognize matching items requires large data sets and long training times. Conversely, training a neural network with triplets of similar objects instead of triplets of identical objects relaxes constraints on the size and content of the data set, making training easier. Moreover, the notion of “similarity” can be almost arbitrary, making it possible to train the neural network to associate objects that aren't visually similar. For instance, the neural network can be trained to associate a suit with a tie, which is not possible with training on identical objects. And because the neural network is trained to recognize similar items, it can also recognize unfamiliar items if they are similar enough to the training data. This is a technical improvement over other neural networks, which can only recognize identical items, and over collaborative filtering systems, which can only recognize items for which they have enough data.................................... Please see Fig. 2 and 4-5. Abstract. SUTIC et al. (US 20190050682 A1)- In an example method for training image signal processors, a reconstructed image is generated via an image signal processor based on a sensor image. An intermediate loss function is generated based on a comparison of an output of one or more corresponding layers of a computer vision network and a copy of the computer vision network. The output of the computer vision network is based on the reconstructed image. An image signal processor is trained based on the intermediate loss function................................... Please see Fig. 1-5. Abstract. HU et al. (US 20220398783 A1)- Disclosed are an image processing method, an image processing device, a neutral network and a training method thereof, and a storage medium. The image processing method includes: obtaining an input image; performing a segmentation process on the input image via a first encoding-decoding network, to obtain a first output feature map and the first segmented image; concatenating the first output feature map with at least one selected from the group consisting of the input image and the first segmented image, to obtain an input of the second encoding-decoding network; and performing a segmentation process on the input of the second encoding-decoding network via a second encoding-decoding network, to obtain the second segmented image. And the first encoding-decoding network and the second encoding-decoding network forms a neural network....................................... Please see Fig. 2-3 and 7 and para. [0089-0091, 0125-0127]. Abstract. DAS et al. (WO 2023282569 A1)- A method for generating an optimal neural network (NN) model may include determining intermediate outputs of the NN model by passing an input dataset through each intermediate exit gate of the plurality of intermediate exit gates, determining an accuracy score for each intermediate exit gate of the plurality of intermediate exit gates based on a comparison of the final output of the NN model with the intermediate output, identifying an earliest intermediate exit gate that produces the intermediate output closer to the final output based on the accuracy score, and generating the optimal NN model by removing remaining layers of the plurality of layers and remaining intermediate exit gates of the plurality of intermediate exit gates located after the determined earliest intermediate exit gate...................................... Please see Fig. 3-6 and 8. Abstract. TSUNODA et al. (US 20220366242 A1)- An information processing apparatus is operable to train a machine learning model that has a hierarchical structure configured by a plurality of hierarchical layers and that is used for recognizing a recognition target in inputted data. An obtaining unit obtains input data and data indicating a ground truth of an output from the machine learning model regarding the input data. A learning unit trains the machine learning model based on an error between the data indicating the ground truth of the output from the machine learning model regarding a specific domain of the input data and at least one output in an intermediate layer of the machine learning model with respect to the input data….................................. Please see Fig. 1-3 and 9-11, and para. [0058]. Abstract. Ukishima et al. (US 20210114368 A1)- There is provided a machine learning model generation device, a machine learning model generation method, a program, an inspection device, an inspection method, and a printing device for inspecting a defect of a printed matter with high accuracy. A machine learning model for detecting the defect of the printed matter is generated by using, as learning input information, at least learning inspection data based on a captured image obtained by capturing an image of a printed matter as an inspection target and second learning reference data based on print digital data and using, as learning output information, at least learning defect information of the learning inspection data estimated by performing comparison processing of the learning inspection data and first learning reference data based on a captured image obtained by capturing the image of the printed matter..................................... Please see Fig. 1-7. Abstract. LILLO (US 20220284261 A1)- Machine learning models are provided that consider, during the process of producing output, various aspects of the training data and/or training process from which the models are created. A machine learning model may generate output (e.g., classification determinations or regression output) that is augmented with information regarding the distribution(s) of the corpus of training data upon which the model was trained, the features extracted from the training data, the resulting determinations made by the model, and/or other information. The augmentation may occur internally while generating the model output, or the output itself may be augmented to include distribution-based data in addition to a model output................................... Please see Fig. 1-2 and 4-6. Abstract. Makhijani et al. (US 11080596 B1)- The present disclosure is directed to filtering co-occurrence data. In one embodiment, a machine learning model can be trained. An output of an intermediate structure of the machine learning model (e.g., an output of an internal layer of a neural network) can be used as a representation of an event. Similarities between representations of events can be determined and used to generate, augment, or modify co-occurrence data.................................. Please see Fig. 1, 4-5. Abstract. Rodríguez-Serrano et al. (US 20170083792 A1)- A system and method provide object localization in a query image based on a global representation of the image generated with a model derived from a convolutional neural network. Representations of annotated images and a query image are each generated based on activations output by a layer of the model which precedes the fully-connected layers of the neural network. A similarity is computed between the query image representation and each of the annotated image representations to identify a subset of the annotated images having the highest computed similarity. Object location information from at least one of the subset of annotated images is transferred to the query image and information is output, based on the transferred object location information................................. Please see Fig. 1-4 and para. [0058-0064 and 0088-0090] Abstract. TESHIMA et al. (US 20220129702 A1)- An image searching apparatus includes: a processor; and a memory, wherein the processor is configured to attach, to an image with a first correct label attached thereto, a second correct label, the first correct label being a correct label attached to each image included in an image dataset for training for use in supervised training, the second correct label being a correct label based on a degree of similarity from a predetermined standpoint; execute main training processing to train a classifier by using the images and one of the first correct label and the second correct label, fine-tune a training state of the classifier; trained by the main training processing, by using the images and the other one of the first correct label and the second correct label; and search, by using the classifier that is fine-tuned, for images similar to a query image..................................... Please see Fig. 1-5. Abstract Gonzales et al. (US 20220383128 A1)- An object analysis system is disclosed herein. The object analysis system may receive an input image that depicts an object. The object analysis system may determine, using a feature extraction model and from the input image, a first feature output that is associated with one or more features of the object. The feature extraction model may be trained based on reference images that depict reference objects that are a type of the object. The object analysis system may determine, using a classification model, that an anomaly status of the object is indicative of the object including an anomaly. The classification model may be trained based on the reference images. The object analysis system may determine, using an anomaly localization model, a location of the anomaly in the input image based on a second feature output of the convolutional neural network encoder. The anomaly localization model may be trained based on the reference images. The object analysis system may perform an action associated with the location of the anomaly...................................... Please see Fig. 1-2 and para. [0034-0042 and 0094]. Abstract. FROLOVA et al. (US 20210081754 A1)- Systems and methods are disclosed for error correction in convolutional neural networks. In one implementation, a first image is received. A first activation map is generated with respect to the first image within a first layer of the convolutional neural network. A correlation is computed between data reflected in the first activation map and data reflected in a second activation map associated with a second image. Based on the computed correlation, a linear combination of the first activation map and the second activation map is used to process the first image within a second layer of the convolutional neural network. An output is provided based on the processing of the first image within the second layer of the convolutional neural network................................. Please see Fig. 1-4. Abstract. OKAWA et al. (US 20220222581 A1)- A creation method for a computer to execute a process includes training a first detection model by using a first training data set; acquiring each of scores of a plurality of pieces of training data included in the first training data set by using the first detection model; creating a second training data set by excluding a part of the training data from the first training data set based on the scores; and training a second detection model by using the second training data set................................... Please see Fig. 1, 9-13 and 18. Abstract. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Aaron Bonansinga whose telephone number is (703) 756-5380 The examiner can normally be reached on Monday-Friday, 9:00 a.m. - 6:00 p.m. ET. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached by phone at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /AARON TIMOTHY BONANSINGA/Examiner, Art Unit 2673 /CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Read full office action

Prosecution Timeline

Jul 02, 2024
Application Filed
May 14, 2026
Non-Final Rejection mailed — §103
Aug 18, 2026
Applicant Interview (Telephonic)
Aug 20, 2026
Examiner Interview Summary
Sep 08, 2026
Response Filed
Nov 05, 2026
Final Rejection mailed — §103 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12789834
VISION-BASED MACHINE LEARNING MODEL FOR AGGREGATION OF STATIC OBJECTS AND SYSTEMS FOR AUTONOMOUS DRIVING
4y 2m to grant Granted Nov 10, 2026
Patent 12793176
REAL TIME LOCAL FILTERING OF ON-SCREEN IMAGES
4y 0m to grant Granted Nov 10, 2026
Patent 12770357
IMAGE PROCESSING METHOD AND IMAGE PROCESSING APPARATUS FOR RECOGNIZING TARGET OBJECT
2y 3m to grant Granted Oct 20, 2026
Patent 12755509
COMPRESSED SPATIAL FREQUENCY TRANSFORM FOR FEATURE TRACKING, IMAGE MATCHING, SEARCH, AND RETRIEVAL
3y 5m to grant Granted Oct 06, 2026
Patent 12741822
System and Method for Determining the Stability of an Object Being Transported on a Conveyor System
3y 1m to grant Granted Sep 22, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

3-4
Expected OA Rounds
79%
Grant Probability
99%
With Interview (+33.3%)
3y 1m (~8m remaining)
Median Time to Grant
Moderate
PTA Risk
Based on 43 resolved cases by this examiner. Grant probability derived from career allowance rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month