DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the Amendment filed December 17, 2025. Claims 1, 8, and 15 are amended. Claims 1-20 are pending in the case. Claims 1, 8, and 15 are the independent claims.
This action is final.
Applicant’s Response
In the amendment filed on December 17, 2025, Applicant amended the claims and provided arguments in response to the rejection of the claims under 35 USC 103 in the previous office action.
Response to Argument/Amendment
Applicant’s amendments to the claims in response to the rejection of the claims under 35 USC 103 are acknowledged, and Applicant’s corresponding arguments have been fully considered. Applicant argues that Hoshino, Miyamoto, Wold, and Inoue do not teach “determine which types of the data augmentation are suitable for training the artificial intelligence model based on the first data set” as recited in the amended independent claims. This argument is persuasive, and the rejection is withdrawn.
New grounds of rejection are provided below.
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims under pre-AIA 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various claims was commonly owned at the time any inventions covered therein were made absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and invention dates of each claim that was not commonly owned at the time a later invention was made in order for the examiner to consider the applicability of pre-AIA 35 U.S.C. 103(c) and potential pre-AIA 35 U.S.C. 102€, (f) or (g) prior art under pre-AIA 35 U.S.C. 103(a).
Claims 1-4, 6-11, 13-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Hoshino et al. (US 20200300778 A1) in view of Miyamoto et al. (US 20240135246 A1), further in view of Wold et al. (US 8244498 B2), further in view of Khonsari et al. (US 20220180123 A1).
With respect to claims, 1, 8, and 15, Hoshino teaches a system for data augmentation to train an artificial intelligence model comprising: a hardware processor; and a memory that stores a computer program product, which, when executed by the hardware processor, causes the hardware processor to perform a method; a computer program product for data augmentation to train an artificial intelligence model, the computer program product comprising a computer readable storage medium having computer readable program code embodied therewith, the program instructions executable by a processor to cause the processor to perform the method (e.g. paragraph 0034-0035, apparatus 300 including CPU/processor 301, storage devices 302, 303; processor controlling apparatus 300; storage device 303 for storing programs, etc.; paragraph 0090, processors reading out and executing computer-executable instructions recorded on storage medium to perform functions of described embodiments); and the computer-implemented method for data augmentation to train an artificial intelligence model comprising:
analyzing a first data set to measure a volume of data in the first data set, and a variation in scope of the volume of data in the first data set to determine deficiencies for training the artificial intelligence model (e.g. paragraph 0004, acquiring storage amount of training data; reference amount of training data set as an amount of the training data necessary for the training unit to train the training model; paragraph 0025, defect data is combination of defect type and positional information in captured image and may include additional detailed information; training data is pair of captured image of inspection target as subject and the defect data; paragraph 0026, displaying storage status of training data for each data classification allowing user to determine whether desirable training data has been stored; data is classified into each type by color and surface roughness of concrete wall surface; paragraphs 0027-0028, user and classifier working together to identify and input defect data in set of target images; paragraph 0029, to train the classifier it is necessary to prepare a sufficient amount of diverse types of the training data; pair of image and defect data used/stored as training data; determining when the amount of training data reaches the sufficient amount; paragraph 0030, displaying (and therefore determining) storage status of the training data for each type, allowing user to grasp the storage status of the training data for each type and acquire the necessary amount of the training data; paragraphs 0031-0032, describing Fig. 2, images in target image group displayed to user, user inputs defect data; window 210 displays progress bar for each type in progress bar display area 220 indicating storage status of the training data; bar of progress bar 221 has not reached upper limit indicated state where amount of training data is not sufficient to execute the training; paragraph 0038, setting reference amount of training data necessary for the training, referring to the amount of training data that needs to be trained to perform detection processing with demanded accuracy; reference amount preset as value indicating a lower limit; defect data required for at least X/reference amount of target images in predetermined size; paragraph 0039, determining whether storage amount of the training data has reached the set reference amount; paragraph 0042, Fig. 4, illustrating stored classification table used to determine whether training data having necessary properties has been stored; paragraph 0044, Fig. 5, setting reference amount of training data necessary for executing the training, such as amount predetermined for each image and each defect type; paragraph 0045, acquiring storage status of training data; paragraph 0047, acquiring properties of images and defect types, storing training data for each classification; based on information acquired through analysis and information in the data classification table, training data stored for each data classification; after the classification, by calculating the data amount for each data classification, data types/classifications included in the training data and the data amount for each classification can be acquired (as shown in Fig. 6B); in S504, the acquisition unit acquires the storage amount of the training data for each classification in this way; paragraph 0048, displaying storage status of training data as shown in Fig. 2; paragraph 0049, determining whether the acquired storage amount of the training data has reached the set reference amount; comprehensively determining whether storage amount of training data has reached predetermined amount for each of the data classifications/whether the storage amount acquired in S504 has reached the reference amount of the training data set in S501; paragraph 0050, determining whether the storage amount of the training data has reached the reference amount for each of the data classifications; paragraph 0053, describing various methods for determining whether the storage amount has reached the reference amount for each data classification; paragraph 0055, Fig. 7B, showing detailed display of storage status of training data, for each data classification, including storage amounts and ratios; paragraph 0065-0068, describing Fig. 10 as illustrating a method similar to that shown in Fig. 5, but augmented to include steps S1001 for analyzing and acquiring information about properties of inspection target images; i.e. the system/method analyzes a set of data/images which are currently stored as training data (analogous to a first data set) to determine various properties of the training data including a measured amount/volume of the data, the various different types/classifications of objects included within the data, and corresponding amounts and ratios for each of these different types classifications of the objects included within the data, where this information includes at least a measured volume of data (i.e. such as the volume/amount of the data as a whole) and information indicating variation in scope of the volume of the data (i.e. such as information corresponding to amounts corresponding to each of the different types/classifications present within the data, and ratios of these within the data as a whole));
augmenting data for the first data set having a volume of data measured failing to meet a threshold value, wherein deficiencies determined in the variation in scope of the volume of data in the first data set are augmented using augmentation methods to provide a second data set of augmented data (e.g. paragraphs 0031-0032, describing Fig. 2, user continuing to perform the defect data input operation while visually checking sequentially displayed images; window 211 indicates display state when defect data is input; pair of defect data and image stored as the training data and bar of progress bar 222 increases; when user inputs defect data, training data is stored for each type; window 212 displays state where defect data input operations have been performed on predetermined number of images and where sufficient amount of training data for each type has been stored; in progress bar display area 240 on window 212, bars of all of the progress bars have reached the upper limits; paragraph 0046, before defect data operations are completed on all images, result of operations performed so far is stored as the training data; paragraph 0052, if the determination unit determines that the storage amount has not reached the reference amount (NO in S506), skipping S507 and S509 and proceeding to S510, and then returning to S502, receiving defect data input operation (i.e. steps S502-S506 of Fig. 5)) for the next inspection target image (and continuing to repeat execution of the above-described process unit the storage amount does reach the reference amount, as shown in Fig. 5); paragraph 0063, while the amount of training data is smaller than the reference/necessary amount, the user continues the defect data input operation to compensate for an insufficient amount of the training data; in second embodiment, based on the storage status of the training data, data classification with which the amount of the training data is insufficient for the reference amount is identified, and an image having a background texture that coincides with/is similar to that of the data classification is selected for input of defect data and storing in training data; paragraph 0065-0068, describing Fig. 10 as illustrating a method similar to that shown in Fig. 5, but augmented to include steps S1001 for analyzing and acquiring information about properties of inspection target images, and then (following determination that the storage amount has not reached the reference amount) S1002 for selecting, from the remaining images in the target image group, an image for which in input of the defect data is to be prompted based on the properties of the inspection target image acquired in S1001 and storage status acquired in S504; as shown in Fig. 11B, training data for part of data classification 1111 is insufficient; to compensate for the insufficiency, inspection target image corresponding to classification 1111 is selected form the image group to support user’s defect data input operation to be preferentially performed on the selected image; selecting images having same properties as insufficient data classification, and preferentially performing input of defect data for these; paragraph 0068, input of defect data preferentially received until storage amount of the data classification 1111 reaches the reference amount; paragraph 0069, preferentially generating training data for data classification with insufficient storage amount; i.e. where the currently stored set of training data has an amount of data measured which fails to meet a threshold (such as the amount of the data as a whole, or an amount of a particular one of the different types/classifications within the data, falling below the set reference amount of data), the data set is augmented by selecting a set of additional training data points (i.e. images plus defect data, where these additional training data points are analogous to a second data set of augmented data, that is data which is selected in order to augment to original set of training data) until the amount of the additional training data points, when taken together with amount of the original set of training data, is sufficient to meet the threshold/reference values); and
training the artificial intelligence model with a combined data set of the first data set and the second data set of augmented data when the first and second data set have an amount of data meeting the threshold value (e.g. paragraph 0004, training unit configured to train training model to detect target from image based on stored training data; reference amount of training data set as amount necessary to train the training model; paragraph 0025, classifier of defect detection method trained using the training data; paragraph 0029, when the amount of training data reaches the sufficient amount, training of the classifier is executed using the stored training data; paragraph 0032, when sufficient amount of training data for each type stored, prompting user to issue instruction for executing the training; paragraph 0039, when storage amount of training data has reached the set reference amount, notifying display control unit to present GUI for issuing instruction for executing training; paragraph 0040, after the storage amount of the training data has reached the set reference amount, training unit trains training model based on the stored training data; paragraph 0046, when storage amount of the data reaches the reference amount, it becomes possible to execute the training to perform classifier-based detection and identification of crack remaining images; paragraph 0051, when the determination unit determines that the storage amount has reached the reference amount, displaying execute training button to prompt user to execute the training; determining whether user has issued instruction for executing the training, and proceeding to execute the training of the classifier/model based on the stored training data; paragraph 0059, apparatus may automatically execute the training without instruction by the user; if the determination unit determines that the storage amount has reached the reference amount, then processing proceeds and the training unit executes the training; by automatically executing the training, it is possible to avoid the training data from being stored more than necessary and to prevent the user from forgetting to execute the training; i.e. once a sufficient amount of additional training data (i.e. analogous to a second data set of augmented data) has been selected to augment the original set of the training data (analogous to the first data set) and the combined amount of both of these is sufficient to meet the threshold/reference values, this combined training data is utilized to train an artificial intelligence model).
Hoshino does not explicitly disclose the augmenting is by automatically using augmentation methods selected for increasing the variation in scope of the volume of data to provide the second data set of augmented data with the volume data and the variation in scope to improve robustness in training of the artificial intelligence model. However, Miyamoto teaches the augmenting is by automatically using augmentation methods selected for increasing the variation in scope of the volume of data to provide the second data set of augmented data with the volume data and the variation in scope (e.g. paragraph 0016, raw data including image data; target portion includes image of a target; performing data augmentation to change at least one of an environment of the target or an imaging condition for the target in an image included in the raw data, allowing generation of various sets of training data usable for generalization learning in training a learning model for image recognition; paragraph 0017-0018, performing data augmentation to change environment of the target by changing at least one of a brightness, a background, or a color tone of the image included in the raw data; this allows generation of sets of training data simulating changes in environments for imaging, such as time and weather; augmentation to change the imaging condition for the target by performing at least one of rotating, inverting, enlarging, reducing, moving, trimming, or filtering of the image included in the raw data; this allows generation of sets of training data simulating different imaging conditions; i.e. in addition to augmenting the data set by including additional data points/images such that threshold amounts for each of the different classifications/types are reached, the dataset may be further augmented by implementing various alterations to the raw data, such as by changing brightness, background, color, performing rotation, inversion, enlargement/reduction, etc. of various elements within images in the raw data, where such augmentation methods are outside of the variation of the scope of the volume of data at least to the extent that the corresponding augmentations may introduce additional variations in scope of image data in the set which were not previously present) to improve robustness in training of the artificial intelligence model (e.g. paragraph 0003, when insufficient volume of training data available, performing data augmentation to increase the data volume; paragraph 0007, generating training data that improves the generalization performance of a learning model; paragraph 0010, generating new data for generalization learning, improving generalization performance; building machine learning model that is robust against domain shifts that occur from distributions of unknown domains).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Hoshino and MIyamoto in front of him to have modified the teachings of Hoshino (directed to machine learning for detecting targets from images), to incorporate the teachings of Miyamoto (directed to machine learning data generation and meta-learning) to include the capability to further perform the augmentation operation, resulting in the second data set (i.e. as taught by Hoshino), by performing various augmentations to the image data included in the raw data, such as by changing brightness, background, color, performing rotation, inversion, enlargement/reduction, etc. of various elements within images in the raw data, resulting in generation of additional sets of training data simulating different imaging conditions as compared to what was originally present in the raw data. One of ordinary skill would have been motivated to perform such a modification in order to generate training data that improves generalization performance of a learning model, such as in an instance where insufficient volume of training data is available, generating new data for generalization learning without being costly, and improving generalization performance in generalization learning as described in Miyamoto (paragraphs 0003, 0007, 0010).
Hoshino and Miyamoto do not explicitly disclose assigning varied weights to change a scope of the first data set, and determining variations in the volume of data not present in the first data set that improve using the varied weights. However, Wold teaches assigning varied weights to change a scope of the first data set, and determining variations in the volume of data not present in the first data set that improve using the varied weights (e.g. col. 7 lines 6-11, measured data represented in graph as plurality of data points, each representative of measured data/observations; col. 7 lines 25-36, data points representative of entries in data matrices; col. 8 lines 23-25, modifying scaling weights/variances to achieve objectives for particular data set, such as devaluing noisy/irrelevant variables or increasing contribution of certain variables; col. 9 lines 19-52, utilizing relationship between variance in data matrices and amount of data in potential data sub-groups/partitions; optimizing particular data partition by overall improvement in combination of variances and function associated with the amount of data in each sub-group; penalty function that encourages approximately equal numbers of observations in each sub-group; adjustable parameter relating to size of sub-groupings; if value is closer to 0, less likely that partition will result in groups of approximately equal sizer; if value is closer to 1, more likely that partition will result in sub-groups of approximately equal size after partition; i.e. various weighting amounts used in a data partition/organization function are utilized to change the scope of the data set and variations in volumes of data, such as by assigning weights which emphasize or deemphasize certain types of data/variables (i.e. to ensure that there is a proper amount of influence from these within the dataset, and to reduce influence of undesirable types), and also by utilizing weights to change the amount/volume of data for different subgroupings within the dataset).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Hoshino, Miyamoto, and Wold in front of him to have modified the teachings of Hoshino (directed to machine learning for detecting targets from images) and Miyamoto (directed to machine learning data generation and meta-learning), to incorporate the teachings of Wold (directed to organizing data) to include the capability to further perform the augmentation operation using assigned weightings for improving the variations in volume of the data set. One of ordinary skill would have been motivated to perform such a modification in order to provide for analysis and clustering of large datasets without the computational intensity and cost associated with earlier approaches, preserve relationships between variables in the datasets, including when data are missing from the dataset, using methods having relatively fast processing time that facilitates relatively fast computation and presentation of data as described in Wold (col. 3 lines 7-41).
Hoshino, Miyamoto, and Wold do not explicitly disclose analyzing the first data set to determine which types of the data augmentation are suitable for training the artificial intelligence model based on the first data set. However, Khonsari teaches analyzing the first data set to determine which types of the data augmentation are suitable for training the artificial intelligence model based on the first data set (e.g. paragraphs 0087-0088, different augmentation transformations result in different accuracies for different types of training data sets; selecting candidate transformations from list and setting probabilities with which selected transformation may be applied to training samples in the initial data set; paragraphs 0099-0101, after processing all transformations, augmenting the original training samples in training data set by applying transformations in updated default augmentation transformations list and updated candidate augmentation transformations list and storing the augmented training samples as a set of final augmented training samples in training data set; after processing all transformations, selecting n transformations from updated default augmentation transformations list and updated candidate augmentation having top n affinity metrics and training machine learning model in accordance with original training samples and training samples augmented using n transformations; for different types of training data sets providing custom default augmentation transformations list and candidate augmentation transformations list taking into consideration the characteristics of training samples in the training data set).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Hoshino, Miyamoto, Wold, and Khonsari in front of him to have modified the teachings of Hoshino (directed to machine learning for detecting targets from images), Wold (directed to organizing data), and Miyamoto (directed to machine learning data generation and meta-learning), to incorporate the teachings of Khonsari (directed to training a machine learning model) to include the capability to analyze the training data set to determine types of data augmentation that are suitable for the model based on the data set (as taught by Khonsari). One of ordinary skill would have been motivated to perform such a modification in order to develop an augmentation technique selection system which is reliable, accurate, and capable of providing fast results with low computational overhead as described in Khonsari (paragraphs 0007-0008).
With respect to claims 2, 9, and 16, Hoshino in view of Miyamoto, further in view of Wold, further in view of Khonsari teaches all of the limitations of claims 1, 8, and 15 as previously discussed, and Hoshino further teaches wherein the artificial intelligence model is for a machine vision application, and further comprising (the system/computer program product) detecting objects from digital images for designation of content employing the artificial intelligence model (e.g. paragraph 0004, model trained to detect detection target from an image; paragraph 0028, classifier trained to support processing for identifying defect type (i.e. included in an image); paragraph 0061, checking for defects in captured image; not limited to specific field, and exemplary embodiment is also applicable to outer appearance inspection for checking a fault such as a scar (detection target), from an image of a product (inspection target) in a factor, and a medical diagnostic operation for checking a lesion (detection target) from a captured image of a human body (inspection target) in a hospital).
With respect to claims 3, 10, and 17, Hoshino in view of Miyamoto, further in view of Wold, further in view of Khonsari all of the limitations of claims 1, 8, and 15 as previously discussed, and Hoshino further teaches wherein the first data set is a plurality of images, wherein one image type is an object type for being analyzed as the first data set (e.g. paragraph 0025, defect data is combination of defect type and positional information in captured image and may include additional detailed information; training data is pair of captured image of inspection target as subject and the defect data; paragraph 0026, stored training data for each data classification; data is classified into each type by color and surface roughness of concrete wall surface; paragraph 0042, describing Fig. 4’s data classification table which illustrates classification information for classifying the properties of the training data, including background features and detection targets/objects; i.e. the stored training data (analogous to a first data set) includes a plurality of training images classified according to types of objects present within the images).
With respect to claims 4, 11, and 18, Hoshino in view of Miyamoto, further in view of Wold, further in view of Khonsari teaches all of the limitations of claims 1, 8, and 15 as previously discussed, and Hoshino further teaches wherein the volume of data in the first data set includes a measurement of the global number of elements in the first data set and an object number of data in the first data set (e.g. paragraph 0044, Fig. 5, setting reference amount of training data necessary for executing the training, such as amount predetermined for each image and each defect type; paragraph 0047, calculating the data amount for each data classification, data types/classifications included in the training data and the data amount for each classification can be acquired (as shown in Fig. 6B); storage amount of the training data for each classification; paragraph 0049, comprehensively determining whether storage amount of training data has reached predetermined amount for each of the data classifications/whether the storage amount acquired in S504 has reached the reference amount of the training data set in S501; paragraph 0050, determining whether the storage amount of the training data has reached the reference amount for each of the data classifications; paragraph 0053, describing various methods for determining whether the storage amount has reached the reference amount for each data classification, including performing the determination independently for each data classification and/or determining whether the total storage amount of the training data has reached the reference amount; paragraph 0055, Fig. 7B, showing detailed display of storage status of training data, for each data classification, including storage amounts and ratios; i.e. the measured amount/volume of the training data may include both a total amount (analogous to a global number of elements in the dataset) and individual amounts corresponding to each type/classification (analogous to an object number of data in the data set)).
With respect to claims 6, 13, and 20, Hoshino in view of Miyamoto, further in view of Wold, further in view of Khonsari teaches all of the limitations of claims 1, 8, and 15 as previously discussed, and Hoshino and Miyamoto further teach wherein the first data set includes at least one image (e.g. Hoshino paragraph 0025, defect data is combination of defect type and positional information in captured image and may include additional detailed information; training data is pair of captured image of inspection target as subject and the defect data; paragraph 0026, stored training data for each data classification; data is classified into each type by color and surface roughness of concrete wall surface; paragraph 0042, describing Fig. 4’s data classification table which illustrates classification information for classifying the properties of the training data, including background features and detection targets/objects; Miyamoto paragraph 0012, raw data including target portion and non-target portion; data augmentation to change non-target portion; changing non-target portion as appropriate, without changing the target portion; target portion directly affects learning target and is portion targeted by machine learning task; paragraph 0016, raw data including image data; target portion includes image of a target).
Miyamoto additionally teaches the augmenting data for the first data set includes a method selected from the group consisting of de-texturing, de-coloring, edge enhancement, a flip/rotate image analysis, cropping of the image, downscaling of the image, upscaling of the image, color conversion of the image, noise variation directed to color of the image, coarse dropout of the image, SMOTE sampling of the image, sample pairing of the image, mixup analysis of the image and combinations thereof (e.g. paragraph 0017-0018, performing data augmentation to change environment of the target by changing at least one of a brightness, a background, or a color tone of the image included in the raw data; this allows generation of sets of training data simulating changes in environments for imaging, such as time and weather; augmentation to change the imaging condition for the target by performing at least one of rotating, inverting, enlarging, reducing, moving, trimming, or filtering of the image included in the raw data; this allows generation of sets of training data simulating different imaging conditions; paragraph 0088, operations for augmenting image data include rotating, inverting, enlarging, reducing, moving, trimming, and filtering of an image included in raw data, changing the brightness of the image included in the raw data (to increase variations of weather and a time period), changing the background, and changing the color tones (to increase variations of weather and a time period)).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Hoshino, Wold, Khonsari, and Miyamoto in front of him to have modified the teachings of Hoshino (directed to machine learning for detecting targets from images), to incorporate the teachings of Miyamoto (directed to machine learning data generation and meta-learning), Khonsari (directed to training a machine learning model), and Wold (directed to organizing data), to include the capability to further perform the augmentation operation, resulting in the second data set (i.e. as taught by Hoshino), by performing various augmentations to the image data included in the raw data, such as by changing brightness, background, color, performing rotation, inversion, enlargement/reduction, etc. of various elements within images in the raw data, resulting in generation of additional sets of training data simulating different imaging conditions as compared to what was originally present in the raw data. One of ordinary skill would have been motivated to perform such a modification in order to reason as generate training data that improves generalization performance of a learning model, such as in an instance where insufficient volume of training data is available, generating new data for generalization learning without being costly, and improving generalization performance in generalization learning as described in Miyamoto (paragraphs 0003, 0007, 0010).
With respect to claims 7 and 14, Hoshino in view of Miyamoto, further in view of Wold, further in view of Khonsari all of the limitations of claims 1 and 8 as previously discussed, and Hoshino further teaches the method further comprising recalculating size of the combined data set of the first data set and the second data set to determine if the threshold value has been met (e.g. paragraph 0046, before defect data operations are completed on all images, result of operations performed so far is stored as the training data; paragraph 0047, in S504, acquiring the storage amount of the training data for each classification; paragraph 0049, in S506, determining whether the storage amount acquired in S504 has reached the reference amount of the training data set in S501; paragraph 0052, if the determination unit determines that the storage amount has not reached the reference amount (NO in S506), skipping S507 and S509 and proceeding to S510, and then returning to S502, receiving defect data input operation (i.e. steps S502-S506 of Fig. 5)) for the next inspection target image (and continuing to repeat execution of the above-described process unit the storage amount does reach the reference amount, as shown in Fig. 5); paragraph 0063, while the amount of training data is smaller than the reference/necessary amount, the user continues the defect data input operation to compensate for an insufficient amount of the training data; in second embodiment, based on the storage status of the training data, data classification with which the amount of the training data is insufficient for the reference amount is identified, and an image having a background texture that coincides with/is similar to that of the data classification is selected for input of defect data and storing in training data; paragraph 0065-0068, describing Fig. 10 as illustrating a method similar to that shown in Fig. 5; i.e. as shown in Figs. 5 and 10, the process includes repeatedly performing the process of annotating and adding images to the training data until the stored amount exceeds the threshold amount, where this includes repeating the step of acquiring storage status of the training data S504 and determining whether the storage amount has reached the reference amount S506, analogous to recalculating size of the combined data set of the first data set (i.e. the original set of training data) and the second data set (i.e. training data added while repeatedly performing the processes of Figs. 5 and 10) to determine if the threshold value has been met).
Claims 5, 12, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Hoshino in view of Miyamoto, further in view of Wold, further in view of Khonsari, further in view of Tamilselvam (US 20210264283 A1).
With respect to claims 5, 12, and 19, Hoshino in view of Miyamoto, further in view of Wold, further in view of Khonsari teaches all of the limitations of claims 1, 8, and 15 as previously discussed. Hoshino and Miyamoto do not explicitly disclose wherein the variation in scope of the volume of data in the first data set is measured by a method selected from the group consisting of object area histogram analysis, object rotation histogram analysis, grey history analysis, similarity distribution and combinations thereof.
However, Tamilselvam teaches wherein the variation in scope of the volume of data in the first data set is measured by a method selected from the group consisting of object area histogram analysis, object rotation histogram analysis, grey history analysis, similarity distribution and combinations thereof (e.g. paragraph 0016, training datasets including images; paragraph 0026, measuring amount of variance, variability, or diversity for each aspect within training dataset, such as using known measurement techniques including similarity measures, cosine similarity, clustering techniques, affinity measurements, class distribution measures, and the like; paragraph 0039, identifying class distribution or variance in data for particular classes to generate policies identifying which aspects or classes of the training data need to be augmented to result in desired variance or distribution for that class).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Hoshino, Miyamoto, Wold, Khonsari, and Tamilselvam in front of him to have modified the teachings of Hoshino (directed to machine learning for detecting targets from images), Wold (directed to organizing data), Khonsari (directed to training a machine learning model), and Miyamoto (directed to machine learning data generation and meta-learning), to incorporate the teachings of Tamilselvam (directed to dataset creation for deep learning models) to include the capability to further measure the variation in the amount of data in the first data set using a similarity distribution method (as taught by Tamilselvam). One of ordinary skill would have been motivated to perform such a modification in order to ensure sufficient variability across a machine learning training dataset as described in Tamilselvam (paragraph 0019).
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. “The use of patents as references is not limited to what the patentees describe as their own inventions or to the problems with which they are concerned. They are part of the literature of the art, relevant for all they contain,” In re Heck, 699 F.2d 1331, 1332-33, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting in re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (GCPA 1968)). Further, a reference may be relied upon for all that it would have reasonably suggested to one having ordinary skill the art, including nonpreferred embodiments. Merck & Co, v. Biocraft Laboratories, 874 F.2d 804, 10 USPQ2d 1843 (Fed. Cir.), cert, denied, 493 U.S. 975 (1989). See also Upsher-Smith Labs. v. Pamlab, LLC, 412 F,3d 1319, 1323, 75 USPQ2d 1213, 1215 (Fed. Cir, 2005): Celeritas Technologies Ltd. v. Rockwell International Corp., 150 F.3d 1354, 1361, 47 USPQ2d 1516, 1522-23 (Fed. Cir. 1998).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEREMY L STANLEY whose telephone number is (469)295-9105. The examiner can normally be reached on Monday-Friday from 9:00 AM to 5:00 PM CST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Al Kawsar, can be reached at telephone number (571) 270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR for authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/JEREMY L STANLEY/
Primary Examiner, Art Unit 2127