DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The action is in response to the amendment filed on 9/04/2025. Claims 1-7, 11-15, and 17-22 have been amended. Claims 8-10 and 16 are cancelled. Claims 1-7, 11-15, and 17-22 are pending and have been examined.
Claim Rejections - 35 USC § 112
The rejections under 35 USC § 112 to Claims 1-18 and 20-22 are WITHDRAWN in view of Applicant’s amendments to Claims 1-7, 11-15, 17-18, 20-22.
Claim Rejections - 35 USC § 101
The rejections under 35 USC § 101 to Claims 1-19 and 22 are WITHDRAWN in view of Applicant’s amendments to Claims 1, 2, 19, and 22.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-7, 11-15, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Kuchnik et al. “Efficient Augmentation via Data Subsampling”, hereinafter “Kuchnik” in view of Koh et al. “Understanding Black-box Predictions via Influence Functions”, from applicant IDS, hereinafter “Koh”, further in view of Barshan et al. (U.S. Patent Application Publication No. US 20210103829 A1), hereinafter “Barshan”, further in view of Ando et al. (U.S. Patent Application Publication No. US 20190377982 A1), hereinafter “Ando”.
Regarding Claim 1, Kuchnik teaches:
A data adjustment device comprising:
wherein the information processing apparatus includes processing circuitry (Kuchnik, p. 13, ¶2, “We perform experiments in Python” demonstrates that Kuchnik performs their method on a computer, in which processor, memory, and storage devices are inherent) configured to
measure a degree of influence of learning data on learning in a neural network... the learning data being used for the learning, and (Kuchnik, p. 13, ¶1, “influence can then be measured for each training point, and can therefore be used as scores”)
adjust the learning data (Kuchnik, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded”), wherein the processing circuitry for adjusting the learning data is further configured to
acquire second data… the second data corresponding to data measured as having a degree of influence above a second threshold higher than the first threshold (Second threshold of degree of influence is the top k values, Kuchnik, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded”, p. 3, ¶3, “augmenting 5, 10, and 25 percent of the data.. via our proposed policies (to be discussed in Section 4)”, Section 4, p. 4, ¶3, “model influence, by which augmentation scores are generated… we select a subset… by ordering the points… and taking the top k values”), wherein data having a degree of influence above the second threshold improves model identification accuracy (Training data is used to improve model accuracy and data is subsampled to find most influential data which contributes to how best a model will be able to predict, Kuchnik, p. 3, paragraph 2, “subsampling is performed with the ultimate aim being to retain the accuracy”), and
generate third data using learning data having a degree of influence above the second threshold as a template for generating the additional the third data (Data generated from an augmentation uses high-influence data as a template when performing an augmentation such as rotation, Kuchnik, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded by applying class-preserving transformations, such as rotations or crops for images, to the original data points.”),
add one or more of the second data or the third data to the learning data (Kuchnik, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded”), and
automatically execute relearning of the neural network using the added learning data having increased high-influence data to improve model accuracy without human intervention (cross validation does automatic relearning, Kuchnik, p. 6, paragraph 3, “We controlled for augmentation-induced regularization by performing a simple cross validation sweep for the regularization parameter λ each time the model was re-trained”, p. 1, Abstract, “90% reduction in augmentation set size while maintaining the accuracy gains of standard data augmentation”).
Kuchnik does not expressly teach:
an information processing apparatus; and
a terminal device
using an influence function that calculates parameter changes without relearning
exclude first data measured as having a degree of influence below a first threshold
… from the terminal device or a database…
However, Koh teaches:
using an influence function that calculates parameter changes without relearning (Koh, p. 2, col. 1, paragraph 4, “we form a quadratic approximation… we can linearly approximate the parameter change due to removing z by computing ˆθ−z − ˆθ ≈ − 1 n Iup,params(z), without retraining the model”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the influence function of Koh with the data augmentation via subsampling of Kuchnik. The motivation to do so would be able to find the influence of data points more efficiently without the need for retraining (Koh, p. 2, col. 1, paragraph 3, “retraining the model for each removed z is prohibitively slow. Fortunately, influence functions give us an efficient approximation”).
Kuchnik in view of Koh does not expressly teach:
an information processing apparatus; and
a terminal device
exclude first data measured as having a degree of influence below a first threshold
… from the terminal device or a database…
However, Barshan teaches:
an information processing apparatus and a terminal device (Barshan, ¶50, “FIG. 1 illustrates a computing environment 100, which may be used to implement and/or execute any of the methods described herein. In some embodiments, the computing environment 100 may be implemented by any of a conventional personal computer, a computer dedicated to managing network resources, a network device and/or an electronic device (such as, but not limited to, a mobile device, a tablet device, a server, a controller unit, a control device, etc”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use an information processing apparatus and a terminal device for executing the method, as does Barshan, in the invention of Kuchnik. The motivation to do so would be to be able to take advantage of the benefits of efficient data augmentation for machine learning on these devices.
exclude first data measured as having a degree of influence below a first threshold (first threshold is highest value of training points determined to have a lowest normalized indicator of influence, Barshan, ¶9, “determining one or more training data points, from the plurality of training data points, having a highest normalized indicator of influence, and determining one or more training data points, from the plurality of training data points, having a lowest normalized indicator of influence”, ¶63, “the user may wish to identify training data points causing erroneous predictions and remove those labeled training data points from the database of labeled training data points”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Barshan excluding data with a low degree of influence and the influence measuring with data augmentation taught by Kuchnik. The modification would have been motivated to reduce erroneous predictions (Barshan, ¶63, “the user may wish to identify training data points causing erroneous predictions and remove those labeled training data points from the database of labeled training data points”).
Kuchnik in view of Koh and Barshan does not expressly teach:
… from the terminal device or a database…
However, Ando teaches:
… from the terminal device or a database… (Ando, ¶140, “One or more embodiments provide a processing method for generating learning data, which may include: a step of specifying requirement information for generating learning data, based on request information for making a request for learning; and a step of transmitting the requirement information to a device that generates the learning data”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Ando transmitting a request for new data to an external device and the addition of new data through augmentation from Kuchnik. The modification would have been to obtain more data for a larger training set.
Regarding Claim 2, Kuchnik teaches:
A data adjustment device comprising:
processing circuitry (Kuchnik, p. 13, ¶2, “We perform experiments in Python” demonstrates that Kuchnik performs their method on a computer, in which processor, memory, and storage devices are inherent) configured to
measure a degree of influence of learning data on learning of a model by machine learning… the learning data being used for the learning (Kuchnik, p. 13, ¶1, “influence can then be measured for each training point, and can therefore be used as scores”); and
adjust the learning data (Kuchnik, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded”), wherein the processing circuitry for adjusting the learning data is further configured to
acquire new data… corresponding to learning data measured as having a degree of influence above the second threshold (Threshold of degree of influence is the top k values, Kuchnik, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded”, p. 3, ¶3, “augmenting 5, 10, and 25 percent of the data.. via our proposed policies (to be discussed in Section 4)”, Section 4, p. 4, ¶3, “model influence, by which augmentation scores are generated… we select a subset… by ordering the points… and taking the top k values”), wherein data having a degree of influence above the second threshold improves model identification accuracy (Training data is used to improve model accuracy and data is subsampled to find most influential data which contributes to how best a model will be able to predict, Kuchnik, p. 3, paragraph 2, “subsampling is performed with the ultimate aim being to retain the accuracy”),
generate additional high-influence data using learning data having a degree of influence above the second threshold as a template for generating the additional high-influence data (Data generated from an augmentation uses high-influence data as a template when performing an augmentation such as rotation causing the data generated to also be high-influence, Kuchnik, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded by applying class-preserving transformations, such as rotations or crops for images, to the original data points.”),
add one or more of the acquired new data or the generated additional high- influence data to the learning data to create an optimized training dataset (Kuchnik, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded”), and
automatically execute relearning of the model using the optimized training dataset with increased high-influence data to improve model accuracy without human intervention (cross validation does automatic relearning, Kuchnik, p. 6, paragraph 3, “We controlled for augmentation-induced regularization by performing a simple cross validation sweep for the regularization parameter λ each time the model was re-trained”, p. 1, Abstract, “90% reduction in augmentation set size while maintaining the accuracy gains of standard data augmentation”).
Kuchnik does not expressly teach:
using an influence function that calculates parameter changes without relearning
exclude learning data measured as having a degree of influence below a first threshold
… from an external device or a database…
However, Koh teaches:
using an influence function that calculates parameter changes without relearning (Koh, p. 2, col. 1, paragraph 4, “we form a quadratic approximation… we can linearly approximate the parameter change due to removing z by computing ˆθ−z − ˆθ ≈ − 1 n Iup,params(z), without retraining the model”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the influence function of Koh with the data augmentation via subsampling of Kuchnik. The motivation to do so would be able to find the influence of data points more efficiently without the need for retraining (Koh, p. 2, col. 1, paragraph 3, “retraining the model for each removed z is prohibitively slow. Fortunately, influence functions give us an efficient approximation”).
Kuchnik in view of Koh does not expressly teach:
exclude learning data measured as having a degree of influence below a first threshold
… from an external device or a database…
However, Barshan teaches:
exclude learning data measured as having a degree of influence below a first threshold (first threshold is highest value of training points determined to have a lowest normalized indicator of influence, Barshan, ¶9, “determining one or more training data points, from the plurality of training data points, having a highest normalized indicator of influence, and determining one or more training data points, from the plurality of training data points, having a lowest normalized indicator of influence”, ¶63, “the user may wish to identify training data points causing erroneous predictions and remove those labeled training data points from the database of labeled training data points”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Barshan excluding data with a low degree of influence and the influence measuring with data augmentation taught by Kuchnik. The modification would have been motivated to reduce erroneous predictions (Barshan, ¶63, “the user may wish to identify training data points causing erroneous predictions and remove those labeled training data points from the database of labeled training data points”).
Kuchnik in view of Koh and Barshan does not expressly teach:
… from an external device or a database…
However, Ando teaches:
… from an external device or a database… (Ando, ¶140, “One or more embodiments provide a processing method for generating learning data, which may include: a step of specifying requirement information for generating learning data, based on request information for making a request for learning; and a step of transmitting the requirement information to a device that generates the learning data”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Ando transmitting a request for new data to an external device and the addition of new data through augmentation from Kuchnik. The modification would have been to obtain more data for a larger training set.
Regarding Claim 3, Kuchnik in view of Koh, Barshan, and Ando teaches the data adjustment device of Claim 2 as referenced above. Kuchnik further teaches:
measure the degree of influence on a basis of a loss function (Kuchnik, p. 4, ¶4, “One method to obtain augmentation scores is the loss at a point in the training set”).
Regarding Claim 4, Kuchnik in view of Koh, Barshan, and Ando teaches the data adjustment device of Claim 2 as referenced above. Kuchnik further teaches:
measure the degree of influence using a technique allowing for measuring the degree of influence (Leave One-Out influence is a technique for measuring influence, p. 4, ¶5, “We also explore policies based on Leave-One-Out (LOO) influence, which measures the influence that a training data point has against its own loss when it is removed from the training set.”).
Regarding Claim 5, Kuchnik in view of Koh, Barshan, and Ando teaches the data adjustment device of Claim 4 as referenced above. Koh further teaches:
measure the degree of influence using an influence function (Koh, p. 1, Abstract, “we use influence functions” , p. 2, col. 1, paragraph 4, “influence functions give us an efficient approximation. The idea is to compute the parameter change if z were upweighted by some small ϵ”)
Regarding Claim 6, Kuchnik teaches the data adjustment device of Claim 2 as referenced above. Kuchnik further teaches:
measure the degree of influence of the predetermined data on a basis of a difference between a case of the learning data and a case of excluding the predetermined data from the learning data (Every piece of data is predetermined to be excluded from the training set during the process of measuring its influence, p. 4, ¶5, “We also explore policies based on Leave-One-Out (LOO) influence, which measures the influence that a training data point has against its own loss when it is removed from the training set”).
Regarding Claim 7, Kuchnik in view of Koh, Barshan, and Ando teaches the data adjustment device of Claim 2 as referenced above. Barshan further teaches:
exclude a first piece of data measured as having a degree of influence below a first threshold from the learning data (first threshold is highest value of training points determined to have a lowest normalized indicator of influence, Barshan, ¶9, “determining one or more training data points, from the plurality of training data points, having a highest normalized indicator of influence, and determining one or more training data points, from the plurality of training data points, having a lowest normalized indicator of influence”, ¶63, “the user may wish to identify training data points causing erroneous predictions and remove those labeled training data points from the database of labeled training data points”).
Regarding Claim 11, Kuchnik in view of Koh, Barshan, and Ando teaches the data adjustment device of Claim 2 as referenced above. Kuchnik further teaches:
add the new data acquired from the external device to the learning data (New data created from augmentation is added to learning data, Kuchnik, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded by applying class-preserving transformations, such as rotations or crops for images, to the original data points.”, p. 3, ¶3, “we seek to make data augmentation more efficient by providing effective policies for subsampling the original training dataset”).
Kuchnik does not expressly teach:
transmit request information to an external device, the request information being used to request the new data
However, Ando teaches:
transmit request information to an external device, the request information being used to request the new data (Ando, ¶140, “One or more embodiments provide a processing method for generating learning data, which may include: a step of specifying requirement information for generating learning data, based on request information for making a request for learning; and a step of transmitting the requirement information to a device that generates the learning data”)
Regarding Claim 12, Kuchnik in view of Koh, Barshan, and Ando teaches the data adjustment device of Claim 2 as referenced above. Kuchnik further teaches:
add the new data acquired from a memory to the learning data, the memory being configured to store data (p. 13, ¶2, “We perform experiments in Python” demonstrates that Kuchnik performs their method on a computer, in which processor, memory, and storage devices are inherent. The storage device can store and retrieve the new data)
Regarding Claim 13, Kuchnik in view of Koh, Barshan, and Ando teaches the data adjustment device of Claim 2 as referenced above. Kuchnik further teaches:
generate the new data and add the generated new data to the learning data (New data is generated through data augmentation, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded”, p.13, ¶1, “applying augmentations to the training set”)
Regarding Claim 14, Kuchnik in view of Koh, Barshan, and Ando teaches the data adjustment device of Claim 13 as referenced above. Kuchnik further teaches:
generate the new data using the second piece of data and add the generated new data to the learning data (New data is generated through augmentation using the second piece of data and is added to training set, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded”, p. 3, ¶3, “augmenting 5, 10, and 25 percent of the data.. via our proposed policies (to be discussed in Section 4)”, Section 4, p. 4, ¶3, “model influence, by which augmentation scores are generated… we select a subset… by ordering the points… and taking the top k values”, p.6, ¶3, “In terms of augmentations, we consider three examples: translation, rotation, and crop.”).
Regarding Claim 15, Kuchnik in view of Koh, Barshan, and Ando teaches the data adjustment device of Claim 13 as referenced above. Kuchnik further teaches:
generate the new data using data augmentation and add the generated new data to the learning data (p. 1, ¶1, “Data augmentation is a process in which the training set is expanded by applying class-preserving transformations, such as rotations or crops for images, to the original data points.”, Section 4, Augmentation Set Selection policies, p. 4, ¶3, “we select a subset S ⊆ D either by ordering the points based on their scores and taking the top k values”)
Regarding Claim 17, Kuchnik in view of Koh, Barshan, and Ando teaches the data adjustment device of Claim 2 as referenced above. Koh further teaches:
measure the degree of influence of data included in the learning data used for learning in a neural network (Kuchnik, p. 13, ¶1, “Once training is complete, both loss and influence can then be measured for each training point, and can therefore be used as scores.”).
Regarding Claim 18, Kuchnik in view of Koh, Barshan, and Ando teaches the data adjustment device of Claim 2 as referenced above. Koh further teaches:
execute learning processing using the learning data subjected to the adjustment (After augmenting a percentage of the learning data, the learning data is used to train a model and the accuracy after augmentation is shown, p. 2, ¶2, “we can maintain 99.86% or more of the full augmentation accuracy while only augmenting 10% of the dataset”, p. ¶3, we report test accuracies from augmenting 5, 10, and 25 percent of the data… via our proposed policies”, p. 3, Table 1 shows accuracy after applying augmentation based on policy showing that learning processing occurred after adjustment)
Regarding Claim 19, Kuchnik teaches:
A data adjustment method executing processing comprising:
measuring a degree of influence of data included in learning data on learning of a model by machine learning… the learning data being used for the learning; (Kuchnik, p. 13, ¶1, “influence can then be measured for each training point, and can therefore be used as scores”); and
adjusting the learning data (Kuchnik, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded”), wherein adjusting the learning data includes
acquiring new data… corresponding to learning data measured as having a degree of influence above the second threshold (Threshold of degree of influence is the top k values, Kuchnik, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded”, p. 3, ¶3, “augmenting 5, 10, and 25 percent of the data.. via our proposed policies (to be discussed in Section 4)”, Section 4, p. 4, ¶3, “model influence, by which augmentation scores are generated… we select a subset… by ordering the points… and taking the top k values”), wherein data having a degree of influence above the second threshold improves model identification accuracy (Training data is used to improve model accuracy and data is subsampled to find most influential data which contributes to how best a model will be able to predict, Kuchnik, p. 3, paragraph 2, “subsampling is performed with the ultimate aim being to retain the accuracy”),
generating additional high-influence data by creating new data similar to learning data having a degree of influence above the second threshold, wherein the generated data increases the proportion of high-influence training examples (Data generated from an augmentation uses high-influence data as a template when performing an augmentation such as rotation causing the data generated to also be high-influence which increases the proportion of high-influence examples, Kuchnik, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded by applying class-preserving transformations, such as rotations or crops for images, to the original data points.”),
adding one or more of the acquired new data or the generated additional high- influence data to the learning data to create an optimized training dataset (Kuchnik, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded”), and
automatically executing relearning of the model using the optimized training dataset with increased high-influence data to improve model accuracy without human intervention (cross validation does automatic relearning, Kuchnik, p. 6, paragraph 3, “We controlled for augmentation-induced regularization by performing a simple cross validation sweep for the regularization parameter λ each time the model was re-trained”, p. 1, Abstract, “90% reduction in augmentation set size while maintaining the accuracy gains of standard data augmentation”).
Kuchnik does not expressly teach:
using an influence function that calculates parameter changes without relearning
excluding learning data measured as having a degree of influence below a first threshold to remove data that contributes less to model accuracy improvement
… from an external device or a database…
However, Koh teaches:
using an influence function that calculates parameter changes without relearning (Koh, p. 2, col. 1, paragraph 4, “we form a quadratic approximation… we can linearly approximate the parameter change due to removing z by computing ˆθ−z − ˆθ ≈ − 1 n Iup,params(z), without retraining the model”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the influence function of Koh with the data augmentation via subsampling of Kuchnik. The motivation to do so would be able to find the influence of data points more efficiently without the need for retraining (Koh, p. 2, col. 1, paragraph 3, “retraining the model for each removed z is prohibitively slow. Fortunately, influence functions give us an efficient approximation”).
Kuchnik in view of Koh does not expressly teach:
excluding learning data measured as having a degree of influence below a first threshold to remove data that contributes less to model accuracy improvement
… from an external device or a database…
However, Barshan teaches:
excluding learning data measured as having a degree of influence below a first threshold to remove data that contributes less to model accuracy improvement (first threshold is highest value of training points determined to have a lowest normalized indicator of influence, Barshan, ¶9, “determining one or more training data points, from the plurality of training data points, having a highest normalized indicator of influence, and determining one or more training data points, from the plurality of training data points, having a lowest normalized indicator of influence”, ¶63, “the user may wish to identify training data points causing erroneous predictions and remove those labeled training data points from the database of labeled training data points”, the lower the indicator of influence the less impactful the data is to accuracy improvement)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Barshan excluding data with a low degree of influence and the influence measuring with data augmentation taught by Kuchnik. The modification would have been motivated to reduce erroneous predictions (Barshan, ¶63, “the user may wish to identify training data points causing erroneous predictions and remove those labeled training data points from the database of labeled training data points”).
Kuchnik in view of Koh and Barshan does not expressly teach:
… from an external device or a database…
However, Ando teaches:
… from an external device or a database… (Ando, ¶140, “One or more embodiments provide a processing method for generating learning data, which may include: a step of specifying requirement information for generating learning data, based on request information for making a request for learning; and a step of transmitting the requirement information to a device that generates the learning data”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Ando transmitting a request for new data to an external device and the addition of new data through augmentation from Kuchnik. The modification would have been to obtain more data for a larger training set.
Claims 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Ando et al., (U.S. Patent Application Publication No. US 20190377982 A1), hereinafter “Ando”, in view of Kuchnik et al. “Efficient Augmentation via Data Subsampling”, from applicant IDS, hereinafter “Kuchnik”.
Regarding Claim 20, Ando teaches:
A terminal device (¶322, “The computer software product is stored in a recording medium (e.g. a ROM/RAM, a magnetic disk, or an optical disc) that contains some instructions for causing a terminal device (which may be a personal computer, a server, or a network device, for example) to execute the methods described in the embodiments”) comprising:
receive request information from an external device having a machine learning model, the request information requesting learning data to be used for machine learning (¶8, “a processing method for generating learning data, which may include: a step of specifying requirement information for generating learning data, based on request information for making a request for learning; and a step of transmitting the requirement information to a device that generates the learning data”)
transmit data collected as data corresponding to the request information to the external device (¶18, “The above-described processing method may further include a step of acquiring, from the device, the learning data acquired through learning performed based on the requirement information”).
Ando does not expressly teach:
wherein the requested learning data corresponds to learning data having a degree of influence above a predetermined threshold on learning in the machine learning model, the learning data with high degree of influence contributing more to improving model identification accuracy
wherein the transmitted data enables the external device to create an optimized training dataset with increased high-influence data for automatically retraining the machine learning model to improve model accuracy without human intervention
However, Kuchnik teaches:
wherein the requested learning data corresponds to learning data having a degree of influence above a predetermined threshold on learning in the machine learning model (Threshold of degree of influence is the top k values, Kuchnik, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded”, p. 3, ¶3, “augmenting 5, 10, and 25 percent of the data.. via our proposed policies (to be discussed in Section 4)”, Section 4, p. 4, ¶3, “model influence, by which augmentation scores are generated… we select a subset… by ordering the points… and taking the top k values”), the learning data with high degree of influence contributing more to improving model identification accuracy (Training data is used to improve model accuracy and data is subsampled to find most influential data which contributes to how best a model will be able to predict, Kuchnik, p. 3, paragraph 2, “subsampling is performed with the ultimate aim being to retain the accuracy”)
wherein the transmitted data enables the external device to create an optimized training dataset with increased high-influence data for automatically retraining the machine learning model to improve model accuracy without human intervention (cross validation does automatic relearning, Kuchnik, p. 6, paragraph 3, “We controlled for augmentation-induced regularization by performing a simple cross validation sweep for the regularization parameter λ each time the model was re-trained”, p. 1, Abstract, “90% reduction in augmentation set size while maintaining the accuracy gains of standard data augmentation”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Kuchnik adding data similar to learning data with a high degree and the data generation and transmission taught by Ando. The modification would have been motivated to increase accuracy without having to augment the entire dataset (Kuchnik, Abstract, “we demonstrate that it is possible to significantly reduce the number of data points included in data augmentation while realizing the same accuracy and invariance benefits of augmenting the entire dataset”)
Regarding Claim 21, Ando in view of Kuchnik teaches the terminal device of Claim 20 as referenced above. Kuchnik further teaches:
wherein the learning data requested by the request information includes new data (Data augmentation adds new data, Kuchnik, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded by applying class-preserving transformations, such as rotations or crops for images, to the original data points) in which a degree of influence on the learning in the machine learning model is equal to or higher than a predetermined reference (Top k values selected for data augmentation will have a degree of influence equal to or higher than the lowest k value, Kuchnik, p. 3, ¶3, “augmenting 5, 10, and 25 percent of the data.. via our proposed policies (to be discussed in Section 4)”, Section 4, Augmentation Set Selection Policies, p. 4, ¶3, “model influence, by which augmentation scores are generated… we select a subset S ⊆ D… by ordering the points based on their scores and taking the top k values”).
Claims 22 is rejected under 35 U.S.C. 103 as being unpatentable over Kuchnik et al., “Efficient Augmentation via Data Subsampling”, from applicant IDS, hereinafter “Kuchnik” in view of Koh et al., “Understanding Black-box Predictions via Influence Functions”, from applicant IDS, hereinafter “Koh”, further in view of Ando et al., (U.S. Patent Application Publication No. US 20190377982 A1), hereinafter “Ando”.
Regarding Claim 22, Kuchnik teaches:
an information processing apparatus comprising:
a model trained by using machine learning;
processing circuitry (Kuchnik, p. 13, ¶2, “We perform experiments in Python” demonstrates that Kuchnik performs their method on a computer, in which processor, memory, and storage devices are inherent) configured to
measure a degree of influence of learning data on the machine learning using… the learning data being used for the machine learning (Kuchnik, p. 13, ¶1, “influence can then be measured for each training point, and can therefore be used as scores”);
acquire new learning data on a basis of the degree of influence, wherein the processing circuitry for acquiring new data is further configured to
acquire the new learning data… corresponding to learning data measured as having a degree of influence above a threshold (Threshold of degree of influence is the top k values, Kuchnik, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded”, p. 3, ¶3, “augmenting 5, 10, and 25 percent of the data.. via our proposed policies (to be discussed in Section 4)”, Section 4, p. 4, ¶3, “model influence, by which augmentation scores are generated… we select a subset… by ordering the points… and taking the top k values”), wherein data having a degree of influence above the threshold improves model identification accuracy (Training data is used to improve model accuracy and data is subsampled to find most influential data which contributes to how best a model will be able to predict, Kuchnik, p. 3, paragraph 2, “subsampling is performed with the ultimate aim being to retain the accuracy”), and
generate additional high-influence learning data using learning data having a degree of influence above the threshold as a template for generating the additional high-influence data, wherein the generated learning data increases the proportion of high-influence training examples (Data generated from an augmentation uses high-influence data as a template when performing an augmentation such as rotation causing the data generated to also be high-influence which increases the proportion of high-influence examples, Kuchnik, p. 1, ¶1, “Data augmentation is a process in which the training set is expanded by applying class-preserving transformations, such as rotations or crops for images, to the original data points.”), and;
automatically retrain the model using the acquired new learning data to create an optimized model with improved accuracy without human intervention (cross validation does automatic relearning, Kuchnik, p. 6, paragraph 3, “We controlled for augmentation-induced regularization by performing a simple cross validation sweep for the regularization parameter λ each time the model was re-trained”, p. 1, Abstract, “90% reduction in augmentation set size while maintaining the accuracy gains of standard data augmentation”).
Kuchnik does not expressly teach:
using an influence function that calculates parameter changes without relearning
… from an external device or a database…
However, Koh teaches:
using an influence function that calculates parameter changes without relearning (Koh, p. 2, col. 1, paragraph 4, “we form a quadratic approximation… we can linearly approximate the parameter change due to removing z by computing ˆθ−z − ˆθ ≈ − 1 n Iup,params(z), without retraining the model”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the influence function of Koh with the data augmentation via subsampling of Kuchnik. The motivation to do so would be able to find the influence of data points more efficiently without the need for retraining (Koh, p. 2, col. 1, paragraph 3, “retraining the model for each removed z is prohibitively slow. Fortunately, influence functions give us an efficient approximation”).
Kuchnik in view of Koh does not expressly teach:
… from an external device or a database…
However, Ando teaches:
… from an external device or a database… (Ando, ¶140, “One or more embodiments provide a processing method for generating learning data, which may include: a step of specifying requirement information for generating learning data, based on request information for making a request for learning; and a step of transmitting the requirement information to a device that generates the learning data”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Ando transmitting a request for new data to an external device and the addition of new data through augmentation from Kuchnik. The modification would have been to obtain more data for a larger training set.
Response to Arguments
103
Argument 1: Kuchnik’s data augmentation is not the same as claimed invention’s generating high-influence data using high influence data as a template
Applicant asserts that selecting data points based on influence scores then applying standard augmentations does not teach generate additional high-influence data using learning data having a degree of influence above the second threshold as a template for generating the additional high-influence data. However, Kuchnik does teach this limitation because it selects high influence data based on influence scores, then generates new data based on the high influence data as a template e.g. rotating an image uses the original data piece as a template. Kuchnik explains transformations expand the training set showing that new data is being added, the data being added is also high-influence data because it is the same as the original high-influence data but with a transformation applied. There is nothing recited in the claimed language that shows these two processes are different.
Regarding applicant assertion that the claimed generative process creates entirely new training examples that resemble the original high influence data, rather than merely applying geometric transformations to existing data points, this is exactly what Kuchnik does. Kuchnik generates new training examples that resemble the original high influence data, Kuchnik generating a new piece of data by applying a transformation such as rotation on a highly influential piece of data clearly teaches this.
Argument 2: The claimed generative approach produces qualitatively different training data compared to Kuchnik’s transformation based augmentations, resulting in superior dataset optimization for neural network training.
Regarding applicant assertion that Kuchnik does not show generating additional high-influence data that increases the proportion of high-influence training examples that possess beneficial characteristics identified in the original high influence data, Kuchnik does show generating new high influence data through augmentation where augmentations will have characteristics identified in the original high influence data. This results in superior dataset optimization as shown in Kuchnik where the same accuracy benefits of data augmentation are gained with less augmentations needed (Kuchnik, Abstract).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JESSE CHEN COULSON whose telephone number is (571)272-4716. The examiner can normally be reached Monday-Friday 8:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JESSE C COULSON/
Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights