Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-19 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-16 of U.S. Patent No. US 12,073,320 B2.
Although the claims at issue are not identical, they are not patentably distinct from each other. Please see the comparison table below.
Claims of this application
U.S. Patent No. US 12,073,320 B2
1. A system for training neural networks comprising:
a neural network;
a labeled sample pool comprising:
at least one training sample;
a plurality of labeled samples;
a processor; and
a system memory, the system memory storing instructions that, when executed by the processor, cause the processor to:
add the at least one training example to the labeled sample pool, the labeled sample pool having the plurality of labeled samples;
sample from the labeled sample pool to build a training set, the training set having a training set size;
wherein the sampling is based on a defined probability distribution over the labeled sample pool, the defined probability distribution having an inverse relation to a number of times the at least one training example has passed through the neural network and the defined probability distribution having a variable sharpness of distribution, and
train the neural network with the training set.
1. A method of incrementally training neural networks comprising:
defining, with a computing system, a probability distribution over a labeled sample pool, the labeled sample pool comprising a plurality of labeled samples, wherein the probability distribution is defined such that a probability of selecting any given labeled sample from the labeled sample pool is inversely proportional to a number of times the any given labeled sample has been previously selected;
generating, with the computing system, a first training set by selecting a first set of labeled samples from the labeled sample pool based on the probability distribution, the first training set having a training set size;
training, with the computing system, a neural network with the first training set;
adding, with the computing system, one or more additional labeled samples to the labeled sample pool;
after the adding, generating, with the computing system, a second training set by selecting a second set of labeled samples from the labeled sample pool based on the probability distribution, the second training set having the training set size; and
training, with the computing system, the neural network with the second training set.
2. The system of claim 1, wherein the system memory stores further instructions that, when executed by the processor, causes the processor to:
continuously repeat until a determination to cease continuously repeating, the steps of:
adding the at least one training example to the labeled sample pool,
sampling from the labeled sample pool to build another training set, the another training set having another training set size; and
training the neural network with the another training set.
2. The method of claim 1, further comprising:
continuously repeating, with the computing system, until a determination to cease continuously repeating, the steps of:
adding the one or more additional labeled samples to the labeled sample pool,
after the adding, generating another training set by selecting another set of labeled samples from the labeled sample pool based on the probability distribution, the another training set having the training set size; and
training the neural network with the another training set.
3. The system of claim 1, wherein the defined probability distribution is a weighted random distribution.
3. The method of claim 1, wherein the probability distribution is a weighted random distribution.
4. The system of claim 1, wherein the defined probability distribution comprises a hyperparameter controlling the sharpness of the defined probability distribution.
4. The method of claim 1, wherein the probability distribution comprises a hyperparameter controlling a sharpness of the probability distribution.
5. The system of claim 4, wherein the hyperparameter is defined in the defined probability distribution to a value that avoids a uniform distribution over all samples and avoids a one-hot distribution.
5. The method of claim 4, wherein the hyperparameter is defined in the probability distribution to a value that avoids a uniform distribution over all samples and avoids a one-hot distribution.
6. The system of claim 1, wherein the defined probability distribution is modified by a fraction of the labeled sample pool, the fraction of the labeled sample pool having a common dataset label.
6. The method of claim 1 wherein the defined probability distribution is modified by a fraction of the labeled sample pool, the fraction of the labeled sample pool having a common dataset label.
7. The system of claim 1, wherein the training set size is a constant.
7. The method of claim 1 wherein the training set size is a constant.
8. The system of claim 1, wherein the training set size is a function of a size of the labeled sample pool.
8. The method of claim 1 wherein the training set size is a function of a size of the labeled sample pool.
9. The system of claim 1, wherein the training set size is a function of a size of a training sample pool.
9. The method of claim 1 wherein the training set size is a function of a size of a training sample pool.
Claim 1-6
Claims 10-15, which are analogous to claims 1-6 quoted above.
Claim 1
Claim 16, which is analogous to claim 1 quoted above.
Although the claims at issue are not identical, they are not patentably distinct from each other because the reference application claim includes the limitations of the pending claims with the exception of minor differences in wording and generic computer components (such as a “processor” and “system memory”) arising from differences in the claims’ statutory category, neither of which make the two claim sets patentably distinct. It is noted that while the pending claims recite a “defined probability distribution,” the concept of “defined” is already implicit in the reference patent’s claims, since in order for a distribution to be used by a computer system, it is defined in some form. Furthermore, while the reference patent claims recite “an inverse relation to a number of times the at least one training example has passed through the neural network” rather than “inversely proportional to a number of times the any given labeled sample has been previously selected,” the former is considered to be a broader version of the latter, since the training process covers passing through the neural network (alternatively, if applicant disagrees as to the statement that the former is a broader version of the latter in a strict sense, the Examiner would still consider it to be an obvious variation of the latter due to their overlapping concepts). Finally, the additional recitation of “having a variable sharpness of distribution” does not result in a patentably distinct claim because a distribution that is dependent on a variable (in this case the number of times the any given labeled sample has been previously selected) by definition has a variable sharpness.
The above analysis for claims 1-9 of this application is also applied to claims 10-18 of this application, which recite limitations that are the same or substantially the same as those of claims 1-9 of this application, addressed above. On the same basis, the above analysis for claim 1 of this application is also applied to claim 19 of this application.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
1. Claims 1-5, 7, 10-14, 16, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Castro et al., arXiv:1807.09536v2 [cs.CV] 3 Sep 2018 (“Castro”) (cited in an IDS) in view of London (US 11,200,511 B1) (cited in an IDS).
As to claim 1, Castro teaches a system for training neural networks comprising:
a neural network; [Abstract: “We address this issue with our approach to learn deep neural networks incrementally, using new data and only a small exemplar set corresponding to samples from the old classes.”]
a labeled sample pool comprising: at least one training sample; a plurality of labeled samples; [§ 3.1, paragraph 1 “When a new class or set of classes is added to the current model, a subset with the most representative samples from them is selected and stored in the representative memory… The first setup considers a memory with a limited capacity of K samples. As the capacity of the memory is independent of the number of classes, the more classes stored, the fewer samples retained per class.” That is, as illustrated in FIG. 2, the combination of “new samples” and the “memory” samples that was used in the previous training iteration is considered to be a “labeled sample pool.” Moreover, the samples are used for training. Further regarding “labeled” and “labeled samples,” § 4, paragraph 2 teaches: “As our approach uses two loss functions, i.e., classification and distillation, we need two labels for each sample, associated with the two losses. For classification, we use the one-hot vector which indicates the class appearing in the image… Each image of our training set will have a classification label and F distillation labels.”]
a processor; and a system memory, the system memory storing instructions that, when executed by the processor, cause the processor to: [§ 8, paragraph 2: “a Titan X Pascal GPU used for this research” (indicating that the method is performed using a computer that includes a GPU). Furthermore, Castro generally teaches that its method includes the execution of complex algorithms such as neural networks, and Castro, § 7, teaches that its method is applied to complex data sets. Since the method of Castro is run using a computer, the instant generic computer limitations are implied by Castro’s disclosure.]
add the at least one training example to the labeled sample pool, the labeled sample pool having the plurality of labeled samples; [The addition of “new samples” as shown in FIG. 2 constitutes adding more labeled samples to the labeled sample pool, which is the combination of new samples and the memory samples.]
sample from the labeled sample pool to build a training set, the training set having a training set size; [As shown in FIG. 2, the process of updating the representative memory repeats itself. For example, § 6, paragraph 2 teaches the examples of “50, 20, 10, 5, and 2 incremental training steps” (i.e., iterations). Therefore, the selection of the samples for the representative memory is performed again in the next incremental step, so as to constitute the generation of a second training set by selecting the samples to be kept in the memory in the next training process. Since this next training set includes samples selected in the previous selection of samples, it is considered to be based on the probability distribution that was used to select the previous samples.] […], and
train the neural network with the training set. [A subsequent training step in the repeating process shown in FIG. 2 constitutes training with the second training set.]
Castro does not explicitly teach “wherein the sampling is based on a defined probability distribution over the labeled sample pool, the defined probability distribution having an inverse relation to a number of times the at least one training example has passed through the neural network and the defined probability distribution having a variable sharpness of distribution.”
London teaches “wherein the sampling is based on a defined probability distribution over the labeled sample pool” [Col. 10, line 64 to col. 11, line 9: “FIG. 2 illustrates example alternative approaches towards sampling training data, according to at least some embodiments. For many types of randomized learning algorithms including traditional versions of stochastic gradient descent, a fixed or unchanging sampling probability distribution with respect to the available training examples 210 may be used, as shown in graph 201A in which sampling probabilities 220 associated with the different training examples… In this approach, the probability that any given training example is selected for a given training iteration is set to a constant value P (such as 1/N if there are N examples in all), which does not change as the training progresses.” Col. 11, lines 10-17: “In contrast, graph 201B represents a snapshot (corresponding to some training iteration K, as indicated in the Y-axis label 222) of the dynamically changing probabilities 228 associated with selecting different training examples when adaptive sampling is used. Higher sampling weights 233 may automatically and efficiently be assigned to training examples that are more “difficult” for the model to learn from, and therefore more useful for the model to learn from.” That is London generally teaches the use of a probability distribution, defined over a sampling pool (analogous to the “labeled example pool” of the instant claim), which is used to sample training data points. The basic technique is the uniform distribution shown in graph 201A, while an extension is “adaptive sampling” technique that uses a non-uniform distribution shown in graph 201B.], “the defined probability distribution having an inverse relation to a number of times the at least one training example has passed through the neural network” [Col. 3, lines 36-39: “As a result of the update to the sampling weight(s), the probability of using a given training example may change from one iteration to another in various embodiments, with the more useful or difficult training observations eventually being assigned higher weights.” The weight update is described in Algorithm 1 described in col. 8, lines 10-20, which teaches that weights are initially set to 1 (i.e., line 1, which states: “initialize sampling weights qi for all examples to 1”) and then line 5 updates the sampling weight (“update sampling weight for it”). Therefore, at the initial stage, the weights are uniformly distributed, as shown in FIG. 2 left side, and then as weights are assigned such that the probability of certain samples being selected become lower than before over successive iterations. Note that in the limitation of “the number of times the at least one training example has passed through the model,” the term “at least one training example” is met by any arbitrary set of training examples. Since each iteration selects and results in the use of at least one training example, the number of iterations performed in Algorithm 1 corresponds to the number of times at least on example is selected. Therefore, the updating of weights such that there are some weights that are lower than other weights (as shown in FIG. 2, right side), results in a decreased sampling probability over time for those lower weights. Therefore, an “inverse relation” is taught in this form, noting that the instant claim does not precisely define what this inverse relation is.] “and the defined probability distribution having a variable sharpness of distribution.” [That is, the sampling weights affect the distribution shape as shown in FIG. 2, right side (item 201B). The weights affect the sharpness of the distribution, such as resulting in bulged parts of the distribution that are higher than the average y-axis value.]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Castro with the teachings of London by modifying Castro to use London’s adaptive sampling technique so as to arrive at the claimed invention. The motivation would have been to enable training examples that are more useful for the model to learn from to be sampled at a higher rate, as suggested by London (col. 11, lines 10-17: “Higher sampling weights 233 may automatically and efficiently be assigned to training examples that are…more useful for the model to learn from”).
As to claim 2, the combination of Castro and London teaches the system of claim 1, wherein the system memory stores further instructions that, when executed by the processor, causes the processor to:
continuously repeat until a determination to cease continuously repeating, the steps of: [As shown in Castro, FIG. 2, the process of updating the representative memory repeats itself. For example, Castro, § 6, paragraph 2 teaches: “We divide the 100 classes into splits of 2, 5, 10, 20, and 50 classes with a random order. Thus, we will have 50, 20, 10, 5, and 2 incremental training steps respectively.” Note that “incremental training steps” here refer to iterations in the process shown in FIG. 2. Therefore, Castro teaches that the process repeats for a set number of iterations, i.e., until a determination to cease continuously repeating when the splits have been exhausted.] adding the at least one training example to the labeled sample pool, sampling from the labeled sample pool to build another training set, the another training set having another training set size; and training the neural network with the another training set. [These steps are substantially the same as the steps of “adding,” “sampling,” and “training” steps in claim 1. Therefore, these limitations are taught by Castro, as modified by London, for the same reasons given for the corresponding limitations in claim 1, in light of the fact that the process in Castro is repetitive, as discussed above, such that the same general steps shown in FIG. 2 of Castro would read on subsequent repetitions involving the same general method. Note that “based on the probability distribution” is taught by London and rendered obvious for the same reason given for the analogous limitation in the parent claim. Furthermore, the feature of “another training set” is taught for the same reasons discussed for the “second training set” in the rejection of claim 1.]
As to claim 3, the combination of Castro and London teaches the system of claim 1, as set forth above.
London further teaches “wherein the probability distribution is a weighted random distribution.” [Col. 11, lines 10-17: “In contrast, graph 201B represents a snapshot (corresponding to some training iteration K, as indicated in the Y-axis label 222) of the dynamically changing probabilities 228 associated with selecting different training examples when adaptive sampling is used. Higher sampling weights 233 may automatically and efficiently be assigned to training examples that are more “difficult” for the model to learn from, and therefore more useful for the model to learn from.” See also Col. 8, lines 10-20: “initialize sampling weights qi for all examples to 1… update sampling weight for it.”]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Castro and London to also arrive at the claimed invention of the instant dependent claim. The motivation for doing so is covered by the motivation given for London in the rejection of the parent independent claim.
As to claim 4, the combination of Castro and London teaches the system of claim 1, as set forth above.
London further teaches “wherein the defined probability distribution comprises a hyperparameter controlling the sharpness of the defined probability distribution.” [Col. 11, lines 10-17: “In contrast, graph 201B represents a snapshot (corresponding to some training iteration K, as indicated in the Y-axis label 222) of the dynamically changing probabilities 228 associated with selecting different training examples when adaptive sampling is used. Higher sampling weights 233 may automatically and efficiently be assigned to training examples that are more “difficult” for the model to learn from, and therefore more useful for the model to learn from.” See also Col. 8, lines 10-20: “initialize sampling weights qi for all examples to 1… update sampling weight for it.” That is, the sampling weights are hyperparameters that affect the distribution shape as shown in FIG. 2, right side (item 201B). They are hyperparameters of the learning process because they are parameters that are outside of the model that is being trained. As shown in FIG. 201B, the weights affect the sharpness of the distribution, such as resulting in bulged parts of the distribution that are higher than the average y-axis value.]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Castro and London to also arrive at the claimed invention of the instant dependent claim. The motivation for doing so is covered by the motivation given for London in the rejection of the parent independent claim.
As to claim 5, the combination of Castro and London teaches the system of claim 4, as set forth above.
London further teaches “wherein the hyperparameter is defined in the defined probability distribution to a value that avoids a uniform distribution over all samples and avoids a one-hot distribution.” [London, Col. 11, lines 10-17, as quoted above, teaches that the hyperparameter causes the distribution in Graph 201B of FIG. 2 to be non-uniform, as compared to Graph 201A, which is a uniform distribution. Furthermore, the use of weights avoids a one-hot distribution as shown in FIG. 2, Graph 201B, wherein a one-hot distribution is avoided because the training examples all generally have a non-zero probability.]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Castro and London to also arrive at the claimed invention of the instant dependent claim. The motivation for doing so is covered by the motivation given for London in the rejection of the parent independent claim.
As to claim 7, the combination of Castro and London teaches the system of claim 1, as set forth above.
London further teaches “wherein the hyperparameter is defined in the probability distribution to a value that avoids a uniform distribution over all samples and avoids a one-hot distribution.” [London, Col. 11, lines 10-17, as quoted above, teaches that the hyperparameter causes the distribution in Graph 201B of FIG. 2 to be non-uniform, as compared to Graph 201A, which is a uniform distribution. Furthermore, the use of weights avoids a one-hot distribution as shown in FIG. 2, Graph 201B, wherein a one-hot distribution is avoided because the training examples all generally have a non-zero probability.]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Castro and London to also arrive at the claimed invention of the instant dependent claim. The motivation for doing so is covered by the motivation given for London in the rejection of the parent independent claim.
As to claims 10-14 and 16, these claims are directed to a method comprising operations that are the same or substantially the same as those of claims 1-5 and 7. Therefore, the rejections made to claims 1-5 and 7 are applied to claims 10-14 and 16.
Furthermore, Castro teaches “with a computer system” for the reasons given for the limitations of the processor and system memory in claim 1.
As to claim 19, this claim is directed to a computer-readable medium for performing that are the same or substantially the same as those of claim 1. Therefore, the rejection made to claim 1 is applied to claim 19.
Furthermore, Castro teaches limitation of “a non-transitory, computer-readable medium comprising instructions that, when executed by a processor, causes the processor to” [§ 8, paragraph 2: “a Titan X Pascal GPU used for this research” (indicating that the method is performed using a computer that includes a GPU). Furthermore, Castro generally teaches that its method includes the execution of complex algorithms such as neural networks, and Castro, § 7, teaches that its method is applied to complex data sets. Since the method of Castro is run using a computer, the instant generic computer limitations are implied.]
2. Claims 6 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Castro in view of London, and further in view of Alabdallah et al., “Using weighted Support Vector Machine to address the imbalanced classes problem of Intrusion Detection System,” KSII Transactions on Internet and Information Systems Vol. 12, No. 10, Oct. 2018 (“Alabdallah”) (cited in an IDS).
As to claim 6, the combination of Castro and London teaches the system of claim 1, but does not teach the further limitations of the instant dependent claim.
Alabdallah teaches “wherein the defined probability distribution is modified by a fraction of the labeled sample pool, the fraction of the labeled sample pool having a common dataset label.” [§ 3.3: “Stratified sampling is a statistical sampling method [24]. It is an alternative to known methods called random sampling. It is used to generate new subsets of data that have the same sample fraction [25] of their classes as in the main corpus. The following equation illustrates the sample fraction: fi = nji/Nj (3) Where 𝑓𝑖 is the fraction of class i in main set and any subset… It guarantees that any subset includes records from all classes and in the same ratio of the main corpus, while the class records selected each time randomly.” That is, the “sample fraction” is used to modify the sampling process (and hence the probability distribution) to thereby implement stratified sampling.]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of the references combined thus far with the teachings of Alabdallah by implementing the stratified sampling technique discussed in Alabdallah, so as to arrive at the claimed invention of the instant dependent claim. The motivation would have been to generate a sample subset that has the same ratio of classes as the base set, as suggested by Alabdallah (§ 3.3, as quoted above: “It guarantees that any subset includes records from all classes and in the same ratio of the main corpus, while the class records selected each time randomly”).
As to claim 15, the further limitations of this claim are the same or substantially the same as those of claim 6. Therefore, the rejection made to claim 6 is applied to claim 15.
3. Claims 8-9 and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Castro in view of London, and further in view of Liu et al., “Mnemonics Training: Multi-Class Incremental Learning without Forgetting,” arXiv:2002.10211v5 [cs.CV] 15 Sep 2020 (“Liu”) (cited in an IDS).
As to claim 8, the combination of Castro and London teaches the system of claim 1, but does not teach the further limitations of this dependent claim.
Liu, in an analogous art, teaches the further limitations. Liu pertains to “multi-class incremental learning without forgetting” (see title), and is therefore in the same field of endeavor as the claimed invention, as well as being pertinent to incremental learning.
In particular, Liu teaches “wherein the training set size is a function of a size of the labeled sample pool.” [§ 4.3: “Typically, the number of exemplars Ei is set to be greatly smaller than that of the original data Di.” That is, the number of exemplars (which is analogous to the training set size) is set based on the size of the original data Di (which is analogous to the labeled sample pool). The Examiner notes that the claim language of “is a function of” does not require any specific relationship or manner in which the function is executed. Therefore, the relationship between the number of exemplars and the size of the original data teaches the limitation of “a function of.”]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of the references combined thus far with the teachings of Liu by implementing the feature that “the training set size is a function of a size of the labeled sample pool.” The motivation would have been to set the size of the training set in a manner that accounts for the size of the original data, as suggested by Liu (see part quoted above).
As to claim 9, the combination of Castro and London teaches the system of claim 1, but does not teach the further limitations of this dependent claim.
Liu, in an analogous art, teaches the further limitations. Liu pertains to “multi-class incremental learning without forgetting” (see title), and is therefore in the same field of endeavor as the claimed invention, as well as being pertinent to incremental learning.
In particular, Liu teaches “wherein the training set size is a function of a size of a training sample pool.” [§ 4.3: “Typically, the number of exemplars Ei is set to be greatly smaller than that of the original data Di.” That is, the number of exemplars (which is analogous to the training set size) is set based on the size of the original data Di (which is analogous to the training sample pool). The Examiner notes that the claim language of “is a function of” does not require any specific relationship or manner in which the function is executed. Therefore, the relationship between the number of exemplars and the size of the original data teaches the limitation of “a function of.”]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of the references combined thus far with the teachings of Liu by implementing the feature that “the training set size is a function of a size of a training sample pool.” The motivation would have been to set the size of the training set in a manner that accounts for the size of the original data, as suggested by Liu (see part quoted above).
As to claims 17-18, the further limitations of this claim are the same or substantially the same as those of claims 8-9, respectively. Therefore, the rejections made to claims 8-9 are applied to claims 17-18, respectively.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. The following documents depict the state of the art.
Liu et al., “Continual Universal Object Detection,” arXiv:2002.05347v1 [cs.CV] 13 Feb 2020 teaches sampling exemplars for incremental learning using a particular methodology.
Chrysakis et al., “Online Continual Learning from Imbalanced Data,” Proceedings of the 37th International Conference on Machine Learning, Online, PMLR 119, 2020 teaches conventional techniques for handling training instances. In this reference, the probability of replaying a certain stored instance is inversely proportional to the number of stored instances of the same class.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YAO DAVID HUANG whose telephone number is (571)270-1764. The examiner can normally be reached Monday - Friday 9:00 am - 5:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached at (571) 270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Y.D.H./Examiner, Art Unit 2124
/MIRANDA M HUANG/Supervisory Patent Examiner, Art Unit 2124
Prosecution Insights