DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Intepretation-112(f)
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means”, or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation of claim 1 uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) are: “extractor unit”, “calculation unit” and “learning unit” in claim 1.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA
35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure,
material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefore, subject to the conditions and requirements of this title.
Step 1:
Claim 1 recites “machine learning apparatus” thus, it is a machine claim.
Claim 2 is machine claim.
Claim 3 is a method claim performed by units.
Claim 4 recites “computer readable non-transitory recording medium” storing a program for execution thus, it is an apparatus claim.
Claims 1-4 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1:
Step 2A Prong 1:
“that continually learns a novel class with fewer samples than a base class, comprising: (except machine learning apparatus)” is a mental step of data comparison.
“a merged feature calculation unit that merges the feature vector of the base class and
the feature vector of the novel class” is a mental step of data operation (fusion) as a merged
feature calculation unit that simply fuses base and novel class feature vectors is a
conventional machine learning technique.
“to calculate a merged feature vector that merges the base class and the novel class”
is a mental step of data operation (fusion).
“and a learning unit that classifies, on a projected space” is a mental step of data operation
(fusion) as a merged feature calculation unit that simply fuses base and
novel class feature vectors is a conventional machine learning technique.
“a query sample of a query set based on a distance between a position of the merged feature vector of the query sample of the query set and a position of a classification weight vector of each class” is a mental step of data comparison.
“and learns a classification weight vector of the novel class to minimize a loss incurred in classification, wherein the novel class feature extraction unit is obtained by subjecting the base class feature extraction unit to self-distillation k times (k is a natural number)”
is a mental step of data adaptation.
Step 2A Prong 2:
“A machine learning apparatus” recited in the preamble does not integrate the judicial exception into a practical application and is merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“a novel class feature extraction unit that extracts a feature vector of the novel class” does not integrate the judicial exception into a practical application and is merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“A machine learning apparatus” recited in the preamble does not amount to significantly more than the judicial exception in the claim and is merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“a novel class feature extraction unit that extracts a feature vector of the novel class” does not amount to significantly more than the judicial exception in the claim and is merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Regarding claim 2:
Step 2A Prong 1:
“wherein the novel class feature extraction unit averages values output by a 1st to a kth generation feature extraction units obtained by subjecting the base class feature extraction unit to self-distillation k times and outputs an average value” is a mental step of data adaptation
Step 2A Prong 2: no additional elements
Step 2B: no additional elements
Regarding claim 3:
Step 2A Prong 1:
“that continually learns a novel class with fewer samples than a base class, comprising” is a mental step of data comparison
“subjecting the base class feature extractor to self-distillation k times (k is a natural number) to obtain a novel class feature extractor” is a mental step of data adaptation.
“merging the feature vector of the base class and the feature vector of the novel class to calculate a merged feature vector that merges the base class and the novel class” is a mental step of data operation (fusion).
“and classifying, on a projected space, a query sample of a query set based on a distance between a position of the merged feature vector of the query sample of the query set and a position of a classification weight vector of each class” is a mental step of data comparison and data operation (fusion).
“and learning a classification weight vector of the novel class to minimize a loss incurred in classification” mental step of data comparison.
Step 2A Prong 2:
“A machine learning method” in the preamble does not integrate the judicial exception into a practical application and is merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“extracting a feature vector of the base class by using a base class feature extractor” does not integrate the judicial exception into a practical application and is merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“A machine learning method” in the preamble does not amount to significantly more than the judicial exception in the claim and is merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“extracting a feature vector of the base class by using a base class feature extractor” does not amount to significantly more than the judicial exception in the claim and is merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Regarding claim 4:
Step 2A Prong 1:
“that continually learns a novel class with fewer samples than a base class” is a mental step of data comparison.
“a module that subjects the base class feature extractor to self-distillation k times (k is a natural number) to obtain a novel class feature extractor (except a module)” a mental step of data operation (fusion).
“a module that merges the feature vector of the base class and the feature vector of the novel class to calculate a merged feature vector that merges the base class and the novel class; and a module that classifies (except a module)” is a mental step of data operation(fusion).
“on a projected space, a query sample of a query set based on a distance between a position of the merged feature vector of the query sample of the query set and a position of a classification weight vector of each class” is a mental step of data comparison.
“and learns a classification weight vector of the novel class to minimize a loss incurred in classification” is a mental step of data comparison.
Step 2A Prong 2:
“A computer readable non-transitory recording medium storing a machine learning program” does not integrate the judicial exception into a practical application and is merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“the program comprising computer-implemented modules that include” does not integrate the judicial exception into a practical application and is merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“a module that extracts a feature vector of the base class by using a base class feature extractor” does not integrate the judicial exception into a practical application and is merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“a module” does not integrate the judicial exception into a practical application and is merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“A computer readable non-transitory recording medium storing a machine learning program”
does not amount to significantly more than the judicial exception in the claim and is merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“the program comprising computer-implemented modules that include” does not amount to significantly more than the judicial exception in the claim and is merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“a module that extracts a feature vector of the base class by using a base class feature extractor” does not amount to significantly more than the judicial exception in the claim and is merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“a module “does not amount to significantly more than the judicial exception in the claim and is merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4 are rejected under 35 U.S.C. 103 as being unpatentable over
Shiyu CHANG et.al (hereinafter CHANG) US 2020/0302230 A1,
In view of Bin Sun et.al (hereinafter Sun) US 2022/0366666 A1.
Regarding claim 1:
CHANG discloses:
- a base class feature extraction unit that extracts a feature vector of the base class;
[0027]: “For each input image, the teacher network 110 yields a teacher region proposal (RPN(T)) 111 used to extract a feature vector 112 and a teacher object prediction 113 identifying one of the previously-learned object classes as corresponding to an object in each region of the teacher RPN.
- a novel class feature extraction unit that extracts a feature vector of the novel class;
[0046]: “the object prediction may be expressed as a specific object probability. the “chair” 0.98 and “banana” 0.95 as the old classes, and “laptop” 0.99 as the new class (after: chair 0.96, banana 0.95), and update the text accordingly.
[0062] : “ FIG. 8 is a flowchart of a method for adding a new class of objects, designated as class A, to an existing object detector, such as the detector used in FIG. 7. This method illustrates obtaining training datasets that can be used for incremental training and is based on what is provided in the input (image or name), the ability to obtain existing training datasets or the need to create a new training dataset.
- a merged feature calculation unit that merges the feature vector of the base class and the feature vector of the novel class to calculate a merged feature vector that merges the base class and the novel class;
[0062] : “ method illustrates obtaining training datasets that can be used for incremental training and is based on what is provided in the input (image or name), the ability to obtain existing training datasets or the need to create a new training dataset.
[0043]: “ The two-stage neural network object detector is iteratively updated so as to balance minimizing a loss of its initial ability to locate and identify objects pertaining to previously-learned object classes and maximizing its ability to additionally locate and to identify the objects pertaining to the additional object class. Ability may be measured by percentage of correct detections among all detections, an average of prediction probabilities, combined with a number of false positive detections or false negative detections, etc.
[0026]: “ An initial version of a two-stage object detector, which is named “Teacher Network (T),” is able to detect (i.e., locate and identify) objects pertaining to the previously-learned object classes. A current version of the two-stage neural network object detector named “Student Network (S)” is trained to detect objects pertaining to a new object class. The following description refers to a single new object class, but plural object classes may be added successively or simultaneously. The student network starts with the parameters from the teacher network, but output from the student network is extended to detect objects pertaining to the new object class.
[BRI: the merging feature vectors means combining the attribute (feature) data from two datasets where a base class dataset and a novel class dataset are merged into a single, unified dataset]
- and learns a classification weight vector of the novel class to minimize a loss incurred in classification,
[0025]: “ employ a reasonable number of model parameters and to have acceptable memory requirements for using the model; and (D) to provide an end-to-end learning mechanism that jointly updates the classifier to identify the new object class and the feature representation to locate an object pertaining to the new object class”,
[0010]:” Exemplary embodiments are directed to methods for incremental learning object detection in images without catastrophic forgetting of previously-learned object classes. The methods include training an existing two-stage neural network object detector to locate and identify objects pertaining to at least one additional object class in images by iteratively updating the two-stage neural network object detector until an overall detection accuracy criterion is met. The updating is performed so as to balance minimizing a loss of an initial ability to locate and identify objects pertaining to the previously-learned one or more object classes and maximizing an ability to locate and identify the objects pertaining to the additional object class”
- wherein the novel class feature extraction unit is obtained by subjecting the base class feature extraction unit to self-distillation k times (k is a natural number).
[0026] “ FIG. 1 schematically represents incremental learning object detection in images without catastrophic forgetting of previously-learned object classes according to an embodiment. An initial version of a two-stage object detector, which is named “Teacher Network (T),” is able to detect (i.e., locate and identify) objects pertaining to the previously-learned object classes. A current version of the two-stage neural network object detector named “Student Network (S)” is trained to detect objects pertaining to a new object class. The following description refers to a single new object class, but plural object classes may be added successively or simultaneously. The student network starts with the parameters from the teacher network, but output from the student network is extended to detect objects pertaining to the new object class”,
[0028] “ Detection accuracy may be estimated based on the teacher network's and student network's outputs. A region proposal network distillation loss 131 quantifying a decrease of student network's ability to locate objects of the previously-learned object classes is based on comparing the teacher region proposal 111 with the student region proposal 121,
[0030]” In one embodiment, a total loss is defined by combining the region proposal network distillation loss, the object identification distillation loss, the region proposal network loss and the object identification loss. The student network may then be updated so as to minimize the total loss.
[BRI: Self-distillation – In this context, the same unit is used as both teacher and student: the unit’s own predictions are used as soft targets to retrain itself. This process encourages the model to refine its feature representations by learning from its own outputs, often reducing intra-class variance and improving generalization]
[0033]:” The two-stage neural network object detector is trained to detect the objects of the object class initially unknown, using the training images one-by-one or batch-by-batch until a predetermined condition is met at 220. The predetermined condition may be completing a predetermined number of iterations, achieving a threshold accuracy or failing to further decrease losses,
[0011] “The predetermined condition is met when the number of training iterations reaches a predetermined number, or when a total loss decrease rate is below a predetermined threshold.
[BRI: The self-distillation process is repeated k times. Each iteration takes the output of the previous iteration as the new target distribution for training. After k iterations, the resulting unit is the novel class feature extraction unit. The “the predetermined threshold” is “k”]
CHANG does not explicitly disclose:
- A machine learning apparatus that continually learns a novel class with fewer samples than a base class, comprising:
- and a learning unit that classifies, on a projected space, a query sample of a query set based on a distance between a position of the merged feature vector of the query sample of the query set and a position of a classification weight vector of each class
However, Sun discloses:
- A machine learning apparatus that continually learns a novel class with fewer samples than a base class, comprising:
[Abstract]: “The feature vector crafting apparatus (18) receiving data after verification of data sources (14-1, . . ., 14-n). The feature vector crafting apparatus (18) automatically generates new feature vectors from new data for forwarding there to an artificial intelligence infrastructure (20)”,
[0007] “According to a first aspect of the present invention there is provided a method of verifying at least one data source that generates new specimen data intended for use in a machine learning infrastructure”,
[0043]” The data source verifying apparatus 12 may comprise processing circuitry or processor 48 and a memory 50 containing instructions executable by the processing circuitry 48 to implement the functionality “.
[BRI: Machine infrastructure is the full stack of physical and virtual components that together run and support computational workloads such as applications, AI models, and data pipelines. Within the modern usage, “machine infrastructure” is the broader system that includes the machine learning apparatus (the compute, storage, and processing hardware/software that runs ML models) along with all the supporting components] needed to develop, train, deploy, and operate those models]
[0107] “ As shown in FIG. 12, after processing data trustworthiness there may occur a core/cloud query for few samples of feature saliency maps from the user terminal UE who found new object according to the feature vector they submitted in the process of establishing data trustworthiness. Then, the core finds a new feature area and crafts new feature vectors the existing machine learning model”
[0066] “ As shown in FIG. 5, the step S10 of obtaining the at least one feature vector further comprises a step 22, operatively being executed by the concluding unit 34, of concluding on the new specimen data being related to a new object upon continuous occurrence of the plurality of feature vectors.”
In [0096]” new objects can appear in the environment like a new moving robot different than the previous one that was identified, e.g., a robot as shown in FIG. 10(c). Then the algorithm according to the present invention will identify the object as a new object and assign a label to it for subsequent handling thereof.
[BRI: the new feature is the novel class determined from few samples of feature saliency maps. A previously identified object is a “base class” and the new object is a “novel class”]
- and a learning unit that classifies, on a projected space, a query sample of a query set based on a distance between a position of the merged feature vector of the query sample of the query set and a position of a classification weight vector of each class,
[0070] “ Generally, the feature vector crafting apparatus 18 supports identification of unknown features in data like images and creation of related new feature vectors for subsequent use in the machine learning infrastructure 20,
[0057]: “ As shown in FIG. 3, the method of verifying at least one data source 14-1, . . . , 14-n further comprises a step S12, operatively being executed by the correlation checking unit 24, of executing a correlation check between the received at least one feature vector and at least one reference feature vector representing trusted specimen data for determination of a correlation measure. As outlined above, the correlation check may use a similarity/distance check
It would be obvious to one of ordinary skill in the art before the effective filing date of the present application to combine CHANG and Sun.
CHANG teaches an incremental learning that generates a novel class (new class) with a base class (previously learned class) and generates loss.
Sun teaches distance between query set and merged feature vector.
One of ordinary skills would be to combine CHANG and Sun that can provide small, expected differences for new object and can automatically verify the trustworthiness of data sources (Sun [0040).
Regarding claim 2:
CHANG discloses:
- wherein the novel class feature extraction unit averages values output by a 1st to a kth generation feature extraction units obtained by subjecting the base class feature extraction unit to self-distillation k times and outputs an average value.
[0027]: “Training images 100 include objects pertaining to the new object class but may also include objects pertaining to previously-learned object classes. These training images are input one-by-one or batch-after-batch to the teacher network 110 and to the student network 120 (i.e., parallel to both networks and in series through the training images). For each input image, the teacher network 110 yields a teacher region proposal (RPN(T)) 111 used to extract a feature vector 112 and a teacher object prediction 113 identifying one of the previously-learned object classes as corresponding to an object in each region of the teacher RPN. Also, for each input image, the student network 120 yields a student region proposal (RPN(S)) 121 used to extract a feature vector 122 and a student object prediction 123 identifying one of the previously-learned object classes or the new object class as corresponding to an object in each region of RPN(S)”,
[0032]” FIG. 2 is a flowchart of method 200 for incremental learning object detection without catastrophic forgetting according to an embodiment. Training images including objects pertaining to an object class unknown to an initial version of a two-stage neural network object detector that is able to detect objects pertaining to at least one previously-learned object class are received 210”,
[0028]:” Detection accuracy may be estimated based on the teacher network's and student network's outputs.
[BRI: Incremental learning typically uses the same model architecture and updates it incrementally with new data, rather than training a completely new model from scratch each time. Within the context of using the same model the detection accuracy estimate is derived from comparing their outputs, it represents a form of self-distillation]
[0028]”A region proposal network distillation loss 131 quantifying a decrease of student network's ability to locate objects of the previously-learned object classes is based on comparing the teacher region proposal 111 with the student region proposal 121. An object identification distillation loss 132 quantifying a decrease of student network's ability to identify the objects of the previously-learned object classes is based on comparing the teacher object prediction 112 with the student object prediction 122.
[0031] The use of the total loss in updating the object detector is a way to balance minimizing its loss of ability to detect objects pertaining to previously-learned objects and maximizing its ability to detect objects pertaining to the newly-added object class. However, there are other ways to achieve this balance; for example, there may be a predefined limit to a loss of ability to detect objects pertaining to any individual previously-learned object classes, to an average loss of ability and/or to the ability to detect objects pertaining to the added object class (in this case the predefined limit is a threshold that must be exceeded), etc.”,
[0029]” Further, comparing the student network's outputs with ground-truth labels associated with the training image enables estimating student network's region proposal loss 141 related to locating objects pertaining to the new object class and student network's object identification loss 142 related to identifying objects pertaining the new object class. In machine learning, the term “ground truth” refers to the training set's classification label or simpler said to the objects in the training images real location and identity.
[0044]” The overall detection accuracy criterion may be assessed by determining a region proposal distillation loss and an object identification distillation loss. “Distillation loss” indicates respective ability's decrease due to adding the new object class. The region proposal distillation loss is based on comparing a region proposal output by an initial version of the two-stage neural network object detector with a current region proposal output by a current version of the two-stage neural network object detector for the same input training image
[0026]” An initial version of a two-stage object detector, which is named Teacher Network (T),” is able to detect (i.e., locate and identify) objects pertaining to the previously-learned object classes
[BRI: the described process is a feature-based knowledge distillation approach where the teacher model’s intermediate feature extractor is used to generate multiple feature vectors, these are aggregated (averaged) over several self-distillation steps, and the resulting averaged feature is used in a combined loss that balances the distillation loss with an “ability” (e.g., task) loss . This averaging can be seen as a form of ensemble over the teacher’s own intermediate outputs]
Regarding claim 3:
CHANG discloses:
- extracting a feature vector of the base class by using a base class feature extractor;
[0027]: “For each input image, the teacher network 110 yields a teacher region proposal (RPN(T)) 111 used to extract a feature vector 112 and a teacher object prediction 113 identifying one of the previously-learned object classes as corresponding to an object in each region of the teacher RPN.
- subjecting the base class feature extractor to self-distillation k times (k is a natural number) to obtain a novel class feature extractor;
[0026] “ FIG. 1 schematically represents incremental learning object detection in images without catastrophic forgetting of previously-learned object classes according to an embodiment. An initial version of a two-stage object detector, which is named “Teacher Network (T),” is able to detect (i.e., locate and identify) objects pertaining to the previously-learned object classes. A current version of the two-stage neural network object detector named “Student Network (S)” is trained to detect objects pertaining to a new object class. The following description refers to a single new object class, but plural object classes may be added successively or simultaneously. The student network starts with the parameters from the teacher network, but output from the student network is extended to detect objects pertaining to the new object class”,
[0028] “ Detection accuracy may be estimated based on the teacher network's and student network's outputs. A region proposal network distillation loss 131 quantifying a decrease of student network's ability to locate objects of the previously-learned object classes is based on comparing the teacher region proposal 111 with the student region proposal 121,
[0030]” In one embodiment, a total loss is defined by combining the region proposal network distillation loss, the object identification distillation loss, the region proposal network loss and the object identification loss. The student network may then be updated so as to minimize the total loss.
[BRI: Self-distillation – In this context, the same unit is used as both teacher and student: the unit’s own predictions are used as soft targets to retrain itself. This process encourages the model to refine its feature representations by learning from its own outputs, often reducing intra-class variance and improving generalization]
[0033]:” The two-stage neural network object detector is trained to detect the objects of the object class initially unknown, using the training images one-by-one or batch-by-batch until a predetermined condition is met at 220. The predetermined condition may be completing a predetermined number of iterations, achieving a threshold accuracy or failing to further decrease losses”,
[0011] “The predetermined condition is met when the number of training iterations reaches a predetermined number, or when a total loss decrease rate is below a predetermined threshold.
[BRI: The self-distillation process is repeated k times. Each iteration takes the output of the previous iteration as the new target distribution for training. After k iterations, the resulting unit is the novel class feature extraction unit. The “the predetermined threshold” is “k”]
- extracting a feature vector of the novel class by using the novel class feature extractor;
[0046]: “the object prediction may be expressed as a specific object probability. the “chair” 0.98 and “banana” 0.95 as the old classes, and “laptop” 0.99 as the new class (after: chair 0.96, banana 0.95), and update the text accordingly”,
[0062] : “ FIG. 8 is a flowchart of a method for adding a new class of objects, designated as class A, to an existing object detector, such as the detector used in FIG. 7. This method illustrates obtaining training datasets that can be used for incremental training and is based on what is provided in the input (image or name), the ability to obtain existing training datasets or the need to create a new training dataset”,
- merging the feature vector of the base class and the feature vector of the novel class to calculate a merged feature vector that merges the base class and the novel class;
[0062] : “ method illustrates obtaining training datasets that can be used for incremental training and is based on what is provided in the input (image or name), the ability to obtain existing training datasets or the need to create a new training dataset.
[0043]: “ The two-stage neural network object detector is iteratively updated so as to balance minimizing a loss of its initial ability to locate and identify objects pertaining to previously-learned object classes and maximizing its ability to additionally locate and to identify the objects pertaining to the additional object class. Ability may be measured by percentage of correct detections among all detections, an average of prediction probabilities, combined with a number of false positive detections or false negative detections, etc.
[0026]: “ An initial version of a two-stage object detector, which is named “Teacher Network (T),” is able to detect (i.e., locate and identify) objects pertaining to the previously-learned object classes. A current version of the two-stage neural network object detector named “Student Network (S)” is trained to detect objects pertaining to a new object class. The following description refers to a single new object class, but plural object classes may be added successively or simultaneously. The student network starts with the parameters from the teacher network, but output from the student network is extended to detect objects pertaining to the new object class.
- and learning a classification weight vector of the novel class to minimize a loss incurred in classification.
[0025]: “ employ a reasonable number of model parameters and to have acceptable memory requirements for using the model; and (D) to provide an end-to-end learning mechanism that jointly updates the classifier to identify the new object class and the feature representation to locate an object pertaining to the new object class”,
[0010]:” Exemplary embodiments are directed to methods for incremental learning object detection in images without catastrophic forgetting of previously-learned object classes. The methods include training an existing two-stage neural network object detector to locate and identify objects pertaining to at least one additional object class in images by iteratively updating the two-stage neural network object detector until an overall detection accuracy criterion is met. The updating is performed so as to balance minimizing a loss of an initial ability to locate and identify objects pertaining to the previously-learned one or more object classes and maximizing an ability to locate and identify the objects pertaining to the additional object class”
CHANG does not explicitly disclose:
- A machine learning method that continually learns a novel class with fewer samples than a base class, comprising:
- and classifying, on a projected space, a query sample of a query set based on a distance between a position of the merged feature vector of the query sample of the query set and a position of a classification weight vector of each class,
However, Sun discloses:
- A machine learning method that continually learns a novel class with fewer samples than a base class, comprising:
[0107] “ As shown in FIG. 12, after processing data trustworthiness there may occur a core/cloud query for few samples of feature saliency maps from the user terminal UE who found new object according to the feature vector they submitted in the process of establishing data trustworthiness. Then, the core finds a new feature area and crafts new feature vectors the existing machine learning model” ,
[0066] “ As shown in FIG. 5, the step S10 of obtaining the at least one feature vector further comprises a step 22, operatively being executed by the concluding unit 34, of concluding on the new specimen data being related to a new object upon continuous occurrence of the plurality of feature vectors.”
[0096]” new objects can appear in the environment like a new moving robot different than the previous one that was identified, e.g., a robot as shown in FIG. 10(c). Then the algorithm according to the present invention will identify the object as a new object and assign a label to it for subsequent handling thereof”,
[BRI: the new feature is the novel class determined from few samples of feature saliency maps. A previously identified object is a “base class” and the new object is a “novel class”]
- and classifying, on a projected space, a query sample of a query set based on a distance between a position of the merged feature vector of the query sample of the query set and a position of a classification weight vector of each class,
[0070] “ Generally, the feature vector crafting apparatus 18 supports identification of unknown features in data like images and creation of related new feature vectors for subsequent use in the machine learning infrastructure 20,
[0057]: “ As shown in FIG. 3, the method of verifying at least one data source 14-1, . . . , 14-n further comprises a step S12, operatively being executed by the correlation checking unit 24, of executing a correlation check between the received at least one feature vector and at least one reference feature vector representing trusted specimen data for determination of a correlation measure. As outlined above, the correlation check may use a similarity/distance check”.
It would be obvious to one of ordinary skill in the art before the effective filing date of the present application to combine CHANG and Sun.
CHANG teaches an incremental learning that generates a novel class (new class) with a base class (previously learned class) and generates loss.
Sun teaches distance between query set and merged feature vector.
One of ordinary skills would be motivated to combine CHANG and Sun that can provide small, expected difference for new object and can automatically verify the trustworthiness of data sources (Sun [0040).
Regarding claim 4:
CHANG discloses:
- a module that extracts a feature vector of the base class by using a base class feature extractor.
[0024” Flowcharts and block diagrams in figures used in the following descriptions illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module”,
[0027]: “For each input image, the teacher network 110 yields a teacher region proposal (RPN(T)) 111 used to extract a feature vector 112 and a teacher object prediction 113 identifying one of the previously-learned object classes as corresponding to an object in each region of the teacher RPN.”,
- a module that subjects the base class feature extractor to self-distillation k times (k is a natural number) to obtain a novel class feature extractor;
[0024” Flowcharts and block diagrams in figures used in the following descriptions illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module”,
[0026] “FIG. 1 schematically represents incremental learning object detection in images without catastrophic forgetting of previously-learned object classes according to an embodiment. An initial version of a two-stage object detector, which is named “Teacher Network (T),” is able to detect (i.e., locate and identify) objects pertaining to the previously-learned object classes. A current version of the two-stage neural network object detector named “Student Network (S)” is trained to detect objects pertaining to a new object class. The following description refers to a single new object class, but plural object classes may be added successively or simultaneously. The student network starts with the parameters from the teacher network, but output from the student network is extended to detect objects pertaining to the new object class”,
[0028] “ Detection accuracy may be estimated based on the teacher network's and student network's outputs. A region proposal network distillation loss 131 quantifying a decrease of student network's ability to locate objects of the previously-learned object classes is based on comparing the teacher region proposal 111 with the student region proposal 121,
[0030]” In one embodiment, a total loss is defined by combining the region proposal network distillation loss, the object identification distillation loss, the region proposal network loss and the object identification loss. The student network may then be updated so as to minimize the total loss.
[BRI: Self-distillation – In this context, the same unit is used as both teacher and student: the unit’s own predictions are used as soft targets to retrain itself. This process encourages the model to refine its feature representations by learning from its own outputs, often reducing intra-class variance and improving generalization]
[0033]:” The two-stage neural network object detector is trained to detect the objects of the object class initially unknown, using the training images one-by-one or batch-by-batch until a predetermined condition is met at 220. The predetermined condition may be completing a predetermined number of iterations, achieving a threshold accuracy or failing to further decrease losses,
[0011] “The predetermined condition is met when the number of training iterations reaches a predetermined number, or when a total loss decrease rate is below a predetermined threshold.
[BRI: The self-distillation process is repeated k times. Each iteration takes the output of the previous iteration as the new target distribution for training. After k iterations, the resulting unit is the novel class feature extraction unit. The “the predetermined threshold” is “k”]
- a module that extracts a feature vector of the novel class by using the novel class feature extractor.
[0024” Flowcharts and block diagrams in figures used in the following descriptions illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module”,
[0046]: “the object prediction may be expressed as a specific object probability. the “chair” 0.98 and “banana” 0.95 as the old classes, and “laptop” 0.99 as the new class (after: chair 0.96, banana 0.95), and update the text accordingly.
[0062] : “ FIG. 8 is a flowchart of a method for adding a new class of objects, designated as class A, to an existing object detector, such as the detector used in FIG. 7. This method illustrates obtaining training datasets that can be used for incremental training and is based on what is provided in the input (image or name), the ability to obtain existing training datasets or the need to create a new training dataset.
- a module that merges the feature vector of the base class and the feature vector of the novel class to calculate a merged feature vector that merges the base class and the novel class;
[0024” Flowcharts and block diagrams in figures used in the following descriptions illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module”,
[0062] : “ method illustrates obtaining training datasets that can be used for incremental training and is based on what is provided in the input (image or name), the ability to obtain existing training datasets or the need to create a new training dataset”,
[0043]: “ The two-stage neural network object detector is iteratively updated so as to balance minimizing a loss of its initial ability to locate and identify objects pertaining to previously-learned object classes and maximizing its ability to additionally locate and to identify the objects pertaining to the additional object class. Ability may be measured by percentage of correct detections among all detections, an average of prediction probabilities, combined with a number of false positive detections or false negative detections, etc.
[0026]: “ An initial version of a two-stage object detector, which is named “Teacher Network (T),” is able to detect (i.e., locate and identify) objects pertaining to the previously-learned object classes. A current version of the two-stage neural network object detector named “Student Network (S)” is trained to detect objects pertaining to a new object class. The following description refers to a single new object class, but plural object classes may be added successively or simultaneously. The student network starts with the parameters from the teacher network, but output from the student network is extended to detect objects pertaining to the new object class.
[BRI: the merging feature vectors means combining the attribute (feature) data from two datasets where a base class dataset and a novel class dataset are merged into a single, unified dataset]
- and learns a classification weight vector of the novel class to minimize a loss incurred in classification.
[0025]: “ employ a reasonable number of model parameters and to have acceptable memory requirements for using the model; and (D) to provide an end-to-end learning mechanism that jointly updates the classifier to identify the new object class and the feature representation to locate an object pertaining to the new object class”,
[0010]:” Exemplary embodiments are directed to methods for incremental learning object detection in images without catastrophic forgetting of previously-learned object classes. The methods include training an existing two-stage neural network object detector to locate and identify objects pertaining to at least one additional object class in images by iteratively updating the two-stage neural network object detector until an overall detection accuracy criterion is met. The updating is performed so as to balance minimizing a loss of an initial ability to locate and identify objects pertaining to the previously-learned one or more object classes and maximizing an ability to locate and identify the objects pertaining to the additional object class”
CHANG does not explicitly disclose:
- that continually learns a novel class with fewer samples than a base class, the program comprising computer-implemented modules that include:
- a module that classifies, on a projected space, a query sample of a query set based on a distance between a position of the merged feature vector of the query sample of the query set and a position of a classification weight vector of each class
However, Sun discloses:
- A computer readable non-transitory recording medium storing a machine learning program that continually learns a novel class with fewer samples than a base class, the program comprising computer-implemented modules that include:
[0092]:
Further to the above, the present invention also relates to a computer program product 56a comprising a computer readable medium, the computer readable medium having computer readable code embodied therein, the computer readable code being configured such that, on execution by a suitable computer or processor 54, the computer or processor is caused to perform a method,
[BRI: a “computer readable medium having computer readable code embodied therein” is generally interpreted to cover a non-transitory recording medium]
[0014] According to an eighth aspect of the present invention there is provided a system for automatic generation of at least one feature vector for use in a machine learning infrastructure from new specimen data generated by at least one data source
[BRI: a module may be a discrete, reusable unit of code or functionality within a ML infrastructure]
[0107] “ As shown in FIG. 12, after processing data trustworthiness there may occur a core/cloud query for few samples of feature saliency maps from the user terminal UE who found new object according to the feature vector they submitted in the process of establishing data trustworthiness. Then, the core finds a new feature area and crafts new feature vectors the existing machine learning model”
[0066] “ As shown in FIG. 5, the step S10 of obtaining the at least one feature vector further comprises a step 22, operatively being executed by the concluding unit 34, of concluding on the new specimen data being related to a new object upon continuous occurrence of the plurality of feature vectors.”
[0096]” new objects can appear in the environment like a new moving robot different than the previous one that was identified, e.g., a robot as shown in FIG. 10(c). Then the algorithm according to the present invention will identify the object as a new object and assign a label to it for subsequent handling thereof.
[BRI: the new feature is the novel class determined from few samples of feature saliency maps. A previously identified object is a “base class” and the new object is a “novel class”]
- a module that classifies, on a projected space, a query sample of a query set based on a distance between a position of the merged feature vector of the query sample of the query set and a position of a classification weight vector of each class
[0007]: “According to a first aspect of the present invention there is provided a method of verifying at least one data source that generates new specimen data intended for use in a machine learning infrastructure”,
[BRI: A ML infrastructure contains a module]
[0070] “ Generally, the feature vector crafting apparatus 18 supports identification of unknown features in data like images and creation of related new feature vectors for subsequent use in the machine learning infrastructure 20,
[0057]: “ As shown in FIG. 3, the method of verifying at least one data source 14-1, . . . , 14-n further comprises a step S12, operatively being executed by the correlation checking unit 24, of executing a correlation check between the received at least one feature vector and at least one reference feature vector representing trusted specimen data for determination of a correlation measure. As outlined above, the correlation check may use a similarity/distance check
It would be obvious to one of ordinary skill in the art before the effective filing date of the present application to combine CHANG and Sun.
CHANG teaches an incremental learning that generates a novel class (new class) with a base class (previously learned class) and generates loss.
Sun teaches distance between query set and merged feature vector.
One of ordinary skills would have been motivated to combine CHANG and Sun that can provide small, expected difference for new object and can automatically verify the trustworthiness of data sources (Sun [0040]).
Conclusion
Any inquiry concerning this communication or earlier communications from the
examiner should be directed to TIRUMALE KRISHNASWAMY RAMESH whose telephone number is (571)272-4605. The examiner can normally be reached by phone.
Examiner interviews are available via telephone, in-person, and video conferencing
using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li B Zhen can be reached on phone (571-272-3768). The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be
obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit:
https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for
information about filing in DOCX format. For additional questions, contact the Electronic
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO
Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TIRUMALE K RAMESH/Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights