Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is in response to the amendments filed 31 March 2026. Claims 1-20 are amended. Claims 1-20 are pending and have been examined.
Response to Arguments
Applicant' s arguments, see pages 17-18, filed 31 March 2026, with respect to the objections to the title and Claim 2 have been fully considered and are persuasive. The objections to the title and Claim 2 have been withdrawn.
APPLICANT'S ARGUMENT: Applicant argues (page 17, paragraph 3) that "The Applicant has amended the Title of the Disclosure, as set forth above. Accordingly, the Applicant requests that the objection to the Title be withdrawn."
Applicant argues (page 18, paragraph 1) that "Claim 2 has been amended to replace 'from an outside' with 'from outside the learning device', as set forth above. Accordingly, the Applicant requests that the objection to claim 2 be withdrawn."
EXAMINER'S RESPONSE: Examiner agrees. The objections to the title of the disclosure and to Claim 2 have been withdrawn in light of arguments and/or amendments.
Applicant' s arguments, see page 18, filed 31 March 2026, with respect to the interpretation of Claims 1, 3-5, 7, 11, 13-15, and 18 under 35 U.S.C. 112(f) have been fully considered and are persuasive. The interpretation of Claims 1, 3-5, 7, 11, 13-15, and 18 under 35 U.S.C. 112(f)has been withdrawn.
APPLICANT'S ARGUMENT: Applicant argues (page 18, paragraph 2) that "claims 1, 3-5, 7, 11, 13-15, and 18 do not invoke claim interpretation under 35 U.S.C. § 112, sixth paragraph."
EXAMINER'S RESPONSE: Examiner agrees. The interpretation of Claims 1, 3-5, 7, 11, 13-15, and 18 under 35 U.S.C. 112(f)has been withdrawn in light of arguments and/or amendments.
Applicant' s arguments, see page 18, filed 31 March 2026, with respect to the interpretation of Claims 1-20 under 35 U.S.C. 112(b) have been fully considered and are persuasive. The rejections of Claims 1-20 under 35 U.S.C. 112(b) have been withdrawn.
APPLICANT'S ARGUMENT: Applicant argues (page 18, paragraph 3) that "Claims 1-20 have been amended, as set forth above. Accordingly, the Applicant requests that the rejection of claims 1-20 under 35 U.S.C. § 112, second paragraph be withdrawn."
EXAMINER'S RESPONSE: Examiner agrees. The rejections of Claims 1-20 under 35 U.S.C. 112(b) have been withdrawn in light of arguments and/or amendments.
Applicant' s arguments, see pages 18-23, filed 31 March 2026, with respect to the rejections of Claims 1-20 under 35 U.S.C. 101 have been fully considered and are persuasive. The rejections to Claims 1-20 under 35 U.S.C. 101 have been withdrawn.
APPLICANT'S ARGUMENT: Applicant argues (page 19, paragraph 2) that "performing learning of a machine learning model is a computationally intensive process involving iterative mathematical optimization (e.g., backpropagation, gradient descent, weight adjustment across potentially millions of parameters) that cannot be practically performed in the human mind. Furthermore, generating provisional correct answer data by running an entire group of unlabeled processing target data through a trained first model is a computational operation that requires executing the trained model on each piece of data in the group which is an operation with no meaningful manual analog."
Applicant argues (page 19, paragraph 3) that "A human cannot select learning data 'based on ... the provisional correct answer data' because that data is a product of computational inference, not of human observation or judgment."
Applicant argues (page 19, paragraph 4) that "The amended independent claims recite performing learning of a machine learning model and generating provisional correct answer data by running unlabeled data through that model which are Al operations that cannot be practically performed in the human mind and therefore do not fall within the mental process grouping. ¶ These limitations ... merely involve mathematical concepts but do not recite them. The claims are therefore eligible and do not require further eligibility analysis."
Applicant argues (page 20, paragraph 3) that "amended independent Claim 1 does not broadly recite 'selecting data' rather it specifies the particular mechanism by which selection is performed: training a first model, generating provisional correct answers from unlabeled data through that model, and selecting learning data based on the two-input combination of the learning data group and the provisional correct answer data."
Applicant argues (page 21, paragraph 1) that "The Office Action analyzed each claim element in isolation while separately ... without considering how these elements interact as an integrated system in which the processor trains a model, runs unlabeled data through that model to generate provisional labels, uses those provisional labels to guide selection, and co-outputs the selected data with the trained model."
Applicant argues (page 20, paragraph 3) that "Amended independent Claim 1 recites a particular solution ... not merely the idea of 'selecting data'. The claim improves ML training technology by automating training data selection through a provisional-labeling mechanism that bridges labeled and unlabeled data domains. Furthermore, the application is particular, limited to a specific pipeline."
EXAMINER'S RESPONSE: Examiner agrees. The rejections to Claims 1-20 under 35 U.S.C. 101 have been withdrawn in light of arguments and/or amendments.
APPLICANT'S ARGUMENT: Applicant argues (page 23, paragraph 3) that "Independent claims 17 and 20 have been amended to recite 'non-transitory computer-readable medium' so as to place the claims 17 and 20 in the statutory category."
EXAMINER'S RESPONSE: Examiner agrees. The rejections to Claims 17 and 20 under 35 U.S.C. 101 have been withdrawn in light of arguments and/or amendments.
Applicant's arguments, see pages 24-27, filed 31 March 2026, with respect to the rejections to Claims 1, 2, 9, and 16-20 under 35 U.S.C. 102(a)(2) and Claims 3-8 and 10-15 under 35 U.S.C. 103 have been fully considered but they are not persuasive.
APPLICANT'S ARGUMENT: Applicant argues (page 25, paragraphs 2-4) that "Kanno describes a learning device in which labeled training data is input to a first model, and a selecting unit evaluates the same labeled training data by comparing an output of the first model with corresponding correct answer data to select appropriate and inappropriate training data. ¶ However, Kanno does not describe inputting unlabeled processing target data to a trained first model, generating provisional correct answer data based on such unlabeled data, and selecting learning data based on the generated provisional correct answer data. ¶ The Applicant respectfully submits that these features of the present invention, which rely on inference using unlabeled target data to guide data selection, are fundamentally different from Kanno's approach of evaluating labeled training data against its own known correct answers."
EXAMINER'S RESPONSE: Examiner respectfully disagrees. As indicated in the rejection of amended Claim 1 below, Kanno is shown to teach inputting processing target data that has no correct answer into the trained first model. Kanno is shown to teach (see [0070] and [0075]) the first model producing provisional correct answer data based on model conference, rather than on supervised label data, that is used for selection of training data for the second, inference model.
APPLICANT'S ARGUMENT: Applicant argues (page 26, paragraph 3) that "dependent Claims 2 and 9 are also not anticipated by Kanno based at least on the dependence on amended independent Claim 1."
Applicant argues (page 26, paragraph 5) that "Kida does not remedy the above-noted deficiency of Kanno. Accordingly, the Applicant submits that dependent claims 3-7 and 10 are not taught, suggested, or rendered obvious over the combination of Kanno and Kida based at least on the dependence on amended independent Claim 1."
Applicant argues (page 27, paragraph 2) that "Metzler does not remedy the above-noted deficiency of Kanno. Accordingly, the Applicant submits that dependent claims 8 and 11-15 are not taught, suggested, or rendered obvious over the combination of Kanno and Metzler based at least on the dependence on amended independent Claim 1."
EXAMINER'S RESPONSE: Examiner respectfully disagrees. As indicated in the rejection of amended Claim 1 below, Kanno is shown to anticipate the invention recited by amended Claim 1. The dependent claims are rejected as indicated below.
Specification
The objection to the title of the invention is withdrawn in light of arguments and/or amendments.
Claim Objections
The objection to Claim 2 for informalities is withdrawn in light of arguments and/or amendments.
Claim Interpretation
Claims 1, 3-5, 7, 11, 13-15, 18 are no longer being interpreted under 35 U.S.C. 112(f) in light of arguments and/or amendments.
Claim Rejections - 35 USC § 112(b)
The rejections of Claims 1-20 under 35 U.S.C. 112(b) are withdrawn in light of arguments and/or amendments.
Claim Rejections - 35 USC § 101
The rejections of Claims 17 and 20 under 35 U.S.C. 101 for non-statutory matter are withdrawn in light of arguments and/or amendments.
The rejections of Claims 1-20 for abstract idea are withdrawn in light of arguments and/or amendments.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 2, 9, and 16-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Kanno (US 2021/0004723 A1, hereinafter "Kanno").
Regarding Claim 1, Kanno teaches:
A learning device, comprising: at least one processor (Kanno, [0027]: "A learning device 100 of the present invention includes a training data storage unit 1, a first learning unit 2, a first model storage unit 3, a selecting unit 4, a second learning unit 5, and a second model storage unit 6" and [0045]: "The first learning unit 2, the selecting unit 4, and the second learning unit 5 are implemented by, for example, a central processing unit (CPU) of a computer that operates according to a learning program") configured to:
perform learning of a first model based on at least one first piece of learning data associated with a correct answer, wherein a learning data group includes the at least one first piece of learning data associated with the correct answer (Kanno, [0010]: "A learning device according to the present invention includes: a training data storage means that stores training data used for generating a first model for determining a category to which given data belongs, the training data being associated with a predetermined correct answer category; a first learning means that executes a first learning process of learning the first model by machine learning using the training data," where Kanno's stored training data corresponds to the instant learning data group);
provide, as an input to the learned first model, processing target data from a processing target data group (Kanno, [0011]: "a learning method according to the present invention is characterized in that a computer including a training data storage means that stores training data used for generating a first model for determining a category to which given data belongs, the training data being associated with a predetermined correct answer category, executes a first learning process of learning the first model by machine learning using the training data"), wherein the processing target data has no correct answer (Kanno, [0031]: "Note that the training data is not limited to the above image data. The user only needs to store training data according to the type of determination for which a model is learned, as the first model, in the training data storage unit 1 in association with a correct answer category," where Kanno's training data is stored according only to the type of determination, corresponding to the instant training data with no correct answer, where Kanno's first and second models determine a data category and an appropriateness, respectively, as in [0028]: "a model for determining a category to which given data belongs is referred to as a first model. In addition, a model for determining whether or not each of training data is appropriate as training data used for learning the first model is referred to as a second model," and where the second model's appropriateness is based on a ranking process of the first model's output rather than a label) and corresponds to data to be processed at a time of inference (Kanno,[0030]: "A case where a model for determining whether or not an object in an image corresponds to a prescribed object is learned as the first model will be exemplified");
generate provisional correct answer data based on an output of the first model and the processing target data (Kanno, [0036]: "The selecting unit 4 reads each piece of training data from the training data storage unit 1. Then, for each correct answer category, the selecting unit 4 determines a category to which each of training data belongs by applying each of the training data to the first model. ... For each correct answer category, regarding each of training data, the selecting unit 4 calculates a difference ... and further sorts the training data based on the difference," where Kanno's grouped and sorted determination data corresponds to the instant provisional correct answer data, and where the appropriateness sorting may not occur according to a supervised signal, as in [0075]: "the selecting unit 4 may sort training data based on the above-described certainty factor. Specifically, the selecting unit 4 may sort training data in ascending order for each correct answer category based on the certainty factor for a correct answer category corresponding to training data, the certainty factor being obtained in the process of determining a category of the training data" and [0070]: "the selecting unit 4 calculates a certainty factor for each category in a process of determining a category to which training data belongs by applying the training data to the first model");
select from the learning data group at least one second piece of learning data for learning of an inference model based on the learning data group and the provisional correct answer data (Kanno, [0039]: "For each correct answer category, the selecting unit 4 selects a predetermined number of pieces of higher training data as appropriate training data and selects a predetermined number of pieces of lower training data as inappropriate training data from the training data sorted in ascending order. ... The training data selected as appropriate training data and the training data selected as inappropriate training data serve as training data (teacher data) for learning the second model");
perform learning of the inference model based on the selected at least one second piece of learning data (Kanno, [0042]: "The second learning unit 5 learns the second model by machine learning using the training data selected as appropriate training data by the selecting unit 4 and the training data selected as inappropriate training data by the selecting unit 4 (in other words, using the training data as teacher data). The second learning unit 5 learns the second model regardless of a correct answer category collectively using the training data selected by the selecting unit 4. Therefore, the second learning unit 5 learns one second model even if there is a plurality of types of correct answer categories," where Kanno's second model correspond to the instant inference model);
output the selected at least one second piece of learning data together with the learned inference model (Kanno, [0080]: "After step S306, the second learning unit 5 stores the second model generated in step S306 in the second model storage unit 6 (step S307)" and [0011]: "a learning method according to the present invention is characterized in that a computer including a training data storage means ... repeats the first learning process, the selecting process, and the second learning process until a prescribed condition is satisfied, and in a case where the second model has been generated in the first learning process, evaluates each of the training data by applying each of the training data to the second model, excludes training data of a prescribed evaluation, and learns the first model," where Kanno's re-use of training data evaluated by the second model for later use in re-training the first model teaches or reasonably suggests outputting the second model together with the appropriate training data from the learning process).
Regarding Claim 16, Kanno teaches:
A generation method (Kanno, [0009]: "an object of the present invention is to provide a learning device, a learning method, and a learning program that can accurately exclude training data inappropriate for learning a model from training data and can learn the model") comprising: by at least one processor of a learning device (Kanno, [0027]: "A learning device 100 of the present invention includes a training data storage unit 1, a first learning unit 2, a first model storage unit 3, a selecting unit 4, a second learning unit 5, and a second model storage unit 6" and [0045]: "The first learning unit 2, the selecting unit 4, and the second learning unit 5 are implemented by, for example, a central processing unit (CPU) of a computer that operates according to a learning program") precisely those steps recited to be performed according to the learning device of Claim 1. Claim 16 is rejected under the same rationale as Claim 1.
Regarding Claim 17, Kanno teaches:
A non-transitory computer-readable medium having stored thereon instructions, when executed by at least one processor, cause the at least one processor to perform operations (Kanno, [0045]: "The first learning unit 2, the selecting unit 4, and the second learning unit 5 are implemented by, for example, a central processing unit (CPU) of a computer that operates according to a learning program" and [0102]: "The learning device 100 according to each of the exemplary embodiments of the present invention is mounted on the computer 1000. Operation of the learning device 100 is stored in the auxiliary storage device 1003 in a form of a learning program" and [0103]: "The auxiliary storage device 1003 is an example of a non-transitory tangible medium") comprising: precisely those steps recited by the method of Claim 1. Claim 17 is rejected under the same rationale as Claim 1.
Regarding Claim 18, Kanno teaches:
An inference device, comprising: at least one processor (Kanno, [0045]: "The first learning unit 2, the selecting unit 4, and the second learning unit 5 are implemented by, for example, a central processing unit (CPU) of a computer that operates according to a learning program" and [0102]: "The learning device 100 according to each of the exemplary embodiments of the present invention is mounted on the computer 1000. Operation of the learning device 100 is stored in the auxiliary storage device 1003 in a form of a learning program") configured to:
receive data as an input to an inference model (Kanno, [0030]: "A case where a model for determining whether or not an object in an image corresponds to a prescribed object is learned as the first model will be exemplified. In this case, for example, a user of the learning device 100 (hereinafter, simply referred to as a user) collects a plurality of pieces of image data. Then, the user stores each piece of image data in the training data storage unit 1 in advance in association with a correct answer category"), wherein the inference model is output from a learning device (Kanno, Fig. 4, S206, "STORE FIRST MODEL IN FIRST MODEL STORAGE UNIT 3"); and
output an inference result (Kanno, [0098]: "In the above description, as the first model, a model for determining whether or not an object in an image corresponds to a prescribed object has been exemplified.... [T]he first model may be a model for classifying a small product whose posture is difficult to fix") that represents a result of a predetermined process comprising precisely those steps recited to be performed according to the learning device of Claim 1. Claim 18 is rejected under the same rationale as Claim 1.
Regarding Claim 19, Kanno teaches:
An inference method, comprising: by at least one processor of an inference device (Kanno, [0098]: "In the above description, as the first model, a model for determining whether or not an object in an image corresponds to a prescribed object has been exemplified. The first model is not limited to such a model. For example, the first model may be a model for classifying a small product whose posture is difficult to fix (for example, a screw) in an image in a case where image data of the image including the small product is given," where an inference device is inherent in Kanno's usage of the product-classifying model) trained according to precisely those steps recited as performed by the inference device Claim 18. Claim 19 is rejected under the same rationale as Claim 18.
Regarding Claim 20, Kanno teaches:
A non-transitory computer-readable medium having stored thereon instructions, when executed by at least one processor (Kanno, [0045]: "The first learning unit 2, the selecting unit 4, and the second learning unit 5 are implemented by, for example, a central processing unit (CPU) of a computer that operates according to a learning program" and [0102]: "The learning device 100 according to each of the exemplary embodiments of the present invention is mounted on the computer 1000. Operation of the learning device 100 is stored in the auxiliary storage device 1003 in a form of a learning program" and [0103]: "The auxiliary storage device 1003 is an example of a non-transitory tangible medium"), cause the at least one processor to perform operations comprising precisely those steps recited by Claim 18. Claim 20 is rejected under the same rationale as Claim 18.
Regarding Claim 2, the rejection of Claim 1 is incorporated. Kanno teaches:
the at least one processor is further configured to receive the learning data group and the processing target data group from outside the learning device (Kanno, [0030]: "a user of the learning device 100 (hereinafter, simply referred to as a user) collects a plurality of pieces of image data. Then, the user stores each piece of image data in the training data storage unit 1 in advance in association with a correct answer category," where Kanno's a user of the learning device corresponds to the instant outside).
Regarding Claim 9, the rejection of Claim 1 is incorporated. Kanno teaches:
wherein the learning data group includes data acquired by a sensor or data generated by a computer (Kanno, [0100]: "the first model may be a model for classifying an object in an image imaged in an environment affected by disturbance (outdoors or the like) in a case where image data of the image is given," where Kanno's image imaged outdoors corresponds to the instant sensor data).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 3-7 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Kanno (US 2021/0004723 A1, hereinafter "Kanno") in view of Kida, et al. (US 2019/0065989 A1, hereinafter "Kida").
Regarding Claim 3, the rejection of Claim 1 is incorporated. Kanno teaches:
the at least one processor is further configured to ... acquire the at least one first piece of learning data from the learning data group (Kanno, [0036]: "The selecting unit 4 reads each piece of training data from the training data storage unit 1. Then, for each correct answer category, the selecting unit 4 determines a category to which each of training data belongs by applying each of the training data to the first model," where Kanno's selecting unit acquires data for reading).
Kanno teaches a data acquiring unit configured to acquire the learning data from the learning data group and a first learning unit configured to perform learning of a first model by using the acquired learning data.
Kanno does not explicitly teach randomly acquire the learning data.
However, Kida teaches:
randomly acquire the ... learning data (Kida, [0021]: "Samples may be selected from a data set in various ways. ... In an example, the samples may be selected randomly from the entire data set. Samples sets 406,408, and 410 each illustrate a possible data subset selected by randomly selecting samples. ... As described in greater detail below, multiple training data subsets may be created from a larger available data set").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Kanno regarding a data acquiring unit configured to acquire the learning data from the learning data group and a first learning unit configured to perform learning of a first model by using the acquired learning data with those of Kida regarding randomly acquire the learning data.
The motivation to do so would be to ensure that the trained model has desired accuracy (Kida, [0021]: "Samples sets ... each illustrate a possible data subset selected by randomly selecting samples. ... These data subsets are used to train, select and estimate performance of various models. Iterating over the multiple data subsets allows a model to be created that has a desired accuracy or sensitivity and complexity of implementation").
Regarding Claim 4, the rejection of Claim 3 is incorporated. The Kanno/Kida combination teaches:
the at least one processor is further configured to perform learning of a second model based on the processing target data as an input and the provisional correct answer data as an output (Kanno, [0042]: "The second learning unit 5 learns the second model by machine learning using the training data selected as appropriate training data by the selecting unit 4 and the training data selected as inappropriate training data by the selecting unit 4 (in other words, using the training data as teacher data). The second learning unit 5 learns the second model regardless of a correct answer category collectively using the training data selected by the selecting unit 4. Therefore, the second learning unit 5 learns one second model even if there is a plurality of types of correct answer categories," where Kanno's selected teacher data is based on the output of the first model, corresponding to the instant provisional, as in [0036]: "the selecting unit 4 determines a category to which each of training data belongs by applying each of the training data to the first model," and where training data may contain image data as a processing target, as in [0031]: "Note that the training data is not limited to the above image data. The user only needs to store training data according to the type of determination for which a model is learned").
Regarding Claim 5, the rejection of Claim 4 is incorporated. The Kanno/Kida combination teaches:
obtain a first inference result from the first model based on the ... acquired at least one first piece of learning data as an input to the first model (Kanno, [0036]: "for each correct answer category, the selecting unit 4 determines a category to which each of training data belongs by applying each of the training data to the first model");
obtain a second inference result from the second model based on the ... acquired at least one first piece of learning data as an input to the second model (Kanno, [0042]: "The second learning unit 5 learns the second model by machine learning using the training data selected as appropriate training data by the selecting unit 4 and the training data selected as inappropriate training data by the selecting unit 4 (in other words, using the training data as teacher data)");
compare the first inference result with the second inference result (Kanno, [0044]: "The learning device 100 repeats a process ... until a prescribed condition is satisfied. ... [A]nother example of this prescribed condition is that in a case where training data selected as appropriate training data or inappropriate training data is applied to the second model, a difference between classification of the training data ('appropriate' or 'inappropriate') and a determination result obtained by applying the training data to the second model is equal to or less than a predetermined threshold value"); and
select the at least one second piece of learning data for the learning of the inference model, based on comparison result of the first inference result with the second inference result (Kanno, [0059]: "After step S203, by applying each piece of each of training data read from the training data storage unit 1 in step S201 to the second model read in step S203, the first learning unit 2 determines whether or not each of the training data is appropriate as training data used for learning the first model. Then, the first learning unit 2 excludes training data that has been determined to be inappropriate from each piece of training data read from the training data storage unit 1 ( step S204)").
Kanno does not explicitly teach randomly acquire the learning data.
However, Kida teaches:
a first inference result ... based on the randomly acquired at least one first piece of learning data as an input to the first model (Kida, [0021]: "Samples may be selected from a data set in various ways. ... In an example, the samples may be selected randomly from the entire data set. Samples sets 406,408, and 410 each illustrate a possible data subset selected by randomly selecting samples. ... As described in greater detail below, multiple training data subsets may be created from a larger available data set" and [0022]: "A subset of the data may be used as training data to train a model to classify a sample. The training data is labeled meaning each sample in the training data includes the sample's correct class").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Kanno regarding a data acquiring unit configured to acquire the learning data from the learning data group and a first learning unit configured to perform learning of a first model by using the acquired learning data with those of Kida regarding randomly acquire the learning data.
The motivation to do so would be to ensure that the trained model has desired accuracy (Kida, [0021]: "Samples sets ... each illustrate a possible data subset selected by randomly selecting samples. ... These data subsets are used to train, select and estimate performance of various models. Iterating over the multiple data subsets allows a model to be created that has a desired accuracy or sensitivity and complexity of implementation").
Regarding Claim 6, the rejection of Claim 5 is incorporated. The Kanno/Kida combination teaches:
the at least one processor is further configured to select the at least one second piece of learning data for which a difference between the first inference result and the second inference result is smaller than a threshold (Kanno, [0062]: "In addition, in step S204, the first learning unit 2 may calculate a numerical value indicating a difference between classification of training data ('appropriate' or 'inappropriate') determined by the selecting unit 4 and a determination result obtained by applying the training data to the second model. Then, in step S103 (see FIG. 3), the first learning unit 2 may determine whether or not a prescribed condition is satisfied depending on whether or not the numerical value indicating the difference is equal to or less than a threshold value").
Regarding Claim 7, the rejection of Claim 5 is incorporated. The Kanno/Kida combination teaches:
output , as the inference model, the first model (Kanno, Fig. 4, S206, "STORE FIRST MODEL IN FIRST MODEL STORAGE UNIT 3" and [0047]: "The first learning unit 2 learns the first model and stores the first model in the first model storage unit 3 (step S101)," where outputting is inherent in Kanno's store step) obtained by repeatedly performing learning (Kanno, Fig. 4, S202, "FIRST ITERATRION" and "SECOND OR SUBSEQUENT ITERATION") together with the selected at least one second piece of learning data (Kanno, [0012]: "a learning program ... including a training data storage means that stores training data used for generating a first model for determining a category to which given data belongs, the training data being associated with a predetermined correct answer category, and is characterized by causing the computer to execute a first learning process of learning the first model");
... select a third piece of the learning data from the learning data group, instead of the learning data not selected, based on the comparison result (Kanno, [0067]: "In step S102, first, the selecting unit 4 reads each piece of training data from the training data storage unit 1 (step S301),"
where the NO branch of Fig. 3 is followed, and where training data is progressively excluded over cycles, as in [0085]: "inappropriate training data is accurately excluded, and the first model is learned");
repeatedly perform learning of the first model (Kanno, Fig. 3, S101, "LEARN FIRST MODEL" and Fig. 4, S202, "FIRST ITERATION" and "SECOND OR SUBSEQUENT ITERATION") based on the selected at least one first piece of learning data and the another piece of the learning data ... selected (Kanno, [0036]: "The selecting unit 4 reads each piece of training data from the training data storage unit 1. Then, for each correct answer category, the selecting unit 4 determines a category to which each of training data belongs by applying each of the training data to the first model," where iterations of the training process exclude inappropriate data, as in [0110]: "in a case where the second model has been generated in the first learning process, the first learning means 72 evaluates each of the training data by applying each of the training data to the second model, excludes training data of a prescribed evaluation, and learns the first model" and [0111]: "With such a configuration, it is possible to accurately exclude training data inappropriate for learning the first model from training data and to learn the first model"), and
obtain a third inference result from the first model based on the learning of the first model, wherein the learning of the first model is performed based on the selected at least one first piece of learning data and the third piece of the learning data (Kanno, Fig. 3, result obtained from the first model at S101 from first or later repeated iteration); and
... perform learning of the second model based on the third inference result of the first model (Kanno, [0069]: "the selecting unit 4 determines a category to which each of training data belongs by applying each of the training data to the first model for each correct answer category (step S303)" and [0079]: "Next to step S305, the second learning unit 5 learns the second model by machine learning collectively using all the pieces of training data selected as appropriate training data by the selecting unit 4 and all the pieces of training data selected as inappropriate training data by the selecting unit 4 (step S306)," where Kanno's first model determination of appropriate/inappropriate corresponds to the instant inference result) ... repeatedly (Kanno, Fig. 3, where cycle of path S101-S103 corresponds to the instant repeatedly).
Kida further teaches:
randomly select a third piece of the learning data (Kida, [0021]: "Samples may be selected from a data set in various ways. ... [M]ultiple training data subsets may be created from a larger available data set" and [0022]: "A subset of the data may be used as training data to train a model to classify a sample. The training data is labeled meaning each sample in the training data includes the sample's correct class. Another subset of the data may be used to test the model to estimate the performance of the selected model")
the another piece of the learning data randomly acquired (Kida, [0021]: "Samples may be selected from a data set in various ways. ... In an example, the samples may be selected randomly from the entire data set. Samples sets 406,408, and 410 each illustrate a possible data subset selected by randomly selecting samples. ... As described in greater detail below, multiple training data subsets may be created from a larger available data set").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the Kanno/Kida combination regarding learning of the first and second models with the further teachings of Kida regarding randomly selects another piece of the learning data instead of the learning data not selected by the data selecting unit and the another piece of the learning data randomly acquired.
The motivation to do so would be to facilitate training models according to desired performance characteristics such as accuracy (Kida, [0021]: "These data subsets are used to train, select and estimate performance of various models. Iterating over the multiple data subsets allows a model to be created that has a desired accuracy or sensitivity and complexity of implementation").
Regarding Claim 10, the rejection of Claim 4 is incorporated. The Kanno/Kida combination teaches:
the at least one processor is further configured to perform the learning of the first model and the second model based on one of regression, a decision tree, a neural network, Bayes, clustering, or time series prediction (Kanno, [0026]: "Note that in each of exemplary embodiments described below, a case where a model for determining a category to which given data belongs is learned by deep learning will be exemplified," where Kanno's deep learning corresponds to the instant using ... a neural network model).
Claims 8 and 11-15 are rejected under 35 U.S.C. 103 as being unpatentable over Kanno (US 2021/0004723 A1, hereinafter "Kanno") in view of Metzler, et al. (US 2019/0178643 A1, hereinafter "Metzler").
Regarding Claim 8, the rejection of Claim 1 is incorporated.
Kanno teaches that the learning data may be image data.
Kanno does not explicitly teach that the learning data is at least any one of RGB data, polarization data, multispectral data, or wavelength data of invisible light.
However, Metzler teaches:
the learning data group includes data that is at least one of RGB data, polarization data, multispectral data, or wavelength data of invisible light (Metzler, [0037]: "a set of artificial images (RGB images, intensity images, depth images, etc.) can be generated from the 3d-model of the object by rendering, which rendered images are then used as training data").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Kanno regarding that the learning data may be image data with those of Metzler regarding the learning data is at least any one of RGB data, polarization data, multispectral data, or wavelength data of invisible light.
The motivation to do so would be to facilitate training using a semi-supervised method where RGB data provides supervision data regarding the modeled object (Metzler, [0107]: "such artificially generated training data can be embodied as a series of numerically rendered RGB-images and/or numerically rendered depth images, representing different states of the object and of the objects environment, for machine learning the classifier 30. Preferably, an at least partially supervised machine learning approach can be applied, which is deriving the supervision information from the 3D-model and/or the parameters of the numerical rendering -- like e.g. environmental conditions, orientations of 3D-model, parameters, proportions from the 3D-model").
Regarding Claim 11, the rejection of Claim 1 is incorporated.
Kanno teaches performing learning of a first model based on learning data.
Kanno does not explicitly teach the at least one processor is further configured to generate the learning data group based on a three-dimensional model of an object.
However, Metzler teaches:
the at least one processor is further configured to generate the learning data group based on a three-dimensional model of an object (Metzler, Fig. 9, depicting steps of generating at 52 and learning at 53, where [0114]: "In block 52, a plurality of numerically rendered images of the virtual 3d-model are generated" and [0115]: "Above steps can be done in an office environment ... e.g. also on a general purpose computer or therefore specifically dedicated machine learning computer" and [0069]: "Numerical rendering engines to generate image data 2 from such 3D-models 1 are known in the art of computer graphics and can be used to provide the numerically rendered images according to the invention, in particular so called photo-realistic rendering engines, preferably comprising ray-tracing, can be utilized for this aspect of the invention").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Kanno regarding the information processing unit performing a process including selection of the learning data on a basis of the learning data group and the input processing target data group with those of Metzler regarding a learning data generating unit configured to generate the learning data group on a basis of a three-dimensional model of an object.
The motivation to do so would be to facilitate usage of trained models in surveying scenarios where real-world objects are automatically identified based on virtual models (Metzler, [0028]: "Such a surveying and/or metrology instrument can in particular provide an autonomous or a semi-autonomous measurement mode, which automatically measures geometrical features of a real world object, which is automatically identified by the classifier in the real world pictures, wherein latter is done according to trained meta information derived from the virtual 3d-model").
Regarding Claim 12, the rejection of Claim 11 is incorporated.
Metzler further teaches:
the at least one processor is further configured to generate the learning data group, wherein the learning data group includes data of a rendering result of the object (Metzler, [0107]: "such artificially generated training data can be embodied as a series of numerically rendered RGB-images and/or numerically rendered depth images, representing different states of the object and of the objects environment"), and
the data of the rendering result includes a simulation result of a state of the object (Metzler, [0107]: "such artificially generated training data can be embodied as a series of numerically rendered RGB-images and/or numerically rendered depth images, representing different states of the object and of the objects environment, for machine learning the classifier 30") as a correct answer (Metzler, [0107]: "an at least partially supervised machine learning approach can be applied, which is deriving the supervision information from the 3D-model and/or the parameters of the numerical rendering -- like e.g. environmental conditions, orientations of 3D-model, parameters, proportions from the 3D-model," where Metzler's supervision information corresponds to the instant correct answer).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the Kanno/Metzler combination regarding the learning data generating unit generates the learning data group including the learning data learning data with the further teachings of Metzler regarding including data of a rendering result of the object having a simulation result of a state of the object as a correct answer.
The motivation to do so would be to facilitate automated training of classification models to be used in automated measurement scenarios based on the generated training data (Metzler, [0107]: "FIG. 7, shows the 3D-Model 1 of an object on the left, based on which a large quantity of virtually generated data 2 as input for training for the classifier 30 can be generated, preferably in a substantially automatic process. ... [A]n at least partially supervised machine learning approach can be applied, which is deriving the supervision information from the 3D-model .... According to the invention those meta data comprises at least an information being relevant for an automated measurement of the object or of a geometric feature of the object by a measuring instrument").
Regarding Claim 13, the rejection of Claim 11 is incorporated. The Kanno/Metzler combination teaches:
wherein the at least one processor is further configured to ... perform learning of a second model based on the processing target data as an input and the provisional correct answer data as an output (Kanno, [0042]: "The second learning unit 5 learns the second model by machine learning using the training data selected as appropriate training data by the selecting unit 4 and the training data selected as inappropriate training data by the selecting unit 4 (in other words, using the training data as teacher data). The second learning unit 5 learns the second model regardless of a correct answer category," where Kanno's appropriate/inappropriate training data corresponds to the instant provisional correct answer).
Metzler further teaches:
perform learning of the first model based on the generated learning data (Metzler, Fig. 9, depicting steps of generating at 52 and learning at 53, where [0114]: "In block 52, a plurality of numerically rendered images of the virtual 3d-model are generated" and [0069]: "Numerical rendering engines to generate image data 2 from such 3D-models 1 are known in the art of computer graphics and can be used to provide the numerically rendered images according to the invention, in particular so called photo-realistic rendering engines, preferably comprising ray-tracing, can be utilized for this aspect of the invention").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the Kanno/Metzler combination regarding a learning device comprising an information processing unit and a learning data generating unit wherein the information processing unit performs selection of the learning data with those of Metzler regarding a first learning unit configured to perform learning of a first model by using the generated learning data.
The motivation to do so would be to facilitate automated training of classification models to be used in automated measurement scenarios based on the generated training data (Metzler, [0107]: "FIG. 7, shows the 3D-Model 1 of an object on the left, based on which a large quantity of virtually generated data 2 as input for training for the classifier 30 can be generated, preferably in a substantially automatic process. ... [A]n at least partially supervised machine learning approach can be applied, which is deriving the supervision information from the 3D-model .... According to the invention those meta data comprises at least an information being relevant for an automated measurement of the object or of a geometric feature of the object by a measuring instrument").
Regarding Claim 14, the rejection of Claim 13 is incorporated.
The Kanno/Metzler combination teaches:
provide, as an input, the generated learning data group to the first model to obtain a first inference result (Kanno, [0036]: "for each correct answer category, the selecting unit 4 determines a category to which each of training data belongs by applying each of the training data to the first model");
provide, as an input, the generated learning data to the second model to obtain a second inference result (Kanno, [0042]: "The second learning unit 5 learns the second model by machine learning using the training data selected as appropriate training data by the selecting unit 4 and the training data selected as inappropriate training data by the selecting unit 4 (in other words, using the training data as teacher data)");
compare the first inference result with the second inference result (Kanno, [0044]: "The learning device 100 repeats a process ... until a prescribed condition is satisfied. ... [A]nother example of this prescribed condition is that in a case where training data selected as appropriate training data or inappropriate training data is applied to the second model, a difference between classification of the training data ('appropriate' or 'inappropriate') and a determination result obtained by applying the training data to the second model is equal to or less than a predetermined threshold value"); and
select the at least one second piece of learning data for learning of the inference model, based on the comparison of the first inference result with the second inference result (Kanno, [0059]: "After step S203, by applying each piece of each of training data read from the training data storage unit 1 in step S201 to the second model read in step S203, the first learning unit 2 determines whether or not each of the training data is appropriate as training data used for learning the first model. Then, the first learning unit 2 excludes training data that has been determined to be inappropriate from each piece of training data read from the training data storage unit 1 ( step S204)").
Regarding Claim 15, the rejection of Claim 14 is incorporated. The Kanno/Metzler combination teaches:
wherein the at least one processor is further configured to specify a condition for the learning data ... based on the at least one second piece of learning data for which the second inference result has a difference from the first inference result smaller than a threshold (Kanno, [0062]: "the first learning unit 2 may calculate a numerical value indicating a difference between classification of training data ('appropriate' or 'inappropriate') determined by the selecting unit 4 and a determination result obtained by applying the training data to the second model. Then, in step S103 (see FIG. 3), the first learning unit 2 may determine whether or not a prescribed condition is satisfied depending on whether or not the numerical value indicating the difference is equal to or less than a threshold value").
Metzler further teaches:
specify a condition for the learning data, to be newly generated (Metzler, [0107]: "such artificially generated training data can be embodied ... representing different states of the object and of the objects environment, for machine learning the classifier 30. Preferably, an at least partially supervised machine learning approach can be applied, which is deriving the supervision information from the 3D-model ... like e.g. environmental conditions, ... proportions from the 3D-model... , etc. which can be provided as corresponding meta data ... for the machine learning process" and [0069]: "A computation unit, e.g. comprising digital processor, memory and input/output interfaces, analyzes this digital picture 3 in such a way to identify and or locate previously trained objects within the picture 3. According to the invention, the previously trained objects are at least partially trained on numerically rendered images 2, which are derived from 3D-model data 1 of the objects, e.g. such a CAD-data, pointcloud-models, or other 3D-computer drawings in various data formats" and [0112]: "In block 50, virtual models of real world objects are used, e.g. generated or loaded from a database, in form of 3d-models").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the Kanno/Metzler combination regarding learning data used as an input for inference of the second inference result having a difference from the first inference result smaller than a threshold with the further teachings of Metzler regarding a condition specifying unit configured to specify a condition of the learning data to be newly generated on a basis of the learning data.
The motivation to do so would be to facilitate a training scenario where a generic model can be trained (Metzler, [0087]: "the invention can be described by establishing a synthetic, pre-trained classifier and/or detector, by deriving a plurality of numerical renderings from a 3D-model and feeding those renderings as training resource for a supervised learning of the classifier and/or detector. The renderings can therein comprise at least one object of interest to be trained, preferably embedded in a virtually generated realistic environment, in particular from a plurality of different views and/or different lightning conditions and/or environmental conditions. Thereby a generic classifier and/or detector can be trained on virtual information").
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBERT N DAY whose telephone number is (703)756-1519. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/R.N.D./Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights