Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. JP2019-178001, filed on 09/27/2019.
Response to Arguments
Applicant’s argument filed 04/03/2026 have been fully considered but they are not persuasive.
Applicant’s Argument: On pages 14-22 of Applicant’s response, applicant states that the claimed invention cannot practically be performed in the human mind and provides Example 39 from 2019 PEG as a reference. Applicant argues that training a neural network does not recite a judicial exception and is eligible because it cannot be performed in the human mind.
Examiner’s Response: Applicant’s argument is not persuasive. The argument regarding example 39 is not applicable since applicant’s claim language is not similar to the claimed invention of example 39. The subject matter eligibility guidance Example 39 does not disclose any abstract ideas in the claim. The claim in Example 39 does not recite any mathematical calculations and the claim do not recite a mental process because the steps are not practically performed in the human mind. The claimed invention is different from Example 39 because the claim invention does recite a judicial exception.
The claimed invention does recite mental processes that can be performed in the human mind with the aid of pen and paper. For example, the process of generating, from the first label, a sample fitting a dataset and the step of generating a sample comprising the first feature are both steps of mental process that can be performed in the human mind. In the claimed invention, “sample” and “feature” are broadly recited without further limiting the scope of the definition. A person having ordinary skills in the arts can generate numeric data sample from a label that consists of a first feature. For example, given a chart showing different labelled defects, a person having ordinary skills in the arts can generate descriptions of the defect such as shape, length, size, and location. The shape and location represent different features and the numeric value is the data sample.
The additional elements of training a ML model comprising of a generator to generate synthetic samples is merely to apply the exception using a generic computer component.
Applicant’s Argument: On pages of 23 Applicant’s response, applicant states the claimed invention improves the technological field of high-quality synthesis of rare defect samples through independent parallel training.
Examiner’s Response: Applicant’s argument is not persuasive. An important consideration in determining whether a claim improves technology is the extent to which the claim covers a particular solution to a problem or a particular way to achieve a desired outcome, as opposed to merely claiming the idea of a solution or outcome (see MPEP 2106.05(a)). The amended claims do not provide sufficient details to describe any technological improvement. If the specifications explicitly set forth an improvement but in a conclusory manner (see MPEP 2106.04(d)(1): a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art), the examiner should not determine the claim improves technology.
During examination, the examiner should analyze the "improvements" consideration by evaluating the specification and the claims to ensure that a technical explanation of the asserted improvement is present in the specification, and that the claim reflects the asserted improvement (see MPEP §2106.05(a)). The MPEP (§2106.05(a)(II)) also warns, “it is important to keep in mind that an improvement in the abstract idea itself (e.g. a recited fundamental economic concept) is not an improvement in technology.” Here, the alleged improvement in the form of “synthesizing two samples generated from a trained model” is an improvement to the abstract idea of a mental process that can be performed in the human mind.
Applicant’s Argument: On pages 23-28 of Applicant’s response, applicant states that Yoo fails to disclose multiple generators that are trained to produce outputs with multiple features.
Examiner’s Response: Applicant’s argument is not persuasive. Yoo (Figure 4) shows 2 different generators that are parallel and trained independently. The first generator is trained on data of the first domain and the second generator is trained on data showing the second domain. Yoo (par. 53) teaches multiple generators that are trained on data belonging to different domains. For example, the generator can generate a fake face image of person B given the input of an image of person A. The image of a face of a person contains multiple features shows as hair, nose, eyes, and skin tone. Therefore, the generated output consists of multiple features. Further, Yoo (par. 69) discloses that different generators are trained on data of different domains and the data of the different domains would consist of different features.
Applicant’s Argument: On pages 27-28 of Applicant’s response, applicant states that Lim fails to disclose synthesis of independently generated pseudo samples.
Examiner’s Response: Applicant’s argument is not persuasive. Yoo discloses parallel training of 2 separate generators. Lim (par 28 & Figure 3) discloses that the foreground and background images are created by the generator. Thus, the images are pseudo samples. Lim (par. 33) further discloses combining the generated images together to form the consolidated image.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1:
Subject Matter Eligibility Analysis Step 1:
Claim 1 recites “A data generation system, comprising” and is thus a machine, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
“to adversarially optimize computational parameters of the first generator to challenge a discrimination accuracy of the first discriminator over successive training cycles, thereby iteratively enhancing generation fidelity of the first generator” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“generating a first ” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“generating a second ” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“generating a new sample of the predetermined type of data by synthesizing the generated second pseudo sample with the generated first pseudo sample, the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit a pseudo value of the second feature attribute and the synthesizing preserving an association between the first and second features” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“ performing convolutional addition of the converted second pseudo sample into the first pseudo sample, ” (a mathematical calculation; performing a convolutional operation of multiplication and addition.)
Claim 1 therefore recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
“a learning apparatus; and a data generation apparatus; wherein the learning apparatus comprises a first processor configured with a first program to perform operations comprising” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
"obtaining a plurality of first learning datasets, each of the plurality of first learning datasets comprising a combination of a first sample of a predetermined type of data comprising a first feature and a first label indicating a type of the first feature” (This step is directed to data gathering, which is understood to be insignificant extra solution activity - see MPEP 2106.05(g))
"obtaining a plurality of second learning datasets, each of the plurality of second learning datasets comprising a combination of a second sample of the predetermined type of data comprising a second feature different from the first feature and a second label indicating a type of the second feature” (This step is directed to data gathering, which is understood to be insignificant extra solution activity - see MPEP 2106.05(g))
“training a first learning model comprising a first generator and a first discriminator through machine learning using the obtained plurality of first learning datasets, the training the first learning model through machine learning comprising training the first generator to generate” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“training a second learning model comprising a second generator through machine learning using the obtained plurality of second learning datasets, the training the second learning model through machine learning comprising training the second generator to generate, ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“the data generation apparatus comprises a second processor configured with a second program to perform operations comprising” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating a first pseudo sample a first input value corresponding to the first label to the trained first generator” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating a second pseudo sample ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 1 is directed to the abstract idea.
Subject Matter Eligibility Analysis Step 2B:
“a learning apparatus; and a data generation apparatus; wherein the learning apparatus comprises a first processor configured with a first program to perform operations comprising” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
"obtaining a plurality of first learning datasets, each of the plurality of first learning datasets comprising a combination of a first sample of a predetermined type of data comprising a first feature and a first label indicating a type of the first feature” (This step is directed to transmitting or receiving information, which is understood to be insignificant extra solution activity and well understood, routine and conventional activity of transmitting and receiving data as identified by the court - see MPEP 2106.05(d))
"obtaining a plurality of second learning datasets, each of the plurality of second learning datasets comprising a combination of a second sample of the predetermined type of data comprising a second feature different from the first feature and a second label indicating a type of the second feature” (This step is directed to transmitting or receiving information, which is understood to be insignificant extra solution activity and well understood, routine and conventional activity of transmitting and receiving data as identified by the court - see MPEP 2106.05(d))
“training a first learning model comprising a first generator and a first discriminator through machine learning using the obtained plurality of first learning datasets, the training the first learning model through machine learning comprising training the first generator to generate” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“training a second learning model comprising a second generator through machine learning using the obtained plurality of second learning datasets, the training the second learning model through machine learning comprising training the second generator to generate, ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“the data generation apparatus comprises a second processor configured with a second program to perform operations comprising” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating a first pseudo sample ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating a second pseudo sample ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not recite significantly more than the abstract idea itself as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 1 is subject-matter ineligible.
Regarding Claim 2:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the predetermined type of data comprises a first component and a second component different from the first component, the second component undergoing predetermined estimation” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
“the first feature comprises the first component” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
“the second feature comprises the second component” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
Regarding Claims 3 and 18:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“challenge the discrimination accuracy of the first discriminator, thereby iteratively enhancing generation fidelity of the first generator” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“the first processor is configured to perform operations such that the training the first generator comprises performing adversarial learning by alternately performing training the first discriminator ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“training the first generator to generate the pseudo sample ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claims 4, 19, and 20:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“ discriminate whether an input sample input into the second discriminator is the second sample obtained from a second learning dataset of the plurality of second learning datasets or the pseudo sample generated by the second generator” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“challenge the discrimination accuracy of the second discriminator, thereby iteratively enhancing generation fidelity of the second generator” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the second learning model further comprises a second discriminator” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
“the first processor is configured to perform operations such that the training the second generator comprises performing adversarial learning by alternately performing training the second discriminator ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“training the second generator to generate the pseudo sample ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 5:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“the ” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“generating the ” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“converting the second pseudo sample to allow a value of the attribute of the second feature included comprised in the second pseudo sample to fit the generated pseudo value and synthesizing the second pseudo sample with the first pseudo sample” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“obtaining a plurality of third learning datasets, each of the plurality of third learning datasets comprising a combination of a sample value of an attribute of the second feature, a reflection degree of the second feature, and the second label” (This step is directed to data gathering, which is understood to be insignificant extra solution activity (2106.05(g) in step 2A prong 2) and well understood, routine and conventional activity of transmitting and receiving data as identified by the court (2106.05(d) in step 2B))
“training a third learning model including comprising a third generator through machine learning using the obtained plurality of third learning datasets, the training the third learning model through machine learning comprising training the third generator to ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating the pseudo value” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 6:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
“the ” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“generating the ” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“converting the second pseudo sample to allow a value of the attribute of the second feature included comprised in the second pseudo sample to fit the generated pseudo value and synthesizing the second pseudo sample with the first pseudo sample” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“obtaining a plurality of third learning datasets, each of the plurality of third learning datasets comprising a combination of a sample value of an attribute of the second feature, a reflection degree of the second feature, and the second label” (This step is directed to data gathering, which is understood to be insignificant extra solution activity (2106.05(g) in step 2A prong 2) and well understood, routine and conventional activity of transmitting and receiving data as identified by the court (2106.05(d) in step 2B))
“generating the pseudo value” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“training a third learning model including comprising a third generator through machine learning using the obtained plurality of third learning datasets, the training the third learning model through machine learning comprising training the third generator to the pseudo value” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“the predetermined type of data comprises a first component and a second component different from the first component, the second component undergoing predetermined estimation” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
“the first feature comprises the first component” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
“the second feature comprises the second component” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
“the predetermined estimation comprises detecting the second component” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
“the second processor is configured to perform operations such that the generating the new sample of the predetermined type of data by synthesizing comprises” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 7:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“ discriminate whether an input value input into the third discriminator is the sample value obtained from a third learning dataset of the plurality of third learning datasets or the pseudo value generated by the third generator” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“challenge the discrimination accuracy of the third discriminator, thereby iteratively enhancing generation fidelity of the third generator” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the third learning model further comprises a third discriminator” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
“the first processor is configured to perform operations such that the training the third generator comprises performing adversarial learning by alternately performing training the third discriminator to ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“training the third generator to generate the pseudo value ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 8:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“the ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein each of the plurality of third learning datasets further comprises the first label” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
“the first processor is configured to perform operations such that the training the third learning model through machine learning comprises training the third generator to generate, from the reflection degree, the second label, and the first label, the pseudo value ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 9:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the reflection degree comprises a set of continuous values” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
Regarding Claim 10:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
“estimating, for each of the plurality of third samples provided to the estimator, an estimate value of an input into the first generator trained to generate a pseudo sample corresponding to each of the plurality of third samples, ” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“generating a third ” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“generating the second sample in each of the plurality of second learning datasets by subtracting the third pseudo sample from each of the plurality of third samples” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“the first processor is configured to; perform operations comprising operation as an estimator; and perform operations such that the obtaining the plurality of second learning datasets comprises” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“obtaining a plurality of third samples of the predetermined type of data, each of the plurality of third samples including comprising the first feature and the second feature” (This step is directed to data gathering, which is understood to be insignificant extra solution activity (2106.05(g) in step 2A prong 2) and well understood, routine and conventional activity of transmitting and receiving data as identified by the court (2106.05(d) in step 2B))
“generating a third pseudo sample” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 11:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“obtaining a plurality of fourth learning datasets, each of the plurality of fourth learning datasets comprising a combination of an input sample value to be provided to the trained first generator and a fourth pseudo sample generated by providing the input sample value to the trained first generator” (This step is directed to data gathering, which is understood to be insignificant extra solution activity (2106.05(g) in step 2A prong 2) and well understood, routine and conventional activity of transmitting and receiving data as identified by the court (2106.05(d) in step 2B))
“training the estimator through machine learning using the obtained plurality of fourth learning datasets, the training the estimator through machine learning comprising training the estimator to ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 12:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the predetermined type of data comprises image data comprising a background and a foreground” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
“the first feature comprises the background” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
“the second feature comprises the foreground” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
Regarding Claim 13:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the predetermined type of data comprises image data comprising a product” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
“the first feature comprises a background comprising the product” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
“the second feature comprises a defect of the product” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
Regarding Claim 14:
The claim recites a system that performs the method as described in claim 1. Therefore, claim 14 is rejected for the same reasons as disclosed for claim 1. The limitations for additional elements of claim 14 are analyzed below.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Please see Step 2A Prong 1 analysis of claim 1
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“A learning apparatus, comprising a processor configured with a program to perform operations comprising” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 15:
Subject Matter Eligibility Analysis Step 1:
Claim 15 recites “A data generation apparatus, comprising a processor configured with a program to perform operations comprising” and is thus a machine, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
“generating , from the first label, a the generated pseudo sample configured to adjust generation parameters based on feedback from the first discriminator to adversarially optimize computational parameters of the first generator to challenge a discrimination accuracy of the first discriminator over successive training cycles, thereby iteratively enhancing generation fidelity of the first generator” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“generating a first ” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“generating , from the second label, a ” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“generating a second ” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“generating a new sample of the predetermined type of data by synthesizing the generated second pseudo sample with the generated first pseudo sample, the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit a pseudo value of the second feature attribute and the synthesizing preserving an association between the first and second features” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“ performing convolutional addition of the converted second pseudo sample into the first pseudo sample, ” (a mathematical calculation; performing a convolutional operation of multiplication and addition.)
Claim 15 therefore recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
"generating a first pseudo sample” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
"generating a second pseudo sample” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“training a first generator through machine learning using a plurality of first learning datasets, each of the plurality of first learning datasets including comprising a combination of a first sample of a predetermined type of data including comprising a first feature and a first label indicating a type of the first feature” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating by the trained first generator,
“training a second generator through machine learning using a plurality of second learning datasets, each of the plurality of second learning datasets comprising a combination of a second sample of the predetermined type of data including comprising a second feature different from the first feature and a second label indicating a type of the second feature, the second generator being trained independently of the first generator” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating by the trained second generator” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 15 is directed to the abstract idea.
Subject Matter Eligibility Analysis Step 2B:
"generating a first pseudo sample” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
"generating a second pseudo sample” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“training a first generator through machine learning using a plurality of first learning datasets, each of the plurality of first learning datasets including comprising a combination of a first sample of a predetermined type of data including comprising a first feature and a first label indicating a type of the first featur” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating by the trained first generator, ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“training a second generator through machine learning using a plurality of second learning datasets, each of the plurality of second learning datasets comprising a combination of a second sample of the predetermined type of data including comprising a second feature different from the first feature and a second label indicating a type of the second feature, the second generator being trained independently of the first generator” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating by the trained second generator” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not recite significantly more than the abstract idea itself as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 15 is subject-matter ineligible.
Regarding Claim 16:
Subject Matter Eligibility Analysis Step 1:
Claim 16 recites “A computer implemented data generation method, comprising” and is thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
“generating, to adversarially optimize computational parameters of the first generator to challenge a discrimination accuracy of the first discriminator over successive training cycles, thereby iteratively enhancing generation fidelity of the first generator” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“generating,
“generating, ” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“generating, ” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“generating, the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit a pseudo value of the second feature attribute and the synthesizing preserving an association between the first and second features” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“ performing convolutional addition of the converted second pseudo sample into the first pseudo sample, ” (a mathematical calculation; performing a convolutional operation of multiplication and addition.)
Claim 16 therefore recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
“training, with a computer, a first generator through machine learning using a plurality of first learning datasets, each of the plurality of first learning datasets comprising a combination of a first sample of a predetermined type of data including comprising a first feature and a first label indicating a type of the first feature” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating, with the computer, by the trained first generator, ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating, with the computer, a first pseudo sample” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“training, with a computer, a second generator through machine learning using a plurality of second learning datasets, the second generator begin trained independently of the first generator, each of the plurality of second learning datasets comprising a combination of a second sample of the predetermined type of data comprising a second feature different from the first feature and a second label indicating a type of the second feature” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating, with the computer, by the trained second generator, ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating, with the computer, a second pseudo sample” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 16 is directed to the abstract idea.
Subject Matter Eligibility Analysis Step 2B:
“training, with a computer, a first generator through machine learning using a plurality of first learning datasets, each of the plurality of first learning datasets comprising a combination of a first sample of a predetermined type of data including comprising a first feature and a first label indicating a type of the first feature” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating, with the computer, by the trained first generator, ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating, with the computer, a first pseudo sample” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“training, with a computer, a second generator through machine learning using a plurality of second learning datasets, the second generator begin trained independently of the first generator, each of the plurality of second learning datasets comprising a combination of a second sample of the predetermined type of data comprising a second feature different from the first feature and a second label indicating a type of the second feature” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating, with the computer, by the trained second generator, ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating, with the computer, a second pseudo sample” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not recite significantly more than the abstract idea itself as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 16 is subject-matter ineligible.
Regarding Claim 17:
The claim recites an article of manufacture that performs the method as described in claim 16. Therefore, claim 17 is rejected for the same reasons as disclosed for claim 16. The limitations for additional elements of claim 17 are analyzed below.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Please see Step 2A Prong 1 analysis of claim 16
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“A non-transitory computer-readable storage medium storing data generation program, which when read and executed, causes a computer to perform operations comprising” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Yoo (US20200151481A1) in view of Lim (US20180293734A1), and Shechtman (US20190251401A1).
Regarding claim 1, Yoo teaches:
“A data generation system, comprising; a learning apparatus; and a data generation apparatus; wherein the learning apparatus comprises a first processor configured with a first program to perform operations comprising” ([abstract, 0097-0100, Figure 8], The reference describes a system that has multiple generators that are trained to receive input data and output synthetic data. The system consists of a training apparatus (learning apparatus) and a data classification apparatus (data generation apparatus). The training apparatus consist of a processor and memory to execute instructions to train the classifier.)
“obtaining a plurality of first learning datasets, each of the plurality of first learning datasets comprising a combination of a first sample of a predetermined type of data ” ([0013, 0050, 0052-0053, 0056], The query input data may be image data, audio data, or fingerprint data (type of data). One or more sensors may capture images (first sample) of a user’s face to be used in user verification. If the input data is an image of a user, the classifier may identify whether the user is “person A” or person “B”. The reference does not explicitly discuss the analysis of the features of the input data, but it is implied that the system identifies whether an image shows “person A” by analyzing the facial features of the person for user identification.)
“obtaining a plurality of second learning datasets, each of the plurality of second learning datasets comprising a combination of a second sample of the predetermined type of data ” ([0013, 0050, 0052-0053, 0057], A new set of training data is received by the system that contains data of a second domain that is different from the first training data. The third training data is categorized as a second domain, which may be images that show “person B”. Third training data contains different features from the first training data because each training data contains data of a different domain. The reference does not explicitly discuss the analysis of the features of the input data and it is implied that the system identifies whether an image shows “person B” of a second domain by analyzing the facial features of the person for user identification.)
“training a first learning model comprising a first generator and a first discriminator through machine learning using the obtained plurality of first learning datasets, the training the first learning model through machine learning comprising training the first generator to generate, to adversarially optimize computational parameters of the first generator to challenge a discrimination accuracy of the first discriminator over successive training cycles, thereby iteratively enhancing generation fidelity of the first generator” ([0053, 0061-0063, 0072], The first generator comprising of a neural network (first learning model) is trained on the first training data that belong to the first domain. The first generator trained on the first training data is used to generate a second training data (pseudo sample) of a second domain. The generator and discriminator operate as adversarial neural network and the performance is enhanced during the training process. During training, the generator generates fake data closer to genuine data and the discriminator is configured to distinguish genuine data from fake data.)
“training a second learning model comprising a second generator through machine learning using the obtained plurality of second learning datasets, the training the second learning model through machine learning comprising training the second generator to generate, ” ([0053, 0061-0063, 0067, Figure 4], The second generator comprising of a neural network (second learning model) is trained on the third training data that belong to the second domain. The second generator trained on the third training data is used to generate a fourth training data (pseudo sample) of a first domain. Domain may be defined as a category consisting of “faces” and the generated sample fits into the fourth training data because both samples describe “faces”. Figure 4 shows the first and second generator being trained separately.)
“the data generation apparatus comprises a second processor configured with a second program to perform operations comprising” ([0099-0100, Figure 9], The data classification apparatus may generate query data of a domain different from a domain including query input data. The data classification apparatus consists of a processor and memory with instructions to perform the operations.)
“generating a first pseudo sample ” ([0053, 0056, Figure 2], A first generator outputs a second training data (first pseudo sample) based on the first input training data. It is implied that the neural network is trained on the individual features of the first input training data to generate new synthetic data that is different from the input data. The generator may generate facial features to describe a person’s face.)
“generating a second pseudo sample ” ([0053, 0057], A second generator outputs a fourth training data (second pseudo sample) based on the third input training data (second input value). It is implied that the neural network is trained on the individual features of the third input training data to generate new synthetic data that is different from the input data. The generator may generate facial features to describe a person’s face.)
“generating a new sample of the predetermined type of data by ” ([0053, 0058, 0080, 0083-0085], The first and second generators generate training data of different domains. The data classification apparatus may be trained on the generated samples from the first and second generator to determine which domain an input data belongs to. The data classification apparatus may receive query input data and may generate first query data (new sample) of the first domain when the query input data is determined to belong to the first domain. A first domain may be “person A” and a facial image (predetermined type of data) may be generated.)
Yoo does not explicitly disclose an implementation of “data comprising a first feature and a first label indicating a type of the first feature”, “data comprising a second feature different from the first feature and a second label indicating a type of the second feature”, “training the first generator to generate, from the first label, a pseudo sample”, “training the second generator to generate, from the second label, a pseudo sample”, “generating a first pseudo sample comprising the first feature by providing a first input value corresponding to the first label”, “generating a second pseudo sample comprising the second feature by providing a second input value corresponding to the second label”, and “generating a new sample of the predetermined type of data by synthesizing the generated second pseudo sample with the generated first pseudo sample, the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit a pseudo value of the second feature attribute and performing convolutional addition of the converted second pseudo sample into the first pseudo sample, the synthesizing preserving an association between the first and second features”. However, Lim discloses in the same field of endeavor:
“obtaining a plurality of first learning datasets, each of the plurality of first learning datasets comprising a combination of a first sample of a predetermined type of data comprising a first feature and a first label indicating a type of the first feature” ([0020-0023, 0028-0029, Figure 3], Training images may contain labels that describe the pixels in the image, such as a label for a crack in a turbine blade or a label for the turbine blade. The spalling 302 and 304 (first feature and label) are shown in training image 308 (learning dataset) as the foreground. In some embodiments, the anomalies of the spalling in the images may be labeled. The training images may be different from each other and can represent different types of spalling and different thermal barrier coating.
“obtaining a plurality of second learning datasets, each of the plurality of second learning datasets comprising a combination of a second sample of the predetermined type of data comprising a second feature different from the first feature and a second label indicating a type of the second feature” ([0020-0023, 0028-0029, Figure 3], In some embodiments, the anomalies in the images may be labeled. The thermal barrier coating 300 (second feature and label) are shown in training image 308 (learning dataset) as the background. The training images may be different from each other and can represent different types of spalling and different thermal barrier coating.
“... training the ” ([0022, 0029], The generator may be trained using labeled images of one or more anomalies. The generator generates anomalies based on pixel characteristics representing the defects in the training images.)
“... training the ” ([0022, 0029], The generator may be trained using labeled images of one or more anomalies. The generator generates undamaged coatings based on the pixel characteristics representing the coatings in the training images.)
“generating a first pseudo sample comprising the first feature by providing a first input value corresponding to the first label ...” ([0022, 0029], The generator may be trained using labeled images of one or more anomalies and the training involves examining the pixel characteristics such as pixel location, and pixel intensity (input value). The generator generates anomalies based on pixel characteristics representing the defects in the training images. The generator generates foreground images representing the anomalies and background images representing the coating.)
“generating a second pseudo sample comprising the second feature by providing a second input value corresponding to the second label ...” ([0022, 0029], The generator may be trained using labeled images of one or more anomalies and the training involves examining the pixel characteristics such as pixel location, and pixel intensity (input value). The generator generates anomalies based on pixel characteristics representing the defects in the training images. The generator generates foreground images representing the anomalies and background images representing the coating.)
“generating a new sample of the predetermined type of data by synthesizing the generated second pseudo sample with the generated first pseudo sample, the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit the synthesizing preserving an association between the first and second features” ([0029, 0032-0033], The generated foreground images (first pseudo sample) and generated background images (second pseudo sample) are combined (synthesizing) by a generator sub-network to create a consolidated images (new sample). The foreground image depicts generator-created anomalies and may have different shape and size. The foreground image is combined with the background image by overlaying the two images and adding the pixel from the foreground image into the background image.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “data comprising a first feature and a first label indicating a type of the first feature”, “data comprising a second feature different from the first feature and a second label indicating a type of the second feature”, “training the first generator to generate, from the first label, a pseudo sample”, “training the second generator to generate, from the second label, a pseudo sample”, “generating a first pseudo sample comprising the first feature by providing a first input value corresponding to the first label”, “generating a second pseudo sample comprising the second feature by providing a second input value corresponding to the second label”, and “generating a new sample of the predetermined type of data by synthesizing the generated second pseudo sample with the generated first pseudo sample, the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit a pseudo value of the second feature attribute and performing convolutional addition of the converted second pseudo sample into the first pseudo sample, the synthesizing preserving an association between the first and second features” from Lim into the teaching of Yoo. Doing so can generate additional training data for the system that combines different size and shape of foreground images with different background objects for anomaly detection (Lim, abstract; par. 14).
Yoo in view of Lim does not explicitly disclose an implementation of “the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit a pseudo value of the second feature attribute and performing convolutional addition of the converted second pseudo sample into the first pseudo sample”. However, Shechtman discloses in the same field of endeavor:
“... the synthesizing comprising converting the second ...” ([0058-0059, 0063-0067], The geometric prediction neural network consists of multiple geometric prediction generators to generate warp parameters. A foreground image and a background image are provided to the generator to determine a predicted warp parameter to transform the foreground object into a more realistic object when combined with the background image. The warped foreground is combined with the background to create a composite image. The compositing process is represented by Equation 1, which involves an elemental multiplication and a summation operation of the images to create the composite image.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit a pseudo value of the second feature attribute and performing convolutional addition of the converted second pseudo sample into the first pseudo sample” from Shechtman into the teaching of Yoo in view of Lim. Doing so can improve the performance of a GAN system to generate realistic composite images by implementing a training process to learn warp parameters that provides geometric alignment of foreground objects with respect to a background image (Shechtman, abstract).
Regarding claim 2, Yoo teaches:
“wherein the predetermined type of data comprises a first component and a second component different from the first component, and the second component undergoing predetermined estimation” ([0050, 0053, 0091], The system may receive image data (predetermined type of data) as the input. The object in the input data may represent different domains. The first input image may represent domain A (first component) and the second input image may represent domain B (second component). Each image, including the images representing the domain B is processed by a classifier to predict the probability (predetermined estimation) that the image belongs to the specific domain.)
“the first feature comprises the first component” ([0053], The first set of training data may be images of person A. The face (first feature) can be analyzed by the system to correctly identify the image to contain person A (first component).)
“the second feature comprises the second component” ([0053], The second set of training data may be images of person B. The face (second feature) can be analyzed by the system to correctly identify the image to contain person B (second component).)
Regarding claims 3 and 18, Yoo teaches:
“the first processor is configured to perform operations such that the training the first generator comprises performing adversarial learning alternately performing training the first discriminator to discriminate whether an input sample input into the first discriminator is the first sample obtained from a first learning dataset of the plurality of first learning datasets or the pseudo sample generated by the first generator” ([0061-0062, 0098, Figure 3], The first generator and the first discriminator is trained on the first training data and fourth training data of the first domain. The first training data is input data and the fourth training data is generated data. The discriminator determines whether the input data is genuine or fake data.)
“training the first generator to generate the pseudo sample to challenge the discrimination accuracy of the first discriminator, thereby iteratively enhancing generation fidelity of the first generator” ([0061-0063], The first generator may have its parameters adjusted based on a cost function to increase the performance. The first discriminator may be trained to improve the performance in distinguishing genuine and fake data. The cost function sets a threshold limit for the performance criteria that the discriminator needs to meet. It is inherent for a GAN system that the generator’s objective is to fool the discriminator into classifying its generated data as real.)
Regarding claims 4, 19, and 20, Yoo teaches:
“wherein the second learning model further comprises a second discriminator” ([0061-0062, Figure 4], A second discriminator is trained with the same neural network and training data as the second generator shown in Figure 4.)
“the first processor is configured to perform operations such that the training the second generator comprises performing adversarial learning alternately performing training the second discriminator to discriminate whether an input sample input into the second discriminator is the second sample obtained from a second learning dataset of the plurality of second learning datasets or the pseudo sample generated by the second generator” ([0061-0062, 0098, Figure 3], The second generator and the second discriminator is trained on the second training data and third training data of the second domain. The third training data is input data containing genuine data and the second training data is generated data. The discriminator determines whether the input data is genuine or fake data.)
“training the second generator to generate the pseudo sample to challenge the discrimination accuracy of the first discriminator, thereby iteratively enhancing generation fidelity of the first generator” ([0061-0063], The second generator may have its parameters adjusted based on a cost function to increase the performance. The second discriminator may be trained to improve the performance in distinguishing genuine and fake data. The cost function sets a threshold limit for the performance criteria that the discriminator needs to meet. It is inherent for a GAN system that the generator’s objective is to fool the discriminator into classifying its generated data as real.)
Regarding claim 5, Yoo teaches:
“obtaining a plurality of third learning datasets, each of the plurality of third learning datasets comprising a combination of a sample value of an attribute of the second feature, ” ([0053, 0074, Figure 4], The training apparatus may receive a fifth training dataset (third learning datasets). The fifth training dataset is generated by the second generator that is trained with training data of the second domain. The second domain may be “person B” and the generator generates a face image. It is inherent that the face image will contain pixels consisting of characteristics such as pixel color and intensity of various facial features.)
“training a third learning model including comprising a third generator through machine learning using the obtained plurality of third learning datasets, the training the third learning model through machine learning comprising training the third generator to generate, ” ([0050, 0055, 0063, 0077, Figure 3], The reference explicitly teaches an embodiment where the data classification system is trained on 2 domains as shown in Figure 3 and implies that the system can be modified to include at least 3 domains. In order to include the addition of the third domain, it would have been obvious to modify Figure 3 to include the steps of receiving a new training data of third domain as input, training a neural network to generate synthetic samples, and providing a third generator with the input to generate new synthetic data. The different generators can translate input data into generated data of a different domain. The training of the third generator consists of translating input data into the first and second domain, which would involve generating attributes of features of the second domain.)
“the second processor is configured to perform operations such that” ([0099-0100, Figure 9], The data classification apparatus may generate query data of a domain different from a domain including query input data. The data classification apparatus consist of a processor and memory with instructions to perform the operations.)
Yoo does not explicitly disclose an implementation of “from the reflection degree and the second label, a pseudo value of the attribute of the second feature fitting the sample value for each of the plurality of third learning datasets”, “generating a pseudo value of the attribute of the second feature by providing, to the trained third generator, a second input value corresponding to the second label and a third input value corresponding to the reflection degree”, and “converting the second pseudo sample to allow a value of the attribute of the second feature included comprised in the second pseudo sample to fit the generated pseudo value and synthesizing the second pseudo sample with the first pseudo sample”. However, Lim discloses in the same field of endeavor of training a generator to produce various foregrounds and backgrounds of training images:
“obtaining a plurality of third learning datasets, each of the plurality of third learning datasets comprising a combination of a sample value of an attribute of the second feature, a ” ([0022, 0028-0029, 0044, Figure 3], The generator sub-network produces a set of new training images of the foreground that consist of anomalies of different shape and sizes (attribute of the second feature). In Figure 3, the system generates a foreground image that has anomaly 315 and 316, which are different from the initial anomalies 302 and 304. The reference does not explicitly teach a rotation or reflection degree of the anomaly in the images. The reference teaches examining pixel characteristics of the image such as location and generating different pixel characteristics.)
“(translation) and the second label, the pseudo value of the attribute of the second feature fitting the sample value for each of the plurality of third learning datasets” ([0022, 0028-0029, 0044, Figure 3], From Figure 3, the anomalies generated by the generator depicts a transformation that change the size and shape of the anomaly. It is implied that that generator was trained to create new attribute values for the labeled anomalies to generate different variations of the anomaly.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “from the reflection degree and the second label, a pseudo value of the attribute of the second feature fitting the sample value for each of the plurality of third learning datasets” from Lim into the teaching of Yoo. Doing so can generate additional training data for the system that combines different size and shape of foreground images with different background objects for anomaly detection (Lim, abstract; par. 14).
Yoo in view of Lim does not explicitly disclose an implementation of “generating a pseudo value of the attribute of the second feature by providing, to the trained third generator, a second input value corresponding to the second label and a third input value corresponding to the reflection degree”, and “converting the second pseudo sample to allow a value of the attribute of the second feature included comprised in the second pseudo sample to fit the generated pseudo value and synthesizing the second pseudo sample with the first pseudo sample”. However, Shechtman discloses in the same field of endeavor of training a generative adversarial network to produce variation of background and foreground images to generate composite images:
“the generating the new sample of the predetermined type of data by synthesizing comprises generating the pseudo value of the attribute of the second feature by providing, to the trained third generator, a second input value corresponding to the second label and a third input value corresponding to the reflection degree” ([0066, 0074, 0148, Figure 2B], The generator is trained to produce new updated warp parameters (pseudo value of the attribute of the second feature) for the objects in the foreground images. The determination of the warp parameters is based on a homograph transformation matrix. By using a homograph transformation matrix, objects can be translated, rotated, and scaled to different positions. The generator receives the initial warp parameters and provide updated warp parameters.)
“converting the second pseudo sample to allow a value of the attribute of the second feature included comprised in the second pseudo sample to fit the generated pseudo value and synthesizing the second pseudo sample with the first pseudo sample” ([0066-0067, Figure 2B], In Figure 2B, the second warp parameters 220b (generated pseudo value) is the updated warp parameters and it is used to transform the object in the foreground image 212b (second pseudo sample). The updated foreground image is combined with the background image 206 (first pseudo sample) to create the updated composite image.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “generating a pseudo value of the attribute of the second feature by providing, to the trained third generator, a second input value corresponding to the second label and a third input value corresponding to the reflection degree”, and “converting the second pseudo sample to allow a value of the attribute of the second feature included comprised in the second pseudo sample to fit the generated pseudo value and synthesizing the second pseudo sample with the first pseudo sample” from Shechtman into the teaching of Yoo in view of Lim. Doing so can generate additional training data that consist of geometric corrections of foreground objects in composite images generated by the system (Shechtman, abstract).
Regarding claim 6, it is a dependent claim of claim 1 and it recites the same limitations as claim 5. Therefore, claim 6 is rejected under the same reasons mention for claim 5. The additional elements of claim 6 is addressed below by using Yoo in view of Lim and Shechtman:
“the predetermined type of data comprises a first component and a second component different from the first component, the second component undergoing predetermined estimation” ([Yoo, 0050, 0053, 0091], The system may receive image data (predetermined type of data) as the input. The object in the input data may represent different domains. The first input image may represent domain A (first component) and the second input image may represent domain B (second component). Each image, including the images representing the domain B is processed by a classifier to predict the probability (predetermined estimation) that the image belongs to the specific domain.)
“the first feature comprises the first component” ([Yoo, 0053], The first set of training data may be images of person A. The face (first feature) can be analyzed by the system to correctly identify the image to contain person A (first component).)
“the second feature comprises the second component” ([Yoo, 0053], The second set of training data may be images of person B. The face (second feature) can be analyzed by the system to correctly identify the image to contain person B (second component).))
“the predetermined estimation comprises detecting the second component” ([Yoo, 0053], The classification system determines which category the object in an image belongs to during the recognition process.)
“the synthesizing comprises generating the pseudo value of the attribute of the second feature by providing, to the trained third generator, a second input value corresponding to the second label and a third input value corresponding to the reflection degree and provided in accordance with a limit of the detection” ([Shechtman, 0106], This claim limitation has already been addressed in claim 5. Claim 6 provides additional elements shown in bold. Shechtman teaches that objects may appear in foreground images that limit the detection of the selected object to warp.)
Regarding claim 7, Yoo in view of Lim teaches:
“wherein the third learning model further comprises a third discriminator, and the first processor is configured to perform operations such that” ([Yoo, 0050, 0055, 0077, 0098, Figure 3], The Yoo reference explicitly teaches an embodiment where the data classification system is trained on 2 domains as shown in Figure 3 and implies that the system can be modified to include at least 3 domains. In order to include the addition of the third domain, it would have been obvious to modify Figure 3 to include the step of training a third generator and discriminator to process the training data from the third domain.)
Yoo in view of Lim does not explicitly disclose an implementation of “the training the third generator comprises performing adversarial learning by alternately performing training the third discriminator to discriminate whether an input value input into the third discriminator is the sample value obtained from a third learning dataset of the plurality of third learning datasets or the pseudo value generated by the third generator”, and “training the third generator to generate the pseudo value of the attribute of the second feature to challenge the discrimination accuracy of the third discriminator, thereby iteratively enhancing generation fidelity of the third generator”. However, Shechtman discloses in the same field of endeavor:
“the training the third generator comprises performing adversarial learning by alternately performing training the third discriminator to discriminate whether an input value input into the third discriminator is the sample value obtained from a third learning dataset of the plurality of third learning datasets or the pseudo value generated by the third generator” ([0074-0075, Figure 2C, Figure 8A], The discriminator is trained to distinguish whether the object in the foreground image modified by the warp parameters is determined to by real or not. Figure 8A shows the iteration of the system to determine whether new object with the updated parameters looks realistic.)
“training the third generator to generate the pseudo value of the attribute of the second feature to challenge the discrimination accuracy of the third discriminator, thereby iteratively enhancing generation fidelity of the third generator” ([0052, 0081], The system is trained to minimize the error loss to be able to generate more realistic composite images using the generated warp parameters. The objective of the geometric prediction neural network is to configure the composite image to fool the discriminator in classifying the composite image as a real image.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “the training the third generator comprises performing adversarial learning by alternately performing training the third discriminator to discriminate whether an input value input into the third discriminator is the sample value obtained from a third learning dataset of the plurality of third learning datasets or the pseudo value generated by the third generator”, and “training the third generator to generate the pseudo value of the attribute of the second feature to challenge the discrimination accuracy of the third discriminator, thereby iteratively enhancing generation fidelity of the third generator” from Shechtman into the teaching of Yoo in view of Lim. Doing so can generate additional training data that consist of geometric corrections of foreground objects in composite images generated by the system (Shechtman, abstract).
Regarding claim 8, Yoo does not explicitly disclose an implementation of “wherein each of the plurality of third learning datasets further comprises the first label”, and “the first processor is configured to perform operations such that the training the third learning model through machine learning comprises training the third generator to generate, from the reflection degree, the second label, and the first label, the pseudo value of the attribute of the second feature fitting the sample value”. However, Lim discloses in the same field of endeavor:
“wherein each of the plurality of third learning datasets further comprises the first label” ([0041], The new training data are generated by the generator as consolidated images that have the defects labelled.)
“the first processor is configured to perform operations such that the training the third learning model through machine learning comprises training the third generator to generate, from the the ” ([0018, 0020, 0022], The input image may be label to identify the defect and the sample. The generator produces different defects in the foreground image.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “wherein each of the plurality of third learning datasets further comprises the first label”, and “the first processor is configured to perform operations such that the training the third leaning model through machine learning comprises training the third generator to generate, from the reflection degree, the second label, and the first label, a pseudo value of the attribute of the second feature fitting the sample value” from Lim into the teaching of Yoo. Doing so can generate additional training data for the system that combines different size and shape of foreground images with different background objects for anomaly detection (Lim, abstract; par. 14).
Yoo in view of Lim does not explicitly disclose an implementation of “the reflection degree” and “a pseudo value”. However, Shechtman discloses in the same field of endeavor of training a generative adversarial network to produce variation of background and foreground images to generate composite images:
“the first processor is configured to perform operations such that the training the third leaning model through machine learning comprises training the third generator to generate, from the reflection degree, the second label, and the first label, the pseudo value of the attribute of the second feature fitting the sample value” ([0066, 0074, 0148, 0151, Figure 2B], The generator is trained to produce new updated warp parameters (pseudo value of the attribute of the second feature) for the objects in the foreground images. The determination of the warp parameters is based on a homograph transformation matrix. By using a homograph transformation matrix, objects can be translated, rotated, and scaled to different positions. The generator receives the initial warp parameters and provide updated warp parameters.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “the reflection degree” and “a pseudo value” from Shechtman into the teaching of Yoo in view of Lim. Doing so can generate additional training data that consist of geometric corrections of foreground objects in composite images generated by the system (Shechtman, abstract).
Regarding claim 9, Yoo in view of Lim does not explicitly disclose an implementation of “the reflection degree comprises a set of continuous values”. However, Shechtman discloses in the same field of endeavor:
“the reflection degree comprises a set of continuous values” ([0042], The warp parameters define how an object can be translated, rotated, and scaled within the boundary of a background image. A numerical value is calculated and stored in the homograph transformation matrix. An object in a foreground image can be rotated to any space within the boundary of the background image and the degree of rotation can be any value between a range of values.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “the reflection degree comprises a set of continuous values” from Shechtman into the teaching of Yoo in view of Lim. Doing so can generate additional training data that consist of geometric corrections of foreground objects in composite images generated by the system (Shechtman, abstract).
Regarding claim 10, Yoo teaches:
“the first processor is configured to; perform operations comprising operation as an estimator; and perform operations such that the obtaining the plurality of second learning datasets comprises obtaining a plurality of third samples of the predetermined type of data, each of the plurality of third samples including comprising the first feature and the second feature” ([0053, 0074, 0098], The training apparatus receives a sixth training dataset that contains images of the second domain. In one embodiment, the training data may be images of people and the second domain refers to images of person “B”. The different training data are images that show person “B” used in user verification system. It is implied that the training images of person “B” contains unique facial features to identify the user.)
“estimating for each of the plurality of third samples provided to the estimator, an estimate value of an input into the first generator trained to generate a pseudo sample corresponding to each of the plurality of third samples, the estimator being trained to estimate, from a pseudo sample generated by the trained first generator, the input provided to the first generator to generate the pseudo sample” ([0072-0074], The first generator is trained on the original input and the sixth training dataset based on an expectation value that is minimized during the training process for the generator. )
“generating a third pseudo sample for each of the plurality of third samples by providing the estimated estimate value to the trained first generator” ([0072], A first generator is trained on the expectation value (estimate value) and once the generator is trained, it can produce training data in the first domain.)
Yoo does not explicitly disclose an implementation of “generating the second sample in each of the plurality of second learning datasets by subtracting the third pseudo sample from each of the plurality of third samples”. However, Lim discloses in the same field of endeavor:
“generating the second sample in each of the plurality of second learning datasets by subtracting the third pseudo sample from each of the plurality of third samples” ([0051], The generator can generate different anomalies (second sample) without the generated background (third pseudo sample) as foreground images. The image may show one or more objects and the generator may generate an image with at least one of the objects only.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “generating the second sample in each of the plurality of second learning datasets by subtracting the third pseudo sample from each of the plurality of third samples” from Lim into the teaching of Yoo. Doing so can generate additional training data for the system that combines different size and shape of foreground images with different background objects for anomaly detection (Lim, abstract; par. 14).
Regarding claim 11, Yoo teaches:
“obtaining a plurality of fourth learning datasets, each of the plurality of fourth learning datasets comprising a combination of an input sample value to be provided to the trained first generator and a fourth pseudo sample generated by providing the input sample value to the trained first generator” ([0074, Figure 3], The trained first generator generates a sixth training data (fourth learning datasets) by receiving the fourth training data (input sample value) as input to generate the sixth training data.)
“training the estimator through machine learning using the obtained plurality of fourth learning datasets, the training the estimator through machine learning comprising training the estimator to estimate, from the fourth pseudo sample, an input provided to the trained first generator to obtain an estimate value fitting the sample value for each of the plurality of fourth learning datasets” ([0072-0074], The first generator is trained on the original input and the sixth training dataset based on an expectation value that is minimized during the training process for the generator. )
Regarding claim 12, Yoo does not explicitly disclose an implementation of “wherein the predetermined type of data comprises image data comprising a background and a foreground; the first feature comprises the background; the second feature comprises the foreground”. However, Lim discloses in the same field of endeavor:
“wherein the predetermined type of data comprises image data comprising a background and a foreground” ([0028], The training images consist of a foreground that shows the anomaly and a background that shows the equipment.)
“the first feature comprises the background” ([0025-0026], The background may show a thermal barrier coating.)
“the second feature comprises the foreground” ([0023], The foreground may show a spalling.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “wherein the predetermined type of data comprises image data comprising a background and a foreground; the first feature comprises the background; the second feature comprises the foreground” from Lim into the teaching of Yoo. Doing so can generate additional training data for the system that combines different size and shape of foreground images with different background objects for anomaly detection (Lim, abstract; par. 14).
Regarding claim 13, Yoo does not explicitly disclose an implementation of “wherein the predetermined type of data comprises image data comprising a product; the first feature comprises a background comprising the product; the second feature comprises a defect of the product”. However, Lim discloses in the same field of endeavor:
“wherein the predetermined type of data comprises image data comprising a product” ([0018], The system detects defects on turbine parts.)
“the first feature comprises a background comprising the product” ([0025-0026], The background may show a thermal barrier coating of a turbine component.)
“the second feature comprises a defect of the product” ([0023], The foreground may show a spalling on the surface of the turbine component.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “wherein the predetermined type of data comprises image data comprising a background and a foreground; the first feature comprises the background; the second feature comprises the foreground” from Lim into the teaching of Yoo. Doing so can generate additional training data for the system that combines different size and shape of foreground images with different background objects for anomaly detection (Lim, abstract; par. 14).
Regarding claim 14:
Claim 14 recites a system (“A learning apparatus, comprising a processor configured with a program to perform operations comprising”) that performs the same process as described in Claim 1. Therefore claim 14 is rejected under the same reasons mention for claim 1. The additional elements of claim 14 is addressed below:
“A learning apparatus, comprising a processor configured with a program to perform operations comprising” ([0097, Figure 8], The training apparatus is shown in Figure 8.)
Regarding claim 15, Yoo teaches:
“A data generation apparatus, comprising a processor configured with a program to perform operations comprising” (abstract, The reference describes a system that has multiple generators that are trained to receive input data and output synthetic data.)
“training a first generator through machine learning using a plurality of first learning datasets, each of the plurality of first learning datasets including comprising a combination of a first sample of a predetermined type of data including ” ([0053, 0056, 0062, Figure 3], A first generator is trained with multiple sets of training data through different iterations. The training data may consist of images of a first domain. The training images may be used for user verification in which the images show person “A” with unique facial features. The reference does not explicitly discuss the analysis of the features of the input data, but it is implied that the system identifies whether an image shows “person A” by analyzing the facial features of the person for user identification.)
“generating by the trained first generator, to adversarially optimize computational parameters of the first generator to challenge a discrimination accuracy of the first discriminator over successive training cycles, thereby iteratively enhancing generation fidelity of the first generator” ([0053, 0056, 0061-0063, 0072, Figure 2], A first generator outputs a second training data (pseudo sample) based on the first input training data. The generator adjusts parameters based on a cost function and the generator is trained to minimize the expectation value, which is based on the discriminator determining the generated sample as fake data. The generator and discriminator operate as adversarial neural network and the performance is enhanced during the training process. During training, the generator generates fake data closer to genuine data and the discriminator is configured to distinguish genuine data from fake data.)
“generating a first pseudo sample ” ([0053, 0056, Figure 2], A first generator outputs a second training data (first pseudo sample) based on the first input training data of the first domain. It is implied that the neural network is trained on the individual features of the first input training data to generate new synthetic data that is different from the input data. The generator may generate facial features to describe a person’s face.)
“training a second generator through machine learning using a plurality of second learning datasets, each of the plurality of second learning datasets comprising a combination of a second sample of the predetermined type of data including ” ([0053, 0057, 0062, Figure 4], A second generator is trained with multiple sets of training data through different iterations. The training data may consist of images of a second domain. The training images may be used for user verification in which the images show person “B” with unique facial features. Domain may be defined as a category consisting of “faces” and the generated sample fits into the fourth training data because both samples describe “faces”. Figure 4 shows the first and second generator being trained separately.)
“generating by the trained second generator, ” ([0057, Figure 2], A second generator outputs a fourth training data (pseudo sample fitting the second sample) based on the third input training data (second learning datasets) of the second domain.)
“generating a second pseudo sample ” ([0053, 0057], A second generator outputs a fourth training data (second pseudo sample) based on the third input training data (second input value) of the second domain. It is implied that the neural network is trained on the individual features of the third input training data to generate new synthetic data that is different from the input data. The generator may generate facial features to describe a person’s face.)
“generating a new sample of the predetermined type of data by ” ([0053, 0058, 0080, 0083-0085], The first and second generators generate training data of different domains. The data classification apparatus may be trained on the generated samples from the first and second generator to determine which domain an input data belongs to. The data classification apparatus may receive query input data and may generate first query data (new sample) of the first domain when the query input data is determined to belong to the first domain. A first domain may be “person A” and a facial image (predetermined type of data) may be generated.)
Yoo does not explicitly disclose an implementation of “data comprising a first feature and a first label indicating a type of the first feature”, “generating by the trained first generator, from the first label, a pseudo sample”, “generating a first pseudo sample comprising the first feature by providing a first input value corresponding to the first label”, “data including comprising a second feature different from the first feature and a second label indicating a type of the second feature”, “generating by the trained second generator, from the second label, a pseudo sample”, “generating a new sample of the predetermined type of data by synthesizing the generated second pseudo sample with the generated first pseudo sample, the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit a pseudo value of the second feature attribute and performing convolutional addition of the converted second pseudo sample into the first pseudo sample, the synthesizing preserving an association between the first and second features”. However, Lim discloses in the same field of endeavor:
“... data comprising a first feature and a first label indicating a type of the first feature” ([0020-0023, 0028-0029, Figure 3], Training images may contain labels that describe the pixels in the image, such as a label for a crack in a turbine blade or a label for the turbine blade. The spalling 302 and 304 (first feature and label) are shown in training image 308 (learning dataset) as the foreground. In some embodiments, the anomalies of the spalling in the images may be labeled. The training images may be different from each other and can represent different types of spalling and different thermal barrier coating.
“... generating by the trained first generator, from the first label, a pseudo sample ...” ([0022, 0029], The generator may be trained using labeled images of one or more anomalies. The generator generates anomalies based on pixel characteristics representing the defects in the training images.)
“... generating a first pseudo sample comprising the first feature by providing a first input value corresponding to the first label ...” ([0022, 0029], The generator may be trained using labeled images of one or more anomalies and the training involves examining the pixel characteristics such as pixel location, and pixel intensity (input value). The generator generates anomalies based on pixel characteristics representing the defects in the training images. The generator generates foreground images representing the anomalies and background images representing the coating.)
“... data comprising a second feature different from the first feature and a second label indicating a type of the second feature ...” ([0020-0023, 0028-0029, Figure 3], In some embodiments, the anomalies in the images may be labeled. The thermal barrier coating 300 (second feature and label) are shown in training image 308 (learning dataset) as the background. The training images may be different from each other and can represent different types of spalling and different thermal barrier coating.
“generating by the trained second generator, from the second label, a pseudo sample ...” ([0022, 0029], The generator may be trained using labeled images of one or more anomalies. The generator generates undamaged coatings based on the pixel characteristics representing the coatings in the training images.)
“generating a new sample of the predetermined type of data by synthesizing the generated second pseudo sample with the generated first pseudo sample, the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit the synthesizing preserving an association between the first and second features” ([0029, 0032-0033], The generated foreground images (first pseudo sample) and generated background images (second pseudo sample) are combined (synthesizing) by a generator sub-network to create a consolidated images (new sample). The foreground image depicts generator-created anomalies and may have different shape and size. The foreground image is combined with the background image by overlaying the two images and adding the pixel from the foreground image into the background image.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “data comprising a first feature and a first label indicating a type of the first feature”, “generating by the trained first generator, from the first label, a pseudo sample”, “generating a first pseudo sample comprising the first feature by providing a first input value corresponding to the first label”, “data including comprising a second feature different from the first feature and a second label indicating a type of the second feature”, “generating by the trained second generator, from the second label, a pseudo sample”, “generating a new sample of the predetermined type of data by synthesizing the generated second pseudo sample with the generated first pseudo sample, the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit a pseudo value of the second feature attribute and performing convolutional addition of the converted second pseudo sample into the first pseudo sample, the synthesizing preserving an association between the first and second features” from Lim into the teaching of Yoo. Doing so can generate additional training data for the system that combines different size and shape of foreground images with different background objects for anomaly detection (Lim, abstract; par. 14).
Yoo in view of Lim does not explicitly disclose an implementation of “the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit a pseudo value of the second feature attribute and performing convolutional addition of the converted second pseudo sample into the first pseudo sample”. However, Shechtman discloses in the same field of endeavor:
“... the synthesizing comprising converting the second ...” ([0058-0059, 0063-0067], The geometric prediction neural network consists of multiple geometric prediction generators to generate warp parameters. A foreground image and a background image are provided to the generator to determine a predicted warp parameter to transform the foreground object into a more realistic object when combined with the background image. The warped foreground is combined with the background to create a composite image. The compositing process is represented by Equation 1, which involves an elemental multiplication and a summation operation of the images to create the composite image.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit a pseudo value of the second feature attribute and performing convolutional addition of the converted second pseudo sample into the first pseudo sample” from Shechtman into the teaching of Yoo in view of Lim. Doing so can improve the performance of a GAN system to generate realistic composite images by implementing a training process to learn warp parameters that provides geometric alignment of foreground objects with respect to a background image (Shechtman, abstract).
Regarding claim 16, Yoo teaches:
“A computer implemented data generation method, comprising” (abstract, The reference describes a system that has multiple generators for generating synthetic data.)
“training, with a computer, a first generator through machine learning using a plurality of first learning datasets, each of the plurality of first learning datasets comprising a combination of a first sample of a predetermined type of data ” ([0053, 0056, 0062, 0103, Figure 3], A first generator is trained with multiple sets of training data through different iterations. The training data may consist of images of a first domain. The training images may be used for user verification in which the images show person “A” with unique facial features. The reference does not explicitly discuss the analysis of the features of the input data, but it is implied that the system identifies whether an image shows “person A” by analyzing the facial features of the person for user identification.)
“generating, with the computer, by the trained first generator, to adjust generation parameters based on feedback from the first discriminator to adversarially optimize computational parameters of the first generator to challenge a discrimination accuracy of the first discriminator over successive training cycles, thereby iteratively enhancing generation fidelity of the first generator” ([0056, 0061-0063, 0103, Figure 2], A first generator outputs a second training data (pseudo sample) based on the first input training data of the first domain. It is implied that the neural network is trained on the individual features of the first input training data to generate new synthetic data that is different from the input data. The generator and discriminator operate as adversarial neural network and the performance is enhanced during the training process. During training, the generator generates fake data closer to genuine data and the discriminator is configured to distinguish genuine data from fake data.)
“generating, with the computer, a first pseudo sample ” ([0053, 0056, 0062, Figure 3], A first generator is trained with multiple sets of training data through different iterations. The training data may consist of images of a first domain. The training images may be used for user verification in which the images show person “A” with unique facial features.)
“training, with the computer, a second generator through machine learning using a plurality of second learning datasets, the second generator being trained independently of the first generator, each of the plurality of second learning datasets comprising a combination of a second sample of the predetermined type of data ” ([0053, 0057, 0062, Figure 4], A second generator is trained with multiple sets of training data through different iterations. The training data may consist of images of a second domain. The training images may be used for user verification in which the images show person “B” with unique facial features. Domain may be defined as a category consisting of “faces” and the generated sample fits into the fourth training data because both samples describe “faces”. Figure 4 shows the first and second generator being trained separately.)
“generating, with the computer, by the trained second generator, ” ([0057], A second generator outputs a fourth training data (pseudo sample) based on the third input training data of the second domain. The neural network is trained on the individual features of the third input training data to generate new synthetic data that is different from the input data.)
“generating, with the computer, a second pseudo sample ” ([0057], A second generator outputs a fourth training data (second pseudo sample) based on the third input training data (second input value) of the second domain. It is implied that the neural network is trained on the individual features of the third input training data to generate new synthetic data that is different from the input data.)
“generating, with the computer, a new sample of the predetermined type of data by ” ([0053, 0058, 0080, 0083-0085], The first and second generators generate training data of different domains. The data classification apparatus may be trained on the generated samples from the first and second generator to determine which domain an input data belongs to. The data classification apparatus may receive query input data and may generate first query data (new sample) of the first domain when the query input data is determined to belong to the first domain. A first domain may be “person A” and a facial image (predetermined type of data) may be generated.)
Yoo does not explicitly disclose an implementation of “data comprising a first feature and a first label indicating a type of the first feature”, “generating, with the computer, by the trained first generator, from the first label, a pseudo sample”, “generating, with the computer, a first pseudo sample comprising the first feature ... by providing a first input value corresponding to the first label”, “data comprising a second feature different from the first feature and a second label indicating a type of the second feature”, “generating, with the computer, by the trained second generator, from the second label, a pseudo sample”, “generating, with the computer, a second pseudo sample comprising the second feature ... by providing a second input value corresponding to the second label”, and “generating a new sample of the predetermined type of data by synthesizing the generated second pseudo sample with the generated first pseudo sample, the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit a pseudo value of the second feature attribute and performing convolutional addition of the converted second pseudo sample into the first pseudo sample, the synthesizing preserving an association between the first and second features”. However, Lim discloses in the same field of endeavor:
“... data comprising a first feature and a first label indicating a type of the first feature” ([0020-0023, 0028-0029, Figure 3], Training images may contain labels that describe the pixels in the image, such as a label for a crack in a turbine blade or a label for the turbine blade. The spalling 302 and 304 (first feature and label) are shown in training image 308 (learning dataset) as the foreground. In some embodiments, the anomalies of the spalling in the images may be labeled. The training images may be different from each other and can represent different types of spalling and different thermal barrier coating.
“... generating, with the computer, by the trained first generator, from the first label, a pseudo sample ...” ([0022, 0029], The generator may be trained using labeled images of one or more anomalies. The generator generates anomalies based on pixel characteristics representing the defects in the training images.)
“generating, with the computer, a first pseudo sample comprising the first feature using the trained first generator by providing a first input value corresponding to the first label ...” ([0022, 0029], The generator may be trained using labeled images of one or more anomalies and the training involves examining the pixel characteristics such as pixel location, and pixel intensity (input value). The generator generates anomalies based on pixel characteristics representing the defects in the training images. The generator generates foreground images representing the anomalies and background images representing the coating.)
“... data comprising a second feature different from the first feature and a second label indicating a type of the second feature” ([0020-0023, 0028-0029, Figure 3], In some embodiments, the anomalies in the images may be labeled. The thermal barrier coating 300 (second feature and label) are shown in training image 308 (learning dataset) as the background. The training images may be different from each other and can represent different types of spalling and different thermal barrier coating.
“generating, with the computer, by the trained sample...” ([0022, 0029], The generator may be trained using labeled images of one or more anomalies and the training involves examining the pixel characteristics such as pixel location, and pixel intensity (input value). The generator generates anomalies based on pixel characteristics representing the defects in the training images. The generator generates foreground images representing the anomalies and background images representing the coating.)
“generating, with the computer, a second pseudo sample comprising the second feature using the trained second generator by providing a second input value corresponding to the second label to the trained second generator” ([0020-0023, 0028-0029, Figure 3], The generator may be trained using labeled images of one or more anomalies. The generator generates undamaged coatings based on the pixel characteristics representing the coatings in the training images.)
“generating a new sample of the predetermined type of data by synthesizing the generated second pseudo sample with the generated first pseudo sample, the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit the synthesizing preserving an association between the first and second features” ([0029, 0032-0033], The generated foreground images (first pseudo sample) and generated background images (second pseudo sample) are combined (synthesizing) by a generator sub-network to create a consolidated image (new sample). The background and foreground images show different features and the consolidated images combines the different features into a single image. The foreground image depicts generator-created anomalies and may have different shape and size. The foreground image is combined with the background image by overlaying the two images and adding the pixel from the foreground image into the background image.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “data comprising a first feature and a first label indicating a type of the first feature”, “generating, with the computer, by the trained first generator, from the first label, a pseudo sample”, “generating, with the computer, a first pseudo sample comprising the first feature ... by providing a first input value corresponding to the first label”, “data comprising a second feature different from the first feature and a second label indicating a type of the second feature”, “generating, with the computer, by the trained second generator, from the second label, a pseudo sample”, “generating, with the computer, a second pseudo sample comprising the second feature ... by providing a second input value corresponding to the second label”, and “generating a new sample of the predetermined type of data by synthesizing the generated second pseudo sample with the generated first pseudo sample, the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit a pseudo value of the second feature attribute and performing convolutional addition of the converted second pseudo sample into the first pseudo sample, the synthesizing preserving an association between the first and second features” from Lim into the teaching of Yoo. Doing so can generate additional training data for the system that combines different size and shape of foreground images with different background objects for anomaly detection (Lim, abstract; par. 14).
Yoo in view of Lim does not explicitly disclose an implementation of “the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit a pseudo value of the second feature attribute and performing convolutional addition of the converted second pseudo sample into the first pseudo sample”. However, Shechtman discloses in the same field of endeavor:
“... the synthesizing comprising converting the second ...” ([0058-0059, 0063-0067], The geometric prediction neural network consists of multiple geometric prediction generators to generate warp parameters. A foreground image and a background image are provided to the generator to determine a predicted warp parameter to transform the foreground object into a more realistic object when combined with the background image. The warped foreground is combined with the background to create a composite image. The compositing process is represented by Equation 1, which involves an elemental multiplication and a summation operation of the images to create the composite image.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “the synthesizing comprising converting the second pseudo sample to enable the second pseudo sample to fit a pseudo value of the second feature attribute and performing convolutional addition of the converted second pseudo sample into the first pseudo sample” from Shechtman into the teaching of Yoo in view of Lim. Doing so can improve the performance of a GAN system to generate realistic composite images by implementing a training process to learn warp parameters that provides geometric alignment of foreground objects with respect to a background image (Shechtman, abstract).
Regarding claim 17:
Claim 17 recites an article of manufacture (“A non-transitory computer-readable storage medium storing data generation program, which when read and executed, causes for causing a computer to perform operations comprising”) that performs the same process as described in Claim 16. Therefore claim 17 is rejected under the same reasons mention for claim 16. The additional elements of claim 17 is addressed below:
“A non-transitory computer-readable storage medium storing data generation program, which when read and executed, causes a computer to perform operations comprising” ([0100, Figure 9],” the data classification apparatus 900 may include a processor 910 and a memory 920”, The data classification apparatus with a memory component is shown in Figure 9.)
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GARY MAC whose telephone number is (703)756-1517. The examiner can normally be reached Monday - Friday 8:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Kawsar can be reached on (571) 270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GARY MAC/Examiner, Art Unit 2127
/ABDULLAH AL KAWSAR/Supervisory Patent Examiner, Art Unit 2127
Prosecution Insights