DETAILED ACTION
Claims 1-19 are presented for examination.
This office action is in response to submission of application on 11/29/2022.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 16 is objected to because of the following informalities: “ The system of claim 5” should be “The system of claim 15”. For examination purposes, the examiner is interpreting “ The system of claim 5” to be “The system of claim 15”. Appropriate correction is required.
Specification
The disclosure is objected to because of the following informalities:
In paragraph 0059, “A second dataset may used in connection…” should be “A second dataset may be used in connection…”.
In paragraph 0069, “sample data that includes an number of false positives” should be “sample data that includes a number of false positives”.
In paragraph 0071, “processes resulting in an processing/ML model” should be “processes resulting in a processing/ML model”.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 4, 5, 6, 9, 10, 11, 13, 15, 16 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (“Discovering New Intents with Deep Aligned Clustering”), hereafter Zhang, in view of Lee et al. (Pub. No.: US 2018/0285771 A1), hereafter Lee.
Regarding claim 1, Zhang discloses:
A method comprising: training, using a supervised learning process, a first learning model with a first dataset (Zhang, Figure 2 and page 3, right column, final paragraph, lines 4-6 “we capture such intent feature information by pre-training the model with the labeled data” teaches pretraining a model with labeled data as training a supervised learning model with a first labeled dataset),
applying the first learning model to a … dataset thereby generating a first learning model output (Zhang, Figure 2 and page 3, right column, final paragraph, last 4 lines “After pre-training, we remove the classifier and use the rest of the network as the feature extractor in the subsequent unsupervised clustering process” and page 4, left column, last paragraph, lines 3-5 “Specifically, we firstly extract intent features of all training data from the pretrained model.” teaches applying the first learning model to a dataset to generate an output),
training, using an unsupervised learning process, a second learning model with the first learning model output thereby generating a clustering output of the second learning model (Zhang, Figure 2 and caption “we perform k-means to produce cluster centroids” teaches performing unsupervised K-means as a second learning model with the first learning model output thereby generating a clustering output of the second learning model),
determining an… assessment based on the clustering output (Zhang, Figure 2 ad page 4, right column, final paragraph, lines 1-3 “We use the cluster validity index (CVI) to evaluate the quality of clusters obtained during each training epoch after clustering” teaches determining a quality assessment based on the clustering output).
While Zhang discloses applying the first learning model to a … dataset thereby generating a first learning model output, they do not explicitly disclose applying the first learning model to a second dataset.
Lee discloses:
applying … learning model to a second dataset (Lee, Fig. 1, ¶[0024] teaches applying a learned classifier to a second dataset).
While Zhang discloses determining an … assessment based on the clustering output, they do not explicitly disclose the assessment to be a bias assessment.
Lee discloses:
determining a bias assessment based on the clustering output (Lee, ¶[0054] teaches determining isolated and abnormal data based on a clustering output as a bias assessment).
Zhang and Lee are analogous art because they are from the same field of endeavor, supervised and unsupervised learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhang to include applying the a learning model to a second dataset and determining a bias assessment based on the clustering output, based on the teachings of Lee. One of ordinary skill in the art would have been motivated to make this modification in order to learn better classifiers, as suggested by Lee (¶[0042]).
Regarding claim 2, Zhang, in view of Lee, discloses the method of claim 1 (and thus the rejection of claim 1 is incorporated). Zhang further discloses:
training, using a third dataset, an … assessment modified learning model using supervised learning (Zhang, Figure 2 , Figure 2 caption “we use G on the pseudo-labels to produce the aligned labels for self-supervised learning.” And page 4, right column, penultimate paragraph “we use the aligned pseudo-labels to perform self-supervised learning under the supervision of the softmax loss” Teaches training an assessment modified self-supervised learning model using supervised learning and a third dataset),
Zhang teaches training, using a third dataset, an … assessment modified learning model using supervised learning but does not disclose the assessment to be a bias assessment.
Lee discloses:
training, using a third dataset, a bias assessment modified learning model using supervised learning (Lee, Fig. 1,¶[0025] and ¶[0054] teaches training a bias assessment modified supervised learning with a third dataset in element 136).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhang to include training, using a third dataset, a bias assessment modified learning model using supervised learning, based on the teachings of Lee. One of ordinary skill in the art would have been motivated to make this modification in order to learn better classifiers, as suggested by Lee (¶[0042]).
Regarding claim 4, Zhang, in view of Lee, discloses the method of claim 1 (and thus the rejection of claim 1 is incorporated) determining bias assessment. Zhang further discloses:
determining a third dataset based in part on the … assessment (Zhang, Figure 2 and caption “we use G on the pseudo-labels to produce the aligned labels for self-supervised learning.” Teaches producing aligned labels as determining a third dataset based on the assessment),
training, using the third data set, a … assessment modified learning model using supervised learning (Zhang, Figure 2 , Figure 2 caption “we use G on the pseudo-labels to produce the aligned labels for self-supervised learning.” And page 4, right column, penultimate paragraph “we use the aligned pseudo-labels to perform self-supervised learning under the supervision of the softmax loss” Teaches training an assessment modified self-supervised learning model using supervised learning and a third dataset),
applying the … assessment modified learning model to the … dataset thereby generating a third learning model output (Zhang, Figure 2 and page 2, left column, penultimate paragraph, lines 4-10 “For each training epoch, we firstly perform k-means on the extracted intent features, and then use the produced cluster assignments as pseudo-labels for training the neural network… so we use the cluster centroids as the targets to obtain the alignment mapping between pseudo-labels in consequent epochs.” Teaches applying the learning model to the datasets in consequent epochs to generate the third learning model outputs),
training, using the unsupervised learning process, a fourth learning model with the third learning model output thereby generating a second clustering output of the fourth learning model (Zhang, Figure 2 and page 2, left column, penultimate paragraph, lines 4-10 “For each training epoch, we firstly perform k-means on the extracted intent features, and then use the produced cluster assignments as pseudo-labels for training the neural network… so we use the cluster centroids as the targets to obtain the alignment mapping between pseudo-labels in consequent epochs.” Teaches applying clustering in consequent epochs to generate the fourth learning model outputs),
determining a second … assessment based on the second clustering output (Zhang, Figure 2 and page 4, right column, final paragraph, lines 1-3 “We use the cluster validity index (CVI) to evaluate the quality of clusters obtained during each training epoch after clustering.” Teaches determining a second assessment for consequent epochs based on the second clustering output),
training, using a fourth dataset, a second bias assessment modified learning model using supervised learning process (Zhang, Figure 2 and page 6, right column, penultimate paragraph, last 3 lines “finding the mapping of produced pseudo-labels between contiguous epochs, which provides stronger supervised signals for representation learning” teaches training between contiguous epochs to produce a second assessment modified learning model using the supervised learning process).
While Zhang discloses the assessment as described above, they do not teach this assessment to be a bias assessment.
Lee discloses:
bias assessment (Lee, ¶[0054] teaches determining isolated and abnormal data based on a clustering output as a bias assessment).
While Zhang discloses applying the … assessment modified learning model to the … dataset thereby generating a third learning model output, they do not explicitly disclose applying the learning model to a second dataset.
Lee discloses:
applying … learning model to the second dataset (Lee, Fig. 1, ¶[0024] teaches applying a learned classifier to a second dataset).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhang to include applying a learning model to a second dataset and bias assessment, based on the teachings of Lee. One of ordinary skill in the art would have been motivated to make this modification in order to learn better classifiers, as suggested by Lee (¶[0042]).
Regarding claim 5, Zhang, in view of Lee, discloses the method of claim 2 (and thus the rejection of claim 2 is incorporated) determining a bias assessment based on the clustering output. Lee further discloses:
automatically determining a problematic cluster where the first model output matches an undesired condition (Lee, ¶[0054] teaches automatically determining a problematic cluster of low confidence where the model output matches an undesired condition).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhang to include automatically determining a problematic cluster where the first model output matches an undesired condition, based on the teachings of Lee. One of ordinary skill in the art would have been motivated to make this modification in order to learn better classifiers, as suggested by Lee (¶[0042]).
Regarding claim 6, Zhang, in view of Lee, discloses the method of claim 5 (and thus the rejection of claim 5 is incorporated). Lee further discloses:
synthesizing data samples based on samples of the problematic cluster (Lee, ¶[0054] and ¶[0052] teaches giving samples of the problematic cluster to a labeler to synthesize data samples).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhang to include synthesizing data samples based on samples of the problematic cluster, based on the teachings of Lee. One of ordinary skill in the art would have been motivated to make this modification in order to learn better classifiers, as suggested by Lee (¶[0042]).
Regarding claim 9, Zhang, in view of Lee, discloses the method of claim 1 (and thus the rejection of claim 1 is incorporated). Lee further discloses:
wherein the first dataset and the second dataset include image data (Lee, ¶[0035]).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhang to include wherein the first dataset and the second dataset include image data, based on the teachings of Lee. One of ordinary skill in the art would have been motivated to make this modification in order to learn better classifiers, as suggested by Lee (¶[0042]).
Regarding Claim 10, Zhang discloses:
A non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors of a computing platform, cause the computing platform to perform operations …(Zhang, page 6, left column, final paragraph, lines 1-4 “We use the pre-trained BERT model (bert-uncased, with 12-layer transformer) implemented in PyTorch (Wolf et al. 2019) as our network backbone, and adopt most of its suggested hyper-parameters for optimization.”).
The rest of claim 10 is substantially similar to claim 1, and thus is rejected on the same basis as claim 1.
Regarding Claim 15, Zhang discloses:
A system comprising of: one or more computer-readable mediums storing instructions that, when executed by the one or more computer processors, cause a computing platform to perform operations comprising…(Zhang, page 6, left column, final paragraph, lines 1-4 “We use the pre-trained BERT model (bert-uncased, with 12-layer transformer) implemented in PyTorch (Wolf et al. 2019) as our network backbone, and adopt most of its suggested hyper-parameters for optimization.”).
The rest of claim 15 is substantially similar to claim 1, and thus is rejected on the same basis as claim 1.
Claims 11 and 16 are substantially similar to claim 2, thus, they are rejected on the same basis as claims 2.
Claims 13 and 18 are substantially similar to claim 5, thus, they are rejected on the same basis as claim 5.
Claims 3, 12, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (“Discovering New Intents with Deep Aligned Clustering”), hereafter Zhang, in view of Lee et al. (Pub. No.: US 2018/0285771 A1), hereafter Lee, in further view of Das et al. (Pub. No.: US 2022/0114490 A1), hereafter Das.
Regarding claim 3, Zhang, in view of Lee, discloses the method of claim 2 (and thus the rejection of claim 2 is incorporated).
Lee discloses training, using a third dataset, a bias assessment modified learning model using supervised learning, but does not teach modifying a first dataset based on the assessment.
Das discloses:
wherein the third dataset is the first dataset modified based on the … assessment (Das, Figure 8, Figure 13, and ¶[0093] teaches modifying the first datasets based on an assessment to form a third retraining dataset).
Zhang, Lee, and Das are analogous art because they are from the same field of endeavor, supervised and unsupervised learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhang, in view of Lee, to include wherein the third dataset is the first dataset modified based on the … assessment, based on the teachings of Das. One of ordinary skill in the art would have been motivated to make this modification in order to generate rich insights, as suggested by Das (¶[0004]).
Claims 12 and 17 are substantially similar to claim 3, thus, they are rejected on the same basis as claim 3.
Claims 7, 8, 14, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (“Discovering New Intents with Deep Aligned Clustering”), hereafter Zhang, in view of Lee et al. (Pub. No.: US 2018/0285771 A1), hereafter Lee, in further view of Abdulaal et al. (Pub. No.: US 2020/0226490 A1), hereafter Abdulaal.
Regarding claim 7, Zhang, in view of Lee, discloses the method of claim 2 (and thus the rejection of claim 2 is incorporated). Lee further discloses:
receiving… bias assessments for a first cluster (Lee, ¶[0054] teaches receiving… bias assessments for a first cluster).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhang to include receiving… bias assessments for a first cluster, based on the teachings of Lee. One of ordinary skill in the art would have been motivated to make this modification in order to learn better classifiers, as suggested by Lee (¶[0042]).
While Lee discloses receiving… bias assessments for a first cluster, they do not disclose receiving, through an interface, … assessments.
Abdulaal discloses:
receiving, through an interface, bias assessments for a first cluster (Abdulaal, Fig. 1, ¶[0046], and [0013] teaches receiving, through an interface, bias assessments for a cluster).
Zhang, Lee, and Abdulaal are analogous art because they are from the same field of endeavor, supervised and unsupervised learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhang, in view of Lee, to include receiving, through an interface, bias assessments for a first cluster, based on the teachings of Abdulaal. One of ordinary skill in the art would have been motivated to make this modification in order to improve the accuracy of the ML classifier, as suggested by Abdulaal (¶[0044]).
Regarding claim 8, Zhang, in view of Lee, in further view of Abdulaal, discloses the method of claim 7 (and thus the rejection of claim 7 is incorporated) receiving, through an interface, bias assessments for a first cluster. Abdulaal further discloses:
presenting a user interface with representative examples from at least one cluster (Abdulaal, ¶[0068] teaches presenting a user interface with representative examples from at least one cluster),
receiving a bias assessment input for the at least one cluster (Abdulaal, ¶[0046] and ¶[0068] teaches receiving a bias assessment input for the at least one cluster).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhang, in view of Lee, to include presenting a user interface with representative examples from at least one cluster and receiving a bias assessment input for the at least one cluster, based on the teachings of Abdulaal. One of ordinary skill in the art would have been motivated to make this modification in order to improve the accuracy of the ML classifier, as suggested by Abdulaal (¶[0044]).
Claims 14 and 19 are substantially similar to claim 7, thus, they are rejected on the same basis as claim 7.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's
disclosure.
U.S. Pub No. 11775863 B2: Wick et al. teaches bias and unsupervised classification.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUMAIRA ZAHIN MAUNI whose telephone number is (703)756-5654. The examiner can normally be reached Monday - Friday, 9 am - 5 pm (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached at (571) 272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.Z.M./Examiner, Art Unit 2141
/HOPE C SHEFFIELD/Primary Examiner, Art Unit 2141