DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR
1.17(e), was filed in this application after final rejection. Since this application is eligible for continued
examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the
finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's
submission filed on 9 September 2025 has been entered.
Response to Amendment
The amendment filed on 9 September 2025 has been entered.
Claims 1-12, 14-16, 18-26 are pending.
Claims 1, 18, 25-26 are amended.
Claims 27-30 are new.
Claims 1-12, 14-16, 18-26, 27-30 will be pending.
Response to Arguments
Applicant's arguments filed on 9 September 2025 have been fully considered, but they are not persuasive.
Applicant’s remarks, regarding the rejections of claims under 35 USC 103, have been fully considered.
Applicant notes Claim 1 is currently amended to include limitations of: "obtaining facial images for a neural network configuration and a neural network training dataset...; augmenting the neural network training dataset using additional images, wherein the additional images comprise synthetic images that represent a specific demographic, wherein the synthetic images expand a number of images for an age band; partitioning the facial images into multiple subgroups...; calculating a multifactor key performance indicator (KPI) per image...; and promoting the neural network configuration and the neural network training dataset..." (emphasis added).
Applicant submits none of the cited references, alone, or in any combination, show or suggest all the elements of currently amended Claim 1. Applicant submits Claim 1, and its corresponding dependent claims, should be deemed allowable. Applicant notes independent Claims 25 and 26 are currently amended in a manner similar to Claim 1 and therefore, Claims 25 and 26 should also be deemed allowable.
Applicant notes Claim 18 is currently amended to recite "wherein the synthetic images are generated and promoted to the production neural network based on a bias and a non-bias in the neural network training dataset" (emphasis added).
Applicant submits none of the cited references show or suggest "wherein the synthetic images are generated and promoted to the production neural network based on a bias and a non- bias in the neural network training dataset." Applicant submits for this reason, as well as the reasons previously stated in the Section 103 remarks regarding parent Claim 1, currently amended Claim 18 should be deemed allowable.
Applicant submits none of the cited references show or suggest the features of Claims 27-30. For this reason, and the reasons previously stated in the Section 103 remarks regarding currently amended parent Claim 1, new Claims 27-30 should be deemed allowable.
Applicant’s arguments have been considered, but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-4, 10-12, 25-26 are rejected under 35 U.S.C. 103 as being unpatentable over Modarresi (U.S. Pre-Grant Publication No. 2015/0213389), in view of Ghosh et al. (U.S. Pre-Grant Publication No. 2020/0293933, hereinafter 'Ghosh') and Srinivas et al. (NPL: "Age, Gender, and Fine-Grained Ethnicity Prediction using Convolutional Neural Networks for the East Asian Face Dataset", hereinafter 'Srinivas').
Regarding claim 1 and analogous claims 25, 26, Modarresi teaches A computer-implemented method for machine learning comprising ([0101] Although exemplary embodiments have been described in terms of systems and methods, it is contemplated that certain functionality described herein may be implemented in software on microprocessors, such as a processors 126 a-n and 128 included in the user devices 134 a-n and analytics server 102, respectively, shown in FIG. 1, and computing devices such as the computer system 1300 illustrated in FIG. 13.; [0102] Aspects of the present invention shown in FIGS. 1-12, or any part(s) or function(s) thereof, may be implemented using hardware, software modules, firmware, tangible computer readable media having logic or instructions stored thereon, or a combination thereof and may be implemented in one or more computer systems or other processing systems.; [0103] FIG. 13 illustrates an example computer system 1300 in which embodiments of the present invention, or portions thereof, may be implemented as computer-readable instructions or code.):
obtaining facial images for a neural network configuration and a neural network training dataset, wherein the neural network training dataset is associated with the neural network configuration ([0026] At this point, test data can be used to calculate the misfit error for each input each variable in a set of input variables. An embodiment divides data into two parts, training data and test data. According to this embodiment, the neural network training dataset is associated with the neural network configuration training data is used to formulate a model (i.e., an algorithm) and the accuracy of the algorithm is checked using the test data.; [0048] As used herein, the term “electronic content” is used to refer to any type of media that can be rendered for display, played on, or used at a computing device, television, or other electronic device. Computing devices include client and server devices such as, but not limited to, servers, desktop computers, laptop computers, smart phones, video game consoles, smart televisions, tablet computers, portable gaming devices, personal digital assistants, etc. obtaining facial images for a neural network configuration and a neural network training dataset Electronic content can include text or multimedia files, such as images, video, audio, or any combination thereof.);
calculating a multifactor key performance indicator (KPI) per image, wherein the calculating is based on analyzing performance of two or more image classifier models ([0005] One embodiment involves analyzing one or more calculating a multifactor key performance indicator (KPI) per image Key Performance Indicators (KPIs) associated with electronic content.; [0024] An embodiment uses of two or more image classifier models Classification and Regression Trees (CART) decision trees as base learners. As an individual base learner can be inaccurate (i.e., produce an inaccurate decision), embodiments combine many base learners (i.e., many decision trees) and use a form of a majority vote, which results in very accurate overall decisions. In certain embodiments, this step uses Classification and Regression Trees (CART) decision trees.; [0048] As used herein, the term “electronic content” is used to refer to any type of media that can be rendered for display, played on, or used at a computing device, television, or other electronic device. Computing devices include client and server devices such as, but not limited to, servers, desktop computers, laptop computers, smart phones, video game consoles, smart televisions, tablet computers, portable gaming devices, personal digital assistants, etc. Electronic content can include text or multimedia files, such as images, video, audio, or any combination thereof.; [0032] calculating is based on analyzing performance Embodiments disclosed herein determine KPIs by analyzing data sets with missing entries, high-dimensional data, quantitative (i.e., numerical) data, qualitative (i.e., categorical) data, and data sets including statistical outliers.); and
Modarresi fails to teach augmenting the neural network training dataset using additional images, wherein the additional images comprise synthetic images that represent a specific demographic, wherein the synthetic images expand a number of images for an age band; partitioning the facial images into multiple subgroups, wherein the multiple subgroups represent demographics with potential for biased training; promoting the neural network configuration and the neural network training dataset to a production neural network, wherein the promoting is based on the multifactor key performance indicator.
Ghosh teaches partitioning the facial images into multiple subgroups, wherein the multiple subgroups represent demographics with potential for biased training ([0195] In certain embodiments, multi-structured big data sources 304 may be dynamically ingested during the data ingestion and processing 1204 phase. In certain embodiments, based upon a particular context, partitioning the facial images into multiple subgroups extraction, parsing, and tagging operations are performed on language, text and images they contain to generate associated datasets 1214.; [0260] In certain embodiments, the AIS impartiality assessment 1342 engine may be configured to multiple subgroups represent demographics with potential for biased training obtain bias ranges of features; compare the identified significant contrasts to obtained bias ranges, and determine which of the identified significant contrasts fall outside the obtained bias ranges.);
promoting the neural network configuration and the neural network training dataset to a production neural network, wherein the promoting is based on the multifactor key performance indicator ([0155] In certain embodiments, the cognitive agent governance and assurance operations may include KPI-driven AI model optimization. In these embodiments, the definition of such KPIs, and the method by which they are used to optimize a particular AI model, is a matter of design choice.; [0164] In turn, the results of the KPI evaluations are then used as feedback to improve the performance of the cognitive process. In certain embodiments, the promoting is based on the multifactor key performance indicator results of the KPI evaluations may be provided as input in step 904 to promoting the neural network configuration and the neural network training dataset to a production neural network determine additional operational and performance parameters related to the cognitive process. In certain embodiments, these additional operational and performance parameters may be used to repeat one or more steps associated with the lifecycle of the cognitive process to revise its functionality, improve its performance, or both.).
Modarresi and Ghosh are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Ghosh to Modarresi before the effective filing date of the claimed invention in order to ensure ML models are designed, implemented, and maintained responsibly and provide auditability of their fairness, robustness, transparency, and interpretability (cf. Ghosh, [0031] Accordingly, certain aspects of the invention likewise reflect there are ethical, moral, and social obligations for researchers, developers, and organizations alike to ensure their ML models are designed, implemented, and maintained responsibly. Various aspects of the invention likewise reflect an appreciation that one approach to achieving responsible design, implementation, and maintenance of ML models is to provide auditability of their fairness, robustness, transparency, and interpretability.).
Srinivas teaches augmenting the neural network training dataset using additional images, wherein the additional images comprise synthetic images that represent a specific demographic ([1. Introduction, pg. 954] Convolutional neural networks are used here to predict demographics and provide a baseline performance for the WEAFD. CNNs have recently found great success in many applications relying on facial imaging, including facial alignment via landmark detection, parsing, identity recognition, attribute recognition, expression detection, and additional images comprise synthetic images that represent a specific demographic demographic estimation including age, gender, and ethnicity. The success of CNNs stem from the ability of deep neural networks to automatically discover features which discriminate and describe the underlying differences in data that are difficult to quantify. Facial data often exhibits such differences through unconstrained pose, lighting, expressions, and occlusions.; [A. Training, Validation and Testing Dataset, pg. 958] For training the age models, we limited the number of images in each class to a maximum of 500. For training the gender models, we limited the number of images in each class to a maximum of 4000. For training the finegrained ethnicity model, we limited the number of images in each class to a maximum of 500. We use augmenting the neural network training dataset using additional images data augmentation during training to increase the number of training images. Data augmentation is achieved by mirroring the image. This doubles the number of images used during training.),
wherein the synthetic images expand a number of images for an age band ([A. Training, Validation and Testing Dataset, pg. 958] We use wherein the synthetic images expand a number of images data augmentation during training to increase the number of training images. Data augmentation is achieved by mirroring the image. This doubles the number of images used during training.; [B. Metadata Collection Procedure, pg. 955] Age, gender and fine-grained ethnicity information were labeled by the Turkers. Age Estimation: Estimate the for an age band age range of the subjects in the image. The choices for age ranges are: 0-2 years, 3-5 years, 6-9 years, 10-12 years, 13-16 years, 17-23 years, 24-39 years, 40-54 years, 55-64 years, and 65+ years.);
Modarresi, Ghosh, and Srinivas are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi and Ghosh, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Srinivas to Modarresi before the effective filing date of the claimed invention in order to use the Wild East Asian Face Dataset (WEAFD) to predict age, gender and fine-grained ethnicity of an individual by providing baseline results using a convolutional neural network (CNN) (cf. Srinivas, [Abstract, pg. 953] This paper explores the difficulty of performing automatic demographic prediction on the East Asian population. We introduce the Wild East Asian Face Dataset (WEAFD), a new and unique dataset, to the research community. This dataset consists primarily of labeled face images of individuals from East Asian countries, including Vietnam, Burma, Thailand, China, Korea, Japan, Indonesia, and Malaysia. East Asian Amazon Mechanical Turk annotators were used to label the age, gender and fine grain ethnicity attributes to reduce the impact of the “other-race effect” and improve quality of annotations. We focus on predicting age, gender and fine-grained ethnicity of an individual by providing baseline results using a convolutional neural network (CNN). Fine-grained ethnicity prediction refers to predicting refined categorization of the human population (Chinese, Japanese, Korean, etc.). Performance of two CNN architectures is presented, highlighting the difficulty of these tasks and showcasing potential design considerations that improve network optimization by promoting region based feature extraction.).
Regarding claim 2, Modarresi, as modified by Ghosh and Srinivas, teaches The method of claim 1.
Ghosh teaches wherein the multifactor key performance indicator (KPI) identifies bias in the training dataset ([0155] In certain embodiments, the cognitive agent governance and assurance operations may include KPI-driven AI model optimization. In these embodiments, the definition of such multifactor key performance indicator (KPI) KPIs, and the method by which they are used to optimize a particular AI model, is a matter of design choice.; [0216] In certain embodiments, the AIS impartiality assessment operation may be performed to identifies bias in the training dataset detect the presence of bias in a particular ML model, such as the opaque model 1332, and if detected, assess its effect on the outcome of an associated cognitive computing operation.; [0164] In turn, the results of the KPI evaluations are then used as feedback to improve the performance of the cognitive process. In certain embodiments, the results of the KPI evaluations may be provided as input in step 904 to determine additional operational and performance parameters related to the cognitive process.).
Modarresi, Ghosh, and Srinivas are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 3, Modarresi, as modified by Ghosh and Srinivas, teaches The method of claim 2.
Ghosh teaches wherein identified bias precludes promotion to the production neural network ([0216] In certain embodiments, the AIS impartiality assessment operation may be performed to identified bias detect the presence of bias in a particular ML model, such as the opaque model 1332, and if detected, assess its effect on the outcome of an associated cognitive computing operation.; [0271] In certain embodiments, the identified significant contrasts may be presented as a cognitive insight 1352 to the developer of the model, such that the developer can precludes promotion to the production neural network modify the opaque model 1332, the training corpus 1302, the model trainer 1304, or a combination thereof, to achieve less biased results. Skilled practitioners of the art will recognize that the described presentation of identified significant contrasts as a cognitive insight 1352 provides the developer of the opaque model 1332 a basis for modifying their model, the training corpus 1302, the model trainer 1304, or a combination thereof.).
Modarresi, Ghosh, and Srinivas are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 4, Modarresi, as modified by Ghosh and Srinivas, teaches The method of claim 2.
Ghosh teaches wherein an absence of identified bias allows promotion to the production neural network ([0216] In certain embodiments, the AIS impartiality assessment operation may be performed to absence of identified bias detect the presence of bias in a particular ML model, such as the opaque model 1332, and if detected, assess its effect on the outcome of an associated cognitive computing operation.; [0271] In certain embodiments, the identified significant contrasts may be presented as a cognitive insight 1352 to the developer of the model, such that the developer can modify the opaque model 1332, the training corpus 1302, the model trainer 1304, or a combination thereof, to achieve less biased results. Skilled practitioners of the art will recognize that the described presentation of identified significant contrasts as a cognitive insight 1352 provides the developer of the opaque model 1332 allows promotion to the production neural network a basis for modifying their model, the training corpus 1302, the model trainer 1304, or a combination thereof.).
Modarresi, Ghosh, and Srinivas are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 10, Modarresi, as modified by Ghosh and Srinivas, teaches The method of claim 1.
Modarresi teaches wherein the training dataset includes facial images ([0026] At this point, test data can be used to calculate the misfit error for each input each variable in a set of input variables. An embodiment divides data into two parts, training dataset training data and test data. According to this embodiment, the training data is used to formulate a model (i.e., an algorithm) and the accuracy of the algorithm is checked using the test data.; [0048] As used herein, the term “electronic content” is used to refer to any type of media that can be rendered for display, played on, or used at a computing device, television, or other electronic device. Computing devices include client and server devices such as, but not limited to, servers, desktop computers, laptop computers, smart phones, video game consoles, smart televisions, tablet computers, portable gaming devices, personal digital assistants, etc. Electronic content can include text or multimedia files, such as includes facial images images, video, audio, or any combination thereof.).
Modarresi, Ghosh, and Srinivas are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 11, Modarresi, as modified by Ghosh and Srinivas, teaches The method of claim 1.
Ghosh teaches further comprising training the production neural network, using the neural network training dataset that is promoted ([0216] In certain embodiments, the AIS impartiality assessment operation may be performed to detect the presence of bias in a particular ML model, such as the opaque model 1332, and if detected, assess its effect on the outcome of an associated cognitive computing operation.; [0271] In certain embodiments, the identified significant contrasts may be presented as a cognitive insight 1352 to the developer of the model, such that the developer can modify the opaque model 1332, the training the production neural network using the neural network training dataset that is promoted training corpus 1302, the model trainer 1304, or a combination thereof, to achieve less biased results. Skilled practitioners of the art will recognize that the described presentation of identified significant contrasts as a cognitive insight 1352 provides the developer of the opaque model 1332 a basis for modifying their model, the training corpus 1302, the model trainer 1304, or a combination thereof.).
Modarresi, Ghosh, and Srinivas are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 12, Modarresi, as modified by Ghosh and Srinivas, teaches The method of claim 11.
Ghosh teaches wherein the neural network training dataset that is promoted enables bias mitigation ([0216] In certain embodiments, the AIS impartiality assessment operation may be performed to detect the presence of bias in a particular ML model, such as the opaque model 1332, and if detected, assess its effect on the outcome of an associated cognitive computing operation.; [0271] In certain embodiments, the identified significant contrasts may be presented as a cognitive insight 1352 to the developer of the model, such that the developer can modify the opaque model 1332, the neural network training dataset that is promoted training corpus 1302, the model trainer 1304, or a combination thereof, to enables bias mitigation achieve less biased results. Skilled practitioners of the art will recognize that the described presentation of identified significant contrasts as a cognitive insight 1352 provides the developer of the opaque model 1332 a basis for modifying their model, the training corpus 1302, the model trainer 1304, or a combination thereof.).
Modarresi, Ghosh, and Srinivas are combinable for the same rationale as set forth above with respect to claim 1.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Modarresi, in view of Ghosh, Srinivas, and further in view of Bekkar et al. (NPL: "Evaluation Measures for Models Assessment over Imbalanced Data Sets", hereinafter 'Bekkar') and Tewari et al. (NPL: "FML: Face Model Learning from Videos", hereinafter 'Tewari').
Regarding claim 5, Modarresi, as modified by Ghosh and Srinivas, teaches The method of claim 1.
Modarresi, as modified by Ghosh and Srinivas, fails to teach wherein the multifactor KPI comprises an F-measure, an ROC- AUC measure, a precision measure, a recall/true positive rate, a false positive rate, a total number of videos measure, a number of positive videos measure, a number of positive frames measure, or a number of negative frames measure.
Bekkar teaches wherein the multifactor KPI comprises an F-measure, an ROC- AUC measure, a precision measure, a recall/true positive rate, a false positive rate, a total number of videos measure, a number of positive videos measure, a number of positive frames measure, or a number of negative frames measure ([Abstract, pg. 27] Imbalanced data learning is one of the challenging problems in data mining; among this matter, founding the right model assessment measures is almost a primary research issue. Skewed class distribution causes a misreading of common evaluation measures as well it lead a biased classification. This article presents a set of alternative for multifactor KPI imbalanced data learning assessment, using a combined measures (G-means, likelihood ratios, Discriminant power, F-measure F-Measure Balanced Accuracy, Youden index, Matthews correlation coefficient), and graphical performance assessment (an ROC- AUC measure ROC curve, Area Under Curve, Partial AUC, Weighted AUC, Cumulative Gains Curve and lift chart, Area Under Lift AUL), that aim to provide a more credible evaluation. We analyze the applications of these measures in churn prediction models evaluation, a well known application of imbalanced data.; [2. Fundamental evaluation measures, pg. 27] In machine learning, the classifier is basically evaluated by a confusion matrix. For a binary class problem a matrix is a square of 2×2 as shown in Table 1; column represents the classifier prediction; while the row is the real value of class label. In imbalanced data context, by convention, the observations of minority class are labelled as positive, whilst and the class label of the majority class observations are labelled negative. Accuracy, the most common metric for classifier evaluation, it assesses the overall effectiveness of the algorithm by estimating the probability of the true value of the class label. On the other hand the error rate=1-accuracy is an estimation of misclassification probability according to model prediction. Intuitively, a precision measure precision is a measure of correctness (i.e., a false positive rate out of positive labeled examples, how many are really a positive examples), while a recall/true positive rate Sensitivity (or Recall) is a measure of completeness or accuracy of positive examples (i.e., how many examples of the positive class were labeled correctly). These two metrics, share an inverse relationship between each other. However, unlike accuracy and error, precision and recall are not sensitive to changes in data distributions. A perfect model will capture all positive examples (Recall = 1), and score as only the examples that are in fact (Precision = 1), from an analytical point of view it is desirable to increase recall without sacrificing accuracy.; [5. Application cases and discussion:, pg. 35] Following this assessment we can affirm that model 1B is the best in class in this case in spite of the lowest Accuracy that it has on the 2nd data set, the Model 2D Basic CHAID reveal the highest KPI in accuracy, Precision and Specificity; AUC are close over different models; while the lift brings a low preference to Model 2B. The Discriminant power and MCC values presents a correlation with Accuracy value, where we observe the lowest value of DP in model 2B, and highest one associated to model 2D which is the same trend of accuracy value.).
Modarresi, Ghosh, Srinivas, and Bekkar are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi, Ghosh, and Srinivas, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Bekkar to Modarresi before the effective filing date of the claimed invention in order to assess imbalanced data and evaluations of model predictions (cf. Bekkar, [Abstract, pg. 27] Imbalanced data learning is one of the challenging problems in data mining; among this matter, founding the right model assessment measures is almost a primary research issue. Skewed class distribution causes a misreading of common evaluation measures as well it lead a biased classification. This article presents a set of alternative for imbalanced data learning assessment, using a combined measures (G-means, likelihood ratios, Discriminant power, F-Measure Balanced Accuracy, Youden index, Matthews correlation coefficient), and graphical performance assessment (ROC curve, Area Under Curve, Partial AUC, Weighted AUC, Cumulative Gains Curve and lift chart, Area Under Lift AUL), that aim to provide a more credible evaluation. We analyze the applications of these measures in churn prediction models evaluation, a well known application of imbalanced data.).
Tewari teaches wherein the multifactor KPI comprises an F-measure, an ROC- AUC measure, a precision measure, a recall/true positive rate, a false positive rate, a total number of videos measure, a number of positive videos measure, a number of positive frames measure, or a number of negative frames measure ([Per-frame Parameter Estimation Network, pg. 10816] We employ a convolutional network to extract low-level features. We then apply a series of convolutions, ReLU, and fully connected layers to regress the a number of positive frames measure, or a number of negative frames measure per-frame parameters p [f] . We refer to the supplemental document for further details.)
Modarresi, Ghosh, Srinivas, Bekkar, and Tewari are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi, Ghosh, Srinivas, and Bekkar, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Tewari to Modarresi before the effective filing date of the claimed invention in order to obtain a more accurate and robust model of facial geometry and albedo (cf. Tewari, [1. Introduction, pg. 10813] From a technical point of view, one of our main contributions is a novel multi-frame consistency loss, which ensures that the face identity and albedo reconstruction is consistent across frames of the same subject. This way we can avoid depth ambiguities present in many monocular approaches and obtain a more accurate and robust model of facial geometry and albedo. Moreover, by imposing orthogonality between our learned face identity model and an existing blendshape expression model, our approach automatically disentangles facial expressions from identity based geometry variations, without resorting to a large set of handcrafted priors.).
Claims 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Modarresi, in view of Ghosh, Srinivas, and further in view of Zhang et al. (NPL: "Mitigating Unwanted Biases with Adversarial Learning", hereinafter 'Zhang').
Regarding claim 6, Modarresi, as modified by Ghosh and Srinivas, teaches The method of claim 1.
Modarresi, as modified by Ghosh and Srinivas, fails to teach wherein the multifactor KPI comprises an equal odds or equal opportunity measure.
Zhang teaches wherein the multifactor KPI comprises an equal odds or equal opportunity measure ([1 INTRODUCTION, pg. 335] Machine learning leverages data to build models capable of assessing the labels and properties of novel data. Unfortunately, the available training data frequently contains biases with respect to things that we would rather not use for decision making. Machine learning builds models faithful to training data and can lead to perpetuating these undesirable biases. For example, systems designed to predict creditworthiness and systems designed to perform analogy completion have been demonstrated to be biased against racial minorities and women respectively. Ideally we would be able to build a model which captures exactly those generalizations from the data which are useful for performing some task which are not discriminatory in a way which the people building those models consider unfair. Work on training machine learning systems that output fair decisions has defined several multifactor KPI useful measurements for fairness: Demographic Parity, comprises an equal odds or equal opportunity measure Equality of Odds, and Equality of Opportunity. These can be imposed as constraints or incorporated into a loss function in order to mitigate disproportional outcomes in the system’s output predictions regarding a protected demographic, such as sex.).
Modarresi, Ghosh, Srinivas, and Zhang are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi, Ghosh, and Srinivas, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Zhang to Modarresi before the effective filing date of the claimed invention in order to mitigate disproportional outcomes in the system’s output predictions regarding a protected demographic, such as sex (cf. Zhang, [1. Introduction, pg. 335] Work on training machine learning systems that output fair decisions has defined several useful measurements for fairness: Demographic Parity, Equality of Odds, and Equality of Opportunity. These can be imposed as constraints or incorporated into a loss function in order to mitigate disproportional outcomes in the system’s output predictions regarding a protected demographic, such as sex.).
Regarding claim 7, Modarresi, as modified by Ghosh and Srinivas, teaches The method of claim 1.
Modarresi, as modified by Ghosh and Srinivas, fails to teach wherein the multifactor KPI identifies models that generalize across one or more of the demographics.
Zhang teaches wherein the multifactor KPI identifies models that generalize across one or more of the demographics ([4 PROPERTIES, pg. 337] We note several properties of the above method that we believe distinguish it from past work. (1) multifactor KPI identifies models that generalize across one or more of the demographics Generality: The above method can be used to enforce demographic parity, eqality of odds, or eqality of opportunity as described in Hardt et al. [5]. Further, it applies without modification to the cases when the output variable and/or protected variable are continuous instead of discrete.).
Modarresi, Ghosh, Srinivas, and Zhang are combinable for the same rationale as set forth above with respect to claim 6.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Modarresi, in view of Ghosh, Srinivas, and further in view of Bist et al. (U.S. Pre-Grant Publication No. 2020/0288206, hereinafter 'Bist').
Regarding claim 8, Modarresi, as modified by Ghosh and Srinivas, teaches The method of claim 1.
Modarresi, as modified by Ghosh and Srinivas, fails to teach wherein the two or more image classifier models operate on the multiple subgroups of facial images.
Bist teaches wherein the two or more image classifier models operate on the multiple subgroups of facial images ([0089] In an embodiment, the present invention provides a method for evaluating media content based on combining two or more image classifier models multi-modal inputs from the participants that include operate on the multiple subgroups of facial images reactions and emotions (captured in form of facial expression) that are recorded in real-time on a frame-by-frame basis. The real time reactions and emotions may be recorded in two different steps or campaigns (with two different sets of people), and which include different participants for each.).
Modarresi, Ghosh, Srinivas, and Bist are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi, Ghosh, and Srinivas, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Bist to Modarresi before the effective filing date of the claimed invention in order to enable a way to add meaningful contextual and personalized information to content, that could then be used for searching, classifying, or analyzing the particular content in a variety of ways (cf. Bist, [0005] In light of above, a method and a system for a scalable platform is provided that enables granular tagging of any multimedia or other web content over connected networks. The method of the invention provides an ability to go in much more granular within a content and enable a way to add meaningful contextual and personalized information to it, that could then be used for searching, classifying, or analyzing the particular content in a variety of ways, and in a variety of applications.).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Modarresi, in view of Ghosh, Srinivas, and further in view of Rohekar et al. (U.S. Pre-Grant Publication No. 2023/0117143, hereinafter 'Rohekar').
Regarding claim 9, Modarresi, as modified by Ghosh and Srinivas, teaches The method of claim 1.
Modarresi, as modified by Ghosh and Srinivas, fails to teach wherein the neural network configuration includes a neural network topology.
Rohekar teaches wherein the neural network configuration includes a neural network topology ([0218] A second exemplary type of neural network is the Convolutional Neural Network (CNN). A CNN is a specialized neural network configuration includes a neural network topology feedforward neural network for processing data having a known, grid-like topology, such as image data. Accordingly, CNNs are commonly used for compute vision and image recognition applications, but they also may be used for other types of pattern recognition such as speech and language processing.).
Modarresi, Ghosh, Srinivas, and Rohekar are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi, Ghosh, and Srinivas, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Rohekar to Modarresi before the effective filing date of the claimed invention in order to compute vision, image recognition and other types of pattern recognition such as speech and language processing (cf. Rohekar, [0218] A second exemplary type of neural network is the Convolutional Neural Network (CNN). A CNN is a specialized feedforward neural network for processing data having a known, grid-like topology, such as image data. Accordingly, CNNs are commonly used for compute vision and image recognition applications, but they also may be used for other types of pattern recognition such as speech and language processing.).
Claims 14-16, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Modarresi, in view of Ghosh, Srinivas, and further in view of Kristensen et al. (U.S. Pre-Grant Publication No. 2021/0286923, hereinafter 'Kristensen').
Regarding claim 14, Modarresi, as modified by Ghosh and Srinivas, teaches The method of claim 1.
Modarresi, as modified by Ghosh and Srinivas, fails to teach wherein the additional images are processed to produce a further multifactor KPI.
Kristensen teaches wherein the additional images are processed to produce a further multifactor KPI ([0027] A sensor model may include a deep neural network (DNN) with any suitable architecture, and may support generative learning. For example, a sensor model may include a generative adversarial network (GANs), a variational autoencoder (VAE), and/or another type of DNN or machine learning model. At a high level, a sensor model may accept some encoded representation of a scene configuration as an input using any number of data structures and/or channels (e.g., concatenated vectors, matrices, tensors, images, etc.).; 0127] Now referring to FIG. 8, FIG. 8 includes a data flow diagram for key performance indicator (KPI) analysis and observation, in accordance with some embodiments of the present disclosure. A the additional images are processed to produce a further multifactor KPI KPI evaluation component may evaluate the performance of the virtual object(s) (e.g., vehicles, robots, etc.). Logs 806 may be generated and passed to re-simulator/simulator 804. The re-simulator/simulator 804 may provide sensor data to the software stack(s) 116 which may be executed using HIL, SIL, or a combination thereof. The KPI evaluation component 802 may use different metrics for each simulation or re-simulation instance. For examples, for re-simulation, KPI evaluation component may provide access to the original re-played CAN data and/or the newly generated CAN data from the software stack(s) 116 (e.g., from HIL or SIL).).
Modarresi, Ghosh, Srinivas, and Kristensen are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi, Ghosh, and Srinivas, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Kristensen to Modarresi before the effective filing date of the claimed invention in order to collect real world data to derive training data used to train a sensor model (cf. Kristensen, [0005] Real-world data and/or virtual data may be collected and used to derive training data (e.g., input scene configurations and/or ground truth sensor data), which may be used to train the sensor model to predict virtual sensor data for a given scene configuration.).
Regarding claim 15, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 14.
Ghosh teaches wherein the additional images are promoted based on the further multifactor KPI ([0155] In certain embodiments, the cognitive agent governance and assurance operations may include based on the further multifactor KPI KPI-driven AI model optimization. In these embodiments, the definition of such KPIs, and the method by which they are used to optimize a particular AI model, is a matter of design choice.; [0216] In certain embodiments, the AIS impartiality assessment operation may be performed to detect the presence of bias in a particular ML model, such as the opaque model 1332, and if detected, assess its effect on the outcome of an associated cognitive computing operation.; [0271] In certain embodiments, the identified significant contrasts may be presented as a cognitive insight 1352 to the developer of the model, such that the developer can modify the opaque model 1332, the additional images are promoted training corpus 1302, the model trainer 1304, or a combination thereof, to achieve less biased results. Skilled practitioners of the art will recognize that the described presentation of identified significant contrasts as a cognitive insight 1352 provides the developer of the opaque model 1332 a basis for modifying their model, the training corpus 1302, the model trainer 1304, or a combination thereof.).
Modarresi, Ghosh, Srinivas, and Kristensen are combinable for the same rationale as set forth above with respect to claim 14.
Regarding claim 16, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 14.
Ghosh teaches wherein the additional images provide neural network training dataset bias mitigation ([0155] In certain embodiments, the cognitive agent governance and assurance operations may include KPI-driven AI model optimization. In these embodiments, the definition of such KPIs, and the method by which they are used to optimize a particular AI model, is a matter of design choice.; [0216] In certain embodiments, the AIS impartiality assessment operation may be performed to detect the presence of bias in a particular ML model, such as the opaque model 1332, and if detected, assess its effect on the outcome of an associated cognitive computing operation.; [0271] In certain embodiments, the identified significant contrasts may be presented as a cognitive insight 1352 to the developer of the model, such that the developer can modify the opaque model 1332, the additional images training corpus 1302, the model trainer 1304, or a combination thereof, to provide neural network training dataset bias mitigation achieve less biased results. Skilled practitioners of the art will recognize that the described presentation of identified significant contrasts as a cognitive insight 1352 provides the developer of the opaque model 1332 a basis for modifying their model, the training corpus 1302, the model trainer 1304, or a combination thereof.).
Modarresi, Ghosh, Srinivas, and Kristensen are combinable for the same rationale as set forth above with respect to claim 14.
Regarding claim 19, Modarresi, as modified by Ghosh and Srinivas, teaches The method of claim 1.
Modarresi, as modified by Ghosh and Srinivas, fails to teach wherein the additional images are generated using a generative adversarial network (GAN).
Kristensen teaches wherein the additional images are generated using a generative adversarial network (GAN) ([0035] Generally, an architecture for the sensor model 120 may be selected to fit the shape of the desired input and output data. Some non-limiting examples of DNNs include perceptron, feed-forward, radial basis, deep feed forward, recurrent, long/short term memory, gated recurrent unit, autoencoder, variational autoencoder, convolutional, deconvolutional, and generative adversarial, to name a few. Some DNNs like GANs may include a convolutional neural network that accepts and evaluates an input image. Moreover, some neural network architectures are designed to accept and operate on an input vector that encodes some type of input information. Further, some neural network architectures—such as using a generative adversarial network (GAN) GANs—may include multiple input channels, which may be used to accept and evaluate multiple input images and/or input vectors. Some generative techniques such as conditional the additional images are generated image synthesis may be applied to generate an output such as a photorealistic image conditioned on some input data.).
Modarresi, Ghosh, Srinivas, and Kristensen are combinable for the same rationale as set forth above with respect to claim 14.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Modarresi, in view of Ghosh, Srinivas, and further in view of Jaipuria et al. (NPL: "Deflating Dataset Bias Using Synthetic Data Augmentation", hereinafter 'Jaipuria').
Regarding claim 18, Modarresi, as modified by Ghosh and Srinivas, teaches The method of claim 1.
Modarresi, as modified by Ghosh and Srinivas, fails to teach wherein the synthetic images are generated and promoted to the production neural network based on a bias and a non-bias in the neural network training dataset.
Jaipuria teaches wherein the synthetic images are generated and promoted to the production neural network based on a bias and a non-bias in the neural network training dataset ([3.1. Revisiting “Name That Dataset”, pg. 3] We chose to apply synthetic images are generated and promoted to the production neural network data augmentation to only these two datasets as they are also used for the lane detection experiments in Section 5 with readily available sim2real data on hand. Fig. 1 compares the confusion matrices of the two classifiers, with and without synthetic data augmentation. Here, the labels 1, 2, 3, 4 and 5 denote the ApolloScape, BDD100K, CULane, Mapillary and TuSimple datasets respectively. Consistent with the motivating hypothesis H, synthetic data augmentation diffuses the strength of the diagonal indicating deflated dataset bias.; [6.2. Experiment Details, pg. 7] As with the other tasks, we train the task network with different percentages of simulated (A + S) and sim2real (A + G) data, starting from 0% to 100% and test on KITTI sequences that were not seen during training (B). We use the based on a bias and a non-bias in the neural network training dataset Root Mean Squared Error (RMSE) metric to determine the performance of the network trained on a particular sim/real or sim2real/real mix,