DETAILED ACTION
Response to Amendment
The amendment filed on 6 April 2026 has been entered.
Claims 1-12, 14-16, 18-26 are pending.
Claims 1, 18, 25-26 are amended.
Claims 31-33 are new.
Claim 19 is cancelled.
Claims 1-12, 14-16, 18, 20-33 will be pending.
Response to Arguments
Applicant's arguments filed on 6 April 2026 have been fully considered, but they are not persuasive.
Applicant’s remarks, regarding the rejections of claims under 35 USC 103, have been fully considered.
Applicant notes Claim 1 is currently amended, to include limitations of: "obtaining facial images ...; augmenting the neural network training dataset using additional images, wherein the additional images comprise computer-generated synthetic images generated using a generative adversarial network (GAN) that represent a specific demographic, wherein the synthetic images expand a number of images for an age band; partitioning the facial images into multiple subgroups ... ; calculating a multifactor key performance indicator (KPI) per image ... ; and promoting the neural network configuration and the neural network training dataset to a production neural network ..." (emphases added).
Applicant submits none of the cited references, alone, or in any combination, show or suggest all the elements of currently amended Claim 1.
Examiner notes Applicant’s arguments, as outlined above, are directed to newly amended claim limitations for which Examiner has not yet made a prima facie case for, rendering Applicant’s arguments moot.
Applicant notes currently amended Claim 18, it recites "wherein the synthetic images are facial images, wherein the facial images are generated and promoted to the production neural network based on a bias and a non-bias in the neural network training dataset" (emphasis added).
Applicant submits Jaipuria does not show or suggest "wherein the synthetic images are facial images", nor does it suggest "augmenting the neural network training dataset using additional images, wherein the additional images comprise computer-generated synthetic images generated using a generative adversarial network (GAN) that represent a specific demographic, wherein the synthetic images expand a number of images for an age band" as recited in currently amended parent Claim 1.
Applicant’s arguments have been considered, but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Allowable Subject Matter
The following is a statement of reasons for the indication of allowable subject matter.
The prior art does not anticipate, or render obvious as a whole, the claim limitations of Claims 5, 33 as disclosed in Applicant’s claims.
Presented with the additional limitations of Claim 5, the prior art fails to make a prima facie case
of obviousness for limitations,
“wherein the multifactor KPI comprises an F- measure, an ROC-AUC measure, a precision measure, a recall/true positive rate, a false positive rate, a total number of videos measure, a number of positive videos measure, a number of positive frames measure, or a number of negative frames measure”,
because the methods and systems for machine learning of Claim 1, shown to be found in the prior arts in the current Office Action (below), cannot be reconciled with the above limitations and an obviousness determination would require impermissible hindsight.
In summary, the references made of record, fail to disclose the required claimed technical features recited by the Claim 5 limitations as a whole. The claim, definite, and enabled by the Specification, is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Presented with the additional limitations of Claim 33, the prior art fails to make a prima facie case
of obviousness for limitations,
“wherein the multifactor key performance indicator (KPI) comprises an F1 score, wherein the F1 score is based on determining a harmonic mean of parameters associated with a test or classification, wherein the parameters include a test precision and a test recall, wherein test precision comprises a fraction of relative instances among a set of instances that are retrieved, wherein the recall associated with the test comprises a fraction of a total number of retrieved instances actually retrieved, wherein the F1 score is calculated for each emotion within the facial images and for each associated demographic category, and wherein the demographic category comprises an age group indicated by the age band”,
because the methods and systems for machine learning of Claim 3, shown to be found in the prior arts in the current Office Action (below), cannot be reconciled with the above limitations and an obviousness determination would require impermissible hindsight.
In summary, the references made of record, fail to disclose the required claimed technical features recited by the Claim 33 limitations as a whole. The claim, definite, and enabled by the Specification, is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-4, 10-12, 14-16, 25-26 are rejected under 35 U.S.C. 103 as being unpatentable over Modarresi (U.S. Pre-Grant Publication No. 2015/0213389), in view of Ghosh et al. (U.S. Pre-Grant Publication No. 2020/0293933, hereinafter 'Ghosh'), Srinivas et al. (NPL: "Age, Gender, and Fine-Grained Ethnicity Prediction using Convolutional Neural Networks for the East Asian Face Dataset", hereinafter 'Srinivas'), and further in view of Kristensen et al. (U.S. Pre-Grant Publication No. 2021/0286923, hereinafter 'Kristensen').
Regarding claim 1 and analogous claims 25, 26, Modarresi teaches A computer-implemented method for machine learning comprising ([0101] Although exemplary embodiments have been described in terms of systems and methods, it is contemplated that certain functionality described herein may be implemented in software on microprocessors, such as a processors 126 a-n and 128 included in the user devices 134 a-n and analytics server 102, respectively, shown in FIG. 1, and computing devices such as the computer system 1300 illustrated in FIG. 13.; [0102] Aspects of the present invention shown in FIGS. 1-12, or any part(s) or function(s) thereof, may be implemented using hardware, software modules, firmware, tangible computer readable media having logic or instructions stored thereon, or a combination thereof and may be implemented in one or more computer systems or other processing systems.; [0103] FIG. 13 illustrates an example computer system 1300 in which embodiments of the present invention, or portions thereof, may be implemented as computer-readable instructions or code.):
obtaining facial images for a neural network configuration and a neural network training dataset, wherein the neural network training dataset is associated with the neural network configuration ([0026] At this point, test data can be used to calculate the misfit error for each input each variable in a set of input variables. An embodiment divides data into two parts, training data and test data. According to this embodiment, the neural network training dataset is associated with the neural network configuration training data is used to formulate a model (i.e., an algorithm) and the accuracy of the algorithm is checked using the test data.; [0048] As used herein, the term “electronic content” is used to refer to any type of media that can be rendered for display, played on, or used at a computing device, television, or other electronic device. Computing devices include client and server devices such as, but not limited to, servers, desktop computers, laptop computers, smart phones, video game consoles, smart televisions, tablet computers, portable gaming devices, personal digital assistants, etc. obtaining facial images for a neural network configuration and a neural network training dataset Electronic content can include text or multimedia files, such as images, video, audio, or any combination thereof.);
calculating a multifactor key performance indicator (KPI) per image, wherein the calculating is based on analyzing performance of two or more image classifier models ([0005] One embodiment involves analyzing one or more calculating a multifactor key performance indicator (KPI) per image Key Performance Indicators (KPIs) associated with electronic content.; [0024] An embodiment uses of two or more image classifier models Classification and Regression Trees (CART) decision trees as base learners. As an individual base learner can be inaccurate (i.e., produce an inaccurate decision), embodiments combine many base learners (i.e., many decision trees) and use a form of a majority vote, which results in very accurate overall decisions. In certain embodiments, this step uses Classification and Regression Trees (CART) decision trees.; [0048] As used herein, the term “electronic content” is used to refer to any type of media that can be rendered for display, played on, or used at a computing device, television, or other electronic device. Computing devices include client and server devices such as, but not limited to, servers, desktop computers, laptop computers, smart phones, video game consoles, smart televisions, tablet computers, portable gaming devices, personal digital assistants, etc. Electronic content can include text or multimedia files, such as images, video, audio, or any combination thereof.; [0032] calculating is based on analyzing performance Embodiments disclosed herein determine KPIs by analyzing data sets with missing entries, high-dimensional data, quantitative (i.e., numerical) data, qualitative (i.e., categorical) data, and data sets including statistical outliers.); and
Modarresi fails to teach augmenting the neural network training dataset using additional images, wherein the additional images comprise computer-generated synthetic images generated using a generative adversarial network (GAN) that represent a specific demographic, wherein the synthetic images expand a number of images for an age band; partitioning the facial images into multiple subgroups, wherein the multiple subgroups represent demographics with potential for biased training; promoting the neural network configuration and the neural network training dataset to a production neural network, wherein the promoting is based on the multifactor key performance indicator.
Ghosh teaches partitioning the facial images into multiple subgroups, wherein the multiple subgroups represent demographics with potential for biased training ([0195] In certain embodiments, multi-structured big data sources 304 may be dynamically ingested during the data ingestion and processing 1204 phase. In certain embodiments, based upon a particular context, partitioning the facial images into multiple subgroups extraction, parsing, and tagging operations are performed on language, text and images they contain to generate associated datasets 1214.; [0260] In certain embodiments, the AIS impartiality assessment 1342 engine may be configured to multiple subgroups represent demographics with potential for biased training obtain bias ranges of features; compare the identified significant contrasts to obtained bias ranges, and determine which of the identified significant contrasts fall outside the obtained bias ranges.);
promoting the neural network configuration and the neural network training dataset to a production neural network, wherein the promoting is based on the multifactor key performance indicator ([0155] In certain embodiments, the cognitive agent governance and assurance operations may include KPI-driven AI model optimization. In these embodiments, the definition of such KPIs, and the method by which they are used to optimize a particular AI model, is a matter of design choice.; [0164] In turn, the results of the KPI evaluations are then used as feedback to improve the performance of the cognitive process. In certain embodiments, the promoting is based on the multifactor key performance indicator results of the KPI evaluations may be provided as input in step 904 to promoting the neural network configuration and the neural network training dataset to a production neural network determine additional operational and performance parameters related to the cognitive process. In certain embodiments, these additional operational and performance parameters may be used to repeat one or more steps associated with the lifecycle of the cognitive process to revise its functionality, improve its performance, or both.).
Modarresi and Ghosh are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Ghosh to Modarresi before the effective filing date of the claimed invention in order to ensure ML models are designed, implemented, and maintained responsibly and provide auditability of their fairness, robustness, transparency, and interpretability (cf. Ghosh, [0031] Accordingly, certain aspects of the invention likewise reflect there are ethical, moral, and social obligations for researchers, developers, and organizations alike to ensure their ML models are designed, implemented, and maintained responsibly. Various aspects of the invention likewise reflect an appreciation that one approach to achieving responsible design, implementation, and maintenance of ML models is to provide auditability of their fairness, robustness, transparency, and interpretability.).
Srinivas teaches augmenting the neural network training dataset using additional images, wherein the additional images comprise computer-generated synthetic images generated using a generative adversarial network (GAN) that represent a specific demographic ([1. Introduction, pg. 954] Convolutional neural networks are used here to predict demographics and provide a baseline performance for the WEAFD. CNNs have recently found great success in many applications relying on facial imaging, including facial alignment via landmark detection, parsing, identity recognition, attribute recognition, expression detection, and additional images comprise synthetic images that represent a specific demographic demographic estimation including age, gender, and ethnicity. The success of CNNs stem from the ability of deep neural networks to automatically discover features which discriminate and describe the underlying differences in data that are difficult to quantify. Facial data often exhibits such differences through unconstrained pose, lighting, expressions, and occlusions.; [A. Training, Validation and Testing Dataset, pg. 958] For training the age models, we limited the number of images in each class to a maximum of 500. For training the gender models, we limited the number of images in each class to a maximum of 4000. For training the finegrained ethnicity model, we limited the number of images in each class to a maximum of 500. We use augmenting the neural network training dataset using additional images data augmentation during training to increase the number of training images. Data augmentation is achieved by mirroring the image. This doubles the number of images used during training.),
wherein the synthetic images expand a number of images for an age band ([A. Training, Validation and Testing Dataset, pg. 958] We use wherein the synthetic images expand a number of images data augmentation during training to increase the number of training images. Data augmentation is achieved by mirroring the image. This doubles the number of images used during training.; [B. Metadata Collection Procedure, pg. 955] Age, gender and fine-grained ethnicity information were labeled by the Turkers. Age Estimation: Estimate the for an age band age range of the subjects in the image. The choices for age ranges are: 0-2 years, 3-5 years, 6-9 years, 10-12 years, 13-16 years, 17-23 years, 24-39 years, 40-54 years, 55-64 years, and 65+ years.);
Modarresi, Ghosh, and Srinivas are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi and Ghosh, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Srinivas to Modarresi before the effective filing date of the claimed invention in order to use the Wild East Asian Face Dataset (WEAFD) to predict age, gender and fine-grained ethnicity of an individual by providing baseline results using a convolutional neural network (CNN) (cf. Srinivas, [Abstract, pg. 953] This paper explores the difficulty of performing automatic demographic prediction on the East Asian population. We introduce the Wild East Asian Face Dataset (WEAFD), a new and unique dataset, to the research community. This dataset consists primarily of labeled face images of individuals from East Asian countries, including Vietnam, Burma, Thailand, China, Korea, Japan, Indonesia, and Malaysia. East Asian Amazon Mechanical Turk annotators were used to label the age, gender and fine grain ethnicity attributes to reduce the impact of the “other-race effect” and improve quality of annotations. We focus on predicting age, gender and fine-grained ethnicity of an individual by providing baseline results using a convolutional neural network (CNN). Fine-grained ethnicity prediction refers to predicting refined categorization of the human population (Chinese, Japanese, Korean, etc.). Performance of two CNN architectures is presented, highlighting the difficulty of these tasks and showcasing potential design considerations that improve network optimization by promoting region based feature extraction.).
Kristensen teaches augmenting the neural network training dataset using additional images, wherein the additional images comprise computer-generated synthetic images generated using a generative adversarial network (GAN) that represent a specific demographic ([0035] Generally, an architecture for the sensor model 120 may be selected to fit the shape of the desired input and output data. Some non-limiting examples of DNNs include perceptron, feed-forward, radial basis, deep feed forward, recurrent, long/short term memory, gated recurrent unit, autoencoder, variational autoencoder, convolutional, deconvolutional, and generative adversarial, to name a few. Some DNNs like GANs may include a convolutional neural network that accepts and evaluates an input image. Moreover, some neural network architectures are designed to accept and operate on an input vector that encodes some type of input information. Further, some neural network architectures—such as using a generative adversarial network (GAN) GANs—may include multiple input channels, which may be used to accept and evaluate multiple input images and/or input vectors. Some generative techniques such as conditional image synthesis may be applied to generated generate an computer-generated synthetic images output such as a photorealistic image conditioned on some input data.).
Modarresi, Ghosh, Srinivas, and Kristensen are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi, Ghosh, and Srinivas, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Kristensen to Modarresi before the effective filing date of the claimed invention in order to collect real world data to derive training data used to train a sensor model (cf. Kristensen, [0005] Real-world data and/or virtual data may be collected and used to derive training data (e.g., input scene configurations and/or ground truth sensor data), which may be used to train the sensor model to predict virtual sensor data for a given scene configuration.).
Regarding claim 2, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 1.
Ghosh teaches wherein the multifactor key performance indicator (KPI) identifies bias in the training dataset ([0155] In certain embodiments, the cognitive agent governance and assurance operations may include KPI-driven AI model optimization. In these embodiments, the definition of such multifactor key performance indicator (KPI) KPIs, and the method by which they are used to optimize a particular AI model, is a matter of design choice.; [0216] In certain embodiments, the AIS impartiality assessment operation may be performed to identifies bias in the training dataset detect the presence of bias in a particular ML model, such as the opaque model 1332, and if detected, assess its effect on the outcome of an associated cognitive computing operation.; [0164] In turn, the results of the KPI evaluations are then used as feedback to improve the performance of the cognitive process. In certain embodiments, the results of the KPI evaluations may be provided as input in step 904 to determine additional operational and performance parameters related to the cognitive process.).
Modarresi, Ghosh, Srinivas, and Kristensen are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 3, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 2.
Ghosh teaches wherein identified bias precludes promotion to the production neural network ([0216] In certain embodiments, the AIS impartiality assessment operation may be performed to identified bias detect the presence of bias in a particular ML model, such as the opaque model 1332, and if detected, assess its effect on the outcome of an associated cognitive computing operation.; [0271] In certain embodiments, the identified significant contrasts may be presented as a cognitive insight 1352 to the developer of the model, such that the developer can precludes promotion to the production neural network modify the opaque model 1332, the training corpus 1302, the model trainer 1304, or a combination thereof, to achieve less biased results. Skilled practitioners of the art will recognize that the described presentation of identified significant contrasts as a cognitive insight 1352 provides the developer of the opaque model 1332 a basis for modifying their model, the training corpus 1302, the model trainer 1304, or a combination thereof.).
Modarresi, Ghosh, Srinivas, and Kristensen are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 4, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 2.
Ghosh teaches wherein an absence of identified bias allows promotion to the production neural network ([0216] In certain embodiments, the AIS impartiality assessment operation may be performed to absence of identified bias detect the presence of bias in a particular ML model, such as the opaque model 1332, and if detected, assess its effect on the outcome of an associated cognitive computing operation.; [0271] In certain embodiments, the identified significant contrasts may be presented as a cognitive insight 1352 to the developer of the model, such that the developer can modify the opaque model 1332, the training corpus 1302, the model trainer 1304, or a combination thereof, to achieve less biased results. Skilled practitioners of the art will recognize that the described presentation of identified significant contrasts as a cognitive insight 1352 provides the developer of the opaque model 1332 allows promotion to the production neural network a basis for modifying their model, the training corpus 1302, the model trainer 1304, or a combination thereof.).
Modarresi, Ghosh, Srinivas, and Kristensen are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 10, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 1.
Modarresi teaches wherein the training dataset includes facial images ([0026] At this point, test data can be used to calculate the misfit error for each input each variable in a set of input variables. An embodiment divides data into two parts, training dataset training data and test data. According to this embodiment, the training data is used to formulate a model (i.e., an algorithm) and the accuracy of the algorithm is checked using the test data.; [0048] As used herein, the term “electronic content” is used to refer to any type of media that can be rendered for display, played on, or used at a computing device, television, or other electronic device. Computing devices include client and server devices such as, but not limited to, servers, desktop computers, laptop computers, smart phones, video game consoles, smart televisions, tablet computers, portable gaming devices, personal digital assistants, etc. Electronic content can include text or multimedia files, such as includes facial images images, video, audio, or any combination thereof.).
Modarresi, Ghosh, Srinivas, and Kristensen are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 11, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 1.
Ghosh teaches further comprising training the production neural network, using the neural network training dataset that is promoted ([0216] In certain embodiments, the AIS impartiality assessment operation may be performed to detect the presence of bias in a particular ML model, such as the opaque model 1332, and if detected, assess its effect on the outcome of an associated cognitive computing operation.; [0271] In certain embodiments, the identified significant contrasts may be presented as a cognitive insight 1352 to the developer of the model, such that the developer can modify the opaque model 1332, the training the production neural network using the neural network training dataset that is promoted training corpus 1302, the model trainer 1304, or a combination thereof, to achieve less biased results. Skilled practitioners of the art will recognize that the described presentation of identified significant contrasts as a cognitive insight 1352 provides the developer of the opaque model 1332 a basis for modifying their model, the training corpus 1302, the model trainer 1304, or a combination thereof.).
Modarresi, Ghosh, Srinivas, and Kristensen are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 12, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 11.
Ghosh teaches wherein the neural network training dataset that is promoted enables bias mitigation ([0216] In certain embodiments, the AIS impartiality assessment operation may be performed to detect the presence of bias in a particular ML model, such as the opaque model 1332, and if detected, assess its effect on the outcome of an associated cognitive computing operation.; [0271] In certain embodiments, the identified significant contrasts may be presented as a cognitive insight 1352 to the developer of the model, such that the developer can modify the opaque model 1332, the neural network training dataset that is promoted training corpus 1302, the model trainer 1304, or a combination thereof, to enables bias mitigation achieve less biased results. Skilled practitioners of the art will recognize that the described presentation of identified significant contrasts as a cognitive insight 1352 provides the developer of the opaque model 1332 a basis for modifying their model, the training corpus 1302, the model trainer 1304, or a combination thereof.).
Modarresi, Ghosh, Srinivas, and Kristensen are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 14, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 1.
Kristensen teaches wherein the additional images are processed to produce a further multifactor KPI ([0027] A sensor model may include a deep neural network (DNN) with any suitable architecture, and may support generative learning. For example, a sensor model may include a generative adversarial network (GANs), a variational autoencoder (VAE), and/or another type of DNN or machine learning model. At a high level, a sensor model may accept some encoded representation of a scene configuration as an input using any number of data structures and/or channels (e.g., concatenated vectors, matrices, tensors, images, etc.).; 0127] Now referring to FIG. 8, FIG. 8 includes a data flow diagram for key performance indicator (KPI) analysis and observation, in accordance with some embodiments of the present disclosure. A the additional images are processed to produce a further multifactor KPI KPI evaluation component may evaluate the performance of the virtual object(s) (e.g., vehicles, robots, etc.). Logs 806 may be generated and passed to re-simulator/simulator 804. The re-simulator/simulator 804 may provide sensor data to the software stack(s) 116 which may be executed using HIL, SIL, or a combination thereof. The KPI evaluation component 802 may use different metrics for each simulation or re-simulation instance. For examples, for re-simulation, KPI evaluation component may provide access to the original re-played CAN data and/or the newly generated CAN data from the software stack(s) 116 (e.g., from HIL or SIL).).
Modarresi, Ghosh, Srinivas, and Kristensen are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 15, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 14.
Ghosh teaches wherein the additional images are promoted based on the further multifactor KPI ([0155] In certain embodiments, the cognitive agent governance and assurance operations may include based on the further multifactor KPI KPI-driven AI model optimization. In these embodiments, the definition of such KPIs, and the method by which they are used to optimize a particular AI model, is a matter of design choice.; [0216] In certain embodiments, the AIS impartiality assessment operation may be performed to detect the presence of bias in a particular ML model, such as the opaque model 1332, and if detected, assess its effect on the outcome of an associated cognitive computing operation.; [0271] In certain embodiments, the identified significant contrasts may be presented as a cognitive insight 1352 to the developer of the model, such that the developer can modify the opaque model 1332, the additional images are promoted training corpus 1302, the model trainer 1304, or a combination thereof, to achieve less biased results. Skilled practitioners of the art will recognize that the described presentation of identified significant contrasts as a cognitive insight 1352 provides the developer of the opaque model 1332 a basis for modifying their model, the training corpus 1302, the model trainer 1304, or a combination thereof.).
Modarresi, Ghosh, Srinivas, and Kristensen are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 16, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 14.
Ghosh teaches wherein the additional images provide neural network training dataset bias mitigation ([0155] In certain embodiments, the cognitive agent governance and assurance operations may include KPI-driven AI model optimization. In these embodiments, the definition of such KPIs, and the method by which they are used to optimize a particular AI model, is a matter of design choice.; [0216] In certain embodiments, the AIS impartiality assessment operation may be performed to detect the presence of bias in a particular ML model, such as the opaque model 1332, and if detected, assess its effect on the outcome of an associated cognitive computing operation.; [0271] In certain embodiments, the identified significant contrasts may be presented as a cognitive insight 1352 to the developer of the model, such that the developer can modify the opaque model 1332, the additional images training corpus 1302, the model trainer 1304, or a combination thereof, to provide neural network training dataset bias mitigation achieve less biased results. Skilled practitioners of the art will recognize that the described presentation of identified significant contrasts as a cognitive insight 1352 provides the developer of the opaque model 1332 a basis for modifying their model, the training corpus 1302, the model trainer 1304, or a combination thereof.).
Modarresi, Ghosh, Srinivas, and Kristensen are combinable for the same rationale as set forth above with respect to claim 1.
Claims 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Modarresi, in view of Ghosh, Srinivas, Kristensen, and further in view of Zhang et al. (NPL: "Mitigating Unwanted Biases with Adversarial Learning", hereinafter 'Zhang').
Regarding claim 6, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 1.
Modarresi, as modified by Ghosh, Srinivas, and Kristensen, fails to teach wherein the multifactor KPI comprises an equal odds or equal opportunity measure.
Zhang teaches wherein the multifactor KPI comprises an equal odds or equal opportunity measure ([1 INTRODUCTION, pg. 335] Machine learning leverages data to build models capable of assessing the labels and properties of novel data. Unfortunately, the available training data frequently contains biases with respect to things that we would rather not use for decision making. Machine learning builds models faithful to training data and can lead to perpetuating these undesirable biases. For example, systems designed to predict creditworthiness and systems designed to perform analogy completion have been demonstrated to be biased against racial minorities and women respectively. Ideally we would be able to build a model which captures exactly those generalizations from the data which are useful for performing some task which are not discriminatory in a way which the people building those models consider unfair. Work on training machine learning systems that output fair decisions has defined several multifactor KPI useful measurements for fairness: Demographic Parity, comprises an equal odds or equal opportunity measure Equality of Odds, and Equality of Opportunity. These can be imposed as constraints or incorporated into a loss function in order to mitigate disproportional outcomes in the system’s output predictions regarding a protected demographic, such as sex.).
Modarresi, Ghosh, Srinivas, Kristensen, and Zhang are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi, Ghosh, Srinivas, and Kristensen, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Zhang to Modarresi before the effective filing date of the claimed invention in order to mitigate disproportional outcomes in the system’s output predictions regarding a protected demographic, such as sex (cf. Zhang, [1. Introduction, pg. 335] Work on training machine learning systems that output fair decisions has defined several useful measurements for fairness: Demographic Parity, Equality of Odds, and Equality of Opportunity. These can be imposed as constraints or incorporated into a loss function in order to mitigate disproportional outcomes in the system’s output predictions regarding a protected demographic, such as sex.).
Regarding claim 7, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 1.
Modarresi, as modified by Ghosh, Srinivas, and Kristensen, fails to teach wherein the multifactor KPI identifies models that generalize across one or more of the demographics.
Zhang teaches wherein the multifactor KPI identifies models that generalize across one or more of the demographics ([4 PROPERTIES, pg. 337] We note several properties of the above method that we believe distinguish it from past work. (1) multifactor KPI identifies models that generalize across one or more of the demographics Generality: The above method can be used to enforce demographic parity, eqality of odds, or eqality of opportunity as described in Hardt et al. [5]. Further, it applies without modification to the cases when the output variable and/or protected variable are continuous instead of discrete.).
Modarresi, Ghosh, Srinivas, Kristensen, and Zhang are combinable for the same rationale as set forth above with respect to claim 6.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Modarresi, in view of Ghosh, Srinivas, Kristensen, and further in view of Bist et al. (U.S. Pre-Grant Publication No. 2020/0288206, hereinafter 'Bist').
Regarding claim 8, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 1.
Modarresi, as modified by Ghosh, Srinivas, and Kristensen, fails to teach wherein the two or more image classifier models operate on the multiple subgroups of facial images.
Bist teaches wherein the two or more image classifier models operate on the multiple subgroups of facial images ([0089] In an embodiment, the present invention provides a method for evaluating media content based on combining two or more image classifier models multi-modal inputs from the participants that include operate on the multiple subgroups of facial images reactions and emotions (captured in form of facial expression) that are recorded in real-time on a frame-by-frame basis. The real time reactions and emotions may be recorded in two different steps or campaigns (with two different sets of people), and which include different participants for each.).
Modarresi, Ghosh, Srinivas, Kristensen, and Bist are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi, Ghosh, Srinivas, and Kristensen, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Bist to Modarresi before the effective filing date of the claimed invention in order to enable a way to add meaningful contextual and personalized information to content, that could then be used for searching, classifying, or analyzing the particular content in a variety of ways (cf. Bist, [0005] In light of above, a method and a system for a scalable platform is provided that enables granular tagging of any multimedia or other web content over connected networks. The method of the invention provides an ability to go in much more granular within a content and enable a way to add meaningful contextual and personalized information to it, that could then be used for searching, classifying, or analyzing the particular content in a variety of ways, and in a variety of applications.).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Modarresi, in view of Ghosh, Srinivas, Kristensen, and further in view of Rohekar et al. (U.S. Pre-Grant Publication No. 2023/0117143, hereinafter 'Rohekar').
Regarding claim 9, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 1.
Modarresi, as modified by Ghosh, Srinivas, and Kristensen, fails to teach wherein the neural network configuration includes a neural network topology.
Rohekar teaches wherein the neural network configuration includes a neural network topology ([0218] A second exemplary type of neural network is the Convolutional Neural Network (CNN). A CNN is a specialized neural network configuration includes a neural network topology feedforward neural network for processing data having a known, grid-like topology, such as image data. Accordingly, CNNs are commonly used for compute vision and image recognition applications, but they also may be used for other types of pattern recognition such as speech and language processing.).
Modarresi, Ghosh, Srinivas, Kristensen, and Rohekar are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi, Ghosh, Srinivas, and Kristensen, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Rohekar to Modarresi before the effective filing date of the claimed invention in order to compute vision, image recognition and other types of pattern recognition such as speech and language processing (cf. Rohekar, [0218] A second exemplary type of neural network is the Convolutional Neural Network (CNN). A CNN is a specialized feedforward neural network for processing data having a known, grid-like topology, such as image data. Accordingly, CNNs are commonly used for compute vision and image recognition applications, but they also may be used for other types of pattern recognition such as speech and language processing.).
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Modarresi, in view of Ghosh, Srinivas, Kristensen, and further in view of Kumar et al. (U.S. Pre-Grant Publication No. 20230214663, hereinafter 'Kumar').
Regarding claim 18, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 1.
Modarresi, as modified by Ghosh, Srinivas, and Kristensen, fails to teach wherein the synthetic images are facial images, wherein the facial images are generated and promoted to the production neural network based on a bias and a non-bias in the neural network training dataset.
Kumar teaches wherein the synthetic images are facial images, wherein the facial images are generated and promoted to the production neural network based on a bias and a non-bias in the neural network training dataset ([0035] Another example application is to generate new synthetic data for underrepresented attributes. For example, a GAN trained to generate generic outputs can be re-trained on a small attribute-specific dataset to make personalized or outputs that exhibit the specific attribute. As one example, this process can be used to extend facial attributes to underrepresented ones, e.g., to generate more wherein the synthetic images are facial images face images with an attribute (e.g., red hair color) that was underrepresented in the original, larger training set. This example use also has implications on and a non-bias fairness or addressing the based on a bias bias in the neural network training dataset in the training set. For example, if a particular attribute is underrepresented in the training set, the method can be used to wherein the facial images are generated and promoted to the production neural network generate more synthetic images for that underrepresented attribute, thereby reducing the bias of the training set. This approach can also be used for creating a balanced or fair evaluation set to provide metrics that cover a desired distribution well.).
Modarresi, Ghosh, Srinivas, Kristensen, and Kumar are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi, Ghosh, Srinivas, and Kristensen, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Kumar to Modarresi before the effective filing date of the claimed invention in order to improve methods for learning a generative model with limited training data, mitigating data requirement (cf. Kumar, [0022] The present disclosure proposes improved methods for learning a generative model with limited training data. In particular, example implementations of the present disclosure leverage a pre-trained GAN model from a related domain and adapt it to the new domain given a set of target examples from the new or target domain. As one example, additional parameters can be added to the pre-trained GAN model and the model can be re-trained on the set of target examples from the target domain. Thus, to mitigate the data requirement, the present disclosure provides systems and methods (some of which may be referred to as “FewShotGAN”) to generate images of a new concept in a few-shot setting.).
Claims 20-24, 29-30 are rejected under 35 U.S.C. 103 as being unpatentable over Modarresi, in view of Ghosh, Srinivas, Kristensen, and further in view of Wilf et al. (U.S. Pre-Grant Publication No. 2019/0005359, hereinafter 'Wilf').
Regarding claim 20, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 1.
Modarresi, as modified by Ghosh, Srinivas, and Kristensen, fails to teach wherein the additional images comprise real images from a specific demographic.
Wilf teaches wherein the additional images comprise real images from a specific demographic ([0030] According to an embodiment of the invention, the process of the face-based personality analysis module may further comprise one or more additional tasks such as: searching for additional images of the subject person by using a name search engine that can be augmented by face recognition by face recognition and analyzing said additional images to enhance the accuracy of predicted personality traits or capabilities.; [0170] Face-based demographic classification (gender, age, ethnicity) is known in prior art. The idea is that improved personality trait/behavior/capability classification may benefit from demographic segmentation. For example, the system additional images comprise real images from a specific demographic collects images of male researchers and female researchers, doing the same with a control group, which in this context may comprise of people that are known not to be researchers. Now we train a classifier for male researcher and one for female researcher (and of course verify during development that we benefit from such segmentation).).
Modarresi, Ghosh, Srinivas, Kristensen, and Wilf are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi, Ghosh, Srinivas, and Kristensen, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Wilf to Modarresi before the effective filing date of the claimed invention in order to predict personality traits based on automated computerized or computer assisted analysis of that person's body images and in particular face images (cf. Wilf, [0006] It is an object of the present invention to provide a system which is capable of predicting personality traits based on automated computerized or computer assisted analysis of that person's body images and in particular face images.; [0007] It is another an object of the present invention to provide an automated method of selecting personality traits and capabilities that can be predicted from face images and predicting such traits and capabilities from one or more face images.).
Regarding claim 21, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 1.
Modarresi, as modified by Ghosh, Srinivas, and Kristensen, fails to teach wherein the additional images comprise real images containing a specific facial characteristic.
Wilf teaches wherein the additional images comprise real images containing a specific facial characteristic ([0030] According to an embodiment of the invention, the process of the face-based personality analysis module may further comprise one or more additional tasks such as: searching for additional images of the subject person by using a name search engine that can be augmented by face recognition by face recognition and analyzing said additional images to enhance the accuracy of predicted personality traits or capabilities.; [0110] Face Landmark Detection module 120 performs geometric rectification and/or frontalization of face images, e.g., by searching for additional images comprise real images containing a specific facial characteristic specific facial key points that are useful in verifying the face pose and later converting the face image into a standard, normalized representation that may represent a frontal pose, and may have an essentially neutral expression. Such key points customarily include the eyes, eye corners, eyebrows, the nose top, the mouth, the chin, etc. as indicated by the black dots on the face image in FIG. 5. Higher level of detail may include dozens of such points which may be used as an image descriptor for the learning and prediction steps.).
Modarresi, Ghosh, Srinivas, Kristensen, and Wilf are combinable for the same rationale as set forth above with respect to claim 20.
Regarding claim 22, Modarresi, as modified by Ghosh, Srinivas, Kristensen, and Wilf, teaches The method of claim 21.
Wilf teaches wherein the specific facial characteristic includes facial expressions ([0030] According to an embodiment of the invention, the process of the face-based personality analysis module may further comprise one or more additional tasks such as: searching for additional images of the subject person by using a name search engine that can be augmented by face recognition by face recognition and analyzing said additional images to enhance the accuracy of predicted personality traits or capabilities.; [0132] specific facial characteristic includes facial expressions Face expression analysis modules 240 further selects face images of neutral expression, in order to avoid biased results of face personality analysis due to extreme expressions. Such expression analysis can be implemented by using techniques such as disclosed by B. Fasel and J. Luettin, Automatic Facial Expression Analysis: A Survey (1999), Pattern Recognition, 36, pp. 259-275, 1999.).
Modarresi, Ghosh, Srinivas, Kristensen, and Wilf are combinable for the same rationale as set forth above with respect to claim 20.
Regarding claim 23, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 1.
Modarresi, as modified by Ghosh, Srinivas, and Kristensen, fails to teach wherein the additional images comprise real images containing a specific image characteristic.
Wilf teaches wherein the additional images comprise real images containing a specific image characteristic ([0030] According to an embodiment of the invention, the process of the face-based personality analysis module may further comprise one or more additional tasks such as: searching for additional images of the subject person by using a name search engine that can be augmented by face recognition by face recognition and analyzing said additional images to enhance the accuracy of predicted personality traits or capabilities.; [0111] Face images are captured in multiple poses and expressions. To facilitate representation, learning and classification we assume that all additional images comprise real images containing a specific image characteristic images are either full frontal or side profile images. In most situations, a large number of available images will allow selecting such images for training and prediction, where such selection can be manual or automatic, using prior art techniques of pose classification.).
Modarresi, Ghosh, Srinivas, Kristensen, and Wilf are combinable for the same rationale as set forth above with respect to claim 20.
Regarding claim 24, Modarresi, as modified by Ghosh, Srinivas, Kristensen, and Wilf, teaches The method of claim 23.
Wilf teaches wherein the image characteristic includes lighting, focus, facial orientation, or resolution ([0111] Face images are captured in multiple poses and expressions. To facilitate representation, learning and classification we assume that all images are either full frontal or side profile images. In most situations, a large number of available images will allow selecting such images for training and prediction, where such selection can be manual or automatic, using prior art techniques of pose classification.; [0130] These similarity-filtered search results are further analyzed to select images of high quality, of neutral expression and of facial orientation appropriate pose (for example full frontal images and side profile images). Face quality module 230 uses quality metrics such as face size, face focus image sharpness to select face select images of resolution good quality.; [0143] SIFT=Scale Invariant Feature Transform image characteristic includes extracts from an image a collection of feature vectors, each of which is invariant to image translation, scaling, and rotation, partially invariant to lighting illumination changes and robust to local geometric distortion.; [0214] A face-based personality analysis module generates description of personality trait as described in the present invention. Face images are captured from video during the call itself and optionally stored for later interactions. Additionally, a name search engine, optionally augmented by face recognition, as described in FIG. 2 hereinabove, brings in additional pictures of the same person, with better quality, controlled pose, illumination etc.; [0125] FIG. 2 describes the face gathering module 110 in further details, according to an embodiment of the invention. Image search engines 210 such as Google Images generate multiple search results upon a name search. Certain filters in the search engine allow returning only images with faces, only images larger than a certain size, etc.).
Modarresi, Ghosh, Srinivas, Kristensen, and Wilf are combinable for the same rationale as set forth above with respect to claim 20.
Regarding claim 29, Modarresi, as modified by Ghosh, Srinivas, Kristensen, and Wilf, teaches The method of claim 21.
Srinivas teaches wherein the specific facial characteristic includes facial markings ([A. Training, Validation and Testing Dataset, pg. 958] We use data augmentation during training to increase the number of training images. Data augmentation is achieved by mirroring the image. This doubles the number of images used during training.; [1. Introduction, pg. 954] Facial data often exhibits such differences through unconstrained pose, lighting, expressions, and occlusions. When combined with differences in age, hair, skin color, and includes facial markings facial marks, it is clear that models trained on this data need to be invariant to such changes and generalize well across facial characteristics.; [B. Metadata Collection Procedure, pg. 955] Age, gender and fine-grained ethnicity information were labeled by the Turkers. Phase 3: In Phase 3, only images labeled as East and West Asians were considered for further labeling. All registered turkers irrespective of their nationality could participate. Turkers labeled information corresponding to wherein the specific facial characteristic soft biometrics such as facial expressions, the presence of facial hair (e.g. beard, stubble, mustache, etc.), head hair color, eyebrow appearance, and image lighting. The dataset exhibits a wide variation in unconstrained pose, lighting, expression, and occlusions as shown in figures 1 and 2. The distribution of subjects in WEAFD for age, gender and fine-grained ethnicity is given in Table I.).
Modarresi, Ghosh, Srinivas, Kristensen, and Wilf are combinable for the same rationale as set forth above with respect to claim 20.
Regarding claim 30, Modarresi, as modified by Ghosh, Srinivas, Kristensen, and Wilf, teaches The method of claim 21.
Srinivas teaches wherein the specific facial characteristic includes one of lighting, focus, facial orientation, or resolution ([A. Training, Validation and Testing Dataset, pg. 958] We use data augmentation during training to increase the number of training images. Data augmentation is achieved by mirroring the image. This doubles the number of images used during training.; [1. Introduction, pg. 954] Facial data often exhibits such differences through unconstrained pose, lighting, expressions, and occlusions. When combined with differences in age, hair, skin color, and facial marks, it is clear that models trained on this data need to be invariant to such changes and generalize well across facial characteristics.; [B. Metadata Collection Procedure, pg. 955] Age, gender and fine-grained ethnicity information were labeled by the Turkers. Phase 3: In Phase 3, only images labeled as East and West Asians were considered for further labeling. All registered turkers irrespective of their nationality could participate. Turkers labeled information corresponding to specific facial characteristic includes one of soft biometrics such as facial expressions, the presence of facial hair (e.g. beard, stubble, mustache, etc.), head hair color, eyebrow appearance, and image lighting. The dataset exhibits a wide variation in facial orientation unconstrained pose, lighting lighting, expression, and occlusions as shown in figures 1 and 2. The distribution of subjects in WEAFD for age, gender and fine-grained ethnicity is given in Table I.).
Modarresi, Ghosh, Srinivas, Kristensen, and Wilf are combinable for the same rationale as set forth above with respect to claim 20.
Claims 27-28 are rejected under 35 U.S.C. 103 as being unpatentable over Modarresi, in view of Ghosh, Srinivas, Kristensen, and further in view of Chun et al. (NPL: "NADS-Net: A Nimble Architecture for Driver and Seat Belt Detection via Convolutional Neural Networks", hereinafter 'Chun').
Regarding claim 27, Modarresi, as modified by Ghosh, Srinivas, and Kristensen, teaches The method of claim 1.
Modarresi, as modified by Ghosh, Srinivas, and Kristensen, fails to teach wherein the demographic includes a seat location.
Chun teaches wherein the demographic includes a seat location ([Data collection] We collected videos of drivers and passengers in a Volvo XC90 research vehicle through on-road driving studies. Over 7 months ranging from Spring to Winter, the total of 100 subjects consented to participate in the study in compliance to the internal review board (IRB) requirements.; [Statistics] It should be noted that all driving sessions were accompanied by a research staff as a safety protocol and, thus, the videos contain some repeated appearances of a few research staffs. To minimize the potential bias in the data, the researchers rotated the duty across the driving sessions. By the safety requirement, the researchers had to sit on the front passenger seat when the vehicle was in motion, but while the vehicle was at park, they moved around to wherein the demographic includes a seat location different seat positions as much as possible to minimize the data bias. Moreover, researchers were asked to wear different clothing and accessories each time.).
Modarresi, Ghosh, Srinivas, Kristensen, and Chun are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi, Ghosh, Srinivas, and Kristensen, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Chun to Modarresi before the effective filing date of the claimed invention in order to provide meaningful insights for the autonomous driving research community and automotive industry for future algorithm development and data collection (cf. Chun, [Abstract] A new convolutional neural network (CNN) architecture for 2D driver/passenger pose estimation and seat belt detection is proposed in this paper. The new architecture is more nimble and thus more suitable for in-vehicle monitoring tasks compared to other generic pose estimation algorithms. The new architecture, named NADS-Net, utilizes the feature pyramid network (FPN) backbone with multiple detection heads to achieve the optimal performance for driver/passenger state detection tasks. The new architecture is validated on a new data set containing video clips of 100 drivers in 50 driving sessions that are collected for this study. The detection performance is analyzed under different demographic, appearance, and illumination conditions. The results presented in this paper may provide meaningful insights for the autonomous driving research community and automotive industry for future algorithm development and data collection.).
Regarding claim 28, Modarresi, as modified by Ghosh, Srinivas, Kristensen, and Chun, teaches The method of claim 27.
Chun teaches wherein the seat location includes a seat location within a vehicle ([Data collection] We collected videos of drivers and passengers in a Volvo XC90 research includes a seat location within a vehicle vehicle through on-road driving studies. Over 7 months ranging from Spring to Winter, the total of 100 subjects consented to participate in the study in compliance to the internal review board (IRB) requirements.; [Statistics] It should be noted that all driving sessions were accompanied by a research staff as a safety protocol and, thus, the videos contain some repeated appearances of a few research staffs. To minimize the potential bias in the data, the researchers rotated the duty across the driving sessions. By the safety requirement, the researchers had to sit on the front passenger seat when the vehicle was in motion, but while the vehicle was at park, they moved around to wherein the seat location different seat positions as much as possible to minimize the data bias. Moreover, researchers were asked to wear different clothing and accessories each time.).
Modarresi, Ghosh, Srinivas, Kristensen, and Chun are combinable for the same rationale as set forth above with respect to claim 27.
Claims 31-32 are rejected under 35 U.S.C. 103 as being unpatentable over Modarresi, in view of Ghosh, Srinivas, Kristensen, Wilf, and further in view of Wang et al. (U.S. Pre-Grant Publication No. 20210019541, hereinafter 'Wang').
Regarding claim 31, Modarresi, as modified by Ghosh, Srinivas, Kristensen, and Wilf, teaches The method of claim 29.
Modarresi, as modified by Ghosh, Srinivas, Kristensen, and Wilf, fails to teach wherein the specific facial characteristic further includes a facial marking comprising a scar.
Wang teaches wherein the specific facial characteristic further includes a facial marking comprising a scar ([0049] The compute components 110 can also implement the discriminators 124. The discriminators 124 can be used to distinguish images, including images in a same or different visual attribute domain. Visual attribute domains can refer to wherein the specific facial characteristic categories or domains of visual features or attributes found on facial images. A facial image can refer to an image that captures or depicts a user's face and/or facial features. Non-limiting examples of visual attribute domains can include an eye glasses domain, a hair domain, a facial expression domain, a skin characteristics domain (e.g., makeup, tattoos, wrinkles, swelling, further includes a facial marking comprising a scar scars, bruising, redness, etc.), a contact lens domain, a facial hair domain (e.g., beard, mustache, eyebrows, etc.), a clothing or garment domain (e.g., hats, scarves, etc.), a specific background domain (e.g., color background, outdoors background, indoors background, daytime background, nighttime background, cluttered background, etc.), a noise domain, an occlusion domain, a brightness or color domain, a facial distortion domain (e.g., periorbital puffiness or swelling, facial edema, facial injury, dental conditions, etc.), and so forth.).
Modarresi, Ghosh, Srinivas, Kristensen, Wilf, and Wang are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Modarresi, Ghosh, Srinivas, Kristensen, and Wilf, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Wang to Modarresi before the effective filing date of the claimed invention in order to allow visual attributes to be transferred to, or removed from, specific images used for face verification, enhancing the robustness of the face verification system (cf. Wang, [0036] In some examples, the approaches herein provide a framework that allows visual attributes to be transferred to, or removed from, specific images used for face verification. This transfer of visual attributes can provide consistency of visual attributes in both probe and reference images. In some cases, such consistency can be implemented by adding one or more visual attributes from the probe image to the corresponding reference image, or removing from the probe image one or more visual attributes that are not included in the reference image. Moreover, to avoid changing facial identity features when transferring visual attributes to an image or removing visual attributes from an image, the approaches herein can implement a face identifier that keeps face identity information unchanged after visual attribute transfers or modifications. The combination of visual attribute consistency and identity maintenance can enhance the robustness of the face verification system.).
Regarding claim 32, Modarresi, as modified by Ghosh, Srinivas, Kristensen, and Wilf, teaches The method of claim 29.
Modarresi, as modified by Ghosh, Srinivas, Kristensen, and Wilf, fails to teach wherein the specific facial characteristic further includes a facial marking comprising a tattoo.
Wang teaches wherein the specific facial characteristic further includes a facial marking comprising a tattoo ([0049] The compute components 110 can also implement the discriminators 124. The discriminators 124 can be used to distinguish images, including images in a same or different visual attribute domain. Visual attribute domains can refer to wherein the specific facial characteristic categories or domains of visual features or attributes found on facial images. A facial image can refer to an image that captures or depicts a user's face and/or facial features. Non-limiting examples of visual attribute domains can include an eye glasses domain, a hair domain, a facial expression domain, a skin characteristics domain (e.g., makeup, further includes a facial marking comprising a tattoo tattoos, wrinkles, swelling, scars, bruising, redness, etc.), a contact lens domain, a facial hair domain (e.g., beard, mustache, eyebrows, etc.), a clothing or garment domain (e.g., hats, scarves, etc.), a specific background domain (e.g., color background, outdoors background, indoors background, daytime background, nighttime background, cluttered background, etc.), a noise domain, an occlusion domain, a brightness or color domain, a facial distortion domain (e.g., periorbital puffiness or swelling, facial edema, facial injury, dental conditions, etc.), and so forth.).
Modarresi, Ghosh, Srinivas, Kristensen, Wilf, and Wang are combinable for the same rationale as set forth above with respect to claim 31.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Orlandi et al. (U.S. Pre-Grant Publication No. 20220072425) teaches determination and measurements of classifier performance.
Ma et al. (NPL: “ElderReact: A Multimodal Dataset for Recognizing Emotional Response in Aging Adults”) teaches multimodal dataset of elders reacting to emotion elicitation stimuli.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAGGIE MAIDO whose telephone number is (703) 756-1953. The examiner can normally be reached M-Th: 6am - 4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached on (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MM/Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights