DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the original filing of 01/22/2024.
Claims 1-20 are pending and have been considered below.
Information Disclosure Statement
The information disclosure statement (IDS) was submitted on 03/11/2025. The information disclosure statement has been partially considered by the examiner. The prior art NPL: Angwin “Machine bias.” In Ethics of Data and Analytics. Auerbach Publication, 254-264. was not found attached.
Drawings
The drawings of 01/22/2024 are objected to under 37 CFR 1.83(a) because they contain details and features which are blurry and hard to read/comprehend (see at least Fig. 3 and 10).
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as "amended." If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either "Replacement Sheet" or "New Sheet" pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The specification is objected to because the attempt to incorporate subject matter into this application by reference to the is ineffective due to incorrect use of numbered reference referrals (see at least paragraphs 69-70, 72). The specification refers to references using bracketed numbers representing references in a reference section (see paragraphs 168-218); however, these reference numbers do not align with the submitted IDS.
The specification is further objected to because it contains an embedded hyperlink and/or other form of browser-executable code. Applicant is required to delete the embedded hyperlink and/or other form of browser-executable code; references to websites should be limited to the top-level domain name without any prefix such as http:// or other browser-executable code. See MPEP § 608.01.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2, 14 and 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 2 and 20, each of claims 2 and 20 recite the limitation: “wherein the decision-making function comprises varying degrees of human intervention”. The term “varying degrees” is a relative term which renders the claim indefinite. The term “varying degrees” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention.
Regarding claim 14, claim 14 recites the limitation “evaluating the deployed model is performed without assumptions about the deployed model”. Use of the term “without assumptions” renders the claim unclear because parent claim 1 states that the evaluation uses the deployed model (claim 1: “wherein the evaluation of the fairness criterion comprises analyzing the audit dataset using the deployed model”); using a model to analyze a dataset necessarily requires information about the model. Having information about a model, such as information regarding its deployment, can broadly be interpreted as having assumptions regarding a model. Therefore it is unclear how evaluating is performed “without assumptions”.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract idea without significantly more.
Regarding independent claims 1 and 15:
Step 1, MPEP 2106.03:
These limitations have been determined, under Step 1, to be statutory categories of invention:
A computer implemented method for measuring fairness, the method comprising [..] (claim 1)
A system comprising: a display; a computing device operably coupled to the display, wherein the computing device comprises at least one processor and memory, the memory having computer-executable instructions stored thereon that, when executed by the at least one processor, cause the at least one processor to [..] (claim 15)
Step 2A Prong One MPEP 2106.04, 2106.04(a):
These limitations represent, under Step 2A Prong One,
mental processes such as concepts that can be practically performed in the human mind, or by a human using pen and paper as a physical aid, including observations, evaluations, judgments and opinions, MPEP 2106.04(a)(2)(III), and
certain methods of organizing human activity including managing personal behavior or relationships or interactions between people, MPEP 2106.04(a)(2)(III);
for example a person analyzing large groups of people to check whether the people have been organized fairly and presenting that in a drawn chart to organize them more fairly:
[..] specifying a fairness criterion on a plurality of population groups, the fairness criterion comprising one or more fairness metrics; [..]
[..] performing an evaluation .. with respect to the fairness criterion, wherein the evaluation of the fairness criterion comprises analyzing .. to predict a respective outcome metric for each of the population groups; [..]
[..] generating a visual diagnostic diagram for facilitating an analysis of potential failures .. with respect to the specified fairness criterion [..]
Step 2A Prong Two, MPEP 2106.04(d):
These limitations represent, under Step 2A Prong Two, mere instructions to implement the abstract idea using generic computing tools, MPEP 2106.05(f):
[..] computer implemented [..] (claim 1)
[..] the system comprising: one or more processors; and a non-transitory, computer-readable medium having instructions recorded thereon that, when executed by the one or more processors, cause operations comprising: [..] (claim 15)
[..] a deployed model [..]
[..] a visual diagnostic diagram [..]
These limitations represent, under Step 2A Prong Two, mere data gathering, MPEP 2106.05:
[..] an audit dataset associated with the deployed model [..]
[..] obtaining a deployed model [..]
[..] generating a visual diagnostic diagram [..]
These limitations represent, under Step 2A Prong Two, mere instructions to apply at a high level of generality, MPEP 2106.05:
[..] the audit dataset is configured to evaluate model fidelity against one or more fairness metrics [..]
[..] wherein the evaluation .. comprises analyzing the audit dataset using the deployed model to predict a respective outcome metric .. [..]
[..] a visual diagnostic diagram for facilitating an analysis of potential failures of the deployed model with respect to the specified fairness criterion [..]
Step 2B, MPEP 2106.05:
These limitations are considered, under Step 2B, insignificant extra-solution activity as being recited at a high level of generality, MPEP 2106.05(d):
[..] computer implemented [..] (claim 1)
[..] the system comprising: one or more processors; and a non-transitory, computer-readable medium having instructions recorded thereon that, when executed by the one or more processors, cause operations comprising: [..] (claim 15)
[..] a deployed model [..]
[..] a visual diagnostic diagram [..]
These limitations are considered, under Step 2B, mere instructions to apply to obtain a solution/outcome, MPEP 2106.05(f):
[..] the audit dataset is configured to evaluate model fidelity against one or more fairness metrics [..]
[..] wherein the evaluation .. comprises analyzing the audit dataset using the deployed model to predict a respective outcome metric .. [..]
[..] a visual diagnostic diagram for facilitating an analysis of potential failures of the deployed model with respect to the specified fairness criterion [..]
These limitations are considered, under Step 2B, insignificant extra-solution activity of data gathering, MPEP 2106.05(g):
[..] an audit dataset associated with the deployed model [..]
[..] obtaining a deployed model [..]
[..] generating a visual diagnostic diagram [..]
Regarding dependent claims 2 and 20, each of these dependent claims further recite limitations wherein the decision-making function is comprising human intervention and where the deployed model is received from. The analysis incorporates the Step 1 and Step 2A Prong One analysis of each respective parent. Further, the additional limitations in each claim represent, under Step 2A Prong Two, mere instructions to implement the abstract idea using generic computing tools, MPEP 2106.05(f) and mere data gathering, MPEP 2106.05; under Step 2B, these additional limitations in each claim are considered insignificant extra-solution activity as being recited at a high level of generality, MPEP 2106.05(d), and insignificant extra-solution activity of data gathering such as selecting a particular type of data, MPEP 2106.05(g).
Regarding dependent claims 3-4, 8-9 and 18-19, each of these dependent claims further recite limitations wherein the decision-making function is unknown or arbitrary (claims 3 and 4) and that the visual diagnostic diagram is an interactive syntax tree (claims 8-9 and 18-19). The analysis incorporates the Step 1 and Step 2A Prong One analysis of each respective parent. Further, the additional limitations in each claim represent, under Step 2A Prong Two, mere instructions to implement the abstract idea using generic computing tools, MPEP 2106.05(f); under Step 2B, these additional limitations in each claim are considered insignificant extra-solution activity as being recited at a high level of generality, MPEP 2106.05(d).
Regarding dependent claims 5, 10, 11, 13-14, 17, each of these dependent claims further recite limitations wherein performing the evaluation of the deployed model comprises an additional step(s) (such as in claims 5 and 10), wherein the step of evaluating is performed in a certain way (claims 13-14 and 17), and wherein method for measuring fairness further comprises another evaluating step (claim 11). The analysis incorporates the Step 1 and Step 2A Prong One analysis of its respective parent. Further, these limitations of each claim represent, under Step 2A Prong One, mental processes such as concepts that can be practically performed in the human mind, or by a human using pen and paper as a physical aid, including observations, evaluations, judgments and opinions, MPEP 2106.04(a)(2)(III), and/or certain methods of organizing human activity including managing personal behavior or relationships or interactions between people, MPEP 2106.04(a)(2)(III). Further, in each claim, under Step 2A Prong Two and Step 2B, all limitations are part of the abstract idea.
Regarding dependent claims 6-7 and 12, each of these dependent claims further recite limitations wherein the audit dataset comprises a particular type of data (claim 6), wherein the outcome data comprises a particular type of data (claim 7), and wherein the fairness criterion is a particular type of data (claim 12). The analysis incorporates the Step 1 and Step 2A Prong One analysis of each respective parent. Further, the additional limitations in each claim represent, under Step 2A Prong Two, mere data gathering, MPEP 2106.05; under Step 2B, these additional limitations in each claim are considered insignificant extra-solution activity of data gathering such as selecting a particular type of data, MPEP 2106.05(g).
Regarding dependent claim 16, this dependent claim further recite limitations of receiving user input, evaluating based on the user input and output a revised output value. The analysis incorporates the Step 1 and Step 2A Prong One analysis of its respective parent. Further, these limitations of the claim represent, under Step 2A Prong One, mental processes such as concepts that can be practically performed in the human mind, or by a human using pen and paper as a physical aid, including observations, evaluations, judgments and opinions, MPEP 2106.04(a)(2)(III), and/or certain methods of organizing human activity including managing personal behavior or relationships or interactions between people, MPEP 2106.04(a)(2)(III). Further, this dependent claim recites wherein the input, evaluating and outputting are by the computing device displaying on a display, limitations which are considered to represent, under Step 2A Prong Two, mere instructions to implement the abstract idea using generic computing tools, MPEP 2106.05(f); under Step 2B, these additional limitations in each claim are considered insignificant extra-solution activity as being recited at a high level of generality, MPEP 2106.05(d).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-7, 10-17 and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Wexler, James, et al. "The what-if tool: Interactive probing of machine learning models." IEEE transactions on visualization and computer graphics 26.1 (2019): 56-65. [WEXLER]
Regarding claim 1, WEXLER discloses a computer implemented method for measuring fairness, the method comprising:
obtaining a deployed model and an audit dataset associated with the deployed model (page 3 3.2 Overall Design: “In TensorBoard mode, users can configure the tool through a dialog box to load a dataset from disk and to query a model being served through TensorFlow Serving.”, load a dataset for auditing and a served/deployed model), wherein the audit dataset is configured to evaluate model fidelity against one or more fairness metrics (page 3 4.1 Exploring Your Data: “users can explore their data as well as perform customizable analyses of their data and model results”, page 1 Abstract: “The What-If Tool lets practitioners test performance in hypothetical situations, analyze the importance of different data features, and visualize model behavior across multiple models and subsets of input data. It also lets practitioners measure systems according to multiple ML fairness metrics.” – evaluate model behavior with reference to fairness metrics);
specifying a fairness criterion on a plurality of population groups, the fairness criterion comprising one or more fairness metrics (page 7 4.3.3 Thresholds and Fairness Optimization Strategies: “Applying a fairness optimization strategy through the tool automatically updates the classification thresholds for each slice individually in order to satisfy a particular fairness definition.”, “In our Census example (Figure 6b), when optimizing the models’ classification thresholds for demographic parity between sexes, the thresholds for male data points is raised and the threshold for female data points is lowered. This adjustment accounts for the large imbalance in the UCI Census dataset between sexes, where men are much more likely to be labeled as high income than women.” – user can interact/specify fairness metric criterion on a plurality of population groups such as men, women, low income, high income);
performing an evaluation of the deployed model with respect to the fairness criterion, wherein the evaluation of the fairness criterion comprises analyzing the audit dataset using the deployed model to predict a respective outcome metric for each of the population groups (page 6-7 4.3 Evaluating Performance and Fairness: “The tools can be used to analyze aggregate model performance as well as compare performance on slices of data.”, “Users can slice their data into subgroups by a single feature available in the dataset, or by the intersection of two features, in order to enable intersectional performance comparisons. For example, in our UCI Census example, users could slice by both sex and race to view intersectional performance, similar to the type of analysis done in [8]. Slicing by features calculates these performance measures for subgroups, which are defined by the unique values in the selected features.”, page 7 4.3.3 Thresholds and Fairness Optimization Strategies: “Applying a fairness optimization strategy through the tool automatically updates the classification thresholds for each slice individually in order to satisfy a particular fairness definition. Again, the results can be seen across the performance table with updates to thresholds, performance metrics and visualizations.” – respective performance outcome metrics are predicted for population groups based on the selected criterion, page 8 Fig. 6); and
generating a visual diagnostic diagram for facilitating an analysis of potential failures of the deployed model with respect to the specified fairness criterion (page 8 Fig. 6).
Regarding claim 2, WEXLER discloses the computer implemented method of claim 1, wherein obtaining the deployed model further comprises receiving the deployed model from a machine learning system, the machine learning system comprising a decision-making function, wherein the decision-making function comprises varying degrees of human intervention (page 9 7. Conclusion and Directions for Future Research: “We have presented the What-If Tool (WIT), which is designed to let ML practitioners explore and probe ML models. WIT enables users to analyze ML system performance on real data and hypothetical scenarios via a graphical user interface. The set of provided visualizations allows users to see a statistical overview of input data and model results, then zoom in to perform intersectional analysis. The tool lets users experiment with hypothetical conditions, with a variety of options that range from direct editing of data points to a novel type of automatic identification of counterfactual examples. It also provides capabilities for assessing and optimizing metrics related to ML fairness, again without any coding.” – machine learning system provides models which are explored in decision making by varying human inputs).
Regarding claim 3, WEXLER discloses computer implemented method of claim 2, wherein the decision-making function is unknown (page 7 4.3.2 Cost Ratio: “When no cost ratio is specified, it defaults to 1.0, at which false positives and false negatives are considered equally undesirable.” – provides an example where decision-making function is not specified/unknown resulting in a default/arbitrary values).
Regarding claim 4, WEXLER discloses computer implemented method of claim 2, wherein the decision-making function is arbitrary (page 7 4.3.2 Cost Ratio: “When no cost ratio is specified, it defaults to 1.0, at which false positives and false negatives are considered equally undesirable.” – provides an example where decision-making function is not specified/unknown resulting in a default/arbitrary values).
Regarding claim 5, WEXLER discloses the computer implemented method of claim 1, wherein performing the evaluation of the deployed model comprises performing a sequence of estimates and generating a confidence set (page 8 Fig. 6: “Performance view of our Census example models broken down by sex, (a) showing performance when the positive classification thresholds are left at their default levels of 0.5 for each sex, and (b) showing performance when the thresholds have been set to achieve demographic parity between sexes. Each confusion matrix refers to a given model: teal for model 1 (neural network) and orange for model 2 (linear classifier). Achieving demographic party on either model requires lowering the threshold for females and raising it for males.” – figures show that as thresholds change, the confidence set estimates for accuracy also change).
Regarding claim 6, WEXLER discloses the computer implemented method of claim 1, wherein the audit dataset comprises data representing a relationship between an outcome metric and a population group (page 7 4.3.3 Thresholds and Fairness Optimization Strategies: “This adjustment accounts for the large imbalance in the UCI Census dataset between sexes, where men are much more likely to be labeled as high income than women.” – dataset represents relationships between outcome labeling and population groups).
Regarding claim 7, WEXLER discloses the computer implemented method of claim 6, wherein the outcome data comprises data representing a relationship between an outcome metric and a plurality of population groups (page 7 4.3.3 Thresholds and Fairness Optimization Strategies: “In our Census example (Figure 6b), when optimizing the models’ classification thresholds for demographic parity between sexes, the thresholds for male data points is raised and the threshold for female data points is lowered.” – outcome data represents relationships between outcome labeling and population groups).
Regarding claim 10, WEXLER discloses the computer implemented method of claim 1, wherein performing an evaluation of the deployed model comprises evaluating the deployed model based on a grammar (page 3 N3. Test hypotheticals without having access to the inner workings of a model: “WIT should treat models as black boxes to help users generate explanations for end-to-end model behavior using hypotheticals to answer questions such as “How would increasing the value of age affect a model’s prediction scores?” or “What would need to change in the data point for a different outcome?”. Hypotheticals allow users to test model performance on perturbations of data points along one or more specified dimensions.” – broadly, evaluating based on ‘grammar’).
Regarding claim 11, WEXLER discloses the computer implemented method of claim 10, further comprising evaluating the deployed model based on a user input and outputting a revised output value (page 8 Fig. 6(b) – adjusted evaluation outputs based on user input).
Regarding claim 12, WEXLER discloses the computer implemented method of claim 1, wherein the fairness criterion is predefined or dynamically varied (page 7 4.3.3 Thresholds and Fairness Optimization Strategies: “In our Census example (Figure 6b), when optimizing the models’ classification thresholds for demographic parity between sexes, the thresholds for male data points is raised and the threshold for female data points is lowered.” – dynamic adjustment of fairness criterion thresholds).
Regarding claim 13, WEXLER discloses the computer implemented method of claim 1, wherein the step of evaluating the deployed model is performed iteratively or continuously (page 1 Introduction: “WIT supports iterative “what-if” exploration through a visual interface.”).
Regarding claim 14, WEXLER discloses the computer implemented method of claim 1, wherein the step of evaluating the deployed model is performed without assumptions about the deployed model (page 3 N3. Test hypotheticals without having access to the inner workings of a model: “Without access to model internals, explanations generated using hypotheticals remain model-agnostic and generalize to perturbations.”).
Regarding claim 15, claim 15 recites limitation similar to claim 1 and is similarly rejected.
Regarding claim 16, WEXLER discloses the system of claim 15, wherein the computing device is further configured to: receive a user input, evaluate the deployed model based on the user input, and output a revised output value by the display (page 1 Introduction: “WIT supports iterative “what-if” exploration through a visual interface.”, page 8 Fig. 6(b) – adjusted evaluation outputs based on user input, page 7 4.5 Data Scaling: “standard laptop”).
Regarding claim 17, claim 17 recites limitations similar to claim 13 and is similarly rejected.
Regarding claim 20, claim 20 recites limitations similar to claim 2 and is similarly rejected.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 8-9 and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over WEXLER in view of Bucklin et al. US 2024/0193481 A1, effective filing 12/10/2021 [BUCKLIN].
Regarding claim 8, WEXLER discloses the computer implemented method of claim 1, wherein the visual diagnostic diagram can be chosen from several different interactive types (page 4 col 1: “The Datapoint Editor enables users to create custom visualizations from both the dataset and model results for a variety of different model understanding tasks.”, page 2 2.2 Flexible visualization problem: “This component allows users to create custom views of input data and model results, with an emphasis on exploring the intersection of multiple attributes.”).
WEXLER fails to explicitly disclose wherein the selected the visual diagnostic diagram is a syntax tree.
BUCKLIN discloses methods for determining fairness of machine models ([0003]). In particular, BUCKLIN discloses interacting with user interfaces for diagnosing fairness iteratively ([0133-139, 0210] Fig. 9) wherein the user interfaces include a syntax tree ([260]: “Information presented in graphical format may include, without limitation, the directed graph of tasks in a modeling procedure”). Therefore it would have been obvious to one having ordinary skill in the art and the teachings of WEXLER and BUCKLIN before them before the effective filing of the claimed invention to combine the use of a syntax tree/directed graph when presenting a visual diagnostic iterative diagram, as taught by BUCKLIN, when the visual diagnostic iterative diagram of WEXLER, yielding the predictable result of the generated visual diagnostic diagram of WEXLER and BUCKLIN including a syntax tree. One would have been motivated to make this combination in order to present additional modular information to the user allowing for easy inclusion of additional performance information, as suggested by BUCKLIN ([0260]).
Regarding claim 9, WEXLER and BUCKLIN disclose the computer implemented method of claim 8, and WEXLER further discloses wherein the syntax tree visual diagnostic diagram is interactive (page 2 2.2 Flexible visualization problem: “This component allows users to create custom views of input data and model results, with an emphasis on exploring the intersection of multiple attributes.”).
Regarding claims 18-19, claims 18-19 recite limitations similar to claims 8-9, respectively, and are similarly rejected.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ding; Haibo et al.
US 12353837 B1
CUSTOMIZABLE FRAMEWORK FOR NATURAL LANGUAGE PROCESSING EXPLAINABILITY
Luqman; Hamzah et al.
US 20200192982 A1
METHODS, COMPUTER READABLE MEDIA, AND SYSTEMS FOR MACHINE TRANSLATION BETWEEN ARABIC AND ARABIC SIGN LANGUAGE
Li; Yancheng et al.
US 20200226489 A1
FACILITATING ONLINE RESOURCE ACCESS WITH BIAS CORRECTED TRAINING DATA GENERATED FOR FAIRNESS-AWARE PREDICTIVE MODELS
Kamkar; Sean Javad et al.
US 20200302524 A1
SYSTEMS AND METHODS FOR MODEL FAIRNESS
Zhang; Yunfeng et al.
US 20210224605 A1
MITIGATING STATISTICAL BIAS IN ARTIFICIAL INTELLIGENCE MODELS
Tristan; Jean-Baptiste Frederic George et al.
US 20210374582 A1
ENHANCED TECHNIQUES FOR BIAS ANALYSIS
Castiglione; Giuseppe Marcello Antonio et al.
US 20220114399 A1
SYSTEM AND METHOD FOR MACHINE LEARNING FAIRNESS TESTING
Kamkar; Sean Javad et al.
US 20220164877 A1
SYSTEMS AND METHODS FOR GENERATING GRADIENT-BOOSTED MODELS WITH IMPROVED FAIRNESS
Lohia; Pranay Kumar et al.
US 20220237415 A1
PRIORITY-BASED, ACCURACY-CONTROLLED INDIVIDUAL FAIRNESS OF UNSTRUCTURED TEXT
Kate; Kiran A et al.
US 20220309073 A1
AUTOMATIC CONVERSION OF DATA WITHIN DATA PIPELINE
Rawat; Ambrish et al.
US 20230289573 A1
FAIRNESS ASSESSMENT FOR DEEP GENERATIVE MODELS
Lang; Ulrich et al.
US 20240202405 A1
METHOD AND SYSTEM FOR ANALYZING AND ESTABLISHING TRUST IN SYSTEMS THAT INCLUDE ARTIFICICAL INTELLIGENCE SYSTEMS
Handte Thomas et al.
WO 2022184693 A1
FIRST AND SECOND COMMUNICATION DEVICES AND METHODS
Goyal, Priya, et al. "Fairness indicators for systematic assessments of visual feature extractors." Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 2022.
Ricci Lara, María Agustina, Rodrigo Echeveste, and Enzo Ferrante. "Addressing fairness in artificial intelligence for medical imaging." nature communications 13.1 (2022): 4581.
Orphanou, Kalia, et al. "Mitigating bias in algorithmic systems—A fish-eye view." ACM Computing Surveys 55.5 (2022): 1-37.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW L TANK whose telephone number is (571)270-1692. The examiner can normally be reached Monday-Thursday 9a-6p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Ell can be reached at 571-270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW L TANK/ Primary Examiner, Art Unit 2141
Prosecution Insights