Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1, 3-15, and 17-25 are presented for examination.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 11-13 and 24-25 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Claim 11 recites the limitation “the iterative generating” in “executing the updated DevOps pipeline script concurrent with the iterative generating.” There is insufficient antecedent basis for this limitation in the claim. Claim 11 recites “iteratively updating a DevOps pipeline script,” but does not recite any “iteratively generating” or “iterative generating” step to which “the iterative generating” can refer. For purposes of examination, “the iterative generating” is interpreted as referring to “the iteratively updating” recited earlier in the claim. Applicant may overcome this rejection by amending “concurrent with the iterative generating” to recite “concurrent with the iteratively updating.”
Claim 24 recites the limitation “the iterative generation” in “execute the updated DevOps pipeline script concurrent with the iterative generation.” There is insufficient antecedent basis for this limitation in the claim. Claim 24 recites “iteratively update a DevOps pipeline script,” but does not recite any “iteratively generate” or “iterative generation” step to which “the iterative generation” can refer. For purposes of examination, “the iterative generation” is interpreted as referring to “the iteratively update” recited earlier in the claim. Applicant may overcome this rejection by amending “concurrent with the iterative generation” to recite “concurrent with the iterative update.”
Claim 24 further recites “identify a constraint that mitigates the new ethical concern by map the intents to corresponding actions using a database of correspondences between known intents and actions.” The phrase “by map the intents to corresponding actions” is grammatically incomplete and renders the metes and bounds of the claim unclear. Applicant may overcome this rejection by amending “by map the intents” to recite “by mapping the intents.”
Dependent claims 12, 13 and 25 are also rejected under 35 U.S.C. 112(b) as being indefinite for failing to cure the deficiencies of their independent claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3-15, and 17-25 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Claims 1, 14 and 15 as drafted, recite a process that, under its broadest reasonable interpretation, covers steps that could reasonably be performed in the mind, including with the aid of pen and paper, but for the recitation of generic computer components. That is, the limitation “determining an ethical concern relating to the development pipeline by: … identifying precepts from the processed input information sources; and expressing the identified precepts as intents using a domain model; identifying a constraint that mitigates the ethical concern by mapping the intents to corresponding actions using a database of correspondences between known intents and actions; and modifying one or more automated processes within the development pipeline based on the identified constraint” as drafted, is a process that, under its broadest reasonable interpretation, recites the abstract idea of mental processes. These limitations encompass a human mind carrying out these functions through observation, evaluation, judgment and/or opinion, or even with the aid of pen and paper. Thus, these limitations recite and fall within the “Mental Processes” grouping of abstract ideas.
This judicial exception is not integrated into a practical application. The claims recite the following additional elements “a computer readable storage medium,” “a hardware processor,” “a system,” “a memory that stores a computer program,” “iteratively generating a development pipeline,” “wherein the development pipeline comprises a DevOps pipeline script,” “wherein the constraint is an entry in the DevOps pipeline script that performs a check to ensure the ethical concern is not triggered during an execution of the development pipeline,” “executing the development pipeline concurrent with the iterative generating, wherein the executing of the development pipeline comprises executing the modified one or more automated processes,” and “obtaining processed input information sources using a miner.” The additional elements “a computer readable storage medium,” “a hardware processor,” “a system,” “a memory that stores a computer program,” “iteratively generating a development pipeline,” “wherein the development pipeline comprises a DevOps pipeline script,” “wherein the constraint is an entry in the DevOps pipeline script that performs a check to ensure the ethical concern is not triggered during an execution of the development pipeline,” and “executing the development pipeline concurrent with the iterative generating, wherein the executing of the development pipeline comprises executing the modified one or more automated processes” are merely instructions to implement an abstract idea on a computer, or merely using a generic computer or computer components as a tool to perform the abstract idea. See MPEP 2106.05(f). The additional element “obtaining processed input information sources using a miner” does nothing more than add insignificant extra-solution activity to the judicial exception, such as data gathering, to perform a task. See MPEP 2106.05(g). Accordingly, the additional elements recited in the claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea, and thus fail to integrate the abstract idea into a practical application.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements “a computer readable storage medium,” “a hardware processor,” “a system,” “a memory that stores a computer program,” “iteratively generating a development pipeline,” “wherein the development pipeline comprises a DevOps pipeline script,” “wherein the constraint is an entry in the DevOps pipeline script that performs a check,” and “executing the development pipeline concurrent with the iterative generating” are generic computer components and instructions used as the tools to perform the abstract idea. See MPEP 2106.05(f). As to the additional element “obtaining processed input information sources using a miner,” the courts have identified that gathering data is well-understood, routine, and conventional activity. See MPEP 2106.05(d). Accordingly, the additional elements recited in the claims cannot provide an inventive concept. Thus, the claims are not patent eligible.
Claims 3 and 17 further define “the DevOps pipeline script includes instructions to build and deploy a project” which is merely instructions to implement an abstract idea on a computer, or merely using a generic computer or computer components as a tool to perform the abstract idea. See MPEP 2106.05(f). Accordingly, the additional element does not integrate the abstract idea into a practical application and does not provide an inventive concept. Thus, the claims are not patent eligible.
Claims 4 and 18 further define the “determining” function set forth in the claims from which they depend, and are thus also considered to recite a mental process that can be reasonably carried out through observation, evaluation, judgment and/or opinion, or even with the aid of pen and paper.
Claims 5 and 19 recite the additional element “the executing of the development pipeline includes executing a new development pipeline based on adding the identified constraint for the new ethical concern” which is merely instructions to implement an abstract idea on a computer, or merely using a generic computer or computer components as a tool to perform the abstract idea. See MPEP 2106.05(f). Accordingly, the additional element does not integrate the abstract idea into a practical application and does not provide an inventive concept. Thus, the claims are not patent eligible.
Claims 6 and 20 further define “the development pipeline develops and deploys a machine learning model” which is merely instructions to implement an abstract idea on a computer, or merely using a generic computer or computer components as a tool to perform the abstract idea. See MPEP 2106.05(f). Accordingly, the additional element does not integrate the abstract idea into a practical application and does not provide an inventive concept. Thus, the claims are not patent eligible.
Claims 7 and 21 further define the “ethical concern” as part of the “determining” function set forth in the claims from which they depend, and are thus also considered to recite a mental process that can be reasonably carried out through observation, evaluation, judgment and/or opinion, or even with the aid of pen and paper.
Claims 8 and 22 further define the “constraint” as part of the “identifying” function set forth in the claims from which they depend, and are thus also considered to recite a mental process that can be reasonably carried out through observation, evaluation, judgment and/or opinion, or even with the aid of pen and paper.
Claim 9 further defines the “determining” function set forth in the claim from which it depends, and is thus also considered to recite a mental process that can be reasonably carried out through observation, evaluation, judgment and/or opinion, or even with the aid of pen and paper.
Claim 10 further defines the “identifying” function set forth in the claim from which it depends, and is thus also considered to recite a mental process that can be reasonably carried out through observation, evaluation, judgment and/or opinion, or even with the aid of pen and paper.
Claims 11 and 24 as drafted, recite a process that, under its broadest reasonable interpretation, covers steps that could reasonably be performed in the mind, including with the aid of pen and paper, but for the recitation of generic computer components. That is, the limitation “determining a new ethical concern that was not known at an initial execution of the DevOps pipeline script, the new ethical concern relating to a bias in training data for the machine learning model … identifying precepts from the processed input information sources; and expressing the identified precepts as intents using a domain model; identifying a constraint that mitigates the new ethical concern by mapping the intents to corresponding actions using a database of correspondences between known intents and actions” as drafted, is a process that, under its broadest reasonable interpretation, recites the abstract idea of mental processes. These limitations encompass a human mind carrying out these functions through observation, evaluation, judgment and/or opinion, or even with the aid of pen and paper. Thus, these limitations recite and fall within the “Mental Processes” grouping of abstract ideas.
This judicial exception is not integrated into a practical application. The claims recite the following additional elements “a hardware processor,” “a memory that stores a computer program,” “iteratively updating a DevOps pipeline script that includes instructions to build and deploy a machine learning model,” “modifying one or more automated processes within the DevOps pipeline script based on adding the identified constraint as an entry in the DevOps pipeline script that performs a check to ensure the new ethical concern is not triggered during an execution of the DevOps pipeline script,” “executing the updated DevOps pipeline script concurrent with the iterative generating, wherein the executing of the DevOps pipeline script comprises executing the modified one or more automated processes,” and “obtaining processed input information sources using a miner.” The generic computer components and instructions are merely instructions to implement the abstract idea on a computer, or merely using a generic computer or computer components as a tool to perform the abstract idea. See MPEP 2106.05(f). The additional element “obtaining processed input information sources using a miner” does nothing more than add insignificant extra-solution activity to the judicial exception, such as data gathering, to perform a task. See MPEP 2106.05(g). Accordingly, the additional elements recited in the claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea, and thus fail to integrate the abstract idea into a practical application.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above, the generic computer components and instructions are used as the tools to perform the abstract idea. See MPEP 2106.05(f). As to the additional element “obtaining processed input information sources using a miner,” the courts have identified that gathering data is well-understood, routine, and conventional activity. See MPEP 2106.05(d). Accordingly, the additional elements recited in the claims cannot provide an inventive concept. Thus, the claims are not patent eligible.
Claims 12 and 25 further define the “constraint” as part of the “identifying” function set forth in the claims from which they depend, and are thus also considered to recite a mental process that can be reasonably carried out through observation, evaluation, judgment and/or opinion, or even with the aid of pen and paper.
Claim 13 further defines the “determining” function set forth in the claim from which it depends, and is thus also considered to recite a mental process that can be reasonably carried out through observation, evaluation, judgment and/or opinion, or even with the aid of pen and paper.
Claim 23 recites the additional element “identify changes in contextual information relating to the development pipeline from one or more artifacts” which does nothing more than add insignificant extra-solution activity to the judicial exception, such as data gathering, to perform a task. See MPEP 2106.05(g). Further, the courts have identified that gathering data is well-understood, routine, and conventional activity. See MPEP 2106.05(d). Accordingly, the additional element does not integrate the abstract idea into a practical application and does not provide an inventive concept. Thus, the claim is not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-6, 9-11, 13-15, 17-20, and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Jagannath (US 20180032322 A1) in view of Zhang (US 11586849 B2) and further in view of Gruber (US 20120016678 A1).
Regarding Claim 1, Jagannath (US 20180032322 A1) teaches:
A computer-implemented method for development pipeline generation, the computer-implemented method comprising:
iteratively generating a development pipeline, including: (Para 0010, "DevOps solutions allow enterprises to quickly design, build, test, deploy, and maintain software applications. DevOps solutions accomplish this by facilitating continuous deployment and release pipeline management, resulting in faster release lifecycles without compromising application quality."; Para 0017, "The DevOps toolchain may include a series of lifecycle stages that aid in the development, deployment, and management of an application through the application’s lifecycle. Each DevOps lifecycle stage may have an associated set of users, user privileges, tasks, DevOps policies, and environments relevant for that stage.") Examiner Comments: Jagannath teaches iteratively generating a development pipeline through repeated DevOps lifecycle stages that develop, deploy, and manage an application over multiple iterations, directly mapping to iteratively generating a development pipeline.
wherein the development pipeline comprises a DevOps pipeline script; (Para 0013, "The DevOps application deployment packages may be provided deployment tool plugins associated with the determined application deployment tools and the deployment tool plugins may execute deployment operations based on deployment properties included in the DevOps application deployment packages to deploy DevOps applications using the determined application deployment tools."; Para 0017, "The DevOps toolchain may include a series of lifecycle stages that aid in the development, deployment, and management of an application through the application’s lifecycle. Each DevOps lifecycle stage may have an associated set of users, user privileges, tasks, DevOps policies, and environments relevant for that stage.") Examiner Comments: Jagannath teaches the development pipeline as a DevOps toolchain of lifecycle stages and deployment packages executed by deployment tool plugins, mapping to the development pipeline comprising a DevOps pipeline script.
modifying one or more automated processes within the development pipeline based on the identified constraint (Para 0035, "redeploying a DevOps application (i.e., removing an application from DevOps application deployment environment 130 and deploying the application again in DevOps application deployment environment 130)."; Para 0036, "Deployment tool plugins 113 may execute deployment operations based on deployment properties included DevOps application deployment packages.") Examiner Comments: Jagannath teaches modifying the automated processes of the pipeline by redeploying the application and by having the deployment tool plugins execute deployment operations based on the deployment properties into which the identified constraint is incorporated.
executing the development pipeline concurrent with the iterative generating, wherein the executing of the development pipeline comprises executing the modified one or more automated processes. (Para 0040, "In some examples, steps of method 300 may be executed substantially concurrently or in a different order than shown in FIG. 3. In some examples, some of the steps of method 300 may, at certain times, be ongoing and/or may repeat."; Para 0035, "Deployment tool plugins 113 may execute deployment operations to deploy DevOps applications in DevOps application deployment environment 130 using application deployment tools 120.") Examiner Comments: Jagannath teaches executing the pipeline concurrent with the iterative generating by running the deployment operations, which are ongoing and repeat, while the lifecycle iterations continue.
Jagannath did not specifically teach
determining an ethical concern relating to the development pipeline; wherein the constraint is an entry in the DevOps pipeline script that performs a check to ensure the ethical concern is not triggered during an execution of the development pipeline; and that the modifying of the one or more automated processes is based on a constraint that mitigates an ethical concern
obtaining processed input information sources using a miner;
identifying precepts from the processed input information sources;
and expressing the identified precepts as intents using a domain model;
by mapping the intents to corresponding actions using a database of correspondences between known intents and actions.
However, Zhang (US 11586849 B2) teaches:
determining an ethical concern relating to the development pipeline by: (Col 1: ln 22-33, "Machine learning, the most common form of AI today, is inherently a form of statistical discrimination. The discrimination can become objectionable to one or more users when it places certain groups represented by the data at systematic advantage and other groups represented by the data at systematic disadvantage."; Col 14: ln 1-7, "The main fairness issue of the German Credit dataset is that it leads to classifiers that may disproportionately penalize young people under 26 years old (e.g., people 25 years old and younger).") Examiner Comments: Zhang teaches determining an ethical concern relating to a development pipeline by evaluating a machine learning model for statistical discrimination and disproportionate penalization of protected groups, which is an ethical concern in an artificial intelligence development pipeline.
identifying a constraint that mitigates the ethical concern by mapping the intents to corresponding actions using a database of correspondences between known intents and actions, wherein the constraint is an entry in the DevOps pipeline script that performs a check to ensure the ethical concern is not triggered during an execution of the development pipeline; (Col 7: ln 34-60, "The independence criterion can require that all analyzed groups represented by the given data receive equal rate of favorable treatment by the machine learning model. The separation criterion can require that the false positive rates and the false negative rates are similar across all groups represented by the given data."; Col 8: ln 50-61, "The metric component 112 can employ disparate impact ratio to filter out one or more machine learning model settings that do not satisfy one or more defined constraint thresholds (e.g., defined by one or more policy makers via the one or more input devices 106 and/or networks 104).") Examiner Comments: Zhang teaches identifying a constraint that mitigates the ethical concern as a fairness criterion and a disparate impact ratio applied against defined constraint thresholds, which, when incorporated into the DevOps pipeline script of Jagannath, reads on the constraint being an entry that performs a check to ensure the ethical concern of bias is not triggered during an execution of the development pipeline.
modifying one or more automated processes within the development pipeline based on the identified constraint; (Col 8: ln 50-61, "The metric component 112 can employ disparate impact ratio to filter out one or more machine learning model settings that do not satisfy one or more defined constraint thresholds (e.g., defined by one or more policy makers via the one or more input devices 106 and/or networks 104).") Examiner Comments: Zhang teaches modifying the automated processes by filtering out one or more machine learning model settings that do not satisfy the identified constraint thresholds, thereby modifying the automated processing of the pipeline based on the identified constraint.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the bias-mitigation constraint of Zhang with the DevOps development pipeline of Jagannath in order to ensure that the automated development and deployment processes account for fairness and societal impact and to reduce the risk of deploying biased machine learning models, because both references address automated development workflows and Zhang expressly adapts to changing societal preferences for ethical artificial intelligence.
Jagannath and Zhang did not specifically teach
obtaining processed input information sources using a miner;
identifying precepts from the processed input information sources;
and expressing the identified precepts as intents using a domain model;
by mapping the intents to corresponding actions using a database of correspondences between known intents and actions;
However, Gruber (US 20120016678 A1) teaches:
obtaining processed input information sources using a miner; identifying precepts from the processed input information sources; and expressing the identified precepts as intents using a domain model; ... by mapping the intents to corresponding actions using a database of correspondences between known intents and actions; (Para 0010, "the intelligent automated assistant systems of the present invention can perform any or all of: actively eliciting input from a user, interpreting user intent, disambiguating among competing interpretations, requesting and receiving clarifying information as needed, and performing (or initiating) actions based on the discerned intent."; Para 0367, "Domain models 1056 component(s) include representations of the concepts, entities, relations, properties, and instances of a domain."; Para 0381, "data from domain model component(s) 1056 may be associated with other model modeling components including ... task flow models 1086") Examiner Comments: Gruber teaches obtaining and processing input sources, interpreting them to identify precepts and intents, expressing the intents using a domain model, and mapping the intents to corresponding actions by way of task flow models associated with the domain model, directly teaching the recited miner, precept, intent, domain-model, and intent-to-action mapping steps.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the intent extraction and domain-model mapping of Gruber into the DevOps development pipeline of Jagannath and Zhang in order to automatically translate raw contextual input sources into actionable pipeline constraints, thereby enabling automatic ethical adaptation of the pipeline.
Regarding Claim 3, Jagannath, Zhang, and Gruber teach the method of Claim 1. Jagannath further teaches:
wherein the DevOps pipeline script includes instructions to build and deploy a project. (Para 0013, "The DevOps application deployment packages may be provided deployment tool plugins associated with the determined application deployment tools and the deployment tool plugins may execute deployment operations based on deployment properties included in the DevOps application deployment packages to deploy DevOps applications using the determined application deployment tools."; Para 0017, "The DevOps toolchain may include a series of lifecycle stages that aid in the development, deployment, and management of an application through the application’s lifecycle. Each DevOps lifecycle stage may have an associated set of users, user privileges, tasks, DevOps policies, and environments relevant for that stage.") Examiner Comments: Jagannath teaches the DevOps pipeline script including instructions to build and deploy a project by generating deployment packages and managing the application through build and deploy lifecycle stages.
Regarding Claim 4, Jagannath, Zhang, and Gruber teach the method of Claim 1.
Zhang further teaches:
wherein the determining of the ethical concern includes a new ethical concern that was not known at an initial execution of the development pipeline. (Col 2: ln 25-38, "An advantage of such a computer program product can be a bias mitigation scheme for machine learning models that can adapt to changes in societal preferences (e.g., as expressed by one or more policy makers).") Examiner Comments: Zhang teaches determining a new ethical concern not known at an initial execution because the bias-mitigation scheme adapts to changes in societal preferences that arise after the model is first evaluated.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the bias-mitigation constraint of Zhang with the DevOps development pipeline of Jagannath in order to ensure that the automated development and deployment processes account for fairness and societal impact and to reduce the risk of deploying biased machine learning models, because both references address automated development workflows and Zhang expressly adapts to changing societal preferences for ethical artificial intelligence.
Regarding Claim 5, Jagannath, Zhang, and Gruber teach the method of Claim 4.
Jagannath further teaches:
wherein the executing of the development pipeline includes executing a new development pipeline based on adding the identified constraint for the new ethical concern. (Para 0035, "redeploying a DevOps application (i.e., removing an application from DevOps application deployment environment 130 and deploying the application again in DevOps application deployment environment 130).") Examiner Comments: Jagannath teaches executing a new development pipeline by redeploying the application after the identified constraint is added, mapping to executing a new pipeline based on adding the constraint for the new ethical concern.
Regarding Claim 6, Jagannath, Zhang, and Gruber teach the method of Claim 1.
Zhang further teaches:
wherein the development pipeline develops and deploys a machine learning model. (Col 1: ln 48-60, "The computer executable components can comprise a model component that can evaluate a machine learning model at a plurality of threshold settings to generate a sample set and can define a relationship between a fairness metric and a utility metric of the machine learning model based on the sample set.") Examiner Comments: Zhang teaches a pipeline that develops and evaluates a machine learning model at a plurality of threshold settings, which in combination with the deployment of Jagannath maps to developing and deploying a machine learning model.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the bias-mitigation constraint of Zhang with the DevOps development pipeline of Jagannath in order to ensure that the automated development and deployment processes account for fairness and societal impact and to reduce the risk of deploying biased machine learning models, because both references address automated development workflows and Zhang expressly adapts to changing societal preferences for ethical artificial intelligence.
Regarding Claim 9, Jagannath, Zhang, and Gruber teach the method of Claim 1.
Zhang further teaches:
wherein the determining of the ethical concern includes identifying changes in contextual information relating to the development pipeline from one or more artifacts. (Col 2: ln 25-38, "An advantage of such a computer program product can be a bias mitigation scheme for machine learning models that can adapt to changes in societal preferences (e.g., as expressed by one or more policy makers).") Examiner Comments: Zhang teaches identifying changes in contextual information relating to the pipeline by adapting to changes in societal preferences expressed by one or more policy makers, which serve as artifacts of changed context.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the bias-mitigation constraint of Zhang with the DevOps development pipeline of Jagannath in order to ensure that the automated development and deployment processes account for fairness and societal impact and to reduce the risk of deploying biased machine learning models, because both references address automated development workflows and Zhang expressly adapts to changing societal preferences for ethical artificial intelligence.
Regarding Claim 10, Jagannath, Zhang, and Gruber teach the method of Claim 1.
Zhang further teaches:
wherein the identifying of the constraint includes looking up the ethical concern in the database to select a predetermined constraint associated with the ethical concern that mitigates the ethical concern. (Col 8: ln 1-17, "Example metrics that can relate to this notion of fairness can include, but are not limited to: statistical parity difference, disparate impact ratio, a combination thereof, and/or the like."; Col 8: ln 50-61, "The metric component 112 can employ disparate impact ratio to filter out one or more machine learning model settings that do not satisfy one or more defined constraint thresholds (e.g., defined by one or more policy makers via the one or more input devices 106 and/or networks 104).") Examiner Comments: Zhang teaches looking up a predefined fairness metric associated with the concern and applying it against defined constraint thresholds, mapping to selecting a predetermined constraint associated with the ethical concern from a database.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the bias-mitigation constraint of Zhang with the DevOps development pipeline of Jagannath in order to ensure that the automated development and deployment processes account for fairness and societal impact and to reduce the risk of deploying biased machine learning models, because both references address automated development workflows and Zhang expressly adapts to changing societal preferences for ethical artificial intelligence.
Regarding Claim 11, Jagannath (US 20180032322 A1) teaches:
A computer-implemented method for DevOps pipeline generation, the computer-implemented method comprising: iteratively updating a DevOps pipeline script that includes instructions to build and deploy a machine learning model, including: (Para 0013, "The DevOps application deployment packages may be provided deployment tool plugins associated with the determined application deployment tools and the deployment tool plugins may execute deployment operations based on deployment properties included in the DevOps application deployment packages to deploy DevOps applications using the determined application deployment tools."; Para 0017, "The DevOps toolchain may include a series of lifecycle stages that aid in the development, deployment, and management of an application through the application’s lifecycle. Each DevOps lifecycle stage may have an associated set of users, user privileges, tasks, DevOps policies, and environments relevant for that stage."; Para 0035, "redeploying a DevOps application (i.e., removing an application from DevOps application deployment environment 130 and deploying the application again in DevOps application deployment environment 130).") Examiner Comments: Jagannath teaches iteratively updating a DevOps pipeline script that includes build and deploy instructions through lifecycle stages and redeployment, which in combination is applied to a machine learning model.
modifying one or more automated processes within the DevOps pipeline script based on adding the identified constraint as an entry in the DevOps pipeline script that performs a check to ensure the new ethical concern is not triggered during an execution of the DevOps pipeline script; and (Para 0013, "The DevOps application deployment packages may be provided deployment tool plugins associated with the determined application deployment tools and the deployment tool plugins may execute deployment operations based on deployment properties included in the DevOps application deployment packages to deploy DevOps applications using the determined application deployment tools."; Para 0036, "Deployment tool plugins 113 may execute deployment operations based on deployment properties included DevOps application deployment packages.") Examiner Comments: Jagannath teaches modifying the automated deployment operations based on adding deployment properties as entries in the deployment package that the deployment tool plugins execute as checks during an execution of the pipeline.
executing the updated DevOps pipeline script concurrent with the iterative generating, wherein the executing of the DevOps pipeline script comprises executing the modified one or more automated processes. (Para 0040, "In some examples, steps of method 300 may be executed substantially concurrently or in a different order than shown in FIG. 3. In some examples, some of the steps of method 300 may, at certain times, be ongoing and/or may repeat."; Para 0035, "Deployment tool plugins 113 may execute deployment operations to deploy DevOps applications in DevOps application deployment environment 130 using application deployment tools 120.") Examiner Comments: Jagannath teaches executing the updated DevOps pipeline script concurrent with the iterative updating by running the deployment operations, which are ongoing and repeat, while the lifecycle iterations continue.
Jagannath did not specifically teach
determining a new ethical concern relating to a bias in training data for the machine learning model that was not known at an initial execution of the DevOps pipeline script; identifying a constraint that mitigates the new ethical concern; obtaining processed input information sources using a miner; identifying precepts from the processed input information sources; expressing the identified precepts as intents using a domain model; and mapping the intents to corresponding actions using a database of correspondences between known intents and actions.
However, Zhang (US 11586849 B2) teaches:
determining a new ethical concern that was not known at an initial execution of the DevOps pipeline script, the new ethical concern relating to a bias in training data for the machine learning model, wherein the determining of the new ethical concern comprises: (Col 14: ln 1-7, "The main fairness issue of the German Credit dataset is that it leads to classifiers that may disproportionately penalize young people under 26 years old (e.g., people 25 years old and younger)."; Col 2: ln 25-38, "An advantage of such a computer program product can be a bias mitigation scheme for machine learning models that can adapt to changes in societal preferences (e.g., as expressed by one or more policy makers).") Examiner Comments: Zhang teaches determining a new ethical concern relating to a bias in training data that was not known at an initial execution because the scheme identifies dataset fairness issues and adapts to changes in societal preferences arising after the model is first evaluated.
identifying a constraint that mitigates the new ethical concern ... (Col 8: ln 50-61, "The metric component 112 can employ disparate impact ratio to filter out one or more machine learning model settings that do not satisfy one or more defined constraint thresholds (e.g., defined by one or more policy makers via the one or more input devices 106 and/or networks 104)."; Col 8: ln 1-17, "Example metrics that can relate to this notion of fairness can include, but are not limited to: statistical parity difference, disparate impact ratio, a combination thereof, and/or the like.") Examiner Comments: Zhang teaches identifying a constraint that mitigates the new ethical concern by employing a disparate impact ratio applied against defined constraint thresholds to filter out settings that fail the constraint.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the bias-mitigation constraint of Zhang with the DevOps pipeline script of Jagannath in order to ensure that the automated development and deployment of the machine learning model accounts for fairness and societal impact, because both references address automated development workflows and Zhang expressly adapts to changing societal preferences for ethical artificial intelligence.
Jagannath and Zhang did not specifically teach
obtaining processed input information sources using a miner;
identifying precepts from the processed input information sources;
expressing the identified precepts as intents using a domain model; and mapping the intents to corresponding actions using a database of correspondences between known intents and actions.
However, Gruber (US 20120016678 A1) teaches:
obtaining processed input information sources using a miner; identifying precepts from the processed input information sources; and expressing the identified precepts as intents using a domain model; ... by mapping the intents to corresponding actions using a database of correspondences between known intents and actions; (Para 0010, "the intelligent automated assistant systems of the present invention can perform any or all of: actively eliciting input from a user, interpreting user intent, disambiguating among competing interpretations, requesting and receiving clarifying information as needed, and performing (or initiating) actions based on the discerned intent."; Para 0367, "Domain models 1056 component(s) include representations of the concepts, entities, relations, properties, and instances of a domain."; Para 0381, "data from domain model component(s) 1056 may be associated with other model modeling components including ... task flow models 1086") Examiner Comments: Gruber teaches obtaining and processing input sources, identifying precepts and intents, expressing the intents using a domain model, and mapping the intents to corresponding actions by way of task flow models associated with the domain model, directly teaching the recited miner, precept, intent, domain-model, and intent-to-action mapping steps.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the intent extraction and domain-model mapping of Gruber into the DevOps pipeline script of Jagannath and Zhang in order to automatically translate raw contextual input sources into actionable pipeline constraints, thereby enabling automatic ethical adaptation of the pipeline.
Regarding Claim 13, Jagannath, Zhang, and Gruber teach the method of Claim 11.
Zhang further teaches:
wherein the determining of the new ethical concern includes identifying changes in contextual information relating to the DevOps pipeline script. (Col 2: ln 25-38, "An advantage of such a computer program product can be a bias mitigation scheme for machine learning models that can adapt to changes in societal preferences (e.g., as expressed by one or more policy makers).") Examiner Comments: Zhang teaches identifying changes in contextual information relating to the DevOps pipeline script by adapting to changes in societal preferences expressed by one or more policy makers.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the bias-mitigation constraint of Zhang with the DevOps pipeline script of Jagannath in order to ensure that the automated development and deployment of the machine learning model accounts for fairness and societal impact, because both references address automated development workflows and Zhang expressly adapts to changing societal preferences for ethical artificial intelligence.
Regarding Claim 14, is a computer program product claim corresponding to the method claim above (Claim 1) and, therefore, is rejected for the same reasons set forth in the rejection of claim 1.
Regarding Claim 15, is a system claim corresponding to the method claim above (Claim 1) and, therefore, is rejected for the same reasons set forth in the rejection of claim 1.
Regarding Claim 17, is a system claim corresponding to the method claim above (Claim 3) and, therefore, is rejected for the same reasons set forth in the rejection of claim 3.
Regarding Claim 18, is a system claim corresponding to the method claim above (Claim 4) and, therefore, is rejected for the same reasons set forth in the rejection of claim 4.
Regarding Claim 19, is a system claim corresponding to the method claim above (Claim 5) and, therefore, is rejected for the same reasons set forth in the rejection of claim 5.
Regarding Claim 20, is a system claim corresponding to the method claim above (Claim 6) and, therefore, is rejected for the same reasons set forth in the rejection of claim 6.
Regarding Claim 24, is a system claim corresponding to the method claim above (Claim 11) and, therefore, is rejected for the same reasons set forth in the rejection of claim 11.
Claims 7, 8, 12, 21, 22, and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Jagannath (US 20180032322 A1) in view of Zhang (US 11586849 B2) and Gruber (US 20120016678 A1) and further in view of Zoldi (US 20230085575 A1).
Regarding Claim 7, Jagannath, Zhang, and Gruber teach the method of Claim 6.
Jagannath, Zhang, and Gruber did not specifically teach
that the ethical concern is a bias of a plurality of biases in training data.
However, Zoldi (US 20230085575 A1) teaches:
wherein the ethical concern is a bias of a plurality of biases in training data for the machine learning model. (Para 0003, "It is not uncommon for the training data to include values or trends that reflect societal bias or other types of bias. This can be due to a variety of reasons, such as the way data was collected or the source of data.") Examiner Comments: Zoldi teaches that the training data can include values or trends reflecting societal bias or other types of bias, mapping to the ethical concern being a bias of a plurality of biases in training data for the machine learning model.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the training-data bias teaching of Zoldi with Jagannath, Zhang, and Gruber in order to specifically address training-data biases within the pipeline, because Zoldi provides a systematic way to identify and eliminate biased features that enhances the general mitigation of Zhang with targeted training-stage constraints.
Regarding Claim 8, Jagannath, Zhang, Gruber, and Zoldi teach the method of Claim 7. Zoldi further teaches:
wherein the identified constraint includes bias mitigation during one of pre-processing of the training data, training of the machine learning model, and post-processing of the machine learning model. (Claim 1, "training the predictive model using the first list and the second list to eliminate bias from the predictive model by removing the features and feature combinations in the combined list as model input and allowed nonlinearities expressed in the predictive model which include features in the first list or combinations of features in the second list of sets of input features."; Para 0013, "After identifying combinations of features whose interaction leads to biased latent features, the model is retrained and the process is repeated until the model has no biased latent feature left.") Examiner Comments: Zoldi teaches bias mitigation during training of the machine learning model by removing biased features and feature combinations and retraining the model until no biased latent feature remains, mapping to bias mitigation during training.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the training-data bias teaching of Zoldi with Jagannath, Zhang, and Gruber in order to specifically address training-data biases within the pipeline, because Zoldi provides a systematic way to identify and eliminate biased features that enhances the general mitigation of Zhang with targeted training-stage constraints.
Regarding Claim 12, Jagannath, Zhang, and Gruber teach the method of Claim 11.
Zoldi further teaches:
wherein the identified constraint includes bias mitigation during one of pre-processing of the training data, training of the machine learning model, and post-processing of the machine learning model. (Claim 1, "training the predictive model using the first list and the second list to eliminate bias from the predictive model by removing the features and feature combinations in the combined list as model input and allowed nonlinearities expressed in the predictive model which include features in the first list or combinations of features in the second list of sets of input features.") Examiner Comments: Zoldi teaches bias mitigation during training of the machine learning model by removing biased features and feature combinations, mapping to bias mitigation during training.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the training-data bias teaching of Zoldi with Jagannath, Zhang, and Gruber in order to specifically address training-data biases within the pipeline, because Zoldi provides a systematic way to identify and eliminate biased features that enhances the general mitigation of Zhang with targeted training-stage constraints.
Regarding Claim 21, is a system claim corresponding to the method claim above (Claim 7) and, therefore, is rejected for the same reasons set forth in the rejection of claim 7.
Regarding Claim 22, is a system claim corresponding to the method claim above (Claim 8) and, therefore, is rejected for the same reasons set forth in the rejection of claim 8.
Regarding Claim 25, is a system claim corresponding to the method claim above (Claim 12) and, therefore, is rejected for the same reasons set forth in the rejection of claim 12.
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Jagannath (US 20180032322 A1) in view of Zhang (US 11586849 B2) and Gruber (US 20120016678 A1) and further in view of Lohia (US 11636386 B2).
Regarding Claim 23, Jagannath, Zhang, and Gruber teach the system of Claim 15.
Jagannath, Zhang, and Gruber did not specifically teach
wherein the computer program further causes the hardware processor to identify changes in contextual information relating to the development pipeline from one or more artifacts.
However, Lohia (US 11636386 B2) teaches:
wherein the computer program further causes the hardware processor to identify changes in contextual information relating to the development pipeline from one or more artifacts. (Col 1: ln 25-47, "identifying one or more instances of bias by observing a change to one or more of the values in the mappings in response to modifying one or more class designations among the data points in the mappings;"; Claim 6, "The at least two classes of data points comprise a majority class and a minority class, distinguished in accordance with a predetermined threshold value.") Examiner Comments: Lohia teaches identifying changes in contextual information by observing changes to values in the data mappings in response to modifying class designations distinguished by a predetermined threshold, mapping to identifying changes in contextual information from one or more artifacts.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the bias-detection teaching of Lohia with Jagannath, Zhang, and Gruber in order to provide a precise mechanism for detecting indirect bias, because Lohia’s correlation and perturbation analysis enables more robust ethical checks within the iterative pipeline.
Response to Arguments
Applicant argues that “amended independent claim 1 of the instant application recites steps that are performed by a machine (e.g., by one or more processors) and the human mind is not equipped to perform these claimed features,”.
Examiner respectfully disagrees. The abstract idea identified in the rejection is the recited process of determining an ethical concern, identifying precepts and expressing them as intents, mapping the intents to corresponding actions, and identifying a constraint that mitigates the ethical concern, which are steps that can be performed in the human mind through observation, evaluation, judgment, and opinion. Reciting that these mental steps are performed by a processor or within a DevOps pipeline script does not remove them from the “Mental Processes” grouping; a claim that recites a mental process performed on a generic computer still recites an abstract idea. The additional machine elements, including the hardware processor, the DevOps pipeline script, the entry that performs a check, the modifying of one or more automated processes, and the executing of the modified processes, are recited at a high level of generality and apply the abstract idea using generic computer components. See MPEP 2106.05(f).
Applicant argues that “the alleged abstract idea is integrated into a practical application”.
Examiner respectfully disagrees. The asserted improvement described in the specification, namely improved pipeline adaptability, execution integrity, and dynamic compliance enforcement, is an improvement to the abstract idea itself, that is, to the ethical evaluation and constraint identification, rather than an improvement to the functioning of a computer or to any other technology or technical field. See MPEP 2106.05(a). The additional elements of the DevOps pipeline script, the entry that performs a check, the modifying of one or more automated processes based on the identified constraint, and the executing of the modified processes are recited at a high level of generality and amount to no more than using a generic computer and a generic pipeline as tools to carry out the abstract ethical-evaluation process, which does not impose meaningful limits on the abstract idea and does not integrate the judicial exception into a practical application. See MPEP 2106.05(f). The claims do not recite a particular technological improvement to how the computer or the pipeline itself operates. Accordingly, the rejection under 35 U.S.C. 101 is maintained.
Applicant argues that “the combination of Jagannath, Zhang, and Gruber does not teach, suggest, or render obvious the constraint is an entry in the DevOps pipeline script that performs a check to ensure the ethical concern is not triggered during execution of the development pipeline,”.
Examiner respectfully disagrees. The rejection is based on the combined teachings of Jagannath, Zhang, and Gruber, and not on any single reference. One cannot show nonobviousness by attacking references individually where the rejection is based on a combination of references. Jagannath teaches the DevOps pipeline script comprising deployment packages that include deployment properties, and further teaches that the deployment tool plugins execute deployment operations based on those deployment properties during an execution of the pipeline. Zhang teaches the substantive content of the constraint, namely a fairness check such as a disparate impact ratio that is applied against one or more defined constraint thresholds to ensure that the ethical concern of bias is not triggered. Incorporating the fairness check of Zhang as a deployment property or entry within the DevOps pipeline script of Jagannath yields the claimed constraint that is an entry in the DevOps pipeline script that performs a check during execution. The combination therefore teaches the argued limitation.
Applicant argues that “modifying one or more automated processes within the development pipeline based on the identified constraint.”
Examiner respectfully disagrees. Zhang expressly teaches employing a disparate impact ratio to filter out one or more machine learning model settings that do not satisfy one or more defined constraint thresholds. Filtering out the settings that fail the identified constraint modifies the automated processing of the model based on the identified constraint. Jagannath further teaches redeploying the application and executing the deployment operations after the deployment properties are incorporated, which teaches executing the modified one or more automated processes. It would have been obvious to one of ordinary skill in the art to combine these teachings for the reasons set forth in the rejection above, namely to ensure that the automated development and deployment processes account for fairness and reduce the risk of deploying biased machine learning models, which is a combination of known elements according to known methods to yield predictable results.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMIR SOLTANZADEH whose telephone number is (571)272-3451. The examiner can normally be reached M-F, 9am - 5pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wei Mui can be reached at (571) 272-3708. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMIR SOLTANZADEH/Examiner, Art Unit 2191 /WEI Y MUI/Supervisory Patent Examiner, Art Unit 2191
Prosecution Insights