DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 5/6/2026 has been entered.
Claims 5-6, 8-9 and 15-16 are cancelled.
Claims 1-4, 7, 10-14 and 17-18 are pending.
Response to Amendment
Applicant’s amendments are acknowledged.
Response to Arguments
Applicant's arguments filed 5/6/2026 have been fully considered in view of further consideration of statutory law, Office policy, precedential common law, and the cited prior art as necessitated by the amendments to the claims, but are not persuasive for the reasons set forth below.
35 USC § 101 Rejections
First, Applicant argues that the “Claims Do Not Recite a Judicial Exception… The claims are not directed at managing or organizing human behavior, but rather to a specific technical mechanism for transforming heterogeneous developer performance data into machine-consumable feature representations and using those representations within machine learning models for evaluation and ranking…
the claimed invention specifically recites pre-processing and transformation of structured and semi-structured input data into numerical representations… Such transformations in claimed invention require machine implementation and cannot be practically performed in the human mind..
Further, the paragraphs [0069]-[0072] of the as-filed specification discloses the use of embedding layers within a trained machine learning model, where the dimensionality of the vector space corresponds to the number of unique categorical values (T1-T5). This reflects a technical improvement in feature representation…
Additionally, the claimed invention includes dynamic feature modification and tuning through preprocessing… These steps are not merely data organization rather they alter the structure and dimensionality of the data in a manner that directly impacts how a machine learning model operates, thereby improving computational efficiency and predictive capability…
Moreover, the claimed invention accounts for data complexity arising from heterogeneous, multi-valued, and multi-dimensional inputs… This handling of data complexity is a technical solution to a machine learning problem, not an abstract organizational activity…
…The re-ranking logic (e.g., prioritizing developers with broader technical exposure despite bug counts) demonstrates multi-dimensional feature interaction handled programmatically, which again cannot be practically executed mentally.
Importantly, the end-to-end pipeline from raw tabular data ingestion, through encoding and feature generation, to machine learning-based ranking and API-driven output delivery (see paragraphs [0076]-[0077] of as filed specification) constitutes a technical data processing architecture. The system enables automated querying (e.g., identifying experts in machine learning) and retrieval of results via API integration, further reinforcing that the claims are directed to computer functionality and intelligent data processing, not to organizing human activity.
In view of the above, the claims do not recite methods of organizing human activity such as managing relationships or human behavior. Instead, they are directed to specific computational techniques involving complex data transformation, feature encoding (including one-hot/n-hot embeddings), dynamic feature modification and tuning, and machine learning-based evaluation…” [Arguments, pages 13-19].
In response, Applicant’s arguments have been considered but are not persuasive. Examiner respectfully disagrees and maintains that, when considered as a whole, the present invention recites abstract ideas. In particular, Examiner maintains that the present invention recites certain methods of organizing human activity.
First, with regard to the argument that the claimed invention specifically recites pre-processing and transformation of structured and semi-structured input data into numerical representations, and that such transformations in claimed invention require machine implementation and cannot be practically performed in the human mind, Examiner observes that whether or not the invention can be practically performed in the human mind is a consideration for the abstract idea grouping of ‘mental processes’, rather than ‘certain methods of organizing human activity’.
Further, with regard to the machine learning and feature tuning aspects of the present invention, Examiner observes that the general thrust of the invention is considered to be directed to ranking and classifying developers into performance categories, rather than machine learning modeling itself. The claimed limitations describe steps for managing personal behavior or relationships or interactions between people, which includes social activities, teaching, and following rules or instructions. Specifically, ranking and classifying developers into performance categories is considered to describe steps for managing personal behavior as well as interactions between people. Thus, claims 1, 11 and 18 are directed to concepts identified as abstract ideas, namely certain methods of organizing human activity. As such, Examiner remains unpersuaded.
Second, Applicant argues that the “claims are integrated into practical application… When properly read in light of the specification (particularly paragraphs [0069]-[0077] of as filed specification), the claims recite a specific, technically constrained data processing pipeline that meaningfully limits any alleged abstract idea and improves the functioning of computer-based machine learning systems. The Examiner's characterization of the claims as merely invoking "Al" or "machine learning" at a high level overlooks the concrete technical mechanisms expressly recited and supported in the specification…
For example, a categorical feature such as programming language is transformed into a high-dimensional binary vector (e.g., Python [100000], Java [010000]). Further, where multiple technologies apply, n-hot encoding is used to represent simultaneous feature presence across multiple dimensions…
Further, the claimed invention explicitly addresses data complexity, feature modification, and feature tuning, which further demonstrates integration into a practical application. The input data includes heterogeneous, multi-dimensional, and multi valued parameters, such as multiple technologies associated with a single developer. The use of one-hot and n-hot encoding generates high-dimensional feature vectors, enabling the system to capture complex interrelationships among features across multiple dimensions. This handling of data complexity is a technical requirement for machine learning systems and is not a conventional or mental process…
Additionally, the claimed invention performs feature modification through structured preprocessing… These steps redefine and restructure the feature space, thereby enabling improved compatibility with machine learning models and enhancing downstream processing.
Moreover, the invention incorporates feature tuning mechanisms, wherein the transformed features are aligned with embedding layers… This ensures that feature representations are adaptively structured to improve model learning behavior, convergence efficiency, and predictive performance. Such tuning directly impacts the operation of the machine learning models and reflects a concrete technological improvement…
This layered modeling approach is not generic. It reflects a multi-stage machine learning pipeline… This improves accuracy, scalability, and interpretability of the system, thereby constituting a practical technological application rather than a mere abstract idea…
As described in paragraphs [0073]-[0075] of as filed specification, developers initially classified within the same performance category are re-ranked based on computed feature interactions… Such operations are inherently computational, requiring structured data representations and algorithmic processing, and therefore integrate the alleged abstract idea into a practical application…
The Examiner analogizes the claims to cases such as Alice, Benson, and TLI Communications… In contrast, the present claims: Require specific feature encoding techniques (one-hot/n-hot embeddings); Define structured transformation of heterogeneous datasets; Utilize embedding-layer-compatible representations; and Implement a multi-model machine learning pipeline with feature-driven ranking logic.
These are not generic instructions to "apply AI," but rather specific improvements in how data is represented, processed, and utilized within machine learning systems. Accordingly, the claims align more closely with decisions such as: Enfish (improved data structure), and McRO (rule-based automation improving computer functionality), as they recite specific computational techniques that improve system performance and capability…” [Arguments, pages 19-30].
In response, Applicant’s arguments have been considered but are not persuasive. As stated in the previous office action, Examiner respectfully disagrees and maintains that the present invention recites an abstract without significantly more.
Examiners evaluate integration into a practical application by: (1) identifying whether there are any additional elements recited in the claim beyond the judicial exception(s); and (2) evaluating those additional elements individually and in combination to determine whether they integrate the exception into a practical application, using one or more of the considerations introduced in subsection I supra, and discussed in more detail in MPEP §§ 2106.04(d)(1), 2106.04(d)(2), 2106.05(a) through (c) and 2106.05(e) through (h).
With regard to the assertion that the present invention demonstrates a practical application because the claimed invention explicitly addresses data complexity, feature modification, and feature tuning, Examiner respectfully disagrees and maintains that the additional elements are not recited at a level of specificity which could be considered to demonstrate significantly more than the judicial exception.
For example, and with regard to the first machine learning model, Examiner observes that the type of machine learning model itself is not specified. Further, the claim does not specify how the model is trained, but instead only states that the model uses parameters in the neural network to generate a low dimensional embedding (i.e. “apply it” to generate this format/outcome). Further, the process in which the model creates feature vectors is not defined. Instead, the claim only states that the feature vectors are generated based on featured vectors generated from the pre-trained model.
Further, limitations including “assessing, by the Al based evaluation system, the one or more feature vectors, based on the first pre-trained machine learning model” (Claim 1), do not provide any detail with regard to how the model itself functions or arrives at the subsequent classification outcomes.
Further still, limitations including “dynamically adjusting, by the Al based evaluation system, a performance evaluation criterion, based on real-time feedback from multiple stakeholders and evolving project requirements”, “modifying the first pre-trained machine learning model with transferable knowledge for a target system to be evaluated, wherein the transferable knowledge corresponds to optimal values associated with the one or more feature vectors corresponding to each of the plurality of performance parameters” and “ tuning the first pre-trained machine learning model using specific characteristics of the target system to create a target model” (Claim 1), do not specify how the machine model adjusts criterion or is itself “modified” or “tuned”, and instead simply state that the adjustment is made based on real-time feedback and requirements.
Further still and with regard to McRO, the court relied on the specification’s explanation of how the particular rules recited in the claim enabled the automation of specific animation tasks that previously could only be performed subjectively by humans, when determining that the claims were directed to improvements in computer animation instead of an abstract idea. McRO, 837 F.3d at 1313-14, 120 USPQ2d at 1100-01 (claims to automatic lip synchronization and facial expression animation were directed to an improvement in computer-related technology and not directed to an abstract idea). In contrast, the court in Affinity Labs of Tex. v. DirecTV, LLC relied on the specification’s failure to provide details regarding the manner in which the invention accomplished the alleged improvement when holding the claimed methods of delivering broadcast content to cellphones ineligible. 838 F.3d 1253, 1263-64, 120 USPQ2d 1201, 1207-08 (Fed. Cir. 2016).
Akin to Affinity Labs, Examiner respectfully maintains that the present specification fails to provide sufficient details regarding the manner in which the invention accomplishes the alleged improvement. For example, with regard to the above-cited ¶¶69-75 of the present specification, Examiner observes that these paragraphs only appear to describe a tabular representation or layout of data, without sufficiently detailing how the data is generated or utilized. Similarly, with regard to the above-cited ¶¶76-77 of the present specification, Examiner observes that these paragraphs merely describe outputs of the machine learning model (e.g. “configured to rank each developer… configured to render output data… based on evaluation of the performance…”), without specifying how the outputs are achieved.
With regard to Enfish, Examiner observes that the claims are directed to clear improvements to computer-related technology, rather than certain methods of organizing human activity, and thus are eligible at Step 1. Enfish, 822 F.3d at 1339, 118 USPQ2d at 1691-92 (claims to a self-referential table for a computer database held eligible at step 1 of the Alice/Mayo test as not directed to an abstract idea). Particularly, the patentee in Enfish argued that its claimed self-referential table for a computer database was an improvement in an existing technology and thus not directed to an abstract idea. Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1336-37, 118 USPQ2d 1684, 1689-90 (Fed. Cir. 2016). The court agreed with the patentee, based on its interpretation of the claimed "means for configuring" under 35 U.S.C. 112(f) as requiring a four-step algorithm that achieved the improvements, as opposed to merely any form of storing tabular data.
In contrast, the present claims do not recite anything akin to a four-step algorithm or an improvement to database functionality. Instead, the present invention only recites inputs and outputs to an unspecified machine learning model, without demonstrating how the outputs are achieved, which Examiner considers to be a drafting effort designed to monopolize the judicial exception (See MPEP 2106.04(d)).
Thus, Examiner respectfully maintains that the present invention recites an abstract idea without significantly more. As such, Examiner remains unpersuaded.
Third, Applicant argues that the “Claims Amount to "Significantly More" than an Abstract Idea…
the Examiner's conclusion under Step 2B that the claims do not recite elements amounting to "significantly more" than the alleged abstract idea. The Examiner's position largely repeats the reasoning provided under Step 2A, Prong 2 and characterizes the claimed elements as generic invocations of "AI" and "machine learning." However, this characterization does not properly consider the specific, non-conventional technical features recited in the claims and supported by the specification (see paragraphs [0069]-[0077] of as filed specification)…
Contrary to the Examiner's assertion, the claims do not merely recite "training a machine learning model." The claims require a specific transformation of heterogeneous input data into structured numerical representations, including: Conversion of raw tabular data (600A) into machine-compatible numerical format (600B); Transformation of ordinal values into numerical scales; and Generation of one-hot and n-hot encoded feature representations for multi valued categorical attributes such as "technology used." These encoding mechanisms fundamentally alter the structure, dimensionality, and representation of the input dataset, enabling compatibility with neural network architectures… it provides a specific technical solution to the problem of processing mixed-type multi-valued datasets, which cannot be effectively handled by conventional data processing techniques. This constitutes a meaningful limitation and a key part of the inventive concept…
Additionally, the claimed invention performs feature modification through structured preprocessing, including conversion of ordinal values into numerical scales and expansion of categorical attributes into multiple feature columns. These steps redefine and restructure the feature space, rather than merely organizing data.
Moreover, the invention incorporates feature tuning mechanisms, wherein the transformed features are aligned with embedding layers based on the number of unique categorical values. This alignment ensures that the feature representations are adaptively structured to improve model learning efficiency, convergence behavior, and predictive performance. Such coordinated handling of complexity, modification and tuning reflects a specific technological improvement in machine learning data preparation and representation, and is not well-understood, routine, or conventional…
This is not a generic "apply ML" scenario. Instead, the claims define a multi-stage, interdependent modeling pipeline, where: The first model operates on specifically engineered feature representations; and The second model utilizes outputs of the first model along with additional computed metrics to perform refined ranking. This layered architecture introduces functional interdependence between models, improving predictive accuracy, scalability, and interpretability. Such an arrangement is neither conventional nor routine and therefore contributes to the inventive concept…
The Examiner has not provided any evidence that such a specific combination of features was well-understood, routine, or conventional at the time of filing. Nor does the cited reasoning under MPEP § 2106.05(a) or § 2106.05(f) address these particularized technical arrangements. Accordingly, the claims cannot be equated with cases such as Alice, Benson, or TLI Communications, which involved generic or conventional implementations lacking technical improvement…
When considered as a whole, the claims recite significantly more than the alleged abstract idea. They define a specific, non-conventional, and technically detailed framework for transforming input data, generating feature representations, and performing multi stage machine learning-based evaluation and ranking. Accordingly, the claims include an inventive concept sufficient to satisfy Step 2B of the Alice/Mayo framework…” [Arguments, pages 20-39].
In response, Applicant’s arguments have been considered but are not persuasive. Examiner respectfully disagrees and maintains that the present invention demonstrates neither an improvement to any particular field of technology nor to the functioning of a computer. First, with regard to the assertion that this characterization does not properly consider the specific, non-conventional technical features recited in the claims and supported by the specification (see paragraphs [0069]-[0077] of as filed specification), Examiner respectfully disagrees for the same reasons as stated in response to the above-argument. In particular, with regard to the above-cited ¶¶69-75 of the present specification, Examiner observes that these paragraphs only appear to describe a tabular representation or layout of data, without sufficiently detailing how the data is generated or utilized. Similarly, with regard to the above-cited ¶¶76-77 of the present specification, Examiner observes that these paragraphs merely describe outputs of the machine learning model (e.g. “configured to rank each developer… configured to render output data… based on evaluation of the performance…”), without specifying how the outputs are achieved.
For example, Examiner observes that the Applicant’s argument highlights elements and steps including, “Conversion of raw tabular data (600A) into machine-compatible numerical format (600B); Transformation of ordinal values into numerical scales; and Generation of one-hot and n-hot encoded feature representations for multi valued categorical attributes such as "technology used."”. However, the argument, claims, and original disclosure of the present invention do not specify how the conversion, transformation, or generation step arrive at these outputs.
With regard to the second machine learning model (recited in dependent claims 2-3 and 12-13), Examiner observes that the claims similarly fail to demonstrate how the ranking output is achieved, other than to say the training process involves assigning weights to features based on a predefined criterion. The claims also do not provide any detail about the model itself, other than simply stating that it is a machine learning model.
Further, and with respect to the assertion that the Examiner has not provided any evidence that such a specific combination of features was well-understood, routine, or conventional at the time of filing… Accordingly, the claims cannot be equated with cases such as Alice, Benson, or TLI Communications, which involved generic or conventional implementations lacking technical improvement…, Examiner respectfully disagrees and observes that citing cases including Alice, Benson, or TLI Communications, amounts to proper evidence that the claimed features were well-understood, routine, or conventional at the time of filing.
Appropriate forms of support include one or more of the following: (a) A citation to an express statement in the specification or to a statement made by an applicant during prosecution that demonstrates the well-understood, routine, conventional nature of the additional element(s); (b) A citation to one or more of the court decisions discussed in Subsection II below as noting the well-understood, routine, conventional nature of the additional element(s); (c) A citation to a publication that demonstrates the well-understood, routine, conventional nature of the additional element(s); and (d) A statement that the examiner is taking official notice of the well-understood, routine, conventional nature of the additional element(s).
Examiner thereby respectfully maintains that the citation of the above-cited court cases satisfies the above-underlined option of the appropriate forms of support for the well-understood, routine, or conventional analysis, as required in MPEP 2106.05(d). Thus, when considered as a whole, the claims are not considered to recite significantly more than the alleged abstract idea. As such, Examiner remains unpersuaded.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-4, 7, 10-14 and 17-18 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Step 1: Claims 1-4, 7, 10-14 and 17-18 are directed to statutory categories, namely a process (claims 1-4, 7 and 10), a machine (claims 11-14 and 17) and an article of manufacture (claim 18).
Step 2A, Prong 1: Claims 1, 11 and 18 in part, recite the following abstract idea:
…A method for evaluating performance of developers …the method comprising: receiving, …, each of a plurality of performance parameters associated with a set of developers, wherein the plurality of performance parameters comprise ordinal values; processing, by… , the ordinal values of the plurality of performance parameters into a neural network passable format using one hot representations; …with exposure to a new environment during an initial training for a classification task associated with the evaluating of performance of developers, wherein…; wherein …is trained using the plurality of performance parameters in the neural network passable format; and wherein training an embedding layer of … using the one hot representations generate a low dimensional embedding; creating… one or more feature vectors corresponding to each of the plurality of performance parameters, based on one or more features determined for each of the plurality of performance parameters, wherein the one or more feature vectors are created based on…, wherein the one or more performance parameters comprise efficiency of a developed product associated with a module developed for a product, complexity of the developed product, types of support received from peers, feedback or rating received from managers, quality of the module developed for the product, and technical skills of each of the set of developers; assessing, …, the one or more feature vectors, based on …; classifying, …, the set of developers into one of a set of performance categories based on the assessing of the one or more feature vectors, wherein the set of performance categories includes an excellent performer category, a good performer category, an average performer category, and a bad performer category; and evaluating, …, the performance of at least one of the set of developers, based on an associated category in the set of performance categories, in response to the classifying; dynamically adjusting … a performance evaluation criterion, based on real-time feedback from multiple stakeholders and evolving project requirements; re-evaluating… the performance of the at least one of the set of developers using the dynamically adjusted performance evaluation criterion; and updating … performance assessment of the at least one of the set of developers to the stakeholders, in response to re-evaluating; modifying … with transferable knowledge for a target system to be evaluated, wherein the transferable knowledge corresponds to optimal values associated with the one or more feature vectors corresponding to each of the plurality of performance parameters; tuning … using specific characteristics of the target system to create a target model; and evaluating the target system performance using the target model to predict system performance of the target system [Claim 1],
…receive each of a plurality of performance parameters associated with a set of developers, wherein the plurality of performance parameters comprise ordinal values; process the ordinal values of the plurality of performance parameters into a neural network passable format using one hot representations; …with exposure to a new environment during an initial training for a classification task associated with the evaluating performance of developers, wherein…; wherein … using the plurality of performance parameters in the neural network passable format; and wherein training an embedding layer of … using the one hot representations generate a low dimensional embedding; create one or more feature vectors corresponding to each of the plurality of performance parameters, based on one or more features determined for each of the plurality of performance parameters, wherein the one or more feature vectors are created based on …, wherein the one or more performance parameters comprise efficiency of a developed product associated with a module developed for a product, complexity of the developed product, types of support received from peers, feedback or rating received from managers, quality of the module developed for the product, and technical skills of each of the set of developers; assess the one or more feature vectors, based on …; classify the set of developers into one of a set of performance categories based on the assessing of the one or more feature vectors, wherein the set of performance categories includes an excellent performer category, a good performer category, an average performer category, and a bad performer category; and evaluate the performance of at least one of the set of developers, based on an associated category in the set of performance categories, in response to the classifying; dynamically adjust… a performance evaluation criterion, based on real-time feedback from multiple stakeholders and evolving project requirements; re-evaluate… the performance of the at least one of the set of developers using the dynamically adjusted performance evaluation criterion; and updating … performance assessment of the at least one of the set of developers to the stakeholders, in response to re-evaluating; modify … with transferable knowledge for a target system to be evaluated, wherein the transferable knowledge corresponds to optimal values associated with the one or more feature vectors corresponding to each of the plurality of performance parameters; tune … using specific characteristics of the target system to create a target model; and evaluate the target system performance using the target model to predict system performance of the target system [Claim 11],
…receiving each of a plurality of performance parameters associated with a set of developers, wherein the plurality of performance parameters comprise ordinal values; processing, by … the ordinal values of the plurality of performance parameters into a neural network passable format using one hot representations; …with exposure to a new environment during an initial training for a classification task associated with the evaluating performance of developers, wherein…; wherein … using the plurality of performance parameters in the neural network passable format; and wherein training an embedding layer of … using the one hot representations generate a low dimensional embedding; creating one or more feature vectors corresponding to each of the plurality of performance parameters, based on one or more features determined for each of the plurality of performance parameters, wherein the one or more feature vectors are created based on…, wherein the one or more performance parameters comprise efficiency of a developed product associated with a module developed for a product, complexity of the developed product, types of support received from peers, feedback or rating received from managers, quality of the module developed for the product, and technical skills of each of the set of developers; assessing the one or more feature vectors, based on…; classifying the set of developers into one of a set of performance categories based on the assessing of the one or more feature vectors, wherein the set of performance categories includes an excellent performer category, a good performer category, an average performer category, and a bad performer category; and evaluating the performance of at least one of the set of developers, based on an associated category in the set of performance categories, in response to the classifying; dynamically adjusting … a performance evaluation criterion, based on real-time feedback from multiple stakeholders and evolving project requirements; re-evaluating… the performance of the at least one of the set of developers using the dynamically adjusted performance evaluation criterion; and updating … performance assessment of the at least one of the set of developers to the stakeholders, in response to re-evaluating; modifying … with transferable knowledge for a target system to be evaluated, wherein the transferable knowledge corresponds to optimal values associated with the one or more feature vectors corresponding to each of the plurality of performance parameters; tuning … using specific characteristics of the target system to create a target model; and evaluating the target system performance using the target model to predict system performance of the target system [Claim 18].
These concepts are not meaningfully different than the following concepts identified by the MPEP:
Concepts relating to certain methods of organizing human activity. The aforementioned limitations describe steps for managing personal behavior or relationships or interactions between people, including social activities, teaching, and following rules or instructions. Specifically, classifying developers into performance categories is considered to describe steps for managing personal behavior as well as interactions between people. As such, claims 1, 11 and 18 are directed to concepts identified as abstract ideas.
The dependent claims recite limitations relative to the independent claims, including, for example:
…wherein evaluating the performance comprises: computing, for each of the set of performance categories, ranks for each developer from the set of developers categorized within an associated performance category…; and ranking each developer from the set of developers for each set of performance categories, based on the computed ranks, to evaluate the performance of each developer from the set of developers [Claims 2 and 12],
… further comprising…, wherein training comprises assigning weights to the one or more features associated with the each of the plurality of performance parameters based on a predefined evaluation criterion [Claims 3 and 13],
…wherein the predefined evaluation criterion comprises one or more of a technical skill in demand and an efficiency of a developed product with respect to bugs identified in the developed product, and wherein high weights are assigned to one or more features associated with at least one of the high demand technical skill as compared to a low demand technical skill and bug-free developed product as compared to the developed product with a plurality of bugs [Claims 4 and 14],
…further comprising: identifying a plurality of bugs associated with a module of a product developed by each of the set of developers; generating a feedback for each of the set of developers, wherein the feedback is generated in response of identifying the plurality of bugs associated with the product developed by each of the set of developers; and evaluating the performance of at least one of the set of developers, based on the feedback [Claims 7 and 17],
…further comprising: modifying the … with transferrable knowledge for a target system to be evaluated, wherein the transferrable knowledge corresponds to optimal values associated with the one or more feature vectors corresponding to each of the plurality of performance parameters; tuning … using specific characteristics of the target system to create a target model; and evaluating the target system performance using the target model to predict system performance of the target system [Claim 9],
…wherein …is configured to receive as input an input observation and an input action and to generate an estimated future reward from the input in accordance with each of the plurality of performance parameters associated with the set of developers [Claim 10].
The limitations of these dependent claims are merely narrowing the abstract idea identified in the independent claims, and thus, the dependent claims also recite abstract ideas.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. In particular, claims 1, 11 and 18 only recite the following additional elements –
…using Artificial Intelligence (AI) … by an AI based evaluation system … ; …the Al based evaluation system…; … training a first machine learning model of the Al based evaluation system… the first machine learning model corresponds to a first pre-trained machine learning model…; the first machine learning model…; …the first pre-trained machine learning model…; …by the Al based evaluation system… the first pre-trained machine learning model…; …by the Al based evaluation system… the first pre-trained machine learning model…; …by the Al based evaluation system…; …by the Al based evaluation system…; …by the Al based evaluation system…; …by the Al based evaluation system…;…by the Al based evaluation system…; [Claim 1],
A system for evaluating performance of developers using Artificial Intelligence (AI), the system comprising :a processor; and a memory communicatively coupled to the processor, wherein the memory stores processor executable instructions, which, on execution, causes the processor to: …; … train a first machine learning model of the Al based evaluation system… the first machine learning model corresponds to a first pre-trained machine learning model…; the first machine learning model is trained…; …the first pre-trained machine learning model…; …by the Al based evaluation system… the first pre-trained machine learning model…; …by the Al based evaluation system… the first pre-trained machine learning model…; …by the Al based evaluation system…; …by the Al based evaluation system…; …by the Al based evaluation system…; …by the Al based evaluation system…;…by the Al based evaluation system…; the first pre-trained machine learning model; … the first pre-trained machine learning model… [Claim 11],
A non-transitory computer-readable medium storing computer-executable instructions for contextually aligning a title of an article with content within the article, the stored instructions, when executed by a processor, cause the processor to perform operations comprising…; … training a first machine learning model of the Al based evaluation system… the first machine learning model corresponds to a first pre-trained machine learning model; …the Al based evaluation system…; … the first pre-trained machine learning model…; … the first pre-trained machine learning model…; the first machine learning model…; …the first pre-trained machine learning model…; …by the Al based evaluation system…; …by the Al based evaluation system…;…by the Al based evaluation system…; the first pre-trained machine learning model; … the first pre-trained machine learning model… [Claim 18].
The dependent claims only recite the following new additional elements –
…based on a second machine learning model… [Claims 2 and 12],
…training the second machine learning model… [Claims 3 and 13],
… the first pre-trained machine learning model corresponds to a Q network, and wherein the Q network… [Claim 10].
The machine learning model, apparatus and executable instructions are recited at a high-level of generality (see MPEP § 2106.05(a)), like the following MPEP example:
iii. Gathering and analyzing information using conventional techniques and displaying the result, TLI Communications, 823 F.3d at 612-13, 118 USPQ2d at 1747-48;
Furthermore, the AI, machine learning and computer-implemented elements are considered to amount to no more than mere instructions to apply the exception using a generic computer component (see MPEP 2106.05(f)), similar to Claim 2 of Example 47 of the July 2024 Subject Matter Eligibility Examples and to the following MPEP examples:
i. A commonplace business method or mathematical algorithm being applied on a general purpose computer, Alice Corp. Pty. Ltd. V. CLS Bank Int’l, 573 U.S. 208, 223, 110 USPQ2d 1976, 1983 (2014); Gottschalk v. Benson, 409 U.S. 63, 64, 175 USPQ 673, 674 (1972); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015);
Accordingly, these additional elements do not integrate the abstract idea into a practical application.
The remaining dependent claims do not recite any new additional elements, and thus do not integrate the abstract idea into a practical application.
Step 2B: Claims 1, 11 and 18 and their underlying limitations, steps, features and terms, considered both individually and as a whole, do not include additional elements that are sufficient to amount to significantly more than the judicial exception for the following reasons:
Independent claims 1, 11 and 18 only recite the following additional elements –
…using Artificial Intelligence (AI) … by an AI based evaluation system … ; …the Al based evaluation system…; … training a first machine learning model of the Al based evaluation system… the first machine learning model corresponds to a first pre-trained machine learning model…; the first machine learning model…; …the first pre-trained machine learning model…; …by the Al based evaluation system… the first pre-trained machine learning model…; …by the Al based evaluation system… the first pre-trained machine learning model…; …by the Al based evaluation system…; …by the Al based evaluation system…; …by the Al based evaluation system…; …by the Al based evaluation system…;…by the Al based evaluation system…; [Claim 1],
A system for evaluating performance of developers using Artificial Intelligence (AI), the system comprising :a processor; and a memory communicatively coupled to the processor, wherein the memory stores processor executable instructions, which, on execution, causes the processor to: …; … train a first machine learning model of the Al based evaluation system… the first machine learning model corresponds to a first pre-trained machine learning model…; the first machine learning model is trained…; …the first pre-trained machine learning model…; …by the Al based evaluation system… the first pre-trained machine learning model…; …by the Al based evaluation system… the first pre-trained machine learning model…; …by the Al based evaluation system…; …by the Al based evaluation system…; …by the Al based evaluation system…; …by the Al based evaluation system…;…by the Al based evaluation system…; the first pre-trained machine learning model; … the first pre-trained machine learning model… [Claim 11],
A non-transitory computer-readable medium storing computer-executable instructions for contextually aligning a title of an article with content within the article, the stored instructions, when executed by a processor, cause the processor to perform operations comprising…; … training a first machine learning model of the Al based evaluation system… the first machine learning model corresponds to a first pre-trained machine learning model; …the Al based evaluation system…; … the first pre-trained machine learning model…; … the first pre-trained machine learning model…; the first machine learning model…; …the first pre-trained machine learning model…; …by the Al based evaluation system…; …by the Al based evaluation system…;…by the Al based evaluation system…; the first pre-trained machine learning model; … the first pre-trained machine learning model… [Claim 18].
These elements do not amount to significantly more than the abstract idea for the reasons discussed in 2A prong 2 with regard to MPEP 2106.05(a) and MPEP 2106.05(f). By the failure of the elements to integrate the abstract idea into a practical application there, the additional elements likewise fail to amount to an inventive concept that is significantly more than an abstract idea here, in Step 2B.
As such, both individually or in combination, these limitations do not add significantly more to the judicial exception.
The remaining dependent claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the dependent claims do not recite any new additional elements other than those mentioned in the independent claims, which amount to no more than mere instructions to apply the exception using a generic computer component (see MPEP 2106.05(f)). As such, these claims are not patent eligible.
Prior Art Considerations
Examiner conducted a thorough search of the body of available prior art (see attached documents regards PTO-892 Notice of Reference Cited and EAST Search History). Notably, Examiner discovered several patent documents that taught aspects of the invention, but no single disclosure taught “every element required by the claims under its broadest reasonable interpretation” [MPEP § 2131] to make a 35 USC § 102 rejection. Further, Examiner considered the individual elements of the recited claims taught across the prior art cited below, but did not find it obvious to combine such disclosures [MPEP § 2142] to make a 35 USC § 103 rejection.
In particular, Wright et al., U.S. Publication No. 2020/0225945 [hereinafter Wright], discloses “Methods, systems, and apparatus, including computer programs encoded on computer storage media, for receiving a source code change; computing a distribution of standard coding durations using a model that takes as input features of source code changes; and computing a representative duration for the code change using the distribution of standard coding durations, wherein the representative duration represents a measure of how long a standard developer defined by the model would take to make the code change” (Wright, Abstract) While Wright discloses several aspects of the present invention and focuses on optimizing likelihoods for starting, ending, and committing coding sessions. However, Wright is silent with respect to optimal feature vectors being transferred to modify an existing model.
Thus, Wright fails to teach, disclose, or suggest “.... modifying the first pre-trained machine learning model with transferable knowledge for a target system to be evaluated, wherein the transferable knowledge corresponds to optimal values associated with the one or more feature vectors corresponding to each of the plurality of performance parameters.....”.
Deshpande, U.S. Publication No. 2011/0173052 [hereinafter Deshpande], discloses an enhanced knowledge management system wherein “A knowledge management module associated with an organization hierarchically arranges knowledge relevant to the organization. The knowledge management module may communicate with databases internal and external to the organization to connect individuals to information regarding elements within the map. In certain embodiments, the management server also includes, for each element in the knowledge management map, information related to the levels of expertise of personnel associated with the organization.” (Deshpande, Abstract).
While Desphande discloses several aspects of the present invention such as performance parameters including feedback received by managers, technical skills of each of a set of developers and types of support received by peers, Desphande does not disclose “.... modifying the first pre-trained machine learning model with transferable knowledge for a target system to be evaluated, wherein the transferable knowledge corresponds to optimal values associated with the one or more feature vectors corresponding to each of the plurality of performance parameters.....”, as stated in the presently amended claims.
Alt et al., U.S. Publication No. 2021/0179118 [hereinafter Alt] discloses a method for determining control parameters for a control system, wherein “The method includes: providing a set of travel trajectories; deriving reward functions from the travel trajectories, using an inverse reinforcement learning method; deriving driver type-specific clusters based on the reward functions; determining control parameters for a particular driver type-specific cluster” (Alt, Abstract).
While Alt discloses some aspects of the present invention including evaluating performance using an inverse learning technique, Desphande does not disclose “.... modifying the first pre-trained machine learning model with transferable knowledge for a target system to be evaluated, wherein the transferable knowledge corresponds to optimal values associated with the one or more feature vectors corresponding to each of the plurality of performance parameters.....”, as stated in the presently amended claims.
Woulfe et al., U.S. Publication No. 2018/0276584 [hereinafter Woulfe] discloses a method of facilitating organizational management using bug data, wherein a “risk factor can be used to determine the quality of the developer's code. The risk factor associated with code produced by a particular developer can be provided to a manager or management system. The risk factor can be used to provide bug-based information to a corporate review and reward process.” (Woulfe, Abstract).
While Woulfe discloses some aspects of the present invention including identifying a plurality of bugs associated with a module of a product developed by each of the set of developers, Woulfe fails to disclose “.... modifying the first pre-trained machine learning model with transferable knowledge for a target system to be evaluated, wherein the transferable knowledge corresponds to optimal values associated with the one or more feature vectors corresponding to each of the plurality of performance parameters.....”, as stated in the presently amended claims.
Tiku et al., U.S. Publication No. 2021/0035013 [hereinafter Tiku], discloses refined user enablement utilizing reinforced learning wherein “A processor may receive profile data associated with a user. The processor may identify, from the profile data, a degree of proficiency of the user. The degree of proficiency may indicate an ability of the user to relay specific information to a second user. The processor may designate the degree of proficiency as a first level. The processor may generate a proposal to increase the first level to a second level. The increase may indicate an increase in the degree of proficiency. The processor may display the proposal to the user” (Tiku, Abstract).
While Tiku discloses some aspects of the present invention including the Q network elements of claim 10, Tiku does not disclose “.... modifying the first pre-trained machine learning model with transferable knowledge for a target system to be evaluated, wherein the transferable knowledge corresponds to optimal values associated with the one or more feature vectors corresponding to each of the plurality of performance parameters.....”, as stated in the presently amended claims.
For the above reasons, Examiner determined the currently pending claims novel and non-obvious given the current search. Amendment to the claims and further search in reaction to such amendment may yield the claims anticipated or obvious in future prosecution, determined at that time.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Strachan et al., U.S. Publication No. 2018/0088939 discloses timing estimations for application lifecycle management work items determined through machine learning.
Tornhill, U.S. Publication No. 2020/0249941 discloses ranking of software code parts.
Burton et al., U.S. Publication No. 2019/0026106 discloses associating software issue reports with changes to code.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS D BOLEN whose telephone number is (408)918-7631. The examiner can normally be reached Monday - Friday 8:00 AM - 5:00 PM PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Patty Munson can be reached on (571) 270-5396. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NICHOLAS D BOLEN/ Examiner, Art Unit 3624
/HAMZEH OBAID/Primary Examiner, Art Unit 3624
Prosecution Insights