DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on and between 3/30/2023 and 10/14/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 8/5/2026 has been entered.
Claims 1, 11 and 21 are currently amended.
Claims 1-22 are pending.
Response to Amendment
Applicant’s amendments are acknowledged.
Response to Arguments
Applicant's arguments filed 8/5/2026 have been fully considered in view of further consideration of statutory law, Office policy, precedential common law, and the cited prior art as necessitated by the amendments to the claims, and are not persuasive for the reasons set forth below.
35 USC § 101 Rejections
First, Applicant argues that, “Performing this step of determining intervals of active work time from point-in-time events is also not practical to do by hand for real point-in-time events associated with software development activity. Activity intervals include precise start and end times associated with every event for every worker, for which there are typically tens to hundreds per day. Furthermore, heuristics to accurately compute activity intervals require comparing other point-in-time and time interval events surrounding each event, making each determination computationally expensive.
As a further illustration of the impracticality of performing this step by hand, the entire business motivation behind this invention is working with incomplete point-in-time data recorded by organizations and their workers. Another solution to tracking worker activity would be to require each worker to log the time they spent on each task with a time logging system to provide a known start time so that you don't have to infer it from other point-in-time events. Many organizations have already decided that having each worker manually log their time creates too much overhead and loss of the very productivity they are trying to measure. Manually performing the steps described in this invention would require an order of magnitude more effort than simply logging time, making it commercially infeasible. The very purpose of this invention is to reduce manual labor, which inherently requires performing the process with a computer” [Arguments, page 8].
In response, Applicant’s arguments are considered but are not persuasive. Examiner respectfully disagrees and maintains that the present claims recite a judicial exception without significantly more. First, and with regard to the assertion that “Performing this step of determining intervals of active work time from point-in-time events is also not practical to do by hand…”, Examiner observes that this argument appears to be directed toward the "mental processes" abstract idea grouping, which is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions.
However, Examiner maintains that the present invention recites abstract ideas in the grouping of “certain methods of organizing human activity”, rather than “mental processes”. The present limitations describe steps for managing personal behavior or relationships or interactions between people, including social activities, teaching, and following rules or instructions. Specifically, determining work effort values based on activity records of a worker is considered to be steps for managing personal behavior. As such, claims 1, 11 and 21 recite concepts identified as abstract ideas in the abstract idea grouping of “certain methods of organizing human activity”. As such, Examiner remains unpersuaded.
35 USC § 102/103 Rejections
First, Applicant argues that “Both Boss and Mohanty describe methods for assessing what a worker did during a particular time period like a work day. They obtain point-in-time events like code changes to determine the scope of worker activity. However, Boss only describes methods for assessing the overall work output and overall activity frequency (e.g., the number of code changes during the day). Boss does not describe a method for determining the current worker activity during each time interval throughout the day, which is complicated and non-obvious for point-in-time events that are missing information about when the activity began. The proposed amendment describes applying heuristics looking at other events before and after point-in-time events to determine the precise interval of associated worker activity. This provides much greater accuracy than simple frequency counts or lists of activities, which is important for assessing the level of worker effort that goes into shorter duration (i.e., intra-day) tasks.
Mohanty does describe classifying activity into time intervals, but those time intervals represent stages of software development, not active worker effort. It is much simpler to determine whether a software project is in the development phase than to determine what point-in-time event someone was working on at a particular time, and inferring active worker effort is not an obvious leap from determining work item stages. Determining active worker effort requires more complex heuristics based on surrounding events. For example, a developer may submit code changes at the end of the day representing work starting in the morning with a single point-in-time event, but also have meetings in the middle of the day. The active coding effort in this scenario occurs both before and after the meetings. This is just one example -- inferring original worker activity intervals with incomplete point-in-time event data accurately with heuristics is not obvious” [Arguments, page 7].
In response, Applicant’s arguments have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Examiner relies upon the newly cited Amit reference to render the above-argued limitation obvious, as detailed in the rejection below. As such, Examiner remains unpersuaded.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-22 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Step 1: Claims 1-22 are directed to statutory categories, namely a process (claims 1-10), a machine (claims 11-20) and an article of manufacture (claims 21-22).
Step 2A, Prong 1: Claims 1, 11 and 21 in part, recite the following abstract idea:
…A method of assessing labor efficiency of knowledge work by means of … of a plurality of event records, the method comprising the process steps of: accessing… of a plurality of event record to obtain a plurality of event records, the event records being related to activities conducted by knowledge workers, each event record containing the identity of each worker associated with an event and either a time interval or a point in time when the event occurred; processing the plurality of event records to generate activity records associated with the time intervals during a time period indicating durations of active work time from the processed event records, the activity records containing information about work activities based on the event records, each activity record including contextual metadata indicative of a task classification, a worker identifier, and at least one of an activity type and a task complexity; the step of processing involving determining, by… intervals of active work time associated with one or more point-in-time-events using heuristics based on other events related to the same worker that occurred before or after the first event and contain points in time or time intervals when the other events occurred; allocating work effort based on the time intervals from the activity records to associate a cost with the activities and to obtain an effort value indicating amount of effort for each activity record; generating a final effort value by adjusting the preliminary effort value based on the contextual metadata included in the activity record; and generating a machine-readable report including the activity records and corresponding effort values, wherein the report is segmented by at least one of time period, task classification, and worker, and is configured for display via… [Claim 1], …A system for assessing labor efficiency of knowledge work, the system comprising:… of a plurality of event records; accessing… of the plurality of event record to obtain a plurality of event records, the event records being related to activities conducted by knowledge workers, each event record containing the identity of each worker associated with an event and either a time interval or a point in time when the event occurred; processing the plurality of event records to generate activity records associated with time intervals during a time period indicating durations of active work time from the processed event records, the activity records containing information about work activities based on the event records, each activity record including contextual metadata indicative of a task classification, a worker identifier, and at least one of an activity type and a task complexity; the operation of processing involving determining, by… intervals of active work time associated with one or more point-in-time-events using heuristics based on other events related to the same worker that occurred before or after the first event and contain points in time or time intervals when the other events occurred; allocating work effort based on the time intervals from the activity records to obtain a preliminary effort value indicating amount of effort for each activity record; and generating a final effort value for each activity record by applying … is configured to adjust the preliminary effort value based on contextual metadata included in the activity record [Claim 11],
…to perform operations, for assessing labor efficiency of knowledge work, comprising: accessing… of the plurality of event record to obtain a plurality of event records, the event records being related to activities conducted by knowledge workers, each event record containing the identity of each worker associated with an event and either a time interval or a point in time when the event occurred; processing the plurality of event records to generate activity records associated with time intervals during a time period indicating durations of active work time from the processed event records, the activity records containing information work activities based on the event records, each activity record including contextual metadata indicative of a task classification, a worker identifier, and at least one of an activity type and a task complexity; the operation of processing involving determining intervals of active work time associated with one or more point-in-time-events using heuristics based on other events related to the same worker that occurred before or after the first event and contain points in time or time intervals when the other events occurred; allocating work effort based on the time intervals from the activity records to obtain an effort value indicating amount of effort for each activity record; and generating a final effort value for each activity record by applying … is configured to adjust the preliminary effort value based on contextual metadata included in the activity record [Claim 21].
These concepts are not meaningfully different than the following concepts identified by the MPEP:
Concepts relating to certain methods of organizing human activity. The aforementioned limitations describe steps for managing personal behavior or relationships or interactions between people, including social activities, teaching, and following rules or instructions. Specifically, determining work effort values based on activity records of a worker is considered to be steps for managing personal behavior. As such, claims 1, 11 and 21 recite concepts identified as abstract ideas.
The dependent claims recite limitations relative to the independent claims, including, for example:
…further comprising the step of classifying the activity records. [Claim 2],
…further comprising the step of enriching the activity records to compute additional information about the activities based on the level of effort that goes into the activities and activity metadata [Claim 3],
…further comprising the step of classifying the enriched activity records [Claim 4],
…further comprising presenting the classified activity records to a user [Claim 5],
…wherein the classified activity records are presented to a user in the form of a report [Claim 6].
The limitations of these dependent claims are merely narrowing the abstract idea identified in the independent claims, and thus, the dependent claims also recite abstract ideas.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. In particular, claims 1, 11 and 21 only recite the following additional elements –
…a processor unit configured to access one or more repositories… the one or more repositories…; …the processor unit…; …a graphical user interface [Claim 1],
…a processor unit including at least one hardware processor configured to access one or more repositories…; and at least one storage device coupled to the at least one hardware processor and for storing instructions that are operable, when executed by the at least one hardware processor, to cause the at least one hardware processor to perform operations comprising…; …the processor unit…; …a trained machine learning model, wherein the trained machine learning model… [Claim 11],
… A computer-readable storage medium storing a program of instructions executable by a machine…; accessing one or more repositories…; …a trained machine learning model, wherein the trained machine learning model… [Claim 21].
The apparatus and executable instructions are recited at a high-level of generality (see MPEP § 2106.05(a)), like the following MPEP example:
iii. Gathering and analyzing information using conventional techniques and displaying the result, TLI Communications, 823 F.3d at 612-13, 118 USPQ2d at 1747-48;
Furthermore, the computer implemented element is considered to amount to no more than mere instructions to apply the exception using a generic computer component (see MPEP 2106.05(f)), like the following MPEP example:
i. A commonplace business method or mathematical algorithm being applied on a general purpose computer, Alice Corp. Pty. Ltd. V. CLS Bank Int’l, 573 U.S. 208, 223, 110 USPQ2d 1976, 1983 (2014); Gottschalk v. Benson, 409 U.S. 63, 64, 175 USPQ 673, 674 (1972); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015);
Accordingly, these additional elements do not integrate the abstract idea into a practical application.
The remaining dependent claims do not recite any new additional elements, and thus do not integrate the abstract idea into a practical application.
Step 2B: Claims 1, 11 and 21 and their underlying limitations, steps, features and terms, considered both individually and as a whole, do not include additional elements that are sufficient to amount to significantly more than the judicial exception for the following reasons:
Independent claims 1, 11 and 21 only recite the following additional elements –
…a processor unit configured to access one or more repositories… the one or more repositories…; …the processor unit…; …a graphical user interface [Claim 1],
…a processor unit including at least one hardware processor configured to access one or more repositories…; and at least one storage device coupled to the at least one hardware processor and for storing instructions that are operable, when executed by the at least one hardware processor, to cause the at least one hardware processor to perform operations comprising…; …the processor unit…; …a trained machine learning model, wherein the trained machine learning model… [Claim 11],
… A computer-readable storage medium storing a program of instructions executable by a machine…; accessing one or more repositories…; …a trained machine learning model, wherein the trained machine learning model… [Claim 21].
These elements do not amount to significantly more than the abstract idea for the reasons discussed in 2A prong 2 with regard to MPEP 2106.05(a) and MPEP 2106.05(f). By the failure of the elements to integrate the abstract idea into a practical application there, the additional elements likewise fail to amount to an inventive concept that is significantly more than an abstract idea here, in Step 2B.
As such, both individually or in combination, these limitations do not add significantly more to the judicial exception.
The remaining dependent claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the dependent claims do not recite any new additional elements other than those mentioned in the independent claims, which amount to no more than mere instructions to apply the exception using a generic computer component (see MPEP 2106.05(f)). As such, these claims are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-7, 10-17 and 20-22 are rejected under 35 U.S.C. 103 as being unpatentable over Boss et al., U.S. Publication No. 2019/0012167 [hereinafter Boss] in view of Amit, U.S. Publication No. 2022/0122025 [hereinafter Amit].
Regarding Claim 1, Boss discloses …A method of assessing labor efficiency of knowledge work by means of a processor unit configured to access one or more repositories of a plurality of event records, the method comprising the process steps of: accessing the one or more repositories of a plurality of event records to obtain a plurality of event records, the event records being related to activities conducted by knowledge workers, each event record containing the identity of each worker associated with an event and either a time interval or a point in time when the event occurred (Boss, ¶ 41, Human resources data 208 may include data regarding a developer's salary. For example, a first developer may be deemed an effective developer and make a low salary, while a second developer that is twice as effective (discloses labor efficiency of knowledge work) may receive a ten times greater salary, which would mean the first developer may be more effective per unit of money. Moreover, there may be obtained from an entity's human resources department further developer information, including: their position, title, and the product they are developing, attrition, salaries, tenure, etc.), (Id., ¶ 26, A further program module 330 provides analytics to classify a developer's activity. Such a module 330 provides for a cognitive classification of a developer. In one embodiment, the module may implement or access a cognitive system, e.g., IBM's Watson® Natural Language Classifier and IBM's Watson® Explorer, or like, cognitive search and content analysis platform that may be leveraged to understand the detailed nature of the work each team member performs. In one embodiment, cognitive algorithms are applied to social data, human resources (HR) data and source code management data to determine exact activities performed by person, such as development, testing, product management, support, operations, etc. In one embodiment, this information may be combined with other data to obtain a detailed understanding of how each developer(s) is(are) spending their time on each task and activity. (discloses obtaining event records and identities for each worker)), (Id., ¶ 45, Retuning to FIG. 1, at 105, there are performed methods to cognitively analyze and classify activities by a team member(s) of the development team being evaluated. The cognitive classification enables users to create and understand cognitively, i.e., in detail, what each team member has done. In one embodiment, the system tracks on a periodic basis, e.g., daily, the activities (work) relating to the team member (developer), for example, relating to the code that was checked in by the team member, and how that user interacted and/or communicated, e.g., what was communicated (socially or collaboratively), in the context of development activities, e.g., in a social collaboration network or system, with colleagues while developing code. Such data may be considered unstructured, arrive from varying sources in no defined format, and may include text, speech, or other audio or video format), (Id., ¶ 24, Computing system 300 includes at least a processor 302, a memory 304, e.g., for storing an operating system and program instructions, a network interface 306, a display device 308, an input device 309, and any other features common to a computing device. In some aspects, computing system 300 may, for example, be any computing device that is configured to communicate with a social media web-site 320 or web- or cloud-based server (not shown) over a public or private communications network 99);
processing the plurality of event records to generate activity records associated with the time intervals during a time period indicating durations of active work time from the processed event records, the activity records containing information about work activities based on the event records, each activity record including contextual metadata indicative of a task classification, a worker identifier, and at least one of an activity type and a task complexity (Id., ¶ 45, Retuning to FIG. 1, at 105, there are performed methods to cognitively analyze and classify activities by a team member(s) of the development team being evaluated. The cognitive classification enables users to create and understand cognitively, i.e., in detail, what each team member has done. In one embodiment, the system tracks on a periodic basis, e.g., daily, the activities (work) relating to the team member (developer), (discloses event records indicating durations of active work time) for example, relating to the code that was checked in by the team member, and how that user interacted and/or communicated, e.g., what was communicated (socially or collaboratively), in the context of development activities, e.g., in a social collaboration network or system, with colleagues while developing code. Such data may be considered unstructured, arrive from varying sources in no defined format, and may include text, speech, or other audio or video format), (Id., ¶ 47, The interaction data and developer's activities, obtained at step 105, FIG. 1, may be combined to make various determinations relating to the developer and that developer's team. For example, the analysis methods determine how a developer's day(s) is(are) spent, e.g., whether/when and how often new code was checked in, (discloses generating activity records based on the event records) whether a developer was chatting with colleagues all day, or was given free coffee or tea, etc. This determination may be combined with other data obtained from the productivity reservoir or repository 250. For example, the developer's delivery of code may be combined with that developer's financial and other personal information and/or product financial information, e.g., revenue, and used to classify the developer's productivity. There is a trade-off between various activities the developer can be engaged in. For example, a developer who does nothing else during the day but generate lines of code would have one classification or role assigned, while a developer that develops efficient algorithms that reduces lines of code yet takes a lot of daily work breaks (discloses records indicating durations of active work times) and/or is more interactive or collaborative may be assigned a different role or classification), (Id., ¶ 38, FIG. 2 shows a linking of the various data sources 200 providing data and metadata for the tool to ingest and provide recommendations for improving productivity from the developers of the software development team. The data sources provide structured and unstructured data/metadata that is input and stored in a storage device, i.e., a productivity data reservoir 250. In one embodiment, the data/metadata is stored in the form of one or more productivity matrices (not shown) in the productivity data reservoir 250. More specifically, the productivity data reservoir is created by ingesting and transforming data and metadata for key product performance metrics across the company or like entity), (Id., ¶ 26, A further program module 330 provides analytics to classify a developer's activity (discloses task classifications) Such a module 330 provides for a cognitive classification of a developer. In one embodiment, the module may implement or access a cognitive system, e.g., IBM's Watson® Natural Language Classifier and IBM's Watson® Explorer, or like, cognitive search and content analysis platform that may be leveraged to understand the detailed nature of the work each team member performs. In one embodiment, cognitive algorithms are applied to social data, human resources (HR) data and source code management data to determine exact activities performed by person, such as development, testing, product management, support, operations, etc. In one embodiment, this information may be combined with other data to obtain a detailed understanding of how each developer(s) is(are) spending their time on each task and activity), (Id., Fig. 2, figure depicts metadata including a worker identifier and activity type (i.e. software changes));
PNG
media_image1.png
331
406
media_image1.png
Greyscale
allocating work effort based on the time intervals from the activity records to obtain a preliminary effort value indicating amount of effort for each activity record (Id., ¶ 41, Human resources data 208 may include data regarding a developer's salary. For example, a first developer may be deemed an effective developer and make a low salary, while a second developer that is twice as effective may receive a ten times greater salary, which would mean the first developer may be more effective per unit of money (discloses associating a cost with the developer activities to obtain an effort value (i.e. effectiveness per unit of money)). Moreover, there may be obtained from an entity's human resources department further developer information, including: their position, title, and the product they are developing, attrition, salaries, tenure, etc.), (Id., ¶ 33, In one embodiment, a cognitive training system/recommender program module 380 is stored in memory 350 and implements a model training algorithm that builds the knowledgebase 360 based on developer(s) activities and personal productivity, and product performance/financials over time. The model aggregates data for all developer's of specific development teams as will be described with respect to the method of FIG. 1), (Id., ¶ 62, FIG. 4 shows an example dashboard display 400 providing an operational read-out of productivity index 402 for a particular individual or team, with an indication of example top improvement areas, 410, 420, 430. The dashboard 400, in one embodiment, depicts an operational or benchmarking type approach by indicating comparisons of particular KPI attributes for a developer's team with other teams having average and more productive levels. In particular, data from the outputs of the model algorithms are formatted for display as shown in example dashboard 400 to provide visualizations for example KPI attributes 410 (“Attrition %” attrition rate attribute), 420 (“% Collocated” attribute), and 430 (“$k Cost Compensation per HC” or salary attribute). For example, from the dashboard 400 of FIG. 4, there is depicted for each improvement area 410-430, a developer's team's performance over a historical time period plotted against other teams that may have a similar composition, e.g., an average performing development team or compared against the best performing team. Via the dashboard, a user can track the performance for the displayed KPIs attributes over time can further show whether one individual or team's performance indicator goals have been met. Thus, for first example improvement area 410 (business objective parameter is “per cent attrition”), there is shown a developer's team's performance over a historical time period 411, and showing their performance as compared to a performance 414 averaged across many development teams, and further differentiated against the best performing team 417);
generating a final effort value by adjusting the preliminary effort value based on the contextual metadata included in the activity record (Id., ¶ 36, Based on the input data, the trained system/recommender module 350 takes the KPIs and builds the model that determines which of those KPI would be most predictive and/or important to address to increase productivity. The model learns, over time, which input parameters have the most improved effect on the output parameters for a particular productivity. The machine learning algorithms implemented in the model provides a set of predictive and descriptive data elements at a predetermined confidence level), (Id., ¶ 38, FIG. 2 shows a linking of the various data sources 200 providing data and metadata for the tool to ingest and provide recommendations for improving productivity from the developers of the software development team. The data sources provide structured and unstructured data/metadata that is input and stored in a storage device, i.e., a productivity data reservoir 250. In one embodiment, the data/metadata is stored in the form of one or more productivity matrices (not shown) in the productivity data reservoir 250. More specifically, the productivity data reservoir is created by ingesting and transforming data and metadata for key product performance metrics across the company or like entity), (Id., ¶ 58, In one embodiment, given the current and historical data that has been collected from the systems that measure user's behavior and personal productivity and/or product performance, a computer run model is generated that is configured to find which team(s) were most productive, and indicate which attributes (i.e., performance indicators) had the most impact upon the most productive team. In one embodiment, machine learning algorithms (such as multivariate linear models) are trained to establish quantitative relationships between the profiles of most effective development team (data derived from steps 102 and 105) and desired performance objectives (based on the model weights data input by the user at step 116 or determined at 120, FIG. 1). Support vector machines, multilinear models and regression based models can alternatively be trained and applied at step 118 to produce similar results based on the developers' personal and productivity data relating to the product and software product financial data received (discloses generating and adjusting an effort value)), (Id., ¶ 69, In on embodiment, in the computing system of FIG. 3, multivariate linear machine learning algorithms are implemented and the model is continuously updated and trained with most positive outcomes to correlate and learn and recognize the most important attributes (parameters) that most positively increases team productivity. These positive predicting attributes are alternatively referred to as key performance indicators (KPI) and may change over time. Consequently, the recommendations (KPI's) and their target values such as shown in recommendation table 500 of FIG. 5, may be change over time);
and generating a machine-readable report including the activity records and corresponding effort values, wherein the report is segmented by at least one of time period, task classification, and worker, and is configured for display via a graphical user interface (Id., ¶ 62, FIG. 4 shows an example dashboard display 400 providing an operational read-out of productivity index 402 for a particular individual or team, with an indication of example top improvement areas, 410, 420, 430. The dashboard 400, in one embodiment, depicts an operational or benchmarking type approach by indicating comparisons of particular KPI attributes for a developer's team with other teams having average and more productive levels. In particular, data from the outputs of the model algorithms are formatted for display as shown in example dashboard 400 to provide visualizations for example KPI attributes 410 (“Attrition %” attrition rate attribute), 420 (“% Collocated” attribute), and 430 (“$k Cost Compensation per HC” or salary attribute). For example, from the dashboard 400 of FIG. 4, there is depicted for each improvement area 410-430, a developer's team's performance over a historical time period plotted against other teams that may have a similar composition, e.g., an average performing development team or compared against the best performing team. Via the dashboard, a user can track the performance for the displayed KPIs attributes over time can further show whether one individual or team's performance indicator goals have been met. Thus, for first example improvement area 410 (business objective parameter is “per cent attrition”), there is shown a developer's team's performance over a historical time period 411, and showing their performance as compared to a performance 414 averaged across many development teams, and further differentiated against the best performing team 417), (Id., Fig. 4, figure depicts a report configured for GUI display segmented by quarterly time periods and including separate elements for tracking effort/improvements in each classified task)
PNG
media_image2.png
536
658
media_image2.png
Greyscale
While suggested in at least Fig. 4 and related text, Boss does not explicitly disclose … the step of processing involving determining, by the processor unit, intervals of active work time associated with one or more point-in-time-events using heuristics based on other events related to the same worker that occurred before or after the first event and contain points in time or time intervals when the other events occurred…
However, Amit discloses … the step of processing involving determining, by the processor unit, intervals of active work time associated with one or more point-in-time-events using heuristics based on other events related to the same worker that occurred before or after the first event and contain points in time or time intervals when the other events occurred… (Amit, ¶ 2, Effort estimation is the process used to predict the amount of effort (e.g., developer hours) needed to develop a software application. The predicted amount of effort can then be used as basis for predicting project costs and for determining an optimal allocation of software developer time. An estimate for a new project can be typically derived by considering characteristics of the new project as well as characteristics of previous similar project), (Id., ¶ 15, In some embodiments, computing a given task completion duration for a given task includes identifying a most recent previous software task completed by the given developer, and computing an amount of time between the given task and the identified most recent task (discloses determining intervals of active work time associated with point-in-time task completion events)), (Id., ¶ 48, Systems implementing embodiments of the present invention can investigate the gross time needed to complete a task (i.e., as opposed to the net-time). One reason for this is that data used by embodiments described herein to compute task time durations includes work interruptions such as coffee breaks, weekends, and vacation. While some of these periods are easy to identify and remove (e.g., weekends), the result may only be partial, and the data can become contaminated with biases and processing. Another reason is that since work interruptions will typically be part of future tasks, they should be considered when computing time estimates), (Id., ¶ 202, Labeling functions are typically heuristics that can be computed. (discloses heuristic event labeling) In some embodiments, task effort estimation model 26 may comprise a labeling function that classifies a given developer ID 82 as a bot if the given developer ID was found in more than 1,000 task commits 34 commits during a consecutive 12 month period. While 99.8% of the observed developers fell below this threshold, developers having certain behavior can reach it (e.g., by having many small commits, no code review process, more than a full-time work)).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the labor efficiency and activity record elements of Boss to include the heuristic event labeling elements of Amit in the analogous art of analysis of software development task effort estimation.
The motivation for doing so would have been to provide an “task effort estimation model 26 [that] can be designed to use machine learning techniques that improve performance with larger sets of task commits 34” (Amit, ¶ 158), wherein such improvements would benefit Boss’ method which seeks to provide “a computer-implemented method for improving the effectiveness and efficiency of software development operations” [Amit, ¶ 158; Boss, ¶ 6].
Regarding Claim 2, the combination of Boss and Amit discloses…The method as claimed in claim 1…
Boss further discloses …further comprising the step of classifying the activity records (Boss, ¶ 26, A further program module 330 provides analytics to classify a developer's activity. (discloses classifying activity records) Such a module 330 provides for a cognitive classification of a developer. In one embodiment, the module may implement or access a cognitive system, e.g., IBM's Watson® Natural Language Classifier and IBM's Watson® Explorer, or like, cognitive search and content analysis platform that may be leveraged to understand the detailed nature of the work each team member performs. In one embodiment, cognitive algorithms are applied to social data, human resources (HR) data and source code management data to determine exact activities performed by person, such as development, testing, product management, support, operations, etc. In one embodiment, this information may be combined with other data to obtain a detailed understanding of how each developer(s) is(are) spending their time on each task and activity).
Regarding Claim 3, the combination of Boss and Amit discloses…The method as claimed in claim 1…
Boss further discloses …further comprising the step of enriching the activity records by computing additional information about the activities based on the level of effort that goes into the activities and activity metadata (Boss, ¶ 37, In one aspect, the tool 300 runs a method 100 such as described with respect to FIG. 1. As shown at a first step 102, FIG. 1, the method continually ingests data and metadata for the tool, and transforms the ingested data/metadata (discloses enriching activity records using activity metadata) into a productivity data reservoir which may form the developer knowledgebase 360), (Id., ¶ 38, FIG. 2 shows a linking of the various data sources 200 providing data and metadata for the tool to ingest and provide recommendations for improving productivity from the developers of the software development team. The data sources provide structured and unstructured data/metadata that is input and stored in a storage device, i.e., a productivity data reservoir 250. In one embodiment, the data/metadata is stored in the form of one or more productivity matrices (not shown) in the productivity data reservoir 250. More specifically, the productivity data reservoir is created by ingesting and transforming data and metadata for key product performance metrics across the company or like entity), (Id., ¶ 47, The interaction data and developer's activities, obtained at step 105, FIG. 1, may be combined to make various determinations relating to the developer and that developer's team. For example, the analysis methods determine how a developer's day(s) is(are) spent, e.g., whether/when and how often new code was checked in, whether a developer was chatting with colleagues all day, or was given free coffee or tea, etc. This determination may be combined with other data obtained from the productivity reservoir or repository 250. For example, the developer's delivery of code may be combined with that developer's financial and other personal information and/or product financial information, e.g., revenue, and used to classify the developer's productivity. There is a trade-off between various activities the developer can be engaged in. For example, a developer who does nothing else during the day but generate lines of code would have one classification or role assigned, while a developer that develops efficient algorithms that reduces lines of code yet takes a lot of daily work breaks and/or is more interactive or collaborative may be assigned a different role or classification (discloses effort level)).
Regarding Claim 4, the combination of Boss and Amit discloses…The method as claimed in claim 3…
Boss further discloses …further comprising the step of classifying the enriched activity records (Boss, ¶ 47, The interaction data and developer's activities, obtained at step 105, FIG. 1, may be combined to make various determinations relating to the developer and that developer's team. For example, the analysis methods determine how a developer's day(s) is(are) spent, e.g., whether/when and how often new code was checked in, whether a developer was chatting with colleagues all day, or was given free coffee or tea, etc. This determination may be combined with other data obtained from the productivity reservoir or repository 250. For example, the developer's delivery of code may be combined with that developer's financial and other personal information and/or product financial information, e.g., revenue, and used to classify the developer's productivity (discloses classifying enriched activity records). There is a trade-off between various activities the developer can be engaged in. For example, a developer who does nothing else during the day but generate lines of code would have one classification or role assigned, while a developer that develops efficient algorithms that reduces lines of code yet takes a lot of daily work breaks and/or is more interactive or collaborative may be assigned a different role or classification).
Regarding Claim 5, the combination of Boss and Amit discloses…The method as claimed in claim 2…
Boss further discloses …further comprising presenting the classified activity records to a user (Boss, ¶ 45, Retuning to FIG. 1, at 105, there are performed methods to cognitively analyze and classify activities by a team member(s) of the development team being evaluated. The cognitive classification enables users to create and understand cognitively, i.e., in detail, what each team member has done. In one embodiment, the system tracks on a periodic basis, e.g., daily, the activities (work) relating to the team member (developer), for example, relating to the code that was checked in by the team member, and how that user interacted and/or communicated, e.g., what was communicated (socially or collaboratively), in the context of development activities, e.g., in a social collaboration network or system, with colleagues while developing code. Such data may be considered unstructured, arrive from varying sources in no defined format, and may include text, speech, or other audio or video format), (Id., ¶ 61, Returning to 118, FIG. 1, having determined the quantitative relationships between developers' profiles and aggregate productivity function, in one embodiment, the method continues to 125, where, based on the determined quantitative relationships between developers' profiles and aggregate productivity function, the method provides operational read-out and assessment. In one embodiment, the system leverages reporting dashboards to provide a visual display of an overall productivity index with recommended improvement areas for the development team).
Regarding Claim 6, the combination of Boss and Amit discloses… The method as claimed in claim 6…
Boss further discloses …further comprising presenting the classified activity records to a user (Boss, ¶ 45, Retuning to FIG. 1, at 105, there are performed methods to cognitively analyze and classify activities by a team member(s) of the development team being evaluated. The cognitive classification enables users to create and understand cognitively, i.e., in detail, what each team member has done. In one embodiment, the system tracks on a periodic basis, e.g., daily, the activities (work) relating to the team member (developer), for example, relating to the code that was checked in by the team member, and how that user interacted and/or communicated, e.g., what was communicated (socially or collaboratively), in the context of development activities, e.g., in a social collaboration network or system, with colleagues while developing code. Such data may be considered unstructured, arrive from varying sources in no defined format, and may include text, speech, or other audio or video format), (Id., ¶ 61, Returning to 118, FIG. 1, having determined the quantitative relationships between developers' profiles and aggregate productivity function, in one embodiment, the method continues to 125, where, based on the determined quantitative relationships between developers' profiles and aggregate productivity function, the method provides operational read-out and assessment. In one embodiment, the system leverages reporting dashboards to provide a visual display of an overall productivity index with recommended improvement areas for the development team).
Regarding Claim 7, the combination of Boss and Amit discloses…The method as claimed in claim 4…
Boss further discloses …wherein the classified enriched activity records are presented to a user (Boss, ¶ 45, Retuning to FIG. 1, at 105, there are performed methods to cognitively analyze and classify activities by a team member(s) of the development team being evaluated. The cognitive classification enables users to create and understand cognitively, i.e., in detail, what each team member has done. In one embodiment, the system tracks on a periodic basis, e.g., daily, the activities (work) relating to the team member (developer), for example, relating to the code that was checked in by the team member, and how that user interacted and/or communicated, e.g., what was communicated (socially or collaboratively), in the context of development activities, e.g., in a social collaboration network or system, with colleagues while developing code. Such data may be considered unstructured, arrive from varying sources in no defined format, and may include text, speech, or other audio or video format), (Id., ¶ 61, Returning to 118, FIG. 1, having determined the quantitative relationships between developers' profiles and aggregate productivity function, in one embodiment, the method continues to 125, where, based on the determined quantitative relationships between developers' profiles and aggregate productivity function, the method provides operational read-out and assessment. In one embodiment, the system leverages reporting dashboards to provide a visual display of an overall productivity index with recommended improvement areas for the development team), (Id., ¶ 62, FIG. 4 shows an example dashboard display 400 providing an operational read-out of productivity index 402 for a particular individual or team, with an indication of example top improvement areas, 410, 420, 430. The dashboard 400, in one embodiment, depicts an operational or benchmarking type approach by indicating comparisons of particular KPI attributes for a developer's team with other teams having average and more productive levels. In particular, data from the outputs of the model algorithms are formatted for display as shown in example dashboard 400 to provide visualizations for example KPI attributes 410 (“Attrition %” attrition rate attribute), 420 (“% Collocated” attribute), and 430 (“$k Cost Compensation per HC” or salary attribute). For example, from the dashboard 400 of FIG. 4, there is depicted for each improvement area 410-430, a developer's team's performance over a historical time period plotted against other teams that may have a similar composition, e.g., an average performing development team or compared against the best performing team. Via the dashboard, a user can track the performance for the displayed KPIs attributes over time can further show whether one individual or team's performance indicator goals have been met. Thus, for first example improvement area 410 (business objective parameter is “per cent attrition”), there is shown a developer's team's performance over a historical time period 411, and showing their performance as compared to a performance 414 averaged across many development teams, and further differentiated against the best performing team 417).
Regarding Claim 10, the combination of Boss and Amit discloses…The method as claimed in claim 1…
Boss further discloses …wherein the knowledge work is software development work (Boss, ¶ 1, The present invention generally relates to systems and methods for maximizing the productivity of software development teams comprising plural individuals).
Regarding Claim 11, Boss discloses …A system for assessing labor efficiency of knowledge work, the system comprising: a processor unit including at least one hardware processor configured to access one or more repositories of the plurality of event records; and at least one storage device coupled to the at least one hardware processor and for storing instructions that are operable, when executed by the at least one hardware processor, to cause the at least one hardware processor to perform operations comprising: accessing the one or more repositories of the plurality of event records to obtain a plurality of event records, the event records being related to activities conducted by knowledge workers, each event record containing the identity of each worker associated with an event, and either a time interval or a point in time when the event occurred (Boss, ¶ 41, Human resources data 208 may include data regarding a developer's salary. For example, a first developer may be deemed an effective developer and make a low salary, while a second developer that is twice as effective (discloses labor efficiency of knowledge work) may receive a ten times greater salary, which would mean the first developer may be more effective per unit of money. Moreover, there may be obtained from an entity's human resources department further developer information, including: their position, title, and the product they are developing, attrition, salaries, tenure, etc.), (Id., ¶ 26, A further program module 330 provides analytics to classify a developer's activity. Such a module 330 provides for a cognitive classification of a developer. In one embodiment, the module may implement or access a cognitive system, e.g., IBM's Watson® Natural Language Classifier and IBM's Watson® Explorer, or like, cognitive search and content analysis platform that may be leveraged to understand the detailed nature of the work each team member performs. In one embodiment, cognitive algorithms are applied to social data, human resources (HR) data and source code management data to determine exact activities performed by person, such as development, testing, product management, support, operations, etc. In one embodiment, this information may be combined with other data to obtain a detailed understanding of how each developer(s) is(are) spending their time on each task and activity. (discloses obtaining event records and identities for each worker)), (Id., ¶ 45, Retuning to FIG. 1, at 105, there are performed methods to cognitively analyze and classify activities by a team member(s) of the development team being evaluated. The cognitive classification enables users to create and understand cognitively, i.e., in detail, what each team member has done. In one embodiment, the system tracks on a periodic basis, e.g., daily, the activities (work) relating to the team member (developer), for example, relating to the code that was checked in by the team member, and how that user interacted and/or communicated, e.g., what was communicated (socially or collaboratively), in the context of development activities, e.g., in a social collaboration network or system, with colleagues while developing code. Such data may be considered unstructured, arrive from varying sources in no defined format, and may include text, speech, or other audio or video format), (Id., ¶ 6, there is provided a computer-implemented method for improving the effectiveness and efficiency of software development operations. The method comprises: for each of a plurality of teams, receiving, at a hardware processor, activities data representing each team member's productivity relating to a product currently being developed; and receiving, at the hardware processor, further activities data relating to each team member's collaborative interactions; and performing, at the hardware processor, a cognitive classification of the activities for each of members of each team, and aggregating classifications of team members of each team to generate a respective individual team profile; generating, at the hardware processor, a respective productivity index function representing desired performance objectives for the team; and inputting, using the hardware processor, an individual team profile data and corresponding team productivity function index data to a model trained to correlate the individual team profile to learned positive attributes associated with a most productive development team of the plurality of teams; and generating, using the trained model, an output recommending which performance attributes of an individual team can be improved based on the model's correlating that team's data with the learned positive predicting attributes for increasing productivity of that team), (Id., ¶ 24, Computing system 300 includes at least a processor 302, a memory 304, e.g., for storing an operating system and program instructions, a network interface 306, a display device 308, an input device 309, and any other features common to a computing device. In some aspects, computing system 300 may, for example, be any computing device that is configured to communicate with a social media web-site 320 or web- or cloud-based server (not shown) over a public or private communications network 99);
processing the plurality of event records to generate activity records associated with the time intervals during a time period indicating durations of active work time from the processed event records, the activity records containing information about work activities based on the event records, each activity record including contextual metadata indicative of a task classification, a worker identifier, and at least one of an activity type and a task complexity (Id., ¶ 45, Retuning to FIG. 1, at 105, there are performed methods to cognitively analyze and classify activities by a team member(s) of the development team being evaluated. The cognitive classification enables users to create and understand cognitively, i.e., in detail, what each team member has done. In one embodiment, the system tracks on a periodic basis, e.g., daily, the activities (work) relating to the team member (developer), (discloses event records indicating durations of active work time) for example, relating to the code that was checked in by the team member, and how that user interacted and/or communicated, e.g., what was communicated (socially or collaboratively), in the context of development activities, e.g., in a social collaboration network or system, with colleagues while developing code. Such data may be considered unstructured, arrive from varying sources in no defined format, and may include text, speech, or other audio or video format), (Id., ¶ 47, The interaction data and developer's activities, obtained at step 105, FIG. 1, may be combined to make various determinations relating to the developer and that developer's team. For example, the analysis methods determine how a developer's day(s) is(are) spent, e.g., whether/when and how often new code was checked in, (discloses generating activity records based on the event records) whether a developer was chatting with colleagues all day, or was given free coffee or tea, etc. This determination may be combined with other data obtained from the productivity reservoir or repository 250. For example, the developer's delivery of code may be combined with that developer's financial and other personal information and/or product financial information, e.g., revenue, and used to classify the developer's productivity. There is a trade-off between various activities the developer can be engaged in. For example, a developer who does nothing else during the day but generate lines of code would have one classification or role assigned, while a developer that develops efficient algorithms that reduces lines of code yet takes a lot of daily work breaks (discloses records indicating durations of active work times) and/or is more interactive or collaborative may be assigned a different role or classification), (Id., ¶ 38, FIG. 2 shows a linking of the various data sources 200 providing data and metadata for the tool to ingest and provide recommendations for improving productivity from the developers of the software development team. The data sources provide structured and unstructured data/metadata that is input and stored in a storage device, i.e., a productivity data reservoir 250. In one embodiment, the data/metadata is stored in the form of one or more productivity matrices (not shown) in the productivity data reservoir 250. More specifically, the productivity data reservoir is created by ingesting and transforming data and metadata for key product performance metrics across the company or like entity), (Id., ¶ 26, A further program module 330 provides analytics to classify a developer's activity (discloses task classifications) Such a module 330 provides for a cognitive classification of a developer. In one embodiment, the module may implement or access a cognitive system, e.g., IBM's Watson® Natural Language Classifier and IBM's Watson® Explorer, or like, cognitive search and content analysis platform that may be leveraged to understand the detailed nature of the work each team member performs. In one embodiment, cognitive algorithms are applied to social data, human resources (HR) data and source code management data to determine exact activities performed by person, such as development, testing, product management, support, operations, etc. In one embodiment, this information may be combined with other data to obtain a detailed understanding of how each developer(s) is(are) spending their time on each task and activity), (Id., Fig. 2, figure depicts metadata including a worker identifier and activity type (i.e. software changes));
PNG
media_image1.png
331
406
media_image1.png
Greyscale
allocating work effort based on the time intervals from the activity records to obtain a preliminary effort value indicating amount of effort for each activity record (Id., ¶ 41, Human resources data 208 may include data regarding a developer's salary. For example, a first developer may be deemed an effective developer and make a low salary, while a second developer that is twice as effective may receive a ten times greater salary, which would mean the first developer may be more effective per unit of money (discloses associating a cost with the developer activities to obtain an effort value (i.e. effectiveness per unit of money)). Moreover, there may be obtained from an entity's human resources department further developer information, including: their position, title, and the product they are developing, attrition, salaries, tenure, etc.), (Id., ¶ 33, In one embodiment, a cognitive training system/recommender program module 380 is stored in memory 350 and implements a model training algorithm that builds the knowledgebase 360 based on developer(s) activities and personal productivity, and product performance/financials over time. The model aggregates data for all developer's of specific development teams as will be described with respect to the method of FIG. 1), (Id., ¶ 62, FIG. 4 shows an example dashboard display 400 providing an operational read-out of productivity index 402 for a particular individual or team, with an indication of example top improvement areas, 410, 420, 430. The dashboard 400, in one embodiment, depicts an operational or benchmarking type approach by indicating comparisons of particular KPI attributes for a developer's team with other teams having average and more productive levels. In particular, data from the outputs of the model algorithms are formatted for display as shown in example dashboard 400 to provide visualizations for example KPI attributes 410 (“Attrition %” attrition rate attribute), 420 (“% Collocated” attribute), and 430 (“$k Cost Compensation per HC” or salary attribute). For example, from the dashboard 400 of FIG. 4, there is depicted for each improvement area 410-430, a developer's team's performance over a historical time period plotted against other teams that may have a similar composition, e.g., an average performing development team or compared against the best performing team. Via the dashboard, a user can track the performance for the displayed KPIs attributes over time can further show whether one individual or team's performance indicator goals have been met. Thus, for first example improvement area 410 (business objective parameter is “per cent attrition”), there is shown a developer's team's performance over a historical time period 411, and showing their performance as compared to a performance 414 averaged across many development teams, and further differentiated against the best performing team 417);
and generating a final effort value for each activity record by applying a trained data model, wherein the trained data model is configured to adjust the preliminary effort value based on contextual metadata included in the activity record (Id., ¶ 36, Based on the input data, the trained system/recommender module 350 takes the KPIs and builds the model that determines which of those KPI would be most predictive and/or important to address to increase productivity. The model learns, over time, which input parameters have the most improved effect on the output parameters for a particular productivity. The machine learning algorithms implemented in the model provides a set of predictive and descriptive data elements at a predetermined confidence level. (discloses trained machine learning model)), (Id., ¶ 58, In one embodiment, given the current and historical data that has been collected from the systems that measure user's behavior and personal productivity and/or product performance, a computer run model is generated that is configured to find which team(s) were most productive, and indicate which attributes (i.e., performance indicators) had the most impact upon the most productive team. In one embodiment, machine learning algorithms (such as multivariate linear models) are trained to establish quantitative relationships between the profiles of most effective development team (data derived from steps 102 and 105) and desired performance objectives (based on the model weights data input by the user at step 116 or determined at 120, FIG. 1). Support vector machines, multilinear models and regression based models can alternatively be trained and applied at step 118 to produce similar results based on the developers' personal and productivity data relating to the product and software product financial data received (discloses generating and adjusting an effort value)), (Id., ¶ 69, In on embodiment, in the computing system of FIG. 3, multivariate linear machine learning algorithms are implemented and the model is continuously updated and trained with most positive outcomes to correlate and learn and recognize the most important attributes (parameters) that most positively increases team productivity. These positive predicting attributes are alternatively referred to as key performance indicators (KPI) and may change over time. Consequently, the recommendations (KPI's) and their target values such as shown in recommendation table 500 of FIG. 5, may be change over time).
While suggested in at least Fig. 4 and related text, Boss does not explicitly disclose … the operation of processing involving determining, by the processor unit, intervals of active work time associated with one or more point-in-time-events using heuristics based on other events related to the same worker that occurred before or after the first event and contain points in time or time intervals when the other events occurred…
However, Amit discloses … the operation of processing involving determining, by the processor unit, intervals of active work time associated with one or more point-in-time-events using heuristics based on other events related to the same worker that occurred before or after the first event and contain points in time or time intervals when the other events occurred… (Amit, ¶ 2, Effort estimation is the process used to predict the amount of effort (e.g., developer hours) needed to develop a software application. The predicted amount of effort can then be used as basis for predicting project costs and for determining an optimal allocation of software developer time. An estimate for a new project can be typically derived by considering characteristics of the new project as well as characteristics of previous similar project), (Id., ¶ 15, In some embodiments, computing a given task completion duration for a given task includes identifying a most recent previous software task completed by the given developer, and computing an amount of time between the given task and the identified most recent task (discloses determining intervals of active work time associated with point-in-time task completion events)), (Id., ¶ 48, Systems implementing embodiments of the present invention can investigate the gross time needed to complete a task (i.e., as opposed to the net-time). One reason for this is that data used by embodiments described herein to compute task time durations includes work interruptions such as coffee breaks, weekends, and vacation. While some of these periods are easy to identify and remove (e.g., weekends), the result may only be partial, and the data can become contaminated with biases and processing. Another reason is that since work interruptions will typically be part of future tasks, they should be considered when computing time estimates), (Id., ¶ 202, Labeling functions are typically heuristics that can be computed. (discloses heuristic event labeling) In some embodiments, task effort estimation model 26 may comprise a labeling function that classifies a given developer ID 82 as a bot if the given developer ID was found in more than 1,000 task commits 34 commits during a consecutive 12 month period. While 99.8% of the observed developers fell below this threshold, developers having certain behavior can reach it (e.g., by having many small commits, no code review process, more than a full-time work)).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the labor efficiency and activity record elements of Boss to include the heuristic event labeling elements of Amit in the analogous art of analysis of software development task effort estimation for the same reasons as stated for claim 1.
Regarding Claims 12-17 and 20, these claims recite substantially the same limitations as stated in claims 2-7 and 10, respectively, and are rejected for the same reasons as stated above.
Regarding Claim 21, Boss discloses … A computer-readable storage medium storing a program of instructions executable by a machine to perform operations, for assessing labor efficiency of knowledge work, comprising: accessing one or more repositories of the plurality of event records to obtain a plurality of event records, the event records being related to activities conducted by knowledge workers, each event record containing the identity of each worker associated with an event, and either a time interval or a point in time when the event occurred (Boss, ¶ 41, Human resources data 208 may include data regarding a developer's salary. For example, a first developer may be deemed an effective developer and make a low salary, while a second developer that is twice as effective (discloses labor efficiency of knowledge work) may receive a ten times greater salary, which would mean the first developer may be more effective per unit of money. Moreover, there may be obtained from an entity's human resources department further developer information, including: their position, title, and the product they are developing, attrition, salaries, tenure, etc.), (Id., ¶ 26, A further program module 330 provides analytics to classify a developer's activity. Such a module 330 provides for a cognitive classification of a developer. In one embodiment, the module may implement or access a cognitive system, e.g., IBM's Watson® Natural Language Classifier and IBM's Watson® Explorer, or like, cognitive search and content analysis platform that may be leveraged to understand the detailed nature of the work each team member performs. In one embodiment, cognitive algorithms are applied to social data, human resources (HR) data and source code management data to determine exact activities performed by person, such as development, testing, product management, support, operations, etc. In one embodiment, this information may be combined with other data to obtain a detailed understanding of how each developer(s) is(are) spending their time on each task and activity. (discloses obtaining event records and identities for each worker)), (Id., ¶ 45, Retuning to FIG. 1, at 105, there are performed methods to cognitively analyze and classify activities by a team member(s) of the development team being evaluated. The cognitive classification enables users to create and understand cognitively, i.e., in detail, what each team member has done. In one embodiment, the system tracks on a periodic basis, e.g., daily, the activities (work) relating to the team member (developer), for example, relating to the code that was checked in by the team member, and how that user interacted and/or communicated, e.g., what was communicated (socially or collaboratively), in the context of development activities, e.g., in a social collaboration network or system, with colleagues while developing code. Such data may be considered unstructured, arrive from varying sources in no defined format, and may include text, speech, or other audio or video format), (Id., ¶ 79, The present invention may be a system, a method, and/or a computer program product at any possible technical detail level of integration. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention), (Id., ¶ 24, Computing system 300 includes at least a processor 302, a memory 304, e.g., for storing an operating system and program instructions, a network interface 306, a display device 308, an input device 309, and any other features common to a computing device. In some aspects, computing system 300 may, for example, be any computing device that is configured to communicate with a social media web-site 320 or web- or cloud-based server (not shown) over a public or private communications network 99);
processing the plurality of event records to generate activity records associated with the time intervals during a time period indicating durations of active work time from the processed event records, the activity records containing information about work activities based on the event records, each activity record including contextual metadata indicative of a task classification, a worker identifier, and at least one of an activity type and a task complexity (Id., ¶ 45, Retuning to FIG. 1, at 105, there are performed methods to cognitively analyze and classify activities by a team member(s) of the development team being evaluated. The cognitive classification enables users to create and understand cognitively, i.e., in detail, what each team member has done. In one embodiment, the system tracks on a periodic basis, e.g., daily, the activities (work) relating to the team member (developer), (discloses event records indicating durations of active work time) for example, relating to the code that was checked in by the team member, and how that user interacted and/or communicated, e.g., what was communicated (socially or collaboratively), in the context of development activities, e.g., in a social collaboration network or system, with colleagues while developing code. Such data may be considered unstructured, arrive from varying sources in no defined format, and may include text, speech, or other audio or video format), (Id., ¶ 47, The interaction data and developer's activities, obtained at step 105, FIG. 1, may be combined to make various determinations relating to the developer and that developer's team. For example, the analysis methods determine how a developer's day(s) is(are) spent, e.g., whether/when and how often new code was checked in, (discloses generating activity records based on the event records) whether a developer was chatting with colleagues all day, or was given free coffee or tea, etc. This determination may be combined with other data obtained from the productivity reservoir or repository 250. For example, the developer's delivery of code may be combined with that developer's financial and other personal information and/or product financial information, e.g., revenue, and used to classify the developer's productivity. There is a trade-off between various activities the developer can be engaged in. For example, a developer who does nothing else during the day but generate lines of code would have one classification or role assigned, while a developer that develops efficient algorithms that reduces lines of code yet takes a lot of daily work breaks (discloses records indicating durations of active work times) and/or is more interactive or collaborative may be assigned a different role or classification), (Id., ¶ 38, FIG. 2 shows a linking of the various data sources 200 providing data and metadata for the tool to ingest and provide recommendations for improving productivity from the developers of the software development team. The data sources provide structured and unstructured data/metadata that is input and stored in a storage device, i.e., a productivity data reservoir 250. In one embodiment, the data/metadata is stored in the form of one or more productivity matrices (not shown) in the productivity data reservoir 250. More specifically, the productivity data reservoir is created by ingesting and transforming data and metadata for key product performance metrics across the company or like entity), (Id., ¶ 26, A further program module 330 provides analytics to classify a developer's activity (discloses task classifications) Such a module 330 provides for a cognitive classification of a developer. In one embodiment, the module may implement or access a cognitive system, e.g., IBM's Watson® Natural Language Classifier and IBM's Watson® Explorer, or like, cognitive search and content analysis platform that may be leveraged to understand the detailed nature of the work each team member performs. In one embodiment, cognitive algorithms are applied to social data, human resources (HR) data and source code management data to determine exact activities performed by person, such as development, testing, product management, support, operations, etc. In one embodiment, this information may be combined with other data to obtain a detailed understanding of how each developer(s) is(are) spending their time on each task and activity), (Id., Fig. 2, figure depicts metadata including a worker identifier and activity type (i.e. software changes));
PNG
media_image1.png
331
406
media_image1.png
Greyscale
allocating work effort based on the time intervals from the activity records to obtain a preliminary effort value indicating amount of effort for each activity record (Id., ¶ 41, Human resources data 208 may include data regarding a developer's salary. For example, a first developer may be deemed an effective developer and make a low salary, while a second developer that is twice as effective may receive a ten times greater salary, which would mean the first developer may be more effective per unit of money (discloses associating a cost with the developer activities to obtain an effort value (i.e. effectiveness per unit of money)). Moreover, there may be obtained from an entity's human resources department further developer information, including: their position, title, and the product they are developing, attrition, salaries, tenure, etc.), (Id., ¶ 33, In one embodiment, a cognitive training system/recommender program module 380 is stored in memory 350 and implements a model training algorithm that builds the knowledgebase 360 based on developer(s) activities and personal productivity, and product performance/financials over time. The model aggregates data for all developer's of specific development teams as will be described with respect to the method of FIG. 1), (Id., ¶ 62, FIG. 4 shows an example dashboard display 400 providing an operational read-out of productivity index 402 for a particular individual or team, with an indication of example top improvement areas, 410, 420, 430. The dashboard 400, in one embodiment, depicts an operational or benchmarking type approach by indicating comparisons of particular KPI attributes for a developer's team with other teams having average and more productive levels. In particular, data from the outputs of the model algorithms are formatted for display as shown in example dashboard 400 to provide visualizations for example KPI attributes 410 (“Attrition %” attrition rate attribute), 420 (“% Collocated” attribute), and 430 (“$k Cost Compensation per HC” or salary attribute). For example, from the dashboard 400 of FIG. 4, there is depicted for each improvement area 410-430, a developer's team's performance over a historical time period plotted against other teams that may have a similar composition, e.g., an average performing development team or compared against the best performing team. Via the dashboard, a user can track the performance for the displayed KPIs attributes over time can further show whether one individual or team's performance indicator goals have been met. Thus, for first example improvement area 410 (business objective parameter is “per cent attrition”), there is shown a developer's team's performance over a historical time period 411, and showing their performance as compared to a performance 414 averaged across many development teams, and further differentiated against the best performing team 417);
and generating a final effort value for each activity record by applying a trained machine learning model, wherein the trained machine learning model is configured to adjust the preliminary effort value based on contextual metadata included in the activity record (Id., ¶ 36, Based on the input data, the trained system/recommender module 350 takes the KPIs and builds the model that determines which of those KPI would be most predictive and/or important to address to increase productivity. The model learns, over time, which input parameters have the most improved effect on the output parameters for a particular productivity. The machine learning algorithms implemented in the model provides a set of predictive and descriptive data elements at a predetermined confidence level. (discloses trained machine learning model)), (Id., ¶ 58, In one embodiment, given the current and historical data that has been collected from the systems that measure user's behavior and personal productivity and/or product performance, a computer run model is generated that is configured to find which team(s) were most productive, and indicate which attributes (i.e., performance indicators) had the most impact upon the most productive team. In one embodiment, machine learning algorithms (such as multivariate linear models) are trained to establish quantitative relationships between the profiles of most effective development team (data derived from steps 102 and 105) and desired performance objectives (based on the model weights data input by the user at step 116 or determined at 120, FIG. 1). Support vector machines, multilinear models and regression based models can alternatively be trained and applied at step 118 to produce similar results based on the developers' personal and productivity data relating to the product and software product financial data received (discloses generating and adjusting an effort value)), (Id., ¶ 69, In on embodiment, in the computing system of FIG. 3, multivariate linear machine learning algorithms are implemented and the model is continuously updated and trained with most positive outcomes to correlate and learn and recognize the most important attributes (parameters) that most positively increases team productivity. These positive predicting attributes are alternatively referred to as key performance indicators (KPI) and may change over time. Consequently, the recommendations (KPI's) and their target values such as shown in recommendation table 500 of FIG. 5, may be change over time).
While suggested in at least Fig. 4 and related text, Boss does not explicitly disclose … the operation of processing involving determining intervals of active work time associated with one or more point-in-time-events using heuristics based on other events related to the same worker that occurred before or after the first event and contain points in time or time intervals when the other events occurred…
However, Amit discloses … the operation of processing involving determining intervals of active work time associated with one or more point-in-time-events using heuristics based on other events related to the same worker that occurred before or after the first event and contain points in time or time intervals when the other events occurred… (Amit, ¶ 2, Effort estimation is the process used to predict the amount of effort (e.g., developer hours) needed to develop a software application. The predicted amount of effort can then be used as basis for predicting project costs and for determining an optimal allocation of software developer time. An estimate for a new project can be typically derived by considering characteristics of the new project as well as characteristics of previous similar project), (Id., ¶ 15, In some embodiments, computing a given task completion duration for a given task includes identifying a most recent previous software task completed by the given developer, and computing an amount of time between the given task and the identified most recent task (discloses determining intervals of active work time associated with point-in-time task completion events)), (Id., ¶ 48, Systems implementing embodiments of the present invention can investigate the gross time needed to complete a task (i.e., as opposed to the net-time). One reason for this is that data used by embodiments described herein to compute task time durations includes work interruptions such as coffee breaks, weekends, and vacation. While some of these periods are easy to identify and remove (e.g., weekends), the result may only be partial, and the data can become contaminated with biases and processing. Another reason is that since work interruptions will typically be part of future tasks, they should be considered when computing time estimates), (Id., ¶ 202, Labeling functions are typically heuristics that can be computed. (discloses heuristic event labeling) In some embodiments, task effort estimation model 26 may comprise a labeling function that classifies a given developer ID 82 as a bot if the given developer ID was found in more than 1,000 task commits 34 commits during a consecutive 12 month period. While 99.8% of the observed developers fell below this threshold, developers having certain behavior can reach it (e.g., by having many small commits, no code review process, more than a full-time work)).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the labor efficiency and activity record elements of Boss to include the heuristic event labeling elements of Amit in the analogous art of analysis of software development task effort estimation for the same reasons as stated for claim 1.
Regarding Claim 22, this claim recites substantially the same limitations as stated in claim 10, and is rejected for the same reasons as stated above.
Claims 8-9 and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Boss in view of Amit and in further view of Mohanty et al., U.S. Publication No. 2023/0066141 [hereinafter Mohanty].
Regarding Claim 8, the combination of Boss and Amit discloses …The method as claimed in claim 1…
While suggested in at least Fig. 4 and related text, Boss does not explicitly disclose … further comprising the step of processing the event data to add or change metadata related to each event to obtain preprocessed event data.
However, Mohanty discloses … further comprising the step of processing the event data to add or change metadata to form updated event records (Mohanty, ¶ 96, Referring back to FIG. 4, the activity tracker 214 includes an activity watcher 404 that monitors the various stages (e.g., the plurality of stages 216) of the development of the software product. The activity watcher 404 may be configured to track and monitor user input and actions performed using the service application 112 at each stage (e.g., the define stage 216a, the design stage 216b, the development stage 216c, and deployment stage 216d) of the plurality of stages 216. In other words, the activity watcher 404 is configured to track a plurality of activities executed during the plurality of stages 216 of the development of the software product), (Id., ¶ 77, The user action designer 202 records (e.g., captures) a first plurality of user actions performed by the user on the UI. In other words, the first plurality of user actions include user actions associated with the plurality of stages 216 (e.g., one of the plurality of stages 216) and performed by the user on the UI rendered on the first user device 102a. The user action designer 202 may generate, based on the recorded first plurality of user actions, metadata associated with the first plurality of user actions. The generated metadata may include, but is not limited to, the selection of the first set of operations, the selection of the first technology of the second plurality of technologies, provision of a set of execution parameters for the execution of the first set of operations, or the like. The set of execution parameters (e.g., configuration parameters) may refer to one or more parameters/parameter values provided (e.g., entered) by the user (discloses execution parameter metadata generated by user configuration parameter input) (e.g., the plurality of users) for the execution of the first set of operations. Examples of the execution parameters are provided in later figures. Further, the first plurality of user actions may include any other action performed by the user for the execution of the first set of operations. For example, the first plurality of user actions may include entry of time schedule (discloses event records) (e.g., a start time and/or stop time) for the execution of the first set of operations. The first plurality of user actions may further include provision of one or more parameters (or parameter values) for the execution of the first set of operations. For example, if the first set of operations corresponds to data ingestion, the first plurality of user actions may include one or more user actions associated with a definition of data source from which data is to be extracted, one or more user actions associated with data transformation to be performed on the extracted data, or the like. Similarly, if the first set of operations corresponds to repository management or code commits, the first plurality of user actions may include one or more user actions associated with an entry of an identifier of a repository to which the code is to be committed), (Id., ¶ 78, In an exemplary scenario, the user may select a data source (e.g., comma separated values or CSV file) and a data ingestion technology (e.g., DataBricks®, Hadoop®, or the like) for developing or implementing a data pipeline for the software product. In such a scenario, the generated metadata may be indicative of the selection of the data source (e.g., CSV file) and the data ingestion technology (e.g., DataBricks®, Hadoop®, or the like) for the development of the data pipeline for the software product by the user (e.g., the plurality of users). In another exemplary scenario, the user may select a deployment mode/deployment technology (e.g., Docker®, Kubernetes®, or Terraform®) and a cloud technology (e.g., Microsoft Azure®, Amazon AWS®, or Google Cloud Platform®) for deploying the software product. In such a scenario, the generated metadata may be indicative of the selection of the cloud technology and the deployment mode (e.g., Docker®, Kubernetes®, or Terraform®) by the user for the deployment of the software product. For the sake of brevity, it is assumed that the first plurality of user actions may include the selection of the first set of operations and the first technology 11).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the labor efficiency and activity record elements of Boss and the heuristic event labeling elements of Amit to include the metadata changing elements of Mohanty in the analogous art of analysis of value stream for software products.
The motivation for doing so would have been to provide “improved tracking of development of software products” (Mohanty, ¶ 5), wherein such improvements would benefit Amit’s method which seeks to provide an “task effort estimation model 26 [that] can be designed to use machine learning techniques that improve performance with larger sets of task commits 34” (Amit, ¶ 158), and wherein such improvements would further benefit Boss’ method which seeks to provide “a computer-implemented method for improving the effectiveness and efficiency of software development operations” [Mohanty, ¶ 5; Amit, ¶ 158; Boss, ¶ 6].
Regarding Claim 9, the combination of Boss, Amit and Mohanty discloses…The method as claimed in claim 8…
While suggested in at least Fig. 4 and related text of Boss, the combination of Boss and Amit does not explicitly disclose …wherein the step of generating activity records is performed with the updated event records.
However, Mohanty discloses … wherein the step of generating activity records is performed with the updated event records (Mohanty, ¶ 78, In an exemplary scenario, the user may select a data source (e.g., comma separated values or CSV file) and a data ingestion technology (e.g., DataBricks®, Hadoop®, or the like) for developing or implementing a data pipeline for the software product. In such a scenario, the generated metadata may be indicative of the selection of the data source (e.g., CSV file) and the data ingestion technology (e.g., DataBricks®, Hadoop®, or the like) for the development of the data pipeline for the software product by the user (e.g., the plurality of users). In another exemplary scenario, the user may select a deployment mode/deployment technology (e.g., Docker®, Kubernetes®, or Terraform®) and a cloud technology (e.g., Microsoft Azure®, Amazon AWS®, or Google Cloud Platform®) for deploying the software product. In such a scenario, the generated metadata may be indicative of the selection of the cloud technology and the deployment mode (e.g., Docker®, Kubernetes®, or Terraform®) by the user for the deployment of the software product. For the sake of brevity, it is assumed that the first plurality of user actions may include the selection of the first set of operations and the first technology 11), (Id., ¶ 79, The user action designer 202 stores, in the user action catalog 203, the metadata associated with the first plurality of user actions. The user action script compiler 204 may generate a first set of user action scripts based on the metadata stored in the user action catalog 203. (discloses calculating activity records with the preprocessed event data obtained through metadata changes) The generated first set of user action scripts may include data, application program interfaces (APIs), code, database scripts, configuration files, or the like that correspond to the metadata. In one example, the first set of user action scripts may be indicative of defined ideas, features, and/or business requirements for documentation of the define stage 216a of the product development of the software product. In another example, the first set of user action scripts may be indicative of one or more UI design mock-ups for designing different facets of the software product. In another example, the first set of user action scripts may be indicative of the selection of a data source (e.g., a CSV file) and a data ingestion technology (e.g., Databricks®) for an implementation of the data pipeline. The first set of user action scripts may correspond to a format or a programming language (e.g., a proprietary language, an open-source language, or the like) natively supported by the service application 112).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the labor efficiency and activity record elements of Boss and the heuristic event labeling elements of Amit to include the metadata changing elements of Mohanty in the analogous art of analysis of value stream for software products for the same reasons as stated for claim 8.
Regarding Claims 18-19, these claims recite substantially the same limitations as stated in claims 8-9, respectively, and are rejected for the same reasons as stated above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Dyer et al., U.S. Publication No. 2017/0301039 discloses devices, systems, and methods of activity-based monitoring and incentivization.
Silverstein et al., U.S. Publication No. 2022/0012666 discloses a system and method for optimum alternative recommendation for personal productivity efficiency.
Fanning et al., U.S. Publication No. 2024/0095027 discloses a software development quality assessment.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS D BOLEN whose telephone number is (408)918-7631. The examiner can normally be reached Monday - Friday 8:00 AM - 5:00 PM PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Patty Munson can be reached at (571) 270-5396. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NICHOLAS D BOLEN/ Examiner, Art Unit 3624 /PATRICIA H MUNSON/Supervisory Patent Examiner, Art Unit 3624
Prosecution Insights