DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on and between 6/25/2025 and 4/30/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Notice to Applicant
Claims 1, 16 and 20 are presently amended.
Claims 1-20 are pending.
Response to Amendment
Applicant’s amendments are acknowledged.
Response to Arguments
Applicant' s arguments filed 3/5/2026 have been fully considered in view of further consideration of statutory law, Office policy, precedential common law, and the cited prior art as necessitated by the amendments to the claims, and are not persuasive for the reasons set forth below.
35 USC § 101 Rejections
First, Applicant argues that “In the Examiner's Step 2A Prong One analysis, the Examiner states that the claimed invention, is comparable to certain methods of organizing human activity and recites an abstract idea... Applicant respectfully asserts that no independent claim explicitly sets forth an abstract idea in such a manner…
As amended, independent claims 1, 16, and 20 do not recite teaching, mentoring, interpersonal relationships, following rules, managing personal behavior, managing social interactions… Instead, claim 1 recites a specific computerized processing architecture involving instantiating an interactive assessment form in which each form field is mapped to a corresponding assessment parameter; continuously monitoring user engagement actions such as cursor hovering over a form field or entering/deleting text in a form field, analyzing an impactful corpus catalog that is expressly claimed as a structured repository of electronically stored catalog entries containing asset metadata; filtering the corpus using a semantic-similarity calculation that evaluates user-entered text against metadata within a mapped metadata field corresponding to the mapped assessment parameter; and providing guidance information through the same interactive assessment form, where guidance generation includes recomputing parameter-specific and overall impact scores and updating a corresponding indicator based on further form interaction.
Therefore, because the claims, as amended, are not certain methods for organizing human activity, the analysis under the Mayo/Alice Test should end…” [Arguments, pages 12-15].
In response, Applicant’s arguments are considered but are not persuasive. Examiner respectfully disagrees and maintains that the present claims recite an abstract idea. In particular, when considered as a whole, Examiner observes that the present invention sets forth techniques for providing guidance information to users in order to enhance the impact of their research and initiatives.
Examiner respectfully maintains that the claimed limitations set forth concepts relating to certain methods of organizing human activity. The claimed limitations describe steps for managing personal behavior or relationships or interactions between people, including social activities, teaching, and following rules or instructions. Specifically, providing guidance and recommendations to a user for research papers is considered to set forth steps for teaching as well as for managing personal behavior. Thus, claims 1, 16 and 20 recite concepts identified as abstract ideas. As such, Examiner remains unpersuaded.
Second, Applicant argues that “…Applicant asserts that any such judicial exception is integrated into a practical application… Specifically, the independent claims, as amended, integrate all concepts therein into the practical applications. As to amended independent claims 1, 16, and 20, the concepts therein are integrated into a practical application of a interactive, closed-loop computational process that continuously monitors and transforms user input events into updated impact-assessment indicators through a structured corpus and mapped-field semantic filtering pipeline and displays said indicators to the user during runtime. The amended claims therefore recite more than mere information processing or display; they recite a specific improvement to the functioning of the computer-implemented guidance engine and a concrete practical application of the recited operations within a defined technological workflow…
Thus, like the patent at issue in Enfish, the concepts in the claims are integrated into a practical application. Further, the practical application is one that cannot be performed in the human mind and is necessarily only able to be performed in a computing environment. See MPEP § 2106.06(b). The human mind is not capable of monitoring the hovering of a cursor over an interactive form field tied to a corresponding asset parameter. Applicant respectfully asserts that such actions are clearly a practical application into which the concepts in amended independent claims 1, 16, and 20 are integrated and are not "no more than adding insignificantly extra-solution activity to the judicial exception"…” [Arguments, pages 15-17].
In response, Applicant’s arguments are considered but are not persuasive. Examiner respectfully disagrees and maintains that the present invention recites a judicial exception without significantly more. Examiners evaluate integration into a practical application by: (1) identifying whether there are any additional elements recited in the claim beyond the judicial exception(s); and (2) evaluating those additional elements individually and in combination to determine whether they integrate the exception into a practical application, using one or more of the considerations introduced in subsection I supra, and discussed in more detail in MPEP §§ 2106.04(d)(1), 2106.04(d)(2), 2106.05(a) through (c) and 2106.05(e) through (h).
Examiner observes that the above arguments do not appear to reference any of the recited additional elements and are therefore unpersuasive with respect to demonstrating either a practical application, or an improvement to the functioning of the computer-implemented guidance engines as alleged above.
With regard to Enfish, the claims were found to improve technology and are not directed to a judicial exception. Particularly, the claims recite a self-referential table for a computer database and are directed to an improvement in computer capabilities, and therefore not directed to an abstract idea. In contrast, the present claims do not recite any such computer or database elements which could be considered to demonstrate significantly more than the judicial exception. The consideration of whether certain activities (e.g. hovering a mouse/cursor over a form field) can be performed in the human mind is relevant for determining if an invention recites an abstract idea in the grouping of mental processes (which Examiner has not alleged), rather than for whether or not the claims demonstrate significantly more than the judicial exception. As such, Examiner remains unpersuaded.
Third, Applicant argues that “"Evaluating additional elements to determine whether they amount to an inventive concept requires considering them both individually and in combination to ensure that they amount to significantly more than the judicial exception itself. Because this approach considers all claim elements, the Supreme Court has noted that "it is consistent with the general rule that patent claims 'must be considered as a whole." [citation omitted] Consideration of the elements in combination is particularly important, because even if an additional element does not amount to significantly more on its own, it can still amount to significantly more when considered in combination with the other elements of the claim". MPEP § 2106.
In view of the above, Applicant respectfully asserts that the Examiner has failed to establish a prima facie case of patent ineligibility under 35 U.S.C. § 101, and respectfully requests withdrawal of the rejection…” [Arguments, pages 17-18].
In response, Applicant’s arguments are considered but are not persuasive. Applicant's arguments fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the claims define a patentable invention without specifically pointing out how the rejection fails to establish a prima facie case of patent ineligibility under 35 U.S.C. § 101, as alleged. Examiner observed that the above-argument does not appear to make any reference to the rejection itself. As such, Examiner remains unpersuaded.
35 USC § 102/103 Rejections
First, Applicant argues that “During the Examiner Interview, the amendments were discussed, and it was generally agreed that the amendments overcome the currently-cited art, but would require further search and consideration. As such, Applicant respectfully asserts that Sengupta fails to disclose or suggest at least the limitations of the amended independent claims.
Thus, independent claims 1, 16, and 20 are patentable over Sengupta, as Sengupta fails to show "each and every element as set forth in the claim". Accordingly, withdrawal of this rejection is respectfully requested…” [Arguments, pages 18-19].
In response, Applicant’s arguments are considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Step 1: Claims 1-20 are directed to statutory categories, namely a process (claims 1-15), an article of manufacture (claims 16-19) and a machine (claim 20).
Step 2A, Prong 1: Claims 1, 16 and 20 in part, recite the following abstract idea:
…A method for providing guidance, the method comprising: detecting an initiation, by an organization user, of an impact assessment program; instantiating an interactive assessment form comprising a set of form fields, wherein each form field is mapped to a corresponding assessment parameter; presenting, through the impact assessment program, the interactive assessment form to the organization user; continuously monitoring interactions, by the organization user, with the interactive assessment form to identify an engagement action, wherein the engagement action comprises at least a hovering of an electronically displayed cursor over a form field of the form fields or an entering or deleting of text in the form field of the form fields; analyzing, based on the engagement action and to obtain guidance information, an impactful corpus catalog comprising a set of catalog entries electronically stored in a data structure, wherein the impactful corpus catalog is a structured repository of asset metadata, and wherein the analyzing comprises filtering the impactful corpus catalog by a semantic-similarity calculation which evaluates text entered in the form field of the form fields against the asset metadata within a metadata field corresponding to the corresponding assessment parameter; providing, through the interactive assessment form, the guidance information to the organization user, wherein providing the guidance information comprises: computing, based on the analyzing, a parameter-specific impact score for the corresponding assessment parameter and an overall impact score; providing the parameter-specific impact score for the corresponding assessment parameter and the overall impact score to the organization user; receiving, from the organization user and based on the parameter-specific impact score and the overall impact score, an update to the form field of the set of form fields within the interactive assessment form; and computing, based on the update to the form field of the set of form fields, an updated parameter-specific impact score for the corresponding assessment parameter, and an updated overall impact score [Claim 1],
…perform a method for providing guidance, the method comprising: detecting an initiation, by an organization user, of an impact assessment program; instantiating an interactive assessment form comprising a set of form fields, wherein each form field is mapped to a corresponding assessment parameter; presenting, through the impact assessment program, the interactive assessment form to the organization user; continuously monitoring interactions, by the organization user, with the interactive assessment form to identify an engagement action, wherein the engagement action comprises at least a hovering of an electronically displayed cursor over a form field of the form fields or an entering or deleting of text in the form field of the form fields; analyzing, based on the engagement action and to obtain guidance information, an impactful corpus catalog comprising a set of catalog entries electronically stored in a data structure, wherein the impactful corpus catalog is a structured repository of asset metadata, and wherein the analyzing comprises filtering the impactful corpus catalog by a semantic-similarity calculation which evaluates text entered in the form field of the form fields against the asset metadata within a metadata field corresponding to the corresponding assessment parameter; providing, through the interactive assessment form, the guidance information to the organization user, wherein providing the guidance information comprises: computing, based on the analyzing, a parameter-specific impact score for the corresponding assessment parameter and an overall impact score; providing the parameter-specific impact score for the corresponding assessment parameter and the overall impact score to the organization user; receiving, from the organization user and based on the parameter-specific impact score and the overall impact score, an update to the form field of the set of form fields within the interactive assessment form; and computing, based on the update to the form field of the set of form fields, an updated parameter-specific impact score for the corresponding assessment parameter, and an updated overall impact score [Claim 16],
… and an insight service operatively connected to …configured to perform a method for providing guidance, the method comprising: detecting an initiation, by an organization user operating …, of an impact assessment program executing on …; instantiating an interactive assessment form comprising a set of form fields, wherein each form field is mapped to a corresponding assessment parameter; presenting, through the impact assessment program, the interactive assessment form to the organization user; continuously monitoring interactions, by the organization user, with the interactive assessment form to identify an engagement action, wherein the engagement action comprises at least a hovering of an electronically displayed cursor over a form field of the form fields or an entering or deleting of text in the form field of the form fields; analyzing, based on the engagement action and to obtain guidance information, an impactful corpus catalog comprising a set of catalog entries electronically stored in a data structure, wherein the impactful corpus catalog is a structured repository of asset metadata, and wherein the analyzing comprises filtering the impactful corpus catalog by a semantic-similarity calculation which evaluates text entered in the form field of the form fields against the asset metadata within a metadata field corresponding to the corresponding assessment parameter; providing, through the interactive assessment form, the guidance information to the organization user, wherein providing the guidance information comprises: computing, based on the analyzing, a parameter-specific impact score for the corresponding assessment parameter and an overall impact score; providing the parameter-specific impact score for the corresponding assessment parameter and the overall impact score to the organization user; receiving, from the organization user and based on the parameter-specific impact score and the overall impact score, an update to the form field of the set of form fields within the interactive assessment form; and computing, based on the update to the form field of the set of form fields, an updated parameter-specific impact score for the corresponding assessment parameter, and an updated overall impact score [Claim 20].
These concepts are not meaningfully different than the following concepts identified by the MPEP:
Concepts relating to certain methods of organizing human activity. The aforementioned limitations describe steps for managing personal behavior or relationships or interactions between people, including social activities, teaching, and following rules or instructions. Specifically, providing guidance and recommendations to a user for research papers is considered to set forth steps for teaching as well as for managing personal behavior. As such, claims 1, 16 and 20 recite concepts identified as abstract ideas.
The dependent claims recite limitations relative to the independent claims, including, for example:
…wherein the guidance information comprises at least one recommendation directed to enhancing an impact of one selected from a group comprising a research paper that has yet to be drafted, and a research initiative that has yet to be pursued, by the organization user [Claims 2 and 17],
…wherein the engagement action reflects a hovering over of a form field in the set of form fields [Claims 3 and 18],
…wherein analyzing, based on the engagement action, the impactful corpus catalog to obtain the guidance information, comprises: making a determination, based on a search across the set of form fields, that at least one form field, in the set of form fields, is a non-empty form field; extracting, based on the determination and from the at least one form field, a field input to obtain at least one field input; filtering, based on the at least one field input, the impactful corpus catalog to identify a catalog entry subset in the set of catalog entries; and analyzing asset metadata, maintained across the catalog entry subset, to obtain the guidance information [Claims 4 and 19],
…wherein the form field is one selected from a group comprising included in, and excluded from, the at least one form field [Claim 5],
…wherein analyzing, based on the engagement action, the impactful corpus catalog to obtain the guidance information, further comprises: prior to filtering the impactful corpus catalog: mapping the at least one form field, respectively, to at least one filtering assessment parameter in a set of assessment parameters, wherein the impactful corpus catalog is filtered further based on the at least one filtering assessment parameter [Claim 6],
…wherein analyzing, based on the engagement action, the impactful corpus catalog to obtain the guidance information, further comprises: prior to making the determination: mapping the form field to a guiding assessment parameter in a set of assessment parameters, wherein the asset metadata belongs to a metadata field matching the guiding assessment parameter [Claim 7],
…wherein monitoring the interactions, by the organization user, with the interactive assessment form further identifies a second engagement action [Claim 8],
…wherein the second engagement action reflects an editing of a second form field in the set of form fields [Claim 9],
…further comprising: for one selected from a group comprising prior to, and after, providing the guidance information: identifying an assessment parameter mapped to the second form field; extracting a second field input from the second form field; filtering, based on the assessment parameter and the second field input, the impactful corpus catalog to identify a second catalog entry subset in the set of catalog entries; obtaining an overall corpus catalog comprising a second set of catalog entries; computing a parameter-specific impact score based on a first cardinality of the second catalog entry subset and a second cardinality of the second set of catalog entries; computing an overall impact score based on a set of parameter-specific impact scores comprising the parameter-specific impact score; and updating the interactive assessment form using the parameter-specific impact score and the overall impact score [Claim 10],
…wherein the interactive assessment form further comprises a set of parameter-specific impact score indicators, and wherein updating the interactive assessment form comprises: identifying a parameter-specific impact score indicator, in the set of parameter- specific impact score indicators, mapped to the assessment parameter; and replacing, with the parameter-specific impact score, a previous parameter-specific impact score displayed by the parameter-specific impact score indicator [Claim 11],
…wherein the interactive assessment form further comprises an overall impact score indicator, and wherein updating the interactive assessment form further comprises: replacing, with the overall impact score, a previous impact score displayed by the overall impact score indicator [Claim 12],
…wherein the second form field is a complex form field, and the method further comprises: after updating the interactive assessment form: identifying, from metadata collectively maintained across the second catalog entry subset, a metadata subset associated with a metadata field matching the assessment parameter; analyzing the metadata subset to obtain second guidance information; and providing, through the interactive assessment form, the second guidance information to the organization user [Claim 13],
…wherein the second form field is one selected from a group comprising a same form field, and a different form field, as the form field [Claim 14],
…wherein analyzing, based on the engagement action, the impactful corpus catalog to obtain the guidance information, comprises: mapping the form field to a guiding assessment parameter in a set of assessment parameters; making a determination, based on a search across the set of form fields, that each form field, in the set of form fields, is an empty form field; and analyzing, based on the determination, asset metadata across the set of catalog entries to obtain the guidance information, wherein the asset metadata belongs to a metadata field matching the guiding assessment parameter [Claim 15],
The limitations of these dependent claims are merely narrowing the abstract idea identified in the independent claims, and thus, the dependent claims also recite abstract ideas.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. In particular, claims 1, 16 and 20 only recite the following additional elements –
Claim 1 recites no additional elements.
A non-transitory computer readable medium (CRM) comprising computer readable program code, which when executed by a computer processor, enables the computer processor to… [Claim 16],
A system, the system comprising: a client device… the client device, and comprising a computer processor… the client device… the client device… [Claim 20].
The apparatus and executable instructions are recited at a high-level of generality (see MPEP § 2106.05(a)), like the following MPEP example:
iii. Gathering and analyzing information using conventional techniques and displaying the result, TLI Communications, 823 F.3d at 612-13, 118 USPQ2d at 1747-48;
Furthermore, the computer implemented element is considered to amount to no more than mere instructions to apply the exception using a generic computer component (see MPEP 2106.05(f)), like the following MPEP example:
i. A commonplace business method or mathematical algorithm being applied on a general purpose computer, Alice Corp. Pty. Ltd. V. CLS Bank Int’l, 573 U.S. 208, 223, 110 USPQ2d 1976, 1983 (2014); Gottschalk v. Benson, 409 U.S. 63, 64, 175 USPQ 673, 674 (1972); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015);
Accordingly, these additional elements do not integrate the abstract idea into a practical application.
The remaining dependent claims do not recite any new additional elements, and thus do not integrate the abstract idea into a practical application.
Step 2B: Claims 1, 16 and 20 and their underlying limitations, steps, features and terms, considered both individually and as a whole, do not include additional elements that are sufficient to amount to significantly more than the judicial exception for the following reasons:
Independent claims 1, 16 and 20 only recite the following additional elements –
Claim 1 recites no additional elements.
A non-transitory computer readable medium (CRM) comprising computer readable program code, which when executed by a computer processor, enables the computer processor to… [Claim 16],
A system, the system comprising: a client device… the client device, and comprising a computer processor… the client device… the client device… [Claim 20].
These elements do not amount to significantly more than the abstract idea for the reasons discussed in 2A prong 2 with regard to MPEP 2106.05(a) and MPEP 2106.05(f). By the failure of the elements to integrate the abstract idea into a practical application there, the additional elements likewise fail to amount to an inventive concept that is significantly more than an abstract idea here, in Step 2B.
As such, both individually or in combination, these limitations do not add significantly more to the judicial exception.
The remaining dependent claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the dependent claims do not recite any new additional elements other than those mentioned in the independent claims, which amount to no more than mere instructions to apply the exception using a generic computer component (see MPEP 2106.05(f)). As such, these claims are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 16-18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Sengupta et al., U.S. Publication No. 2015/0206055 [hereinafter Sengupta] in view of Malhotra et al., U.S. Publication No. 2019/0340945 [hereinafter Malhotra].
Regarding Claim 1, Sengupta discloses …A method for providing guidance, the method comprising: detecting an initiation, by an organization user, of an impact assessment program (Sengupta, ¶ 80, Once the feedback is received 135, the initial automated analysis 110/120 may be re-run. For example, if the humans suggested additional external data, new hypotheses, new patterns, new subsets of data with higher p-values, etc., each of these may enable improved automated analysis. After the automated analysis is completed in light of the human-feedback, the system may go through an additional human-feedback step. (discloses imitation of an impact assessment program) The automated-analysis through human feedback cycle may be carried out as many times as necessary to get optimal analysis results. The feedback cycle may be terminated after a set number of times or if the results do not improve significantly after a feedback cycle or if no significant new feedback is received during a given human feedback step. The feedback cycle need not be a monolithic process. For example, if a human feedback only affects part of the overall analysis, that part may be reanalyzed automatically based on the feedback without affecting the rest of the analysis), (Id., ¶ 81, As the analysis is improved based on human feedback, a learning algorithm can evaluate which human feedback had the most impact on the results and which feedback had minor or even negative impact on the results. (discloses impact assessment) As this method clearly links specific human feedback to specific impacts on the results of the analysis, the learning algorithms have a rich source of data to train on. Eventually, these learning algorithms would themselves be able to suggest improvement opportunities which could be directly leveraged in the automated analysis phase), (Id., ¶ 123, FIGS. 6A-6B illustrate user interfaces that enable a user to share (FIG. 6A) or download (FIG. 6B) a story. FIG. 6A illustrates that a user can share the story with other users in the organization and either authorize them to view the story or edit the story as well. (discloses organization users) Any edits made by authorized users can be seen by every other user. The user can use the `History` link below each graph to revert to his preferred version of the story. BeyondCore users can be grouped into organizations and, in some embodiments, users can share stories only with people within their organization. If a user does not see a specific user in their share screen, the user may confirm that the specific user has registered with BeyondCore and is in the same organization as the user himself. Additional options to include allowing or denying editing capabilities of story 610, granting or revoking access to the story 620, selecting users with whom he wants to share his story 630 (e.g., via a drop down list that includes every user in the viewing user's organization that has registered with BeyondCore), sharing the story 650, and so on. Further, as illustrated in FIG. 6B, a user can download (e.g., via UI element 670) and email a static HTML version of the story to other licensed users in his organization. In some embodiments, only the main story (excluding the recommendations panel) is available through this HTML file. Users may also download the story in other formats such as PowerPoint, Word and pdf files);
instantiating an interactive assessment form comprising a set of form fields, wherein each form field is mapped to a corresponding assessment parameter; presenting, through the impact assessment program, the interactive assessment form to the organization user (Id., ¶ 25, FIGS. 3A-3E are screen shots illustrating the evolution of a story (an analysis project), according to some embodiments), (Id., ¶ 114, FIG. 3C illustrates a user interface for selecting the business outcome (the variable) that the user wishes to analyze. This is typically the KPI or metric in the user's dashboards and reports, e.g. revenue or cost. This page lists all the numeric or binary (e.g. Male/Female) columns in the user's data that have sufficient variability. If the user does not see a variable that is expected, the user may verify that the variable has numerical values and not text. If a variable has only a few values (e.g. 1, 2, 5), BeyondCore will treat it as a categorical variable instead of numeric. In most cases this is desired and statistically appropriate. To change a variable from categorical to numeric, the user may go to the Data Setup page and manually filter or reformat data (see advanced options at FIG. 4A). Returning to FIG. 3C, the user may click user element 345 to select the business outcome. (which would be the y-axis of the graph, or the number the user wants to predict)), (Id., Fig. 3C, Figure depicts an interactive assessment form comprising a set of form fields mapped to assessment parameters);
PNG
media_image1.png
299
448
media_image1.png
Greyscale
continuously monitoring interactions, by the organization user, with the interactive assessment form to identify an engagement action, wherein the engagement action comprises at least a hovering of an electronically displayed cursor over a form field of the form fields or an entering or deleting of text in the form field of the form fields (Id., ¶ 187, Referring to FIG. 11B, what-if scenario analysis enables a user to compare predicted outcomes for different values of a variable under specified conditions. The user interface may allow the user to select 1140 the variable that the user wishes to compare different outcomes for (e.g., in the example of 1140, what acquisition channel should the user use?). The user may also choose 1145 the variables to constrain based on. The user may also click `View Graph` 1150 to see the analysis. The user may also specify 1155 conditions under which the user wants to compare the what-if variable (in this case, Females with income between 61500 and 81500)), (Id., ¶ 188, As illustrated in FIG. 11C, the Predictive Analysis shows the expected outcome under specified circumstances and explains the reasons behind the prediction. The user may additionally select variables to constrain based on 1160, conditions 1165 for which to predict the outcome (in this case, Californian females who are 20 to 25 years old with income between 61500 and 81500), and an option to view the graph with the analysis 1170. The graph itself includes the overall average 1175-a, reasons behind the prediction 1175-b (the user may hover or mouse over to see additional details) (discloses monitoring interactions to identify an engagement action), and predicted outcome 1175-c. The prediction is based on the automatically learned features of the data set, the automated analysis approach for which has been described above), (Id., ¶ 82, The human feedback patterns could also be analyzed to detect deterministic patterns that may or may not be context specific. For example, if local rainfall patterns turn out to be a common external variable for retail analyses, the software may automatically start including this data in similar analyses. Similarly, if humans frequently combine behavior patterns noticed on Saturdays and Sundays to create a higher p-value pattern for weekends, the software could learn to treat weekends and weekdays differently in its analyses);
analyzing, based on the engagement action and to obtain guidance information, an impactful corpus catalog comprising a set of catalog entries electronically stored in a data structure, wherein the impactful corpus catalog is a structured repository of asset metadata… (Id., ¶ 43, Various types of automated analysis have been described previously by the inventors. For example, in the context of document processing by operators, one goal may be to find documents that are similar in some way in order to identify underlying patterns of operator behavior. A search can be conducted for segments of the data which share as few as one or more similar field or parameter values. For example, a database of loan applications can be searched for applicants between 37 and 39 years of age. Any pair of applications from this sample might be no more similar than a randomly chosen pair from the population. However, this set of applications can be statistically analyzed to determine whether certain loan officers are more likely to approve loans from this section of the population), (Id., ¶ 44, Alternatively, it may not be necessary to find even one very similar parameter. Large segments of the population may be aggregated for analysis using criteria such as "applicants under 32 years old" or "applicants earning more than $30,000 per year." Extending this methodology one step further, a single analysis can be conducted on the sample consisting of the entire population), (Id., ¶ 79, Some humans may try to submit large volumes of suggestions hoping that at least one of them works. Others may even write computer code to generate many suggestions. As long as the computation resources needed to evaluate such suggestions is minimal, this is not a significant problem and may even contribute to the overall objective of useful analysis. To reduce the computational cost of the evaluation of suggestions, such suggestions may first be tested against a subset of the overall data. Suggestions would only be incorporated while analyzing the overall data if the suggestion enabled a significant improvement when used to analyze the subset data. To further save computation expenses, multiple suggestions evaluated on the subset data may be combined before the corresponding updated analysis is run on the complete data. (discloses obtaining guidance information based on an automated analysis of an impactful corpus catalog) Additionally, computation resources could be allocated to different users via a quota system, and users could optionally "purchase" more using their rewards from previous suggestions), (Id., ¶ 116, FIG. 3E illustrates a user interface that allows a user to further customize a BeyondCore story. The user may edit the story name, row labels, column labels, and the like. For example, UI element 360 allows a user to specify a story title, UI element 362 allows a user to specify how BeyondCore should refer to a row in the data, (discloses structured asset metadata) UI element 364 enables the user to specify the unit of the y-axis of his graphs (the outcome variable), UI element 368 enables the user to access advanced options or further customize a story);
and providing, through the interactive assessment form, the guidance information to the organization user, wherein providing the guidance information comprises: computing, based on the analyzing, a parameter-specific impact score for the corresponding assessment parameter and an overall impact score; (Id., ¶ 86, Once the automated analysis with human feedback is completed, the data could be presented to expert analysts 140 for further enhancement. Such analysts would have the benefit of the following: [0087] lists of hypotheses detected automatically as well as proposed by humans; [0088] results of how well the data fit various regression models detected automatically as well as proposed by humans; [0089] specific subsets of data with high p-values, corresponding to automatically or manually detected patterns (discloses parameter specific scores), and corresponding manually proposed causal links; [0090] votes and tags indicating agreement from communities such as customers or employees; and [0091] other valuable context information), (Id., ¶ 104, The impact of each variable combination typically is determined by the behavior of a variable combination with respect to the outcome and by the population of the variable combination. In one approach, automated analysis learns the normative behavior for each variable combination as it relates to the outcome. For example it may learn that Men in California spend more while 18 to 25 year olds who buy over the Mobile channel spend less than usual in general (here amount spent is the outcome). But a specific transaction may be for a Male 18 to 25 years old from California who purchased goods over the Mobile channel. By observing the norm for each variable combination in isolation and in combinations across multiple transactions, we can learn the "net impact" (the behavior) of a variable combination. This is the positive or negative impact of the variable combination on the observed outcome, net of the impact of all other variable combinations that may also be affecting that specific transaction. This allows automated analysis to learn a behavior metric that is similar to obtaining a regression coefficient in a regression analysis, but which can be learned via the search-based approach described above with reference to FIG. 1, instead of running a regression analysis. In an alternative approach, a type of regression analysis is run for the outcome with respect to all of the variable combinations being considered. For each variable combination, there will be a regression term (the impact) that equals the regression coefficient (the behavior) multiplied by the population. (discloses overall impact score) Behavior may also be measured in terms of correlation coefficients, net-effect impact net of all other variables, or any other suitable metric that captures how the variable combination affects the outcome or how the outcome trends as a function of the variable combinations. Population may also be measured in terms of counts (i.e., number of observations), whether or not something occurred, frequency/percentage of overall population, or relative frequencies of observations. The overall impact of a variable combination depends on both its behavior (i.e., how strongly does that variable combination affect the outcome) and its population (i.e., how much of that variable combination exists in the data set of interest). These impacts, behaviors and populations can then be used to analyze the data set in different ways), (Id., ¶ 109, Predictive graphs 220 illustrate an outcome of predictive analysis that selects the Descriptive graphs 210 to be displayed as well as to make Prescriptive recommendations 240. Expert users can access the predictive capabilities directly from the `Choose a graph` feature), (Id., ¶ 110, Diagnostic graphs 230 highlight multiple unrelated factors (i.e., variable combinations) that contribute to an outcome or visual pattern displayed in a graph. For a Descriptive graph 210, BeyondCore automatically checks for what other factors might be contributing to the pattern. For example, a hospital that is doing badly may actually have far more emergency patients and that is why it is doing badly. Diagnostic graphs 230 help ensure that the patterns the user focuses on are real and not accidents of the data), (Id., ¶ 111, Prescriptive graphs 240 provide a means for the user to communicate to BeyondCore which of the variables are actionable (things that can be changed easily) and whether the user wants to maximize or minimize the outcome. BeyondCore can then look at millions (typically) of possibilities for changing variables, conducts Predictive analysis, recommends specific actions, quantifies the expected impact, and explains the reasoning behind the recommendations);
providing the parameter-specific impact score for the corresponding assessment parameter and the overall impact score to the organization user receiving, from the organization user and based on the parameter-specific impact score and the overall impact score, an update to the form field of the set of form fields within the interactive assessment form (Id., ¶ 89, specific subsets of data with high p-values, corresponding to automatically or manually detected patterns (discloses parameter specific scores), and corresponding manually proposed causal links; [0090] votes and tags indicating agreement from communities such as customers or employees; and [0091] other valuable context information), (Id., ¶ 104, The impact of each variable combination typically is determined by the behavior of a variable combination with respect to the outcome and by the population of the variable combination. In one approach, automated analysis learns the normative behavior for each variable combination as it relates to the outcome. For example it may learn that Men in California spend more while 18 to 25 year olds who buy over the Mobile channel spend less than usual in general (here amount spent is the outcome). But a specific transaction may be for a Male 18 to 25 years old from California who purchased goods over the Mobile channel. By observing the norm for each variable combination in isolation and in combinations across multiple transactions, we can learn the "net impact" (the behavior) of a variable combination. This is the positive or negative impact of the variable combination on the observed outcome, net of the impact of all other variable combinations that may also be affecting that specific transaction. This allows automated analysis to learn a behavior metric that is similar to obtaining a regression coefficient in a regression analysis, but which can be learned via the search-based approach described above with reference to FIG. 1, instead of running a regression analysis. In an alternative approach, a type of regression analysis is run for the outcome with respect to all of the variable combinations being considered. For each variable combination, there will be a regression term (the impact) that equals the regression coefficient (the behavior) multiplied by the population. (discloses overall impact score) Behavior may also be measured in terms of correlation coefficients, net-effect impact net of all other variables, or any other suitable metric that captures how the variable combination affects the outcome or how the outcome trends as a function of the variable combinations. Population may also be measured in terms of counts (i.e., number of observations), whether or not something occurred, frequency/percentage of overall population, or relative frequencies of observations. The overall impact of a variable combination depends on both its behavior (i.e., how strongly does that variable combination affect the outcome) and its population (i.e., how much of that variable combination exists in the data set of interest). These impacts, behaviors and populations can then be used to analyze the data set in different ways), (Id., ¶ 52, Local search methods operate by considering a given sample, and repeatedly modifying it with the goal of raising the metric. This continues until the metric is higher for the sample under consideration than for any nearby samples (a local optimum). The notion of proximity is complex for samples of the sort we are discussing. The "modify" step in the algorithm will change the restrictions defining the current sample. This can consist of widening or tightening the restriction on one field, or adding a restriction on a new field, (discloses updating form fields on an assessment form) or removing the restriction on a restricted field. For example, if we consider a sample consisting of "Loan applications from females aged 30-40" and calculate the metric to be X, we could then calculate the metric for "females", "females aged 30-50", "females aged 20-40", "people aged 30-40", and others. Each of these metrics will be compared to X and the search algorithm will continue);
and computing, based on the update to the form field of the set of form fields, an updated parameter-specific impact score for the corresponding assessment parameter, and an updated overall impact score (Id., ¶ 52, Local search methods operate by considering a given sample, and repeatedly modifying it with the goal of raising the metric. This continues until the metric is higher for the sample under consideration than for any nearby samples (a local optimum). The notion of proximity is complex for samples of the sort we are discussing. The "modify" step in the algorithm will change the restrictions defining the current sample. This can consist of widening or tightening the restriction on one field, or adding a restriction on a new field, (discloses updating form fields on an assessment form) or removing the restriction on a restricted field. For example, if we consider a sample consisting of "Loan applications from females aged 30-40" and calculate the metric to be X, we could then calculate the metric for "females", "females aged 30-50", "females aged 20-40", "people aged 30-40", and others. (discloses computing updated impact scores) Each of these metrics will be compared to X and the search algorithm will continue), (Id., ¶ 53, Because the metrics are highest for samples with acute variances, samples obtained using parameter values which are responsible for the unusual behavior will have the highest scores. Much larger and much smaller samples will have lower scores. As the search algorithm runs, the sample under consideration will "evolve" to contain the features that are causing the discrepancy in operator processing while not containing unrelated random information. Of course, the search will cease on one local maximum. If the local search is repeated multiple times from random starting samples, many samples with peak metrics can be identified in the data).
While suggested in at least Fig. 1 and related text, Sengupta does not explicitly disclose …and wherein the analyzing comprises filtering the impactful corpus catalog by a semantic-similarity calculation which evaluates text entered in the form field of the form fields against the asset metadata within a metadata field corresponding to the corresponding assessment parameter.
However, Malhotra discloses …and wherein the analyzing comprises filtering the impactful corpus catalog by a semantic-similarity calculation which evaluates text entered in the form field of the form fields against the asset metadata within a metadata field corresponding to the corresponding assessment parameter (Malhotra, ¶ 20, Before data set 142 is analyzed, server system 130 may prompt users to provide profile information in one of a number of ways. For example, server system 130 may have provided a web page with a text field for one or more of the above-referenced types of information. In response to receiving profile information from a user's device, server system 130 stores the information in an account that is associated with the user and that is associated with credential data that is used to authenticate the user to server system 130 when the user attempts to log into server system 130 at a later time. Each text string provided by a user may be stored in association with the field into which the text string was entered. For example, if a user enters “Sales Manager” in a job title field, then “Sales Manager” is stored in association with type data that indicates that “Sales Manager” is a job title (discloses evaluating text in form fields). As another example, if a user enters “Java programming” in a skills field, then “Java programming” is stored in association with type data that indicates that “Java programming” is a skill), (Id., ¶ 32, In an embodiment, some text may be considered more relevant than other text in determining whether (or how much) a particular skill is associated with a corresponding learning resource. For example, if a title or metadata (discloses evaluating form field text against asset metadata) of a learning resource indicates a first skill, then it is more likely that the learning resource is dedicated to teaching that first skill than in teaching a second skill that is mentioned only in the body of the learning resource. As another example, if multiple online reviews of a learning resource indicate a first skill, then it is more likely that the learning resource is dedicated to teaching that first skill than in teaching a second skill that is not mentioned in any of the online reviews. 101, One method to generate embeddings includes artificial neural networks. In the context of linguistics, word embedding, when used as the underlying input representation, have been shown to boost performance in natural language processing (NLP) tasks, such as syntactic parsing and sentiment analysis. Word embedding aims to quantify and categorize semantic similarities between linguistic items (discloses semantic-similarity calculations) based on their distributional properties in large samples of language data. The underlying idea that a word is characterized by “the company it keeps.”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the guidance information elements of Sengupta to include the semantic similarity elements of Malhotra in the analogous art of automatic generation and personalization of learning paths.
The motivation for doing so would have been to provide “a computer-related improvement over prior learning assistance computer systems…” wherein “Embodiments consider attributes and/or activity of individual users and their respective career goals in crafting individualized learning paths” (Malhotra, ¶ 112), wherein such improvements would have benefitted Sengupta’s method which provides an improved system wherein “ the user's interaction with the multi-screen reporting are tracked. In some embodiments, the user's interaction with the multi-screen reporting are tracked in a manner that is auditable. In some embodiments, changes to the multi-screen reporting resulting from the user's interaction with the multi-screen reporting are sharable with other users” [Malhotra, ¶ 112; Sengupta, ¶ 129].
Regarding Claim 2, the combination of Sengupta and Malhotra discloses …the method of claim 1…
Sengupta further discloses …wherein the guidance information comprises at least one recommendation directed to enhancing an impact of one selected from a group comprising a research paper that has yet to be drafted, and a research initiative that has yet to be pursued, by the organization user (Id., ¶ 76, Existing approaches can be further improved in a number of ways. For example, one embodiment taps a human's social knowledge, something much harder for computers to emulate than specific spatial reasoning. Moreover, we tap the social knowledge in a structured machine-interpretable manner which makes the solution scalable. Humans excel at graph search problems such as geometric folding (or chess-playing) where there are many options at each step. Today, this gives people an advantage in a head-to-head competition, but with rapid advances in technology and falling costs, computers are rapidly catching up. In fact, computer algorithms are now widely considered to outperform humans at the game of chess. However, no amount of increased processor speed will enable a computer to compete in the arena of social cognizance and emotional intelligence. Socialization comes naturally to humans and can be effectively harnessed using our methods), (Id., ¶ 77, Additionally, various embodiments can be non-trivially reward based. By tying a tangible payment to the actual business value created, the system is no longer academic, but can encourage users to spend significant amounts of time generating value. Additionally, a user who seeks to "game" the system by writing computer algorithms to participate is actually contributing to the community in a valid and valuable way. Such behavior is encouraged. This value sharing approach brings the state of the art in crowdsourcing out of the arena of research papers and into the world of business).
Regarding Claim 3, the combination of Sengupta and Malhotra discloses …the method of claim 1…
Sengupta further discloses …wherein the engagement action reflects a hovering over of a form field in the set of form fields (Id., ¶ 188, As illustrated in FIG. 11C, the Predictive Analysis shows the expected outcome under specified circumstances and explains the reasons behind the prediction. The user may additionally select variables to constrain based on 1160, conditions 1165 for which to predict the outcome (in this case, Californian females who are 20 to 25 years old with income between 61500 and 81500), and an option to view the graph with the analysis 1170. The graph itself includes the overall average 1175-a, reasons behind the prediction 1175-b (the user may hover or mouse over to see additional details) (discloses monitoring interactions to identify an engagement action), and predicted outcome 1175-c. The prediction is based on the automatically learned features of the data set, the automated analysis approach for which has been described above), (Id., ¶ 82, The human feedback patterns could also be analyzed to detect deterministic patterns that may or may not be context specific. For example, if local rainfall patterns turn out to be a common external variable for retail analyses, the software may automatically start including this data in similar analyses. Similarly, if humans frequently combine behavior patterns noticed on Saturdays and Sundays to create a higher p-value pattern for weekends, the software could learn to treat weekends and weekdays differently in its analyses).
Regarding Claim 16, Sengupta discloses … A non-transitory computer readable medium (CRM) comprising computer readable program code, which when executed by a computer processor, enables the computer processor to perform a method for providing guidance, the method comprising: detecting an initiation, by an organization user, of an impact assessment program (Sengupta, ¶ 80, Once the feedback is received 135, the initial automated analysis 110/120 may be re-run. For example, if the humans suggested additional external data, new hypotheses, new patterns, new subsets of data with higher p-values, etc., each of these may enable improved automated analysis. After the automated analysis is completed in light of the human-feedback, the system may go through an additional human-feedback step. (discloses imitation of an impact assessment program) The automated-analysis through human feedback cycle may be carried out as many times as necessary to get optimal analysis results. The feedback cycle may be terminated after a set number of times or if the results do not improve significantly after a feedback cycle or if no significant new feedback is received during a given human feedback step. The feedback cycle need not be a monolithic process. For example, if a human feedback only affects part of the overall analysis, that part may be reanalyzed automatically based on the feedback without affecting the rest of the analysis), (Id., ¶ 81, As the analysis is improved based on human feedback, a learning algorithm can evaluate which human feedback had the most impact on the results and which feedback had minor or even negative impact on the results. (discloses impact assessment) As this method clearly links specific human feedback to specific impacts on the results of the analysis, the learning algorithms have a rich source of data to train on. Eventually, these learning algorithms would themselves be able to suggest improvement opportunities which could be directly leveraged in the automated analysis phase), (Id., ¶ 123, FIGS. 6A-6B illustrate user interfaces that enable a user to share (FIG. 6A) or download (FIG. 6B) a story. FIG. 6A illustrates that a user can share the story with other users in the organization and either authorize them to view the story or edit the story as well. (discloses organization users) Any edits made by authorized users can be seen by every other user. The user can use the `History` link below each graph to revert to his preferred version of the story. BeyondCore users can be grouped into organizations and, in some embodiments, users can share stories only with people within their organization. If a user does not see a specific user in their share screen, the user may confirm that the specific user has registered with BeyondCore and is in the same organization as the user himself. Additional options to include allowing or denying editing capabilities of story 610, granting or revoking access to the story 620, selecting users with whom he wants to share his story 630 (e.g., via a drop down list that includes every user in the viewing user's organization that has registered with BeyondCore), sharing the story 650, and so on. Further, as illustrated in FIG. 6B, a user can download (e.g., via UI element 670) and email a static HTML version of the story to other licensed users in his organization. In some embodiments, only the main story (excluding the recommendations panel) is available through this HTML file. Users may also download the story in other formats such as PowerPoint, Word and pdf files), (Id., Claim 29, A computer program product for recommending actions to affect an outcome of a process, the computer program product comprising a non-transitory machine-readable medium storing computer program code for performing a method, the method comprising: receiving from a user an identification of an outcome to be affected; processing a data set containing observations of the process, the observations expressed as values for a plurality of variables and for the outcome, wherein processing the data set determines behaviors for different variable combinations with respect to the outcome, the variable combinations defined by values for one or more of the variables; receiving an identification of one or more actionable variables from the plurality of variables; for pairs of a first variable combination and a second variable combination, wherein the first and second variable combinations are the same except that one or more of the actionable variables take first values in the first variable combination and take different second values in the second variable combination, predicting an impact of changing the actionable variables in the first variable combination from the first values to the second values by applying (a) the behavior of the second variable combination to (b) a population of the first variable combination; and recommending actions to change actionable variables based on the predicted impacts.);
instantiating an interactive assessment form comprising a set of form fields, wherein each form field is mapped to a corresponding assessment parameter; presenting, through the impact assessment program, the interactive assessment form to the organization user (Id., ¶ 25, FIGS. 3A-3E are screen shots illustrating the evolution of a story (an analysis project), according to some embodiments), (Id., ¶ 114, FIG. 3C illustrates a user interface for selecting the business outcome (the variable) that the user wishes to analyze. This is typically the KPI or metric in the user's dashboards and reports, e.g. revenue or cost. This page lists all the numeric or binary (e.g. Male/Female) columns in the user's data that have sufficient variability. If the user does not see a variable that is expected, the user may verify that the variable has numerical values and not text. If a variable has only a few values (e.g. 1, 2, 5), BeyondCore will treat it as a categorical variable instead of numeric. In most cases this is desired and statistically appropriate. To change a variable from categorical to numeric, the user may go to the Data Setup page and manually filter or reformat data (see advanced options at FIG. 4A). Returning to FIG. 3C, the user may click user element 345 to select the business outcome. (which would be the y-axis of the graph, or the number the user wants to predict)), (Id., Fig. 3C, Figure depicts an interactive assessment form comprising a set of form fields mapped to assessment parameters);
PNG
media_image1.png
299
448
media_image1.png
Greyscale
continuously monitoring interactions, by the organization user, with the interactive assessment form to identify an engagement action, wherein the engagement action comprises at least a hovering of an electronically displayed cursor over a form field of the form fields or an entering or deleting of text in the form field of the form fields (Id., ¶ 187, Referring to FIG. 11B, what-if scenario analysis enables a user to compare predicted outcomes for different values of a variable under specified conditions. The user interface may allow the user to select 1140 the variable that the user wishes to compare different outcomes for (e.g., in the example of 1140, what acquisition channel should the user use?). The user may also choose 1145 the variables to constrain based on. The user may also click `View Graph` 1150 to see the analysis. The user may also specify 1155 conditions under which the user wants to compare the what-if variable (in this case, Females with income between 61500 and 81500)), (Id., ¶ 188, As illustrated in FIG. 11C, the Predictive Analysis shows the expected outcome under specified circumstances and explains the reasons behind the prediction. The user may additionally select variables to constrain based on 1160, conditions 1165 for which to predict the outcome (in this case, Californian females who are 20 to 25 years old with income between 61500 and 81500), and an option to view the graph with the analysis 1170. The graph itself includes the overall average 1175-a, reasons behind the prediction 1175-b (the user may hover or mouse over to see additional details) (discloses monitoring interactions to identify an engagement action), and predicted outcome 1175-c. The prediction is based on the automatically learned features of the data set, the automated analysis approach for which has been described above), (Id., ¶ 82, The human feedback patterns could also be analyzed to detect deterministic patterns that may or may not be context specific. For example, if local rainfall patterns turn out to be a common external variable for retail analyses, the software may automatically start including this data in similar analyses. Similarly, if humans frequently combine behavior patterns noticed on Saturdays and Sundays to create a higher p-value pattern for weekends, the software could learn to treat weekends and weekdays differently in its analyses);
analyzing, based on the engagement action and to obtain guidance information, an impactful corpus catalog comprising a set of catalog entries electronically stored in a data structure, wherein the impactful corpus catalog is a structured repository of asset metadata… (Id., ¶ 43, Various types of automated analysis have been described previously by the inventors. For example, in the context of document processing by operators, one goal may be to find documents that are similar in some way in order to identify underlying patterns of operator behavior. A search can be conducted for segments of the data which share as few as one or more similar field or parameter values. For example, a database of loan applications can be searched for applicants between 37 and 39 years of age. Any pair of applications from this sample might be no more similar than a randomly chosen pair from the population. However, this set of applications can be statistically analyzed to determine whether certain loan officers are more likely to approve loans from this section of the population), (Id., ¶ 44, Alternatively, it may not be necessary to find even one very similar parameter. Large segments of the population may be aggregated for analysis using criteria such as "applicants under 32 years old" or "applicants earning more than $30,000 per year." Extending this methodology one step further, a single analysis can be conducted on the sample consisting of the entire population), (Id., ¶ 79, Some humans may try to submit large volumes of suggestions hoping that at least one of them works. Others may even write computer code to generate many suggestions. As long as the computation resources needed to evaluate such suggestions is minimal, this is not a significant problem and may even contribute to the overall objective of useful analysis. To reduce the computational cost of the evaluation of suggestions, such suggestions may first be tested against a subset of the overall data. Suggestions would only be incorporated while analyzing the overall data if the suggestion enabled a significant improvement when used to analyze the subset data. To further save computation expenses, multiple suggestions evaluated on the subset data may be combined before the corresponding updated analysis is run on the complete data. (discloses obtaining guidance information based on an automated analysis of an impactful corpus catalog) Additionally, computation resources could be allocated to different users via a quota system, and users could optionally "purchase" more using their rewards from previous suggestions), (Id., ¶ 116, FIG. 3E illustrates a user interface that allows a user to further customize a BeyondCore story. The user may edit the story name, row labels, column labels, and the like. For example, UI element 360 allows a user to specify a story title, UI element 362 allows a user to specify how BeyondCore should refer to a row in the data, (discloses structured asset metadata) UI element 364 enables the user to specify the unit of the y-axis of his graphs (the outcome variable), UI element 368 enables the user to access advanced options or further customize a story);
and providing, through the interactive assessment form, the guidance information to the organization user, wherein providing the guidance information comprises: computing, based on the analyzing, a parameter-specific impact score for the corresponding assessment parameter and an overall impact score; (Id., ¶ 86, Once the automated analysis with human feedback is completed, the data could be presented to expert analysts 140 for further enhancement. Such analysts would have the benefit of the following: [0087] lists of hypotheses detected automatically as well as proposed by humans; [0088] results of how well the data fit various regression models detected automatically as well as proposed by humans; [0089] specific subsets of data with high p-values, corresponding to automatically or manually detected patterns (discloses parameter specific scores), and corresponding manually proposed causal links; [0090] votes and tags indicating agreement from communities such as customers or employees; and [0091] other valuable context information), (Id., ¶ 104, The impact of each variable combination typically is determined by the behavior of a variable combination with respect to the outcome and by the population of the variable combination. In one approach, automated analysis learns the normative behavior for each variable combination as it relates to the outcome. For example it may learn that Men in California spend more while 18 to 25 year olds who buy over the Mobile channel spend less than usual in general (here amount spent is the outcome). But a specific transaction may be for a Male 18 to 25 years old from California who purchased goods over the Mobile channel. By observing the norm for each variable combination in isolation and in combinations across multiple transactions, we can learn the "net impact" (the behavior) of a variable combination. This is the positive or negative impact of the variable combination on the observed outcome, net of the impact of all other variable combinations that may also be affecting that specific transaction. This allows automated analysis to learn a behavior metric that is similar to obtaining a regression coefficient in a regression analysis, but which can be learned via the search-based approach described above with reference to FIG. 1, instead of running a regression analysis. In an alternative approach, a type of regression analysis is run for the outcome with respect to all of the variable combinations being considered. For each variable combination, there will be a regression term (the impact) that equals the regression coefficient (the behavior) multiplied by the population. (discloses overall impact score) Behavior may also be measured in terms of correlation coefficients, net-effect impact net of all other variables, or any other suitable metric that captures how the variable combination affects the outcome or how the outcome trends as a function of the variable combinations. Population may also be measured in terms of counts (i.e., number of observations), whether or not something occurred, frequency/percentage of overall population, or relative frequencies of observations. The overall impact of a variable combination depends on both its behavior (i.e., how strongly does that variable combination affect the outcome) and its population (i.e., how much of that variable combination exists in the data set of interest). These impacts, behaviors and populations can then be used to analyze the data set in different ways), (Id., ¶ 109, Predictive graphs 220 illustrate an outcome of predictive analysis that selects the Descriptive graphs 210 to be displayed as well as to make Prescriptive recommendations 240. Expert users can access the predictive capabilities directly from the `Choose a graph` feature), (Id., ¶ 110, Diagnostic graphs 230 highlight multiple unrelated factors (i.e., variable combinations) that contribute to an outcome or visual pattern displayed in a graph. For a Descriptive graph 210, BeyondCore automatically checks for what other factors might be contributing to the pattern. For example, a hospital that is doing badly may actually have far more emergency patients and that is why it is doing badly. Diagnostic graphs 230 help ensure that the patterns the user focuses on are real and not accidents of the data), (Id., ¶ 111, Prescriptive graphs 240 provide a means for the user to communicate to BeyondCore which of the variables are actionable (things that can be changed easily) and whether the user wants to maximize or minimize the outcome. BeyondCore can then look at millions (typically) of possibilities for changing variables, conducts Predictive analysis, recommends specific actions, quantifies the expected impact, and explains the reasoning behind the recommendations);
providing the parameter-specific impact score for the corresponding assessment parameter and the overall impact score to the organization user receiving, from the organization user and based on the parameter-specific impact score and the overall impact score, an update to the form field of the set of form fields within the interactive assessment form (Id., ¶ 89, specific subsets of data with high p-values, corresponding to automatically or manually detected patterns (discloses parameter specific scores), and corresponding manually proposed causal links; [0090] votes and tags indicating agreement from communities such as customers or employees; and [0091] other valuable context information), (Id., ¶ 104, The impact of each variable combination typically is determined by the behavior of a variable combination with respect to the outcome and by the population of the variable combination. In one approach, automated analysis learns the normative behavior for each variable combination as it relates to the outcome. For example it may learn that Men in California spend more while 18 to 25 year olds who buy over the Mobile channel spend less than usual in general (here amount spent is the outcome). But a specific transaction may be for a Male 18 to 25 years old from California who purchased goods over the Mobile channel. By observing the norm for each variable combination in isolation and in combinations across multiple transactions, we can learn the "net impact" (the behavior) of a variable combination. This is the positive or negative impact of the variable combination on the observed outcome, net of the impact of all other variable combinations that may also be affecting that specific transaction. This allows automated analysis to learn a behavior metric that is similar to obtaining a regression coefficient in a regression analysis, but which can be learned via the search-based approach described above with reference to FIG. 1, instead of running a regression analysis. In an alternative approach, a type of regression analysis is run for the outcome with respect to all of the variable combinations being considered. For each variable combination, there will be a regression term (the impact) that equals the regression coefficient (the behavior) multiplied by the population. (discloses overall impact score) Behavior may also be measured in terms of correlation coefficients, net-effect impact net of all other variables, or any other suitable metric that captures how the variable combination affects the outcome or how the outcome trends as a function of the variable combinations. Population may also be measured in terms of counts (i.e., number of observations), whether or not something occurred, frequency/percentage of overall population, or relative frequencies of observations. The overall impact of a variable combination depends on both its behavior (i.e., how strongly does that variable combination affect the outcome) and its population (i.e., how much of that variable combination exists in the data set of interest). These impacts, behaviors and populations can then be used to analyze the data set in different ways), (Id., ¶ 52, Local search methods operate by considering a given sample, and repeatedly modifying it with the goal of raising the metric. This continues until the metric is higher for the sample under consideration than for any nearby samples (a local optimum). The notion of proximity is complex for samples of the sort we are discussing. The "modify" step in the algorithm will change the restrictions defining the current sample. This can consist of widening or tightening the restriction on one field, or adding a restriction on a new field, (discloses updating form fields on an assessment form) or removing the restriction on a restricted field. For example, if we consider a sample consisting of "Loan applications from females aged 30-40" and calculate the metric to be X, we could then calculate the metric for "females", "females aged 30-50", "females aged 20-40", "people aged 30-40", and others. Each of these metrics will be compared to X and the search algorithm will continue);
and computing, based on the update to the form field of the set of form fields, an updated parameter-specific impact score for the corresponding assessment parameter, and an updated overall impact score (Id., ¶ 52, Local search methods operate by considering a given sample, and repeatedly modifying it with the goal of raising the metric. This continues until the metric is higher for the sample under consideration than for any nearby samples (a local optimum). The notion of proximity is complex for samples of the sort we are discussing. The "modify" step in the algorithm will change the restrictions defining the current sample. This can consist of widening or tightening the restriction on one field, or adding a restriction on a new field, (discloses updating form fields on an assessment form) or removing the restriction on a restricted field. For example, if we consider a sample consisting of "Loan applications from females aged 30-40" and calculate the metric to be X, we could then calculate the metric for "females", "females aged 30-50", "females aged 20-40", "people aged 30-40", and others. (discloses computing updated impact scores) Each of these metrics will be compared to X and the search algorithm will continue), (Id., ¶ 53, Because the metrics are highest for samples with acute variances, samples obtained using parameter values which are responsible for the unusual behavior will have the highest scores. Much larger and much smaller samples will have lower scores. As the search algorithm runs, the sample under consideration will "evolve" to contain the features that are causing the discrepancy in operator processing while not containing unrelated random information. Of course, the search will cease on one local maximum. If the local search is repeated multiple times from random starting samples, many samples with peak metrics can be identified in the data).
While suggested in at least Fig. 1 and related text, Sengupta does not explicitly disclose …and wherein the analyzing comprises filtering the impactful corpus catalog by a semantic-similarity calculation which evaluates text entered in the form field of the form fields against the asset metadata within a metadata field corresponding to the corresponding assessment parameter.
However, Malhotra discloses …and wherein the analyzing comprises filtering the impactful corpus catalog by a semantic-similarity calculation which evaluates text entered in the form field of the form fields against the asset metadata within a metadata field corresponding to the corresponding assessment parameter (Malhotra, ¶ 20, Before data set 142 is analyzed, server system 130 may prompt users to provide profile information in one of a number of ways. For example, server system 130 may have provided a web page with a text field for one or more of the above-referenced types of information. In response to receiving profile information from a user's device, server system 130 stores the information in an account that is associated with the user and that is associated with credential data that is used to authenticate the user to server system 130 when the user attempts to log into server system 130 at a later time. Each text string provided by a user may be stored in association with the field into which the text string was entered. For example, if a user enters “Sales Manager” in a job title field, then “Sales Manager” is stored in association with type data that indicates that “Sales Manager” is a job title (discloses evaluating text in form fields). As another example, if a user enters “Java programming” in a skills field, then “Java programming” is stored in association with type data that indicates that “Java programming” is a skill), (Id., ¶ 32, In an embodiment, some text may be considered more relevant than other text in determining whether (or how much) a particular skill is associated with a corresponding learning resource. For example, if a title or metadata (discloses evaluating form field text against asset metadata) of a learning resource indicates a first skill, then it is more likely that the learning resource is dedicated to teaching that first skill than in teaching a second skill that is mentioned only in the body of the learning resource. As another example, if multiple online reviews of a learning resource indicate a first skill, then it is more likely that the learning resource is dedicated to teaching that first skill than in teaching a second skill that is not mentioned in any of the online reviews. 101, One method to generate embeddings includes artificial neural networks. In the context of linguistics, word embedding, when used as the underlying input representation, have been shown to boost performance in natural language processing (NLP) tasks, such as syntactic parsing and sentiment analysis. Word embedding aims to quantify and categorize semantic similarities between linguistic items (discloses semantic-similarity calculations) based on their distributional properties in large samples of language data. The underlying idea that a word is characterized by “the company it keeps.”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the guidance information elements of Sengupta to include the semantic similarity elements of Malhotra in the analogous art of automatic generation and personalization of learning paths for the same reasons as stated for claim 1.
Regarding Claims 17-18, these claims recite limitations substantially similar to those recited in claims 2-3, respectively, and are rejected for the same reasons as stated above.
Regarding Claim 20, Sengupta discloses …A system, the system comprising: a client device; and an insight service operatively connected to the client device, and comprising a computer processor configured to perform a method for providing guidance, the method comprising: detecting an initiation, by an organization user, of an impact assessment program (Sengupta, ¶ 80, Once the feedback is received 135, the initial automated analysis 110/120 may be re-run. For example, if the humans suggested additional external data, new hypotheses, new patterns, new subsets of data with higher p-values, etc., each of these may enable improved automated analysis. After the automated analysis is completed in light of the human-feedback, the system may go through an additional human-feedback step. (discloses imitation of an impact assessment program) The automated-analysis through human feedback cycle may be carried out as many times as necessary to get optimal analysis results. The feedback cycle may be terminated after a set number of times or if the results do not improve significantly after a feedback cycle or if no significant new feedback is received during a given human feedback step. The feedback cycle need not be a monolithic process. For example, if a human feedback only affects part of the overall analysis, that part may be reanalyzed automatically based on the feedback without affecting the rest of the analysis), (Id., ¶ 81, As the analysis is improved based on human feedback, a learning algorithm can evaluate which human feedback had the most impact on the results and which feedback had minor or even negative impact on the results. (discloses impact assessment) As this method clearly links specific human feedback to specific impacts on the results of the analysis, the learning algorithms have a rich source of data to train on. Eventually, these learning algorithms would themselves be able to suggest improvement opportunities which could be directly leveraged in the automated analysis phase), (Id., ¶ 123, FIGS. 6A-6B illustrate user interfaces that enable a user to share (FIG. 6A) or download (FIG. 6B) a story. FIG. 6A illustrates that a user can share the story with other users in the organization and either authorize them to view the story or edit the story as well. (discloses organization users) Any edits made by authorized users can be seen by every other user. The user can use the `History` link below each graph to revert to his preferred version of the story. BeyondCore users can be grouped into organizations and, in some embodiments, users can share stories only with people within their organization. If a user does not see a specific user in their share screen, the user may confirm that the specific user has registered with BeyondCore and is in the same organization as the user himself. Additional options to include allowing or denying editing capabilities of story 610, granting or revoking access to the story 620, selecting users with whom he wants to share his story 630 (e.g., via a drop down list that includes every user in the viewing user's organization that has registered with BeyondCore), sharing the story 650, and so on. Further, as illustrated in FIG. 6B, a user can download (e.g., via UI element 670) and email a static HTML version of the story to other licensed users in his organization. In some embodiments, only the main story (excluding the recommendations panel) is available through this HTML file. Users may also download the story in other formats such as PowerPoint, Word and pdf files), (Id., ¶ 96, The system, as described in the present invention or any of its components, may be embodied in the form of a computer system. Typical examples of a computer system include a general-purpose computer, a programmed microprocessor, a micro-controller, a peripheral integrated circuit element, and other devices or arrangements of devices that are capable of implementing the steps that constitute the method of the present invention), (Id., ¶ 97, The computer system comprises a computer, an input device, a display unit and the Internet. The computer comprises a microprocessor. The microprocessor can be one or more general- or special-purpose processors such as a Pentium.RTM., Centrino.RTM., Power PC.RTM., and a digital signal processor. The microprocessor is connected to a communication bus. The computer also includes a memory, which may include Random Access Memory (RAM) and Read Only Memory (ROM). The computer system also comprises a storage device, which can be a hard disk drive or a removable storage device such as a floppy disk drive, optical disk drive, and so forth. The storage device can also be other similar means for loading computer programs or other instructions into the computer system);
instantiating an interactive assessment form comprising a set of form fields, wherein each form field is mapped to a corresponding assessment parameter; presenting, through the impact assessment program, the interactive assessment form to the organization user (Id., ¶ 25, FIGS. 3A-3E are screen shots illustrating the evolution of a story (an analysis project), according to some embodiments), (Id., ¶ 114, FIG. 3C illustrates a user interface for selecting the business outcome (the variable) that the user wishes to analyze. This is typically the KPI or metric in the user's dashboards and reports, e.g. revenue or cost. This page lists all the numeric or binary (e.g. Male/Female) columns in the user's data that have sufficient variability. If the user does not see a variable that is expected, the user may verify that the variable has numerical values and not text. If a variable has only a few values (e.g. 1, 2, 5), BeyondCore will treat it as a categorical variable instead of numeric. In most cases this is desired and statistically appropriate. To change a variable from categorical to numeric, the user may go to the Data Setup page and manually filter or reformat data (see advanced options at FIG. 4A). Returning to FIG. 3C, the user may click user element 345 to select the business outcome. (which would be the y-axis of the graph, or the number the user wants to predict)), (Id., Fig. 3C, Figure depicts an interactive assessment form comprising a set of form fields mapped to assessment parameters);
PNG
media_image1.png
299
448
media_image1.png
Greyscale
continuously monitoring interactions, by the organization user, with the interactive assessment form to identify an engagement action, wherein the engagement action comprises at least a hovering of an electronically displayed cursor over a form field of the form fields or an entering or deleting of text in the form field of the form fields (Id., ¶ 187, Referring to FIG. 11B, what-if scenario analysis enables a user to compare predicted outcomes for different values of a variable under specified conditions. The user interface may allow the user to select 1140 the variable that the user wishes to compare different outcomes for (e.g., in the example of 1140, what acquisition channel should the user use?). The user may also choose 1145 the variables to constrain based on. The user may also click `View Graph` 1150 to see the analysis. The user may also specify 1155 conditions under which the user wants to compare the what-if variable (in this case, Females with income between 61500 and 81500)), (Id., ¶ 188, As illustrated in FIG. 11C, the Predictive Analysis shows the expected outcome under specified circumstances and explains the reasons behind the prediction. The user may additionally select variables to constrain based on 1160, conditions 1165 for which to predict the outcome (in this case, Californian females who are 20 to 25 years old with income between 61500 and 81500), and an option to view the graph with the analysis 1170. The graph itself includes the overall average 1175-a, reasons behind the prediction 1175-b (the user may hover or mouse over to see additional details) (discloses monitoring interactions to identify an engagement action), and predicted outcome 1175-c. The prediction is based on the automatically learned features of the data set, the automated analysis approach for which has been described above), (Id., ¶ 82, The human feedback patterns could also be analyzed to detect deterministic patterns that may or may not be context specific. For example, if local rainfall patterns turn out to be a common external variable for retail analyses, the software may automatically start including this data in similar analyses. Similarly, if humans frequently combine behavior patterns noticed on Saturdays and Sundays to create a higher p-value pattern for weekends, the software could learn to treat weekends and weekdays differently in its analyses);
analyzing, based on the engagement action and to obtain guidance information, an impactful corpus catalog comprising a set of catalog entries electronically stored in a data structure, wherein the impactful corpus catalog is a structured repository of asset metadata… (Id., ¶ 43, Various types of automated analysis have been described previously by the inventors. For example, in the context of document processing by operators, one goal may be to find documents that are similar in some way in order to identify underlying patterns of operator behavior. A search can be conducted for segments of the data which share as few as one or more similar field or parameter values. For example, a database of loan applications can be searched for applicants between 37 and 39 years of age. Any pair of applications from this sample might be no more similar than a randomly chosen pair from the population. However, this set of applications can be statistically analyzed to determine whether certain loan officers are more likely to approve loans from this section of the population), (Id., ¶ 44, Alternatively, it may not be necessary to find even one very similar parameter. Large segments of the population may be aggregated for analysis using criteria such as "applicants under 32 years old" or "applicants earning more than $30,000 per year." Extending this methodology one step further, a single analysis can be conducted on the sample consisting of the entire population), (Id., ¶ 79, Some humans may try to submit large volumes of suggestions hoping that at least one of them works. Others may even write computer code to generate many suggestions. As long as the computation resources needed to evaluate such suggestions is minimal, this is not a significant problem and may even contribute to the overall objective of useful analysis. To reduce the computational cost of the evaluation of suggestions, such suggestions may first be tested against a subset of the overall data. Suggestions would only be incorporated while analyzing the overall data if the suggestion enabled a significant improvement when used to analyze the subset data. To further save computation expenses, multiple suggestions evaluated on the subset data may be combined before the corresponding updated analysis is run on the complete data. (discloses obtaining guidance information based on an automated analysis of an impactful corpus catalog) Additionally, computation resources could be allocated to different users via a quota system, and users could optionally "purchase" more using their rewards from previous suggestions), (Id., ¶ 116, FIG. 3E illustrates a user interface that allows a user to further customize a BeyondCore story. The user may edit the story name, row labels, column labels, and the like. For example, UI element 360 allows a user to specify a story title, UI element 362 allows a user to specify how BeyondCore should refer to a row in the data, (discloses structured asset metadata) UI element 364 enables the user to specify the unit of the y-axis of his graphs (the outcome variable), UI element 368 enables the user to access advanced options or further customize a story);
and providing, through the interactive assessment form, the guidance information to the organization user, wherein providing the guidance information comprises: computing, based on the analyzing, a parameter-specific impact score for the corresponding assessment parameter and an overall impact score; (Id., ¶ 86, Once the automated analysis with human feedback is completed, the data could be presented to expert analysts 140 for further enhancement. Such analysts would have the benefit of the following: [0087] lists of hypotheses detected automatically as well as proposed by humans; [0088] results of how well the data fit various regression models detected automatically as well as proposed by humans; [0089] specific subsets of data with high p-values, corresponding to automatically or manually detected patterns (discloses parameter specific scores), and corresponding manually proposed causal links; [0090] votes and tags indicating agreement from communities such as customers or employees; and [0091] other valuable context information), (Id., ¶ 104, The impact of each variable combination typically is determined by the behavior of a variable combination with respect to the outcome and by the population of the variable combination. In one approach, automated analysis learns the normative behavior for each variable combination as it relates to the outcome. For example it may learn that Men in California spend more while 18 to 25 year olds who buy over the Mobile channel spend less than usual in general (here amount spent is the outcome). But a specific transaction may be for a Male 18 to 25 years old from California who purchased goods over the Mobile channel. By observing the norm for each variable combination in isolation and in combinations across multiple transactions, we can learn the "net impact" (the behavior) of a variable combination. This is the positive or negative impact of the variable combination on the observed outcome, net of the impact of all other variable combinations that may also be affecting that specific transaction. This allows automated analysis to learn a behavior metric that is similar to obtaining a regression coefficient in a regression analysis, but which can be learned via the search-based approach described above with reference to FIG. 1, instead of running a regression analysis. In an alternative approach, a type of regression analysis is run for the outcome with respect to all of the variable combinations being considered. For each variable combination, there will be a regression term (the impact) that equals the regression coefficient (the behavior) multiplied by the population. (discloses overall impact score) Behavior may also be measured in terms of correlation coefficients, net-effect impact net of all other variables, or any other suitable metric that captures how the variable combination affects the outcome or how the outcome trends as a function of the variable combinations. Population may also be measured in terms of counts (i.e., number of observations), whether or not something occurred, frequency/percentage of overall population, or relative frequencies of observations. The overall impact of a variable combination depends on both its behavior (i.e., how strongly does that variable combination affect the outcome) and its population (i.e., how much of that variable combination exists in the data set of interest). These impacts, behaviors and populations can then be used to analyze the data set in different ways), (Id., ¶ 109, Predictive graphs 220 illustrate an outcome of predictive analysis that selects the Descriptive graphs 210 to be displayed as well as to make Prescriptive recommendations 240. Expert users can access the predictive capabilities directly from the `Choose a graph` feature), (Id., ¶ 110, Diagnostic graphs 230 highlight multiple unrelated factors (i.e., variable combinations) that contribute to an outcome or visual pattern displayed in a graph. For a Descriptive graph 210, BeyondCore automatically checks for what other factors might be contributing to the pattern. For example, a hospital that is doing badly may actually have far more emergency patients and that is why it is doing badly. Diagnostic graphs 230 help ensure that the patterns the user focuses on are real and not accidents of the data), (Id., ¶ 111, Prescriptive graphs 240 provide a means for the user to communicate to BeyondCore which of the variables are actionable (things that can be changed easily) and whether the user wants to maximize or minimize the outcome. BeyondCore can then look at millions (typically) of possibilities for changing variables, conducts Predictive analysis, recommends specific actions, quantifies the expected impact, and explains the reasoning behind the recommendations);
providing the parameter-specific impact score for the corresponding assessment parameter and the overall impact score to the organization user receiving, from the organization user and based on the parameter-specific impact score and the overall impact score, an update to the form field of the set of form fields within the interactive assessment form (Id., ¶ 89, specific subsets of data with high p-values, corresponding to automatically or manually detected patterns (discloses parameter specific scores), and corresponding manually proposed causal links; [0090] votes and tags indicating agreement from communities such as customers or employees; and [0091] other valuable context information), (Id., ¶ 104, The impact of each variable combination typically is determined by the behavior of a variable combination with respect to the outcome and by the population of the variable combination. In one approach, automated analysis learns the normative behavior for each variable combination as it relates to the outcome. For example it may learn that Men in California spend more while 18 to 25 year olds who buy over the Mobile channel spend less than usual in general (here amount spent is the outcome). But a specific transaction may be for a Male 18 to 25 years old from California who purchased goods over the Mobile channel. By observing the norm for each variable combination in isolation and in combinations across multiple transactions, we can learn the "net impact" (the behavior) of a variable combination. This is the positive or negative impact of the variable combination on the observed outcome, net of the impact of all other variable combinations that may also be affecting that specific transaction. This allows automated analysis to learn a behavior metric that is similar to obtaining a regression coefficient in a regression analysis, but which can be learned via the search-based approach described above with reference to FIG. 1, instead of running a regression analysis. In an alternative approach, a type of regression analysis is run for the outcome with respect to all of the variable combinations being considered. For each variable combination, there will be a regression term (the impact) that equals the regression coefficient (the behavior) multiplied by the population. (discloses overall impact score) Behavior may also be measured in terms of correlation coefficients, net-effect impact net of all other variables, or any other suitable metric that captures how the variable combination affects the outcome or how the outcome trends as a function of the variable combinations. Population may also be measured in terms of counts (i.e., number of observations), whether or not something occurred, frequency/percentage of overall population, or relative frequencies of observations. The overall impact of a variable combination depends on both its behavior (i.e., how strongly does that variable combination affect the outcome) and its population (i.e., how much of that variable combination exists in the data set of interest). These impacts, behaviors and populations can then be used to analyze the data set in different ways), (Id., ¶ 52, Local search methods operate by considering a given sample, and repeatedly modifying it with the goal of raising the metric. This continues until the metric is higher for the sample under consideration than for any nearby samples (a local optimum). The notion of proximity is complex for samples of the sort we are discussing. The "modify" step in the algorithm will change the restrictions defining the current sample. This can consist of widening or tightening the restriction on one field, or adding a restriction on a new field, (discloses updating form fields on an assessment form) or removing the restriction on a restricted field. For example, if we consider a sample consisting of "Loan applications from females aged 30-40" and calculate the metric to be X, we could then calculate the metric for "females", "females aged 30-50", "females aged 20-40", "people aged 30-40", and others. Each of these metrics will be compared to X and the search algorithm will continue);
and computing, based on the update to the form field of the set of form fields, an updated parameter-specific impact score for the corresponding assessment parameter, and an updated overall impact score (Id., ¶ 52, Local search methods operate by considering a given sample, and repeatedly modifying it with the goal of raising the metric. This continues until the metric is higher for the sample under consideration than for any nearby samples (a local optimum). The notion of proximity is complex for samples of the sort we are discussing. The "modify" step in the algorithm will change the restrictions defining the current sample. This can consist of widening or tightening the restriction on one field, or adding a restriction on a new field, (discloses updating form fields on an assessment form) or removing the restriction on a restricted field. For example, if we consider a sample consisting of "Loan applications from females aged 30-40" and calculate the metric to be X, we could then calculate the metric for "females", "females aged 30-50", "females aged 20-40", "people aged 30-40", and others. (discloses computing updated impact scores) Each of these metrics will be compared to X and the search algorithm will continue), (Id., ¶ 53, Because the metrics are highest for samples with acute variances, samples obtained using parameter values which are responsible for the unusual behavior will have the highest scores. Much larger and much smaller samples will have lower scores. As the search algorithm runs, the sample under consideration will "evolve" to contain the features that are causing the discrepancy in operator processing while not containing unrelated random information. Of course, the search will cease on one local maximum. If the local search is repeated multiple times from random starting samples, many samples with peak metrics can be identified in the data).
While suggested in at least Fig. 1 and related text, Sengupta does not explicitly disclose …and wherein the analyzing comprises filtering the impactful corpus catalog by a semantic-similarity calculation which evaluates text entered in the form field of the form fields against the asset metadata within a metadata field corresponding to the corresponding assessment parameter.
However, Malhotra discloses …and wherein the analyzing comprises filtering the impactful corpus catalog by a semantic-similarity calculation which evaluates text entered in the form field of the form fields against the asset metadata within a metadata field corresponding to the corresponding assessment parameter (Malhotra, ¶ 20, Before data set 142 is analyzed, server system 130 may prompt users to provide profile information in one of a number of ways. For example, server system 130 may have provided a web page with a text field for one or more of the above-referenced types of information. In response to receiving profile information from a user's device, server system 130 stores the information in an account that is associated with the user and that is associated with credential data that is used to authenticate the user to server system 130 when the user attempts to log into server system 130 at a later time. Each text string provided by a user may be stored in association with the field into which the text string was entered. For example, if a user enters “Sales Manager” in a job title field, then “Sales Manager” is stored in association with type data that indicates that “Sales Manager” is a job title (discloses evaluating text in form fields). As another example, if a user enters “Java programming” in a skills field, then “Java programming” is stored in association with type data that indicates that “Java programming” is a skill), (Id., ¶ 32, In an embodiment, some text may be considered more relevant than other text in determining whether (or how much) a particular skill is associated with a corresponding learning resource. For example, if a title or metadata (discloses evaluating form field text against asset metadata) of a learning resource indicates a first skill, then it is more likely that the learning resource is dedicated to teaching that first skill than in teaching a second skill that is mentioned only in the body of the learning resource. As another example, if multiple online reviews of a learning resource indicate a first skill, then it is more likely that the learning resource is dedicated to teaching that first skill than in teaching a second skill that is not mentioned in any of the online reviews. 101, One method to generate embeddings includes artificial neural networks. In the context of linguistics, word embedding, when used as the underlying input representation, have been shown to boost performance in natural language processing (NLP) tasks, such as syntactic parsing and sentiment analysis. Word embedding aims to quantify and categorize semantic similarities between linguistic items (discloses semantic-similarity calculations) based on their distributional properties in large samples of language data. The underlying idea that a word is characterized by “the company it keeps.”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the guidance information elements of Sengupta to include the semantic similarity elements of Malhotra in the analogous art of automatic generation and personalization of learning paths for the same reasons as stated for claim 1.
Claims 4-15 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Sengupta in view of Malhotra and in further view of Demaris et al., U.S. Publication No. 2020/0210383 [hereinafter Demaris].
Regarding Claim 4, the combination of Sengupta and Malhotra discloses …the method of claim 3…
Sengupta further discloses …wherein analyzing, based on the engagement action, the impactful corpus catalog to obtain the guidance information, comprises: making a determination, based on a search across the set of form fields, that at least one form field, in the set of form fields, is a non-empty form field (Id., ¶ 115, FIGS. 2-13 illustrate examples and implementations of various aspects of the analysis methodology and frameworks described above. For convenience, these examples and implementations will be referred to as BeyondCore. These examples concern the analysis of a data set for a process, where the term "process" is intended to include any system, business, operation, activity, series of actions, or any other things that can generate a data set. The data set contains observations of the process, which are expressed as values for the outcome of the process and for variables that may affect the process. Depending on the application, the data set may contain at least 100,000 observations, at least 1,000,000 observations or more. The outcome may be directly observed or it may be derived. Typically, the data set will be organized into rows and columns, where each column is a different outcome or variable and each row is a different observation of the outcomes and variables. Typically, not every cell will be filled. That is, some variables may be blank for some observations), (Id., ¶ 41, For example, in the policy data entry example described above, an automated analysis 110/120 could detect that Spanish forms had higher error rates in the gender field but automated analysis may not be able to spot the true underlying reason. A human being however may suggest 130 checking the errors against whether or not the corresponding operator knew Spanish. As indicated by the feedback arrow 135, this would allow the analysis 110/120 to statistically confirm that operators who do not know Spanish exhibit a disproportionately high error rate while selecting the gender for female customers (due to the Mujer=male confusion). In this way, actionable insights can be iteratively developed through a combination of computer analysis and statistically untrained human feedback);
extracting, based on the determination and from the at least one form field, a field input to obtain at least one field input (Id., ¶ 41, For example, in the policy data entry example described above, an automated analysis 110/120 could detect that Spanish forms had higher error rates in the gender field but automated analysis may not be able to spot the true underlying reason. A human being however may suggest 130 checking the errors against whether or not the corresponding operator knew Spanish. As indicated by the feedback arrow 135, this would allow the analysis 110/120 to statistically confirm that operators who do not know Spanish exhibit a disproportionately high error rate while selecting the gender for female customers (due to the Mujer=male confusion). In this way, actionable insights can be iteratively developed through a combination of computer analysis and statistically untrained human feedback), (Id., ¶ 46, These methods can be combined to find a diverse variety of samples to analyze. A sample might consist of the documents with each field similar to a given value for that field, or it might comprise the set of all the documents. In addition, some fields may be restricted to a small or large range, where other fields have no restriction. Each sample may be analyzed with statistical methods to determine whether operators are processing documents consistently), (Id., ¶ 47, There are several statistical hypothesis tests which may be appropriate for making this determination. If the output of the process is binary, such as a loan approval, and the number of documents in the sample under analysis is small, a test such as Fisher's Exact Test may be used. If the output is a number, such as a loan interest rate, and the sample is large, a Chi-Square Test may be used. These tests can be used to determine whether one operator is producing significantly differing output from the remainder of the operators. Alternately, the operators can be split into two groups and these tests can be used to determine whether the operators in the two groups are producing significantly differing output. All possible splits can be analyzed to find the one with the highest statistical significance. Alternately, these tests can be used to determine simply whether the distribution of operator output for this sample is significantly more unusual than what would be expected under the null hypothesis, i.e., all operators making decisions in the same manner)
filtering, based on the at least one field input, the impactful corpus catalog to identify a catalog entry subset in the set of catalog entries (Id., ¶ 79, Some humans may try to submit large volumes of suggestions hoping that at least one of them works. Others may even write computer code to generate many suggestions. As long as the computation resources needed to evaluate such suggestions is minimal, this is not a significant problem and may even contribute to the overall objective of useful analysis. To reduce the computational cost of the evaluation of suggestions, such suggestions may first be tested against a subset of the overall data. Suggestions would only be incorporated while analyzing the overall data if the suggestion enabled a significant improvement when used to analyze the subset data. To further save computation expenses, multiple suggestions evaluated on the subset data may be combined before the corresponding updated analysis is run on the complete data. (discloses obtaining guidance information by filtering to identify a catalog entry subset of an impactful corpus catalog) Additionally, computation resources could be allocated to different users via a quota system, and users could optionally "purchase" more using their rewards from previous suggestions), (Id., ¶ 104, The impact of each variable combination typically is determined by the behavior of a variable combination with respect to the outcome and by the population of the variable combination. In one approach, automated analysis learns the normative behavior for each variable combination as it relates to the outcome. For example it may learn that Men in California spend more while 18 to 25 year olds who buy over the Mobile channel spend less than usual in general (here amount spent is the outcome). But a specific transaction may be for a Male 18 to 25 years old from California who purchased goods over the Mobile channel. By observing the norm for each variable combination in isolation and in combinations across multiple transactions, we can learn the "net impact" (the behavior) of a variable combination. This is the positive or negative impact of the variable combination on the observed outcome, net of the impact of all other variable combinations that may also be affecting that specific transaction. This allows automated analysis to learn a behavior metric that is similar to obtaining a regression coefficient in a regression analysis, but which can be learned via the search-based approach described above with reference to FIG. 1, instead of running a regression analysis. In an alternative approach, a type of regression analysis is run for the outcome with respect to all of the variable combinations being considered. For each variable combination, there will be a regression term (the impact) that equals the regression coefficient (the behavior) multiplied by the population. Behavior may also be measured in terms of correlation coefficients, net-effect impact net of all other variables, or any other suitable metric that captures how the variable combination affects the outcome or how the outcome trends as a function of the variable combinations. Population may also be measured in terms of counts (i.e., number of observations), whether or not something occurred, frequency/percentage of overall population, or relative frequencies of observations. The overall impact of a variable combination depends on both its behavior (i.e., how strongly does that variable combination affect the outcome) and its population (i.e., how much of that variable combination exists in the data set of interest). These impacts, behaviors and populations can then be used to analyze the data set in different ways).
While suggested in at least Fig. 1 and related text, Sengupta does not explicitly disclose …and analyzing asset metadata, maintained across the catalog entry subset, to obtain the guidance information.
However, Demaris discloses …and analyzing asset metadata, maintained across the catalog entry subset, to obtain the guidance information (Demaris, ¶ 20, some computing systems store files on a cloud computing data storage system. However, during authoring and editing of a document, these same files may also be stored on a local disk by the application. Similarly, as discussed in further detail below, some systems have applications that support document collaboration, such as merge, sharing, co-authoring, and other functionalities or capabilities. In some cases, these systems can rely on a synchronization engine (or “sync engine”) that is responsible for detecting and synchronizing changes to files and folders between a local disk and a cloud storage system. In order to accomplish this, the sync engine can track the state of both the file that is stored on the disk and that stored in the cloud, and reconcile those states when it receives information that something has changed. As an example, if a file is edited on a local disk, the sync engine may detect that change, realize it needs to send the change to the cloud, transmit the change, wait for the cloud to respond, and then update its local state information to indicate that the change has been made. In cases where a file is the product of or subject to collaboration, additional and evolving metadata can be associated with the file. In some implementations, the system can be configured to maintain and communicate a set of collaborative metadata for each file. In other words, each file may have or be linked to metadata about the file. Such “collaborative metadata” for each file may be used in part to help maintain the various versions of the file, as well as provide important information (discloses analyzing asset metadata to obtain guidance information) about status, workflow tasks, or other aspects of the file. However, maintaining an up-do-date synchronization of the file as well as its associated collaborative metadata can cause delays during synchronization as well as consume significant amounts of computational resources. In order to help minimize the impact of the synchronization process on system and network resources, the following disclosure proposes a paradigm in which collaborative metadata for a file is essentially separated from the file during synchronization and subsequent user access of the file on a client device. In other words, while users download and/or access files via client device applications and these files continue to be synchronized with a cloud storage system, the collaborative metadata remains in a remote location (e.g., outside of the local device). In the event a user initiates an action in the file that requires access of the collaborative metadata, the proposed content management system (CMS) can respond by providing a portal or other network interface on the local device via which the user can directly access the collaborative metadata that is stored in the cloud. The synchronization process for files thus occurs without requiring transference of large, costly amounts of collaborative metadata to the client device. By enabling users to ‘reach out’ to the cloud for file metadata on an as-needed basis, the system as a whole can operate more smoothly and efficiently).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the impact assessment and guidance elements of Sengupta and the semantic similarity elements of Malhotra to include the metadata analysis elements of Demaris in the analogous art of remote access of metadata for collaborative documents.
The motivation for doing so would have been to provide “new and improved ideas for reducing time lags during synchronization of data as well as improving the experience for users to interact with data about a given document” (Demaris, ¶ 2), wherein such improvements would have benefitted Malhotra’s method to provide “a computer-related improvement over prior learning assistance computer systems…” wherein “Embodiments consider attributes and/or activity of individual users and their respective career goals in crafting individualized learning paths” (Malhotra, ¶ 112),and wherein such improvements would have further benefitted Sengupta’s method which provides an improved system wherein “ the user's interaction with the multi-screen reporting are tracked. In some embodiments, the user's interaction with the multi-screen reporting are tracked in a manner that is auditable. In some embodiments, changes to the multi-screen reporting resulting from the user's interaction with the multi-screen reporting are sharable with other users” [Demaris, ¶ 2; Malhotra, ¶ 112; Sengupta, ¶ 129].
Regarding Claim 5, the combination of Sengupta, Malhotra and Demaris discloses …the method of claim 4…
Sengupta further discloses …wherein the form field is one selected from a group comprising included in, and excluded from, the at least one form field (Id., ¶ 12, because humans cannot easily deal with large volumes of data or complex data, analysts often ignore variables they deem less important. Analysts may easily accidentally ignore a variable that turns out to be key. During an analysis of a credit card application process, it was found that the auditors had ignored the "Time at current address" field in their analysis as it was thought to be a relatively unimportant field. However, it turned out that this field had an exceptionally high error rate (perhaps precisely because operators also figured that the field was unimportant and thus did not pay attention to processing it correctly). Once the high error rate was factored in, this initially ignored field turned out to be a key factor in the overall analysis. Analysts also sometimes initially explore data to get a "sense of it" to help them form their hypotheses. Typically, for large datasets, analysts can only explore subsets of the overall data to detect patterns that would lead them to the right hypotheses or models. If they accidentally look at the wrong subset or fail to review a subset with the clearest patterns, they may easily miss key factors that would affect the accuracy of their analysis).
Regarding Claim 6, the combination of Sengupta, Malhotra and Demaris discloses …the method of claim 4…
Sengupta further discloses …wherein analyzing, based on the engagement action, the impactful corpus catalog to obtain the guidance information, further comprises: prior to filtering the impactful corpus catalog: mapping the at least one form field, respectively, to at least one filtering assessment parameter in a set of assessment parameters, wherein the impactful corpus catalog is filtered further based on the at least one filtering assessment parameter (Id., ¶ 43, Various types of automated analysis have been described previously by the inventors. For example, in the context of document processing by operators, one goal may be to find documents that are similar in some way in order to identify underlying patterns of operator behavior. A search can be conducted for segments of the data which share as few as one or more similar field or parameter values. For example, a database of loan applications can be searched for applicants between 37 and 39 years of age. Any pair of applications from this sample might be no more similar than a randomly chosen pair from the population. However, this set of applications can be statistically analyzed to determine whether certain loan officers are more likely to approve loans from this section of the population), (Id., ¶ 44, Alternatively, it may not be necessary to find even one very similar parameter. Large segments of the population may be aggregated for analysis using criteria such as "applicants under 32 years old" or "applicants earning more than $30,000 per year." Extending this methodology one step further, a single analysis can be conducted on the sample consisting of the entire population), (Id., ¶ 79, Some humans may try to submit large volumes of suggestions hoping that at least one of them works. Others may even write computer code to generate many suggestions. As long as the computation resources needed to evaluate such suggestions is minimal, this is not a significant problem and may even contribute to the overall objective of useful analysis. To reduce the computational cost of the evaluation of suggestions, such suggestions may first be tested against a subset of the overall data. Suggestions would only be incorporated while analyzing the overall data if the suggestion enabled a significant improvement when used to analyze the subset data. To further save computation expenses, multiple suggestions evaluated on the subset data may be combined before the corresponding updated analysis is run on the complete data. (discloses obtaining guidance information based on an automated analysis of an impactful corpus catalog) Additionally, computation resources could be allocated to different users via a quota system, and users could optionally "purchase" more using their rewards from previous suggestions), (Id., ¶ 114, FIG. 3C illustrates a user interface for selecting the business outcome (the variable) that the user wishes to analyze. This is typically the KPI or metric in the user's dashboards and reports, e.g. revenue or cost. This page lists all the numeric or binary (e.g. Male/Female) columns in the user's data that have sufficient variability. If the user does not see a variable that is expected, the user may verify that the variable has numerical values and not text. If a variable has only a few values (e.g. 1, 2, 5), BeyondCore will treat it as a categorical variable instead of numeric. In most cases this is desired and statistically appropriate. To change a variable from categorical to numeric, the user may go to the Data Setup page and manually filter or reformat data (discloses filtering assessment parameters) (see advanced options at FIG. 4A). Returning to FIG. 3C, the user may click user element 345 to select the business outcome. (which would be the y-axis of the graph, or the number the user wants to predict)).
Regarding Claim 7, the combination of Sengupta, Malhotra and Demaris discloses …the method of claim 4…
Sengupta further discloses …wherein analyzing, based on the engagement action, the impactful corpus catalog to obtain the guidance information, further comprises: prior to making the determination: mapping the form field to a guiding assessment parameter in a set of assessment parameters… (Id., ¶ 43, Various types of automated analysis have been described previously by the inventors. For example, in the context of document processing by operators, one goal may be to find documents that are similar in some way in order to identify underlying patterns of operator behavior. A search can be conducted for segments of the data which share as few as one or more similar field or parameter values. For example, a database of loan applications can be searched for applicants between 37 and 39 years of age. Any pair of applications from this sample might be no more similar than a randomly chosen pair from the population. However, this set of applications can be statistically analyzed to determine whether certain loan officers are more likely to approve loans from this section of the population), (Id., ¶ 44, Alternatively, it may not be necessary to find even one very similar parameter. Large segments of the population may be aggregated for analysis using criteria such as "applicants under 32 years old" or "applicants earning more than $30,000 per year." Extending this methodology one step further, a single analysis can be conducted on the sample consisting of the entire population), (Id., ¶ 45, it is possible to analyze sets of data which do not contain all of the information that the operators use to make decisions. In the case of loan applications requiring a personal interview, it would be very hard to conduct a controlled experiment that includes the personal interview. It would also be difficult to search for "similar" interviews. However, we can still search for applications with some parameters similar, and aggregate the statistics across all interviews. It may not be possible to identify any single loan decision as incorrect or suspect, but if, for example, among applicants aged 26-28, earning over $32,000, one loan officer approves 12% of loans and another approves 74% of loans, there may be training or other issues).
While suggested in at least Fig. 1 and related text of Sengupta, the combination of Sengupta and Malhotra does not explicitly disclose …wherein the asset metadata belongs to a metadata field matching the guiding assessment parameter
However, Demaris discloses …wherein the asset metadata belongs to a metadata field matching the guiding assessment parameter (Demaris, ¶ 20, some computing systems store files on a cloud computing data storage system. However, during authoring and editing of a document, these same files may also be stored on a local disk by the application. Similarly, as discussed in further detail below, some systems have applications that support document collaboration, such as merge, sharing, co-authoring, and other functionalities or capabilities. In some cases, these systems can rely on a synchronization engine (or “sync engine”) that is responsible for detecting and synchronizing changes to files and folders between a local disk and a cloud storage system. In order to accomplish this, the sync engine can track the state of both the file that is stored on the disk and that stored in the cloud, and reconcile those states when it receives information that something has changed. As an example, if a file is edited on a local disk, the sync engine may detect that change, realize it needs to send the change to the cloud, transmit the change, wait for the cloud to respond, and then update its local state information to indicate that the change has been made. In cases where a file is the product of or subject to collaboration, additional and evolving metadata can be associated with the file. In some implementations, the system can be configured to maintain and communicate a set of collaborative metadata for each file. In other words, each file may have or be linked to metadata about the file. Such “collaborative metadata” for each file may be used in part to help maintain the various versions of the file, as well as provide important information (discloses analyzing asset metadata to obtain guidance information) about status, workflow tasks, or other aspects of the file. However, maintaining an up-do-date synchronization of the file as well as its associated collaborative metadata can cause delays during synchronization as well as consume significant amounts of computational resources. In order to help minimize the impact of the synchronization process on system and network resources, the following disclosure proposes a paradigm in which collaborative metadata for a file is essentially separated from the file during synchronization and subsequent user access of the file on a client device. In other words, while users download and/or access files via client device applications and these files continue to be synchronized with a cloud storage system, the collaborative metadata remains in a remote location (e.g., outside of the local device). In the event a user initiates an action in the file that requires access of the collaborative metadata, the proposed content management system (CMS) can respond by providing a portal or other network interface on the local device via which the user can directly access the collaborative metadata that is stored in the cloud. The synchronization process for files thus occurs without requiring transference of large, costly amounts of collaborative metadata to the client device. By enabling users to ‘reach out’ to the cloud for file metadata on an as-needed basis, the system as a whole can operate more smoothly and efficiently), (Id., ¶ 26, it may be understood that such workflows rely heavily on metadata that is associated with a file. As many users continue to open and access documents on client device applications, rather than web-based applications, the following implementations describe a mechanism by which collaborative metadata about the document's workflow status or history may remain in the cloud storage system. The following systems and methods offer users the ability to quickly access this remote collaborative metadata while working locally. In other words, work occurring on a client device for a document (e.g., as a local instance) can remain coupled with the collaborative metadata for the document via links available during document access, allowing users to continue to manage workflows and other resultant information about the document. In some implementations, a form can be generated that contains some or all the metadata that is determined to be of relevance for the document, depending on its configuration. That form can for example contain one or more fields, each field mapping to one piece of metadata, as will be discussed in greater detail below).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the impact assessment and guidance elements of Sengupta and the semantic similarity elements of Malhotra to include the metadata analysis elements of Demaris in the analogous art of remote access of metadata for collaborative documents for the same reasons as stated for claim 4.
Regarding Claim 8, the combination of Sengupta, Malhotra and Demaris discloses …the method of claim 4…
Sengupta further discloses …wherein monitoring the interactions, by the organization user, with the interactive assessment form further identifies a second engagement action (Id., ¶ 116, FIG. 3E illustrates a user interface that allows a user to further customize a BeyondCore story. The user may edit the story name, row labels, column labels, and the like. For example, UI element 360 allows a user to specify a story title, UI element 362 allows a user to specify how BeyondCore should refer to a row in the data, UI element 364 enables the user to specify the unit of the y-axis of his graphs (the outcome variable), UI element 368 enables the user to access advanced options or further customize a story).
PNG
media_image2.png
442
642
media_image2.png
Greyscale
Regarding Claim 9, the combination of Sengupta, Malhotra and Demaris discloses …the method of claim 8…
Sengupta further discloses …wherein the second engagement action reflects an editing of a second form field in the set of form fields (Id., ¶ 116, FIG. 3E illustrates a user interface that allows a user to further customize a BeyondCore story. The user may edit the story name, row labels, column labels, and the like. For example, UI element 360 allows a user to specify a story title, UI element 362 allows a user to specify how BeyondCore should refer to a row in the data, UI element 364 enables the user to specify the unit of the y-axis of his graphs (the outcome variable), UI element 368 enables the user to access advanced options or further customize a story).
Regarding Claim 10, the combination of Sengupta, Malhotra and Demaris discloses …the method of claim 9…
Sengupta further discloses …the method further comprising: for one selected from a group comprising prior to, and after, providing the guidance information: identifying an assessment parameter mapped to the second form field (Id., ¶ 116, FIG. 3E illustrates a user interface that allows a user to further customize a BeyondCore story. The user may edit the story name, row labels, column labels, and the like. For example, UI element 360 allows a user to specify a story title, UI element 362 allows a user to specify how BeyondCore should refer to a row in the data, UI element 364 enables the user to specify the unit of the y-axis of his graphs (the outcome variable), UI element 368 enables the user to access advanced options or further customize a story), (Id., ¶ 53, Because the metrics are highest for samples with acute variances, samples obtained using parameter values which are responsible for the unusual behavior will have the highest scores. Much larger and much smaller samples will have lower scores. As the search algorithm runs, the sample under consideration will "evolve" to contain the features that are causing the discrepancy in operator processing while not containing unrelated random information. Of course, the search will cease on one local maximum. If the local search is repeated multiple times from random starting samples, many samples with peak metrics can be identified in the data.
extracting a second field input from the second form field; filtering, based on the assessment parameter and the second field input, the impactful corpus catalog to identify a second catalog entry subset in the set of catalog entries (Id., ¶ 114, FIG. 3C illustrates a user interface for selecting the business outcome (the variable) that the user wishes to analyze. This is typically the KPI or metric in the user's dashboards and reports, e.g. revenue or cost. This page lists all the numeric or binary (e.g. Male/Female) columns in the user's data that have sufficient variability. If the user does not see a variable that is expected, the user may verify that the variable has numerical values and not text. If a variable has only a few values (e.g. 1, 2, 5), BeyondCore will treat it as a categorical variable instead of numeric. In most cases this is desired and statistically appropriate. To change a variable from categorical to numeric, the user may go to the Data Setup page and manually filter or reformat data (discloses filtering assessment parameters) (see advanced options at FIG. 4A). Returning to FIG. 3C, the user may click user element 345 to select the business outcome. (which would be the y-axis of the graph, or the number the user wants to predict));
obtaining an overall corpus catalog comprising a second set of catalog entries (Id., ¶ 79, Some humans may try to submit large volumes of suggestions hoping that at least one of them works. Others may even write computer code to generate many suggestions. As long as the computation resources needed to evaluate such suggestions is minimal, this is not a significant problem and may even contribute to the overall objective of useful analysis. To reduce the computational cost of the evaluation of suggestions, such suggestions may first be tested against a subset of the overall data. Suggestions would only be incorporated while analyzing the overall data if the suggestion enabled a significant improvement when used to analyze the subset data. To further save computation expenses, multiple suggestions evaluated on the subset data may be combined before the corresponding updated analysis is run on the complete data. (discloses obtaining guidance information based on an automated analysis of an impactful corpus catalog) Additionally, computation resources could be allocated to different users via a quota system, and users could optionally "purchase" more using their rewards from previous suggestions);
computing a parameter-specific impact score based on a first cardinality of the second catalog entry subset and a second cardinality of the second set of catalog entries (Id., ¶ 71, Humans may be provided financial or other rewards based on whether their feedback was useful and unique. For example, in the filtering case, a user might be rewarded based on the feedback's usefulness, namely how much better the p-value of their specified subset was than the average p-values of the top 10 subsets previously detected by the software automatically or with the help of humans. A uniqueness criterion may also be easily applied to the reward formula such that a higher reward would be paid if the human-specified subset differed significantly from previously identified subsets. The uniqueness of a user specified set N as compared to each of the previously identified sets S.sub.t may be determined by a formula such as the following: (Number of elements in N-Number of element in N intersect S.sub.t)/(Number of element in N intersect S.sub.t). Other uniqueness and usefulness criteria might be applied instead or in addition (discloses cardinality in catalog subsets)), (Id., ¶ 72, For feedback involving regression models or combinations of fields to be used in the model, a very similar approach combining usefulness and uniqueness can be used. Usefulness can be determined by the improvement in the "fit" of the model while uniqueness can be determined by whether a substantially similar model has already been submitted previously or detected automatically), (Id., ¶ 39, Steps 110 and 120 are the automatic analysis of large data sets and the automatic detection of potentially valuable and meaningful patterns within those data sets. We have previously disclosed multiple approaches to automatically analyzing data to detect underlying patterns and insights. Examples include U.S. Pat. No. 7,849,062 "Identifying and Using Critical Fields in Quality Management" that disclosed means to automatically detect underlying error patterns in data processing operations as well as pending patent application PCT/US2011/033489 "Identifying and Using Critical Fields in Quality Management" that disclose additional approaches to automatically analyzing data to detect underlying patterns. While some of these inventions were described in the context of data processing or human error patterns detection, the underlying methods are also applicable to a broad range of analytics. In U.S. patent application Ser. No. 13/249,168 "Analyzing Large Data Sets to Find Operator Deviation Patterns," we specifically disclosed approaches that allowed the automatic detection of subsets of data with high p-values indicating the high likelihood that the specific subset contained some underlying patterns and that the corresponding data distribution was unlikely to have been random. Thus, the underlying patterns have a higher chance of leading to meaningful actionable insights. These approaches can be applied to analyses including but not limited to customer segmentation (psychographics), sales analysis, marketing campaign optimization, demand forecasting, inventory/resource/supply chain optimization, assortment/product mix optimization, causal analysis, fraud detection, overbilling detection, and risk analysis. All of the foregoing are incorporated by reference herein), (Id., ¶ 52, Local search methods operate by considering a given sample, and repeatedly modifying it with the goal of raising the metric. This continues until the metric is higher for the sample under consideration than for any nearby samples (a local optimum). The notion of proximity is complex for samples of the sort we are discussing. The "modify" step in the algorithm will change the restrictions defining the current sample. This can consist of widening or tightening the restriction on one field, or adding a restriction on a new field, or removing the restriction on a restricted field. For example, if we consider a sample consisting of "Loan applications from females aged 30-40" and calculate the metric to be X, we could then calculate the metric for "females", "females aged 30-50", "females aged 20-40", "people aged 30-40", and others. Each of these metrics will be compared to X and the search algorithm will continue),
computing an overall impact score based on a set of parameter-specific impact scores comprising the parameter-specific impact score; and updating the interactive assessment form using the parameter-specific impact score and the overall impact score (Id., ¶ 53, the metrics are highest for samples with acute variances, samples obtained using parameter values which are responsible for the unusual behavior will have the highest scores. Much larger and much smaller samples will have lower scores. As the search algorithm runs, the sample under consideration will "evolve" to contain the features that are causing the discrepancy in operator processing while not containing unrelated random information. Of course, the search will cease on one local maximum. If the local search is repeated multiple times from random starting samples, many samples with peak metrics can be identified in the data), (Id., ¶ 104, The impact of each variable combination typically is determined by the behavior of a variable combination with respect to the outcome and by the population of the variable combination. In one approach, automated analysis learns the normative behavior for each variable combination as it relates to the outcome. For example it may learn that Men in California spend more while 18 to 25 year olds who buy over the Mobile channel spend less than usual in general (here amount spent is the outcome). But a specific transaction may be for a Male 18 to 25 years old from California who purchased goods over the Mobile channel. By observing the norm for each variable combination in isolation and in combinations across multiple transactions, we can learn the "net impact" (the behavior) of a variable combination. This is the positive or negative impact of the variable combination on the observed outcome, net of the impact of all other variable combinations that may also be affecting that specific transaction. This allows automated analysis to learn a behavior metric that is similar to obtaining a regression coefficient in a regression analysis, but which can be learned via the search-based approach described above with reference to FIG. 1, instead of running a regression analysis. In an alternative approach, a type of regression analysis is run for the outcome with respect to all of the variable combinations being considered. For each variable combination, there will be a regression term (the impact) that equals the regression coefficient (the behavior) multiplied by the population. Behavior may also be measured in terms of correlation coefficients, net-effect impact net of all other variables, or any other suitable metric that captures how the variable combination affects the outcome or how the outcome trends as a function of the variable combinations. Population may also be measured in terms of counts (i.e., number of observations), whether or not something occurred, frequency/percentage of overall population, or relative frequencies of observations. The overall impact of a variable combination depends on both its behavior (i.e., how strongly does that variable combination affect the outcome) and its population (i.e., how much of that variable combination exists in the data set of interest). These impacts, behaviors and populations can then be used to analyze the data set in different ways), (Id., Fig. 8M, Figure depicts updating an interactive assessment form with variable (i.e. parameter) scores and an overall score.
PNG
media_image3.png
467
687
media_image3.png
Greyscale
Regarding Claim 11, the combination of Sengupta, Malhotra and Demaris discloses …the method of claim 10…
Sengupta further discloses …wherein the interactive assessment form further comprises a set of parameter-specific impact score indicators, and wherein updating the interactive assessment form comprises: identifying a parameter-specific impact score indicator, in the set of parameter-specific impact score indicators, mapped to the assessment parameter (Id., ¶ 148, FIGS. 9A-9B illustrate diagnostic graphs that highlight multiple unrelated factors that contribute to an outcome or visual pattern displayed in a graph. (discloses parameter-specific impact score indicators) A diagnostic graph results from a diagnostic analysis performed on a dataset to analyze differences in an outcome between a data set for a process and a subset of the data set. For instance, referring to the graph of FIG. 9B, outcome value 925 represents the outcome for the data set and outcome value 927 represents the outcome for the subset {gender=male}. The diagnostic analysis and the resulting diagnostic graph then presents various contributing factors (represented as 926-a, 926-b, 926-c, 926-d, and 926-e) that caused or can explain a discrepancy (925 versus 927) between the outcome value for the data set and that for the subset. The contributing factors are those variable combinations for which the estimated contribution to the outcome additively explain (sums up to) the difference between the outcome for the data set and for the subset), (Id., ¶ 151, For these pairs, the analysis estimates contributions of the pair to differences in the outcome between the data set and the subset, based on differences in the behaviors of the pair and also based on differences in populations of the pair. In one approach, for each pair, an outcome for each of the two variable combinations is computed as a product of the (a) behavior of that variable combination with respect to the outcome and (b) the population of the subgroup defined by that variable combination. The difference in outcomes for the two pairs is used to assess a contribution of the pair to differences in the outcome between the data set and the subset), (Id., ¶ 152, Differences in the outcome between the data set and the subset is reported based on the estimated contributions for the variable combinations, for example in the form of a diagnostic graph such as the one illustrated in FIG. 9B. Similarly, contributing factor 926-a is due to the variable combination {Gender=Male}, which is example pair 1 in Table 1b above. Contributing factors 926-b et al are due to other variable combinations).
PNG
media_image4.png
472
623
media_image4.png
Greyscale
and replacing, with the parameter-specific impact score, a previous parameter-specific impact score displayed by the parameter-specific impact score indicator (Id., ¶ 150, To determine the drivers of the differences between the data set and the subset, corresponding pairs of variable combinations are considered, where the test variables take the trial values in one of the variable combinations and are not specified in the other variable combination. Examples of pairs of variable combinations for Example 1 of Table 1 a are illustrated in Table 1b), (Id., ¶ 134, The story page shows an executive report based on the results of the analysis. Illustrated in FIG. 8C are four main areas: Home Menu, Toolbar 850, Story 854, and Story Menu 852. Toolbar 850 drives the Story Menu 852 and points to other actions of interest. In some embodiments, the story is shown as soon as the initial analysis is completed. The Story Menu 852 can take different views: table of contents or recommendations. The Story 854 enables the user to scroll through the story that BeyondCore has guided the user to and that the user has optionally actively updated. In some cases, the initial story may be shown even before certain complex computations are completed), (Id., ¶ 169, The automated analysis learns the normative behavior over time for different variable combinations. Then it continues collecting data. The data may not perfectly conform to the learned norms but may be within statistical tolerance. Over time, the analysis may encounter new data where the behaviors or relative populations for certain variable combinations start to deviate significantly from the learned norm. Such cases can be flagged to the untrained human who can intervene if this is a deviation from the norm, or who can indicate that this is a one time deviation from the norm that can be ignored (for example an impact on tourism because of the World Cup), or who can indicate that this is just an evolution of the norm, in which case the automated analysis can adjust its understanding of the normative behavior by updating the learned model based on the new data).
Regarding Claim 12, the combination of Sengupta, Malhotra and Demaris discloses …the method of claim 11…
Sengupta further discloses …wherein the interactive assessment form further comprises an overall impact score indicator, and wherein updating the interactive assessment form further comprises: replacing, with the overall impact score, a previous impact score displayed by the overall impact score indicator (Id., ¶ 104, The impact of each variable combination typically is determined by the behavior of a variable combination with respect to the outcome and by the population of the variable combination. In one approach, automated analysis learns the normative behavior for each variable combination as it relates to the outcome. For example it may learn that Men in California spend more while 18 to 25 year olds who buy over the Mobile channel spend less than usual in general (here amount spent is the outcome). But a specific transaction may be for a Male 18 to 25 years old from California who purchased goods over the Mobile channel. By observing the norm for each variable combination in isolation and in combinations across multiple transactions, we can learn the "net impact" (the behavior) of a variable combination. This is the positive or negative impact of the variable combination on the observed outcome, net of the impact of all other variable combinations that may also be affecting that specific transaction. This allows automated analysis to learn a behavior metric that is similar to obtaining a regression coefficient in a regression analysis, but which can be learned via the search-based approach described above with reference to FIG. 1, instead of running a regression analysis. In an alternative approach, a type of regression analysis is run for the outcome with respect to all of the variable combinations being considered. For each variable combination, there will be a regression term (the impact) that equals the regression coefficient (the behavior) multiplied by the population. Behavior may also be measured in terms of correlation coefficients, net-effect impact net of all other variables, or any other suitable metric that captures how the variable combination affects the outcome or how the outcome trends as a function of the variable combinations. Population may also be measured in terms of counts (i.e., number of observations), whether or not something occurred, frequency/percentage of overall population, or relative frequencies of observations. The overall impact of a variable combination depends on both its behavior (i.e., how strongly does that variable combination affect the outcome) and its population (i.e., how much of that variable combination exists in the data set of interest). These impacts, behaviors and populations can then be used to analyze the data set in different ways), ), (Id., ¶ 134, The story page shows an executive report based on the results of the analysis. Illustrated in FIG. 8C are four main areas: Home Menu, Toolbar 850, Story 854, and Story Menu 852. Toolbar 850 drives the Story Menu 852 and points to other actions of interest. In some embodiments, the story is shown as soon as the initial analysis is completed. The Story Menu 852 can take different views: table of contents or recommendations. The Story 854 enables the user to scroll through the story that BeyondCore has guided the user to and that the user has optionally actively updated. In some cases, the initial story may be shown even before certain complex computations are completed).
Regarding Claim 13, the combination of Sengupta, Malhotra and Demaris discloses …the method of claim 10…
Sengupta further discloses …and providing, through the interactive assessment form, the second guidance information to the organization user (Id., ¶ 79, Some humans may try to submit large volumes of suggestions hoping that at least one of them works. Others may even write computer code to generate many suggestions. As long as the computation resources needed to evaluate such suggestions is minimal, this is not a significant problem and may even contribute to the overall objective of useful analysis. To reduce the computational cost of the evaluation of suggestions, such suggestions may first be tested against a subset of the overall data. Suggestions would only be incorporated while analyzing the overall data if the suggestion enabled a significant improvement when used to analyze the subset data. To further save computation expenses, multiple suggestions evaluated on the subset data may be combined before the corresponding updated analysis is run on the complete data. (discloses guidance information based on an automated analysis of an impactful corpus catalog) Additionally, computation resources could be allocated to different users via a quota system, and users could optionally "purchase" more using their rewards from previous suggestions).
While suggested in at least Fig. 1 and related text of Sengupta, the combination of Sengupta and Malhotra does not explicitly disclose …wherein the second form field is a complex form field, and the method further comprises: after updating the interactive assessment form: identifying, from metadata collectively maintained across the second catalog entry subset, a metadata subset associated with a metadata field matching the assessment parameter; analyzing the metadata subset to obtain second guidance information;
However, Demaris discloses …wherein the second form field is a complex form field, and the method further comprises: after updating the interactive assessment form: identifying, from metadata collectively maintained across the second catalog entry subset, a metadata subset associated with a metadata field matching the assessment parameter (Demaris, ¶ 26, Generally, it may be understood that such workflows rely heavily on metadata that is associated with a file. As many users continue to open and access documents on client device applications, rather than web-based applications, the following implementations describe a mechanism by which collaborative metadata about the document's workflow status or history may remain in the cloud storage system. The following systems and methods offer users the ability to quickly access this remote collaborative metadata while working locally. In other words, work occurring on a client device for a document (e.g., as a local instance) can remain coupled with the collaborative metadata for the document via links available during document access, allowing users to continue to manage workflows and other resultant information about the document. In some implementations, a form can be generated that contains some or all the metadata that is determined to be of relevance for the document, depending on its configuration. That form can for example contain one or more fields, each field mapping to one piece of metadata, as will be discussed in greater detail below), (Id., ¶ 40, The collaborative document 252, stored in a collaborative document repository 250, can include the electronic content for the document itself. In some implementations, the metadata and corresponding metadata-based action options for the collaborative document 252 may be stored in a separate module (here, metadata service 240). The metadata service 240 can provide the second user 204, via the second client system 238, with an online metadata interface 246 for engaging with various workflow options or other metadata-based activities for the selected document. In some implementations, the changes or other information generated in conjunction with the metadata-based activities can be recorded in a collaborative document metadata store 248), (Id., ¶ 56, As shown next in FIG. 5, the menu option 412, when selected, can elicit a response from the system. In some implementations, the system can respond to the selection of the menu option 412 with a presentation of an (optional) metadata options interface 510 that can provide users with a more specialized range of actions that may be performed in conjunction with the available metadata content. Thus, in some implementations, the metadata options interface 510 can offer a plurality of actuatable options for initiation of a variety of tasks that employ or rely on the metadata content in some manner in order to be executed while remaining “in” the context or environment of the native application 340. As an example, the metadata options interface 510 includes a first option 512, a second option 514, and a third option 516. The first option 512 offers the capability to “Add an Action” to the document (or the document workflow), the second option 514 (“Review Workflow Tasks”) offers the capability to view actions that have been added to the document previously and/or by other collaborators of the document, and the third option 516 offers the capability of viewing or accessing additional options relevant to the document workflow or that employ document metadata. The metadata options interface 510 is shown for illustrative purposes only, and any other type of interface or options may be offered or presented to the user in response to a triggering event. In some implementations, the metadata options interface 510 may not be shown, and selection of the menu option 412 can lead directly to an arrangement similar to that described with reference to FIG. 6);
analyzing the metadata subset to obtain second guidance information (Id., ¶ 20, some computing systems store files on a cloud computing data storage system. However, during authoring and editing of a document, these same files may also be stored on a local disk by the application. Similarly, as discussed in further detail below, some systems have applications that support document collaboration, such as merge, sharing, co-authoring, and other functionalities or capabilities. In some cases, these systems can rely on a synchronization engine (or “sync engine”) that is responsible for detecting and synchronizing changes to files and folders between a local disk and a cloud storage system. In order to accomplish this, the sync engine can track the state of both the file that is stored on the disk and that stored in the cloud, and reconcile those states when it receives information that something has changed. As an example, if a file is edited on a local disk, the sync engine may detect that change, realize it needs to send the change to the cloud, transmit the change, wait for the cloud to respond, and then update its local state information to indicate that the change has been made. In cases where a file is the product of or subject to collaboration, additional and evolving metadata can be associated with the file. In some implementations, the system can be configured to maintain and communicate a set of collaborative metadata for each file. In other words, each file may have or be linked to metadata about the file. Such “collaborative metadata” for each file may be used in part to help maintain the various versions of the file, as well as provide important information (discloses analyzing asset metadata to obtain guidance information) about status, workflow tasks, or other aspects of the file. However, maintaining an up-do-date synchronization of the file as well as its associated collaborative metadata can cause delays during synchronization as well as consume significant amounts of computational resources. In order to help minimize the impact of the synchronization process on system and network resources, the following disclosure proposes a paradigm in which collaborative metadata for a file is essentially separated from the file during synchronization and subsequent user access of the file on a client device. In other words, while users download and/or access files via client device applications and these files continue to be synchronized with a cloud storage system, the collaborative metadata remains in a remote location (e.g., outside of the local device). In the event a user initiates an action in the file that requires access of the collaborative metadata, the proposed content management system (CMS) can respond by providing a portal or other network interface on the local device via which the user can directly access the collaborative metadata that is stored in the cloud. The synchronization process for files thus occurs without requiring transference of large, costly amounts of collaborative metadata to the client device. By enabling users to ‘reach out’ to the cloud for file metadata on an as-needed basis, the system as a whole can operate more smoothly and efficiently).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the impact assessment and guidance elements of Sengupta and the semantic similarity elements of Malhotra to include the metadata analysis elements of Demaris in the analogous art of remote access of metadata for collaborative documents for the same reasons as stated for claim 4.
Regarding Claim 14, the combination of Sengupta, Malhotra and Demaris discloses …the method of claim 9…
Sengupta further discloses …wherein the second form field is one selected from a group comprising a same form field, and a different form field, as the form field (Id., ¶ 52, L ocal search methods operate by considering a given sample, and repeatedly modifying it with the goal of raising the metric. This continues until the metric is higher for the sample under consideration than for any nearby samples (a local optimum). The notion of proximity is complex for samples of the sort we are discussing. The "modify" step in the algorithm will change the restrictions defining the current sample. This can consist of widening or tightening the restriction on one field, or adding a restriction on a new field, or removing the restriction on a restricted field. For example, if we consider a sample consisting of "Loan applications from females aged 30-40" and calculate the metric to be X, we could then calculate the metric for "females", "females aged 30-50", "females aged 20-40", "people aged 30-40", and others. Each of these metrics will be compared to X and the search algorithm will continue), (Id., ¶ 53, Because the metrics are highest for samples with acute variances, samples obtained using parameter values which are responsible for the unusual behavior will have the highest scores. Much larger and much smaller samples will have lower scores. As the search algorithm runs, the sample under consideration will "evolve" to contain the features that are causing the discrepancy in operator processing while not containing unrelated random information. Of course, the search will cease on one local maximum. If the local search is repeated multiple times from random starting samples, many samples with peak metrics can be identified in the data).
Regarding Claim 15, the combination of Sengupta and Malhotra discloses …the method of claim 3…
Sengupta further discloses …wherein analyzing, based on the engagement action, the impactful corpus catalog to obtain the guidance information, comprises: mapping the form field to a guiding assessment parameter in a set of assessment parameters (Id., ¶ 43, Various types of automated analysis have been described previously by the inventors. For example, in the context of document processing by operators, one goal may be to find documents that are similar in some way in order to identify underlying patterns of operator behavior. A search can be conducted for segments of the data which share as few as one or more similar field or parameter values. For example, a database of loan applications can be searched for applicants between 37 and 39 years of age. Any pair of applications from this sample might be no more similar than a randomly chosen pair from the population. However, this set of applications can be statistically analyzed to determine whether certain loan officers are more likely to approve loans from this section of the population), (Id., ¶ 44, Alternatively, it may not be necessary to find even one very similar parameter. Large segments of the population may be aggregated for analysis using criteria such as "applicants under 32 years old" or "applicants earning more than $30,000 per year." Extending this methodology one step further, a single analysis can be conducted on the sample consisting of the entire population);
making a determination, based on a search across the set of form fields, that each form field, in the set of form fields, is an empty form field (Id., ¶ 51, In a stable process where there were no deviations from the norm, the variance would be significantly lower than in a process with patterns of deviations from the norm. Any of these metrics, or others, can be used as the basis of a hill climb or other local search method to identify interesting samples of the population that would be most useful to analyze to detect underlying patterns of deviations from norms or fragmented norms. A key property of these metrics is that they are highest for the section of the document population that actually represents the variance in operator behavior. For example, if one operator is not approving loans from males aged 20-30, the metric should be higher for "males aged 20-30" than for "males aged 20-50" and "people aged 20-30."), (Id., ¶ 52, Local search methods operate by considering a given sample, and repeatedly modifying it with the goal of raising the metric. This continues until the metric is higher for the sample under consideration than for any nearby samples (a local optimum). The notion of proximity is complex for samples of the sort we are discussing. The "modify" step in the algorithm will change the restrictions defining the current sample. This can consist of widening or tightening the restriction on one field, or adding a restriction on a new field, or removing the restriction on a restricted field. For example, if we consider a sample consisting of "Loan applications from females aged 30-40" and calculate the metric to be X, we could then calculate the metric for "females", "females aged 30-50", "females aged 20-40", "people aged 30-40", and others. Each of these metrics will be compared to X and the search algorithm will continue), (Id., ¶ 53, Because the metrics are highest for samples with acute variances, samples obtained using parameter values which are responsible for the unusual behavior will have the highest scores. Much larger and much smaller samples will have lower scores. As the search algorithm runs, the sample under consideration will "evolve" to contain the features that are causing the discrepancy in operator processing while not containing unrelated random information. Of course, the search will cease on one local maximum. If the local search is repeated multiple times from random starting samples, many samples with peak metrics can be identified in the data), (Id., ¶ 101, Depending on the application, the data set may contain at least 100,000 observations, at least 1,000,000 observations or more. The outcome may be directly observed or it may be derived. Typically, the data set will be organized into rows and columns, where each column is a different outcome or variable and each row is a different observation of the outcomes and variables. Typically, not every cell will be filled. That is, some variables may be blank for some observations).
While suggested in at least Fig. 1 and related text of Sengupta, the combination of Sengupta and Malhotra does not explicitly disclose …and analyzing, based on the determination, asset metadata across the set of catalog entries to obtain the guidance information, wherein the asset metadata belongs to a metadata field matching the guiding assessment parameter.
However, Demaris discloses …and analyzing, based on the determination, asset metadata across the set of catalog entries to obtain the guidance information, wherein the asset metadata belongs to a metadata field matching the guiding assessment parameter (Demaris, ¶ 20, some computing systems store files on a cloud computing data storage system. However, during authoring and editing of a document, these same files may also be stored on a local disk by the application. Similarly, as discussed in further detail below, some systems have applications that support document collaboration, such as merge, sharing, co-authoring, and other functionalities or capabilities. In some cases, these systems can rely on a synchronization engine (or “sync engine”) that is responsible for detecting and synchronizing changes to files and folders between a local disk and a cloud storage system. In order to accomplish this, the sync engine can track the state of both the file that is stored on the disk and that stored in the cloud, and reconcile those states when it receives information that something has changed. As an example, if a file is edited on a local disk, the sync engine may detect that change, realize it needs to send the change to the cloud, transmit the change, wait for the cloud to respond, and then update its local state information to indicate that the change has been made. In cases where a file is the product of or subject to collaboration, additional and evolving metadata can be associated with the file. In some implementations, the system can be configured to maintain and communicate a set of collaborative metadata for each file. In other words, each file may have or be linked to metadata about the file. Such “collaborative metadata” for each file may be used in part to help maintain the various versions of the file, as well as provide important information (discloses analyzing asset metadata to obtain guidance information) about status, workflow tasks, or other aspects of the file. However, maintaining an up-do-date synchronization of the file as well as its associated collaborative metadata can cause delays during synchronization as well as consume significant amounts of computational resources. In order to help minimize the impact of the synchronization process on system and network resources, the following disclosure proposes a paradigm in which collaborative metadata for a file is essentially separated from the file during synchronization and subsequent user access of the file on a client device. In other words, while users download and/or access files via client device applications and these files continue to be synchronized with a cloud storage system, the collaborative metadata remains in a remote location (e.g., outside of the local device). In the event a user initiates an action in the file that requires access of the collaborative metadata, the proposed content management system (CMS) can respond by providing a portal or other network interface on the local device via which the user can directly access the collaborative metadata that is stored in the cloud. The synchronization process for files thus occurs without requiring transference of large, costly amounts of collaborative metadata to the client device. By enabling users to ‘reach out’ to the cloud for file metadata on an as-needed basis, the system as a whole can operate more smoothly and efficiently), (Id., ¶ 26, Generally, it may be understood that such workflows rely heavily on metadata that is associated with a file. As many users continue to open and access documents on client device applications, rather than web-based applications, the following implementations describe a mechanism by which collaborative metadata about the document's workflow status or history may remain in the cloud storage system. The following systems and methods offer users the ability to quickly access this remote collaborative metadata while working locally. In other words, work occurring on a client device for a document (e.g., as a local instance) can remain coupled with the collaborative metadata for the document via links available during document access, allowing users to continue to manage workflows and other resultant information about the document. In some implementations, a form can be generated that contains some or all the metadata that is determined to be of relevance for the document, depending on its configuration. That form can for example contain one or more fields, each field mapping to one piece of metadata, as will be discussed in greater detail below).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the impact assessment and guidance elements of Sengupta and the semantic similarity elements of Malhotra to include the metadata analysis elements of Demaris in the analogous art of remote access of metadata for collaborative documents for the same reasons as stated for claim 4.
Regarding Claim 19, this claim recites limitations substantially similar to those recited in claim 4, and is rejected for the same reasons as stated above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Olsen et al., U.S. Publication No. 2017/0220535 discloses enterprise writing assistance.
Towle et al., U.S. Publication No. 2015/0104765 discloses adaptive grammar instruction for parallel structures.
Sandholm et al., U.S. Patent No. 10,958,694 discloses sharing content between collocated mobile devices in an ad-hoc private social group.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS D BOLEN whose telephone number is (408)918-7631. The examiner can normally be reached Monday - Friday 8:00 AM - 5:00 PM PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Patty Munson can be reached at (571) 270-5396. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NICHOLAS D BOLEN/Examiner, Art Unit 3624 /HAMZEH OBAID/Primary Examiner, Art Unit 3624 June 19, 2026
Prosecution Insights